The phrase originated during World War II. Lexicographer Eric Partridge attributes it to British army intelligence very early in the war (in the plural illegitimis). The phrase was adopted by US Army general "Vinegar" Joe Stillwell as his motto during the war.[1] It was later further popularized in the US by 1964 presidential candidate Barry Goldwater.
I guess it was just quoted in a Harry Potter book.......my bad!