In main street interpretation of statistical findings, type II errors are often swept under the rug.
A good illustration of the danger in only telling the type I side of the story is [xkcd/1132](http://xkcd.com/1132/), the frequentist looks hilariously stupid, and we all hail the Bayesian smartness.
Yet a closer look from the frequentist view, the type II error of the test: H_0: the sun has gone nova is Pr[detector reports no|the sun has indeed exploded] = 1 - 1/36 > 95%; in other words the power of the test(1-Pr[type II error]) is extremely low (1/36). Even though the test has a reasonably low level alpha=0.05(type I error), it's still pretty useless as its power is also pathetic.
The comic made the frequentist look idiotic, however, he's just not a good frequentist with a sufficient grasp of type II error. In reality, many people are like the freqentist, easily satisfied with a low p-value without thinking about the fundamental usefulness of their tests (power), becoming victims of type II errors.
EDIT: I made a mistake, see eli_gottlieb's comment below for correction.
Actually, the comic specifies a 1/36 chance that the detector lies, no matter which result is true. So the P(detector reports no | sun has in fact exploded) = 1/36, making it a very strong test. The real problem there is that the p-value just doesn't mean what he thinks it means, and he in fact hasn't factored in any prior for how often the sun actually explodes.
Oops, you are right, I am a bad freqentist. Just had the discussion about the comic with a friend verbally, thought we figured it out. I guess it is really easy to get even the most basic concepts applied wrong.
This is one of those fundamental things they should just make mandatory in grade school. I constantly find myself explaining this to people I work with - and even though it's kind of intuitive ("penny wise, pound foolish", "straining gnats to swallow camels"), it helps to have a formal concept like this to hang the concept on.
A good illustration of the danger in only telling the type I side of the story is [xkcd/1132](http://xkcd.com/1132/), the frequentist looks hilariously stupid, and we all hail the Bayesian smartness.
Yet a closer look from the frequentist view, the type II error of the test: H_0: the sun has gone nova is Pr[detector reports no|the sun has indeed exploded] = 1 - 1/36 > 95%; in other words the power of the test(1-Pr[type II error]) is extremely low (1/36). Even though the test has a reasonably low level alpha=0.05(type I error), it's still pretty useless as its power is also pathetic.
The comic made the frequentist look idiotic, however, he's just not a good frequentist with a sufficient grasp of type II error. In reality, many people are like the freqentist, easily satisfied with a low p-value without thinking about the fundamental usefulness of their tests (power), becoming victims of type II errors.
EDIT: I made a mistake, see eli_gottlieb's comment below for correction.