Hypothesis Testing

Terminology for Hypothesis Testing

• Hypothesis tests are based on two hypotheses. The null hypothesis, H_{0}, is a statement about the value of a population parameter (a parameter of the distribution of a random variable) which our data will tell us whether or not to reject. The alternate hypothesis, H_{1}, is what we believe the parameter is if we reject the null hypothesis.

• In general, the null hypothesis is something that we want to show is false. This is because hypothesis tests do not show if something is true, only if something is false, but we can find out something true that we want to know by showing that the null hypothesis is false. To this end, the null hypothesis is usually that a parameter takes a specific value, while the alternate hypothesis is usually that the parameter differs from the specific value, rather than specifying another value.

• A hypothesis test is the means by which we generate a test statistic that directs us to either reject or not reject the null hypothesis.

• The test statistic is a “summary” of the collected data, and should have a sampling distribution specified by the null hypothesis.

One or Two Tailed Tests

Hypothesis tests can be one tailed or two tailed. This depends on H_{1}.

• In a one tailed test, H_{1} takes the form p>x or p<x, where H_{0} is that p=x.
• In a two tailed test, H_{1} takes the form p\neq x, where H_{0} is that p=x.

Significant Data

We reject the null hypothesis when the data we observe is unlikely to have occurred if it were true.

Specifically, we state a significance level \alpha before we perform the hypothesis test, and if the probability of getting the data we got is less than \alpha in a one tail test or less than \dfrac{\alpha}{2} in a two tail test if we assume H_{0} is true, then we reject H_{0}.

The way we test the probability of getting the data is by looking at the sampling distribution of the test statistic, which is set by the null hypothesis.

You will usually be told what significance level to use. Common significance levels include 5\%\;(\alpha)=0.05 and 1\%\;(\alpha)=0.01

Critical Region

The critical region is the set of values of the test statistic that would cause H_{0} to be rejected. The first value inside the critical region is called the critical value. If the test statistic is as extreme or more extreme than the critical value, then we reject H_{0}.

A one tailed test has a single critical region, containing the highest or lowest values. A two tailed test has two critical regions, one containing high values and one containing low values.

You can test whether your data is significant by finding the critical region and seeing if the test statistic falls within it.

Actual Significance Level and p-value

The p-value is the probability of obtaining the results we got if H_{0} is true. If the p-value is less than \alpha (or \dfrac{\alpha}{2} for two tail test) we reject H_{0}. If the p-value is greater than \alpha (or \dfrac{\alpha}{2} for two tail test) we do not reject H_{0}.

The actual significance level is the probability of the data being in the critical region if H_{0} is true. For continuous data, this is the same as the significance level. However, for discrete data, it might differ.