In inferential statistics, we use patterns in the sample data to draw inferences about the population represented, accounting for randomness. There are two basic approaches are:
- Hypothesis testing
- Estimation
Common goal is to conclude on the effect of an independent variable (exposure) on a dependent variable (outcome).
The aim of a statistical test
To reach a scientific decision (“yes” or “no”) on a difference (or effect), on a probabilistic basis, on observed data.
Why significance testing?
Consider an example of gastroenteritis outbreak in Karachi:
“The risk of illness was higher among diners who ate home preserved pickles (RR=3.6).”
Is the association due to chance?
The two hypothesis!
| There is NO difference between the two groups(=no effect) |
Null Hypothesis (H0)(e.g.: RR=1) |
| There is a difference between the two groups(=there is an effect) |
Alternative Hypothesis (H1)(e.g.: RR=3.6) |
When you perform a test of statistical significance you usually reject or do not reject the Null Hypothesis (H0).
Gastroenteritis outbreak in Karachi
- Null hypothesis (H0): “There is no association between consumption of green pickles and gastroenteritis.”
- Alternative hypothesis (H1): “There is an association between consumption of green pickles and gastroenteritis.”
Hypothesis testing and null hypothesis
Tests of statistical significance are applied. If:
Data not consistent with H0:
– H0 can be rejected in favour of some alternative hypothesis H1 (the objective of our study).
Data are consistent with the H0:
– H0 cannot be rejected
You cannot say that the H0 is true. You can only decide to reject it or not reject it.
How to decide when to reject the null hypothesis?
H0 is rejected using reported p value.
p-value is the probability that our result (e.g. a difference between proportions or a RR) or more extreme values could be observed under the null hypothesis.
p values – practicalities
If p value is small, it indicates low degree of compatibility between H0 and the observed data, you reject H0, the test is significant.
If p value is large, there is high degree of compatibility between H0 and the observed data, you don’t reject H0, the test is not significant.
We can never reduce to zero the probability that our result was not observed by chance alone.
Levels of significance – practicalities
p value > 0.05 = H0 non rejected (non significant)
p value ≤ 0.05 = H0 rejected (significant)
BUT:
Give always the exact p-value rather than significant vs non-significant.
Examples from the literature
- ”The limit for statistical significance was set at p=0.05.”
- ”There was a strong relationship (p 1 risk factor,
if < 1 protective factor
(independently from sample size)
CI Magnitude and precision of effect
Comments on p-values and Confidence Intervals
Presence of significance does not prove clinical or biological relevance of an effect. A lack of significance is not necessarily a lack of an effect: “Absence of evidence is not evidence of absence”.
A huge effect in a small sample or a small effect in a large sample can result in identical p values. A statistical test will always give a significant result if the sample is big enough. p values and confidence intervals do not provide any information on the possibility that the observed association is due to bias or confounding.
Read more
Tests of Statistical Significance
Significance Testing and p value – howMed
In inferential statistics, we use patterns in the sample data to draw inferences about the population represented, accounting for randomness. There are two basic approaches are:
Common goal is to conclude on the effect of an independent variable (exposure) on a dependent variable (outcome).
The aim of a statistical test
To reach a scientific decision (“yes” or “no”) on a difference (or effect), on a probabilistic basis, on observed data.
Why significance testing?
Consider an example of gastroenteritis outbreak in Karachi:
“The risk of illness was higher among diners who ate home preserved pickles (RR=3.6).”
Is the association due to chance?
The two hypothesis!
When you perform a test of statistical significance you usually reject or do not reject the Null Hypothesis (H0).
Gastroenteritis outbreak in Karachi
Hypothesis testing and null hypothesis
Tests of statistical significance are applied. If:
Data not consistent with H0:
– H0 can be rejected in favour of some alternative hypothesis H1 (the objective of our study).
Data are consistent with the H0:
– H0 cannot be rejected
You cannot say that the H0 is true. You can only decide to reject it or not reject it.
How to decide when to reject the null hypothesis?
H0 is rejected using reported p value.
p-value is the probability that our result (e.g. a difference between proportions or a RR) or more extreme values could be observed under the null hypothesis.
p values – practicalities
If p value is small, it indicates low degree of compatibility between H0 and the observed data, you reject H0, the test is significant.
If p value is large, there is high degree of compatibility between H0 and the observed data, you don’t reject H0, the test is not significant.
We can never reduce to zero the probability that our result was not observed by chance alone.
Levels of significance – practicalities
p value > 0.05 = H0 non rejected (non significant)
p value ≤ 0.05 = H0 rejected (significant)
BUT:
Give always the exact p-value rather than significant vs non-significant.
Examples from the literature
if < 1 protective factor
(independently from sample size)
CI Magnitude and precision of effect
Comments on p-values and Confidence Intervals
Presence of significance does not prove clinical or biological relevance of an effect. A lack of significance is not necessarily a lack of an effect: “Absence of evidence is not evidence of absence”.
A huge effect in a small sample or a small effect in a large sample can result in identical p values. A statistical test will always give a significant result if the sample is big enough. p values and confidence intervals do not provide any information on the possibility that the observed association is due to bias or confounding.
Read more
Tests of Statistical Significance