If a random sample of 10 people found that 9 were pro-life (i.e., 90%), while another random sample of 1000 people from the same population found that 550 were pro-life (i.e., 55%), which would you find to be more significant? Why?
Between a sample size of 10 people with 9 with pro-life (i.e 90%) and another random sample of 1000 people from the same population but with 50 people with pro-life (i.e., 55%), the later is more significant (Jin, 2016). This is because; the difference between a sample statistic and a hypothetical value is statistically significant if a hypothesis test indicates that it is very unlikely that it occurred by chance. To assess statistical significance, examine the p-value of the test. If the p-value is less than a specified significance level (α) (usually 0.10, 0.05, or 0.01), you can declare the difference to be statistically significant and reject the null hypothesis of the test.
Statistical significance in itself does not imply that its results have practical consequences. If you use a high power test, you can conclude that a small difference from the hypothetical value is statistically significant. However, this small difference may be meaningless for your situation. You should use your expert knowledge to determine if the difference is practically significant.
For example, suppose you are testing whether the population means (μ) the number of people in the sample is 10. If μ is not equal to 10, the power of the test approaches 1 as the sample size increases, and the p-value approaches 0 (Larch & Walde, 2008).
With sufficient observations, even the trivial differences between the hypothetical and actual parameter values tend to become significant. With a large enough sample, you will probably reject the null hypothesis that μ equals 10 people, even if the difference is of no practical importance.
Confidence intervals (if applicable) are often more useful than hypothesis testing because they provide a way to assess practical importance in addition to statistical significance. They help tell you what parameter value is, rather than explaining what it is not.
Jin, Y. (2016). Matching for Cylinder Shape in Point Cloud Using Random Sample Consensus. Journal of KIISE, 43(5), 562-568.
Larch, M., & Walde, J. (2008). Finite sample properties of alternative GMM estimators for random-effects models with spatially correlated errors. The Annals of Regional Science, 43(2), 473-490.
by EssayRoyal, Dec. 6, 2019, 6:23 p.m.