Sampling
INTRODUCTION
Inference is a statistical concept that pertains to drawing conclusions from a subset of evidence (a sample). Let us say the mean score on a leadership measure (a population parameter) was known to be 100.
(Note: In practice, we rarely know the values of population parameters.) Next, we take a random sample of individuals from this population, administer the leadership test, and compute the mean of the sample (a sample statistic). Would the sample mean be exactly equal to the value of the population mean?
Probably not. However, if we take numerous random samples and compute their corresponding sample means, this would create a sampling distribution of means. By definition, the mean of the sampling distribution is the population mean.
In practice, however, researchers do not take numerous samples to estimate the population mean. Your applied business research will likely consist of only one random sample from a known population.
Since one goal of statistics is to draw conclusions from a subset of evidence, how confident are you about your inference to the population? What sample size will you need for your study? At first, the question seems quite simple, since the expected answer is a single value. However, the question is more like “a riddle, wrapped in a mystery, inside an enigma” (Churchill, 1939).
Determining sample size requires, among other things, prior knowledge of the outcome. For example, you need to know the variance estimates of your measures, estimates of correlation coefficients or deltas, and the characteristics of variables primary to your study. One way to increase power is to increase the sample size. Another way is to have the value of the alternative hypothesis further away from the value in the null.
In other words, it is easier to find large differences than small differences. If you are trying to discern a larger effect, the sample size needed can be smaller. To take this to the level needed for a doctoral student, you must understand the relationships among sample size, effect size, the probability of a Type I error (alpha) and the probability of a Type II error (beta). The power of the test is 1 minus the probability of a Type II error.
What size should a sample be to satisfy both practical and statistical significance? Researchers must be able to defend their choices. Increasing the sample size is costly, takes more time, and can be unethical. While many factors are important, one of the most important is the size of the effect, which will be important in making a decision.
This is a measurement issue often related to instrument selected. Cohen’s d, a frequently used effect size, can be explored using tabled value with small, medium, and large sizes. Small is used for critical research, such as in medical research. Medium is quite typical in organizational research. Large effect sizes might be used in exploratory research.
(Note that d is in italics following the statistical abbreviations and symbols guide in the APA manual (p. 183).
Wuensch (n.d.) suggests that you resolve the following issues before deciding on a sample size:
• How much power is desired?
• What is the smallest effect size that would be important?
• What is the level of statistical significance?
• What is the test statistic?
• What is the standard deviation in the population of the outcome variable?
• What is the population correlation (only for correlated sample designs)?
Much of the data required for such a design is often not known in advance (a priori), so one looks into the literature for clues as to statistical values and makes assumptions.
In the absence of values from the literature, one might conduct a pilot study. Statistical power calculators, such as G*Power, allow rich sensitivity analysis of changing numerical values that balance the statistical power and the study’s practical significance.
The classic look-up tables were created by Cohen (1988) and expanded by Murphy and Myors (2009). Another approach, depending on the technique, is to use a heuristic or rule of thumb. Such heuristic rules are frequently used in multiple regression and exploratory factor analysis (EFA) (Tabachnick & Fidell, 2007).
Also of concern are participant response rates. The sample size should be increased by a certain percentage to plan for participants refusing to participate or leaving the study. Refusals are not unusual for those invited to participate in your applied business research survey.
Aside from the statistical properties of your survey, the population sampling method will influence the inferences you will make. For making inferences to the population parameters, the sample must be the product of a random selection process.
Otherwise, the sample is considered biased, which limits your ability to generalize the outcomes of your study to the population. Remember: Nonprobability samples are unacceptable for Capella applied business research projects. A successful design uses a probability sample!
In this unit, you will explore the key considerations for choosing an a priori sample size, including statistical power, statistical significance, and effect size. You will compute the statistical power for the journal article that you discussed in Units 1 and 2. Also, you will learn to use G*Power, a free application that will help you calculate the sample size in advance and statistical power after the study is complete.
