Sampling (statistics)
Sampling is that part of
statistical practice concerned with the selection of individual observations intended to yield some knowledge about a
population of concern, especially for the purposes of
statistical inference. In particular, results from
probability theory and
statistical theory are employed to guide practice.
The sampling process consists of five stages (Makerere University Institute of Statistics & Applied Economics (ISAE)):
* Definition of population of concern
* Specification of a
sampling frame, a
set of items or events that it is possible to measure
* Specification of
sampling method for selecting items or events from the frame
* Sampling and data collecting
* Review of sampling process
Successful statistical practice is based on focused
problem definition. Typically, we seek to take action on some
population, for example when a
batch of material from
production must be released to the customer or sentenced for scrap or rework. Alternatively, we seek knowledge about the
cause system of which the population is an outcome, for example when a researcher performs an experiment on rats with the intention of gaining insights into
biochemistry that can be applied for the benefit of
humans. In the latter case, the population of concern can be difficult to specify, as it is in the case of measuring some physical characteristic such as the
electrical conductivity of
copper.
However, in all cases, time spent in making the population of concern precise is often well spent, often because it raises many issues, ambiguities and questions that would otherwise have been overlooked at this stage.
In the most straightforward case, such as the sentencing of a batch of material from production (
acceptance sampling by lots), it is possible to identify and measure every single item in the population and to include any one of them in our sample. However, in the more general case this is not possible. There is no way to identify all rats in the set of all rats. There is no way to identify every voter at a forthcoming election (in advance of the election).
These imprecise populations are not amenable to sampling in any of the ways below and to which we could apply statistical theory.
As a remedy, we seek a
sampling frame which has the property that we can identify every single element and include any in our sample. For example, in an
electoral poll, possible sampling frames include:
*
Electoral register*
Telephone directory* Shoppers in Anytown, High Street on the Monday afternoon before the election.
The sampling frame must be representative of the population and this is a question outside the scope of statistical theory demanding the judgement of experts in the particular subject matter being studied. All the above frames omit some people who will vote at the next election and contain some people who will not. People not in the frame have no prospect of being sampled. Statistical theory tells us about the uncertainties in extrapolating from a sample to the frame. In extrapolating from frame to population its role is motivational and suggestive.
In defining the frame, practical, economic, ethical and technical issues need to be addressed. The need to obtain timely results may prevent extending the frame far into the future.
The difficulties can be extreme when the population and frame are
disjoint. This is a particular problem in
forecasting where inferences about the future are made from historical
data. In fact, in
1703, when
Jacob Bernoulli proposed to
Gottfried Leibniz the possibility of using historical mortality data to predict the
probability of early death of a living man,
Gottfried Leibniz recognised the problem in replying:
Nature has established patterns originating in the return of events but only for the most part. New illnesses flood the human race, so that no matter how many experiments you have done on corpses, you have not thereby imposed a limit on the nature of events so that in the future they could not vary.Having established the frame, there are a number of ways of organising it to improve efficiency and effectiveness.
In this case, all elements of the frame are treated equally and it is not subdivided or partitioned. One of the sampling methods below is applied to the whole frame.
Where the population embraces a number of distinct categories, the frame can be organised by these categories into separate
strata or
demographics. One of the sampling methods below is then applied to each
stratum separately. Major gains in efficiency (either lower sample sizes or higher precision) can be achieved by varying the
sampling fraction from stratum to stratum. The sample size should be made proportional to the stratum
standard deviation. From the efficiency point of view (i.e. maximum precision for a given sample size) strata should be chosen to have:
*
means which differ substantially from one another
*
variances which are different from one another, and lower than the overall variance
Cluster sampling
Random sampling of a population spread across a large area, eg all of Europe involves a lot of travelling, cost and delay.
Cluster or
area sampling addresses this problem. There are three stages: 1) the target population is divided into many regional clusters (groups) eg London, Berlin, Rome etc 2) a few clusters are randomly selected for study 3) A few subjects are randomly chosen from within a cluster
Quota sampling
In
quota sampling, the population is first segmented into
mutually exclusive sub-groups, just as in
stratified sampling. Then judgement is used to select the subjects or units from each segment based on a specified proportion. For example, an interviewer may be told to sample 200 females and 300 males between the age of 45 and 60.
It is this second step which makes the technique one of non-probability sampling. In quota sampling the selection of the sample is non-
random. For example interviewers might be tempted to interview those who look most helpful. The problem is that these samples may be
biased because not everyone gets a chance of selection. This non-random element is its greatest weakness and quota versus probability has been a matter of controversy for many years.
Within any of the types of frame identified above, a variety of sampling methods can be employed, individually or in combination.
Random sampling
In random sampling, also known as probability sampling, every combination of items from the frame, or stratum, has a known probability of occurring, but these probabilities are not necessarily equal. With any form of sampling there is a risk that the sample may not adequately represent the population but with random sampling there is a large body of statistical theory which quantifies the risk and thus enables an appropriate sample size to be chosen. Furthermore, once the sample has been taken the
sampling error associated with the measured results can be computed. With non-random sampling there is no measure of the associated sampling error. While such methods may be cheaper this is largely meaningless since there is no measure of quality. There are several forms of random sampling. For example, in
simple random sampling, each element has an equal probability of occurring. It may be infeasible in many practical situations. Other examples of probability sampling include
stratified sampling and
multistage sampling.
Selecting (say) every
kth name from the telephone directory is called an
every kth sample, which is an example of
systematic sampling. It is a type of
nonprobability sampling unless the directory itself is
randomized. It is easy to implement and the
stratification induced can make it efficient, but it is especially vulnerable to periodicities in the list. If periodicity is present and the period is a multiple of
k, then
bias will result. It is important that the first name chosen is not simply the first in the list, but is chosen to be the
rth, where
r is a random integer in the range 1,...,
k-1. Every
kth sampling is especially useful for efficient sampling from
databases.
Mechanical sampling
Mechanical sampling is typically used in sampling
solids,
liquids and
gases, using devices such as grabs, scoops,
thief probes, the
coliwasa and
riffle splitter.
Mechanical sampling is generally not
random and is a type of
nonprobability sampling. Care is needed in ensuring that the sample is representative of the frame. Much work in this area was developed by
Pierre Gy.
Convenience sampling
Sometimes called,
grab or
opportunity sampling, this is the method of choosing items arbitrarily and in an unstructured manner from the frame. Though almost impossible to treat rigorously, it is the method most commonly employed in many practical situations. In social science research,
snowball sampling is a similar technique, where existing study subjects are used to recruit more subjects into the sample.
Sample size
Where the frame and population are identical, statistical theory yields exact recommendations on sample size. However, where it is not straightforward to define a frame representative of the population, it is more important to understand the
cause system of which the population are outcomes and to ensure that all sources of variation are embraced in the frame. Large number of observations are of no value if major sources of variation are neglected in the study. In other words, it is taking a sample group that matches the survey category and is easy to survey. Bartlett, Kotrlik, and Higgins (2001) published a paper titled Organizational Research: Determining Appropriate Sample Size in Survey Research Information Technology, Learning, and Performance Journal (avaialble at http://www.osra.org/itlpj/bartlettkotrlikhiggins.pdf) that provides an explanation of Cochran's (1977) formulas. A discussion and illustration of sample size formulas, including the formula for adjusting the sample size for smaller populations, is included. A table is provided that can be used to select the sample size for a research problem based on three alpha levels and a set error rate.
Good data collection involves:
* Following the defined sampling process
* Keeping the data in time order
* Noting comments and other contextual events
* Recording non-responses
After sampling, a review should be held of the exact process followed in sampling, rather than that intended, in order to study any effects that any divergences might have on subsequent analysis. A particular problem is that of
non-responses.
Non-responses
In
survey sampling, many of the individuals identified as part of the sample may be unwilling to participate or impossible to contact. In this case, there is a risk of differences, between (say) the willing and unwilling, leading to
selection bias in conclusions. This is often addressed by follow-up studies which make a repeated attempt to contact the unresponsive and to characterise their similarities and differences with the rest of the frame.
In many situations the sample fraction may be varied by stratum and data will have to be weighted to correctly represent the population. Thus for example, a simple random sample of individuals in the United Kingdom might include some in remote Scottish islands who would be inordinately expensive to sample. A cheaper method would be to use a stratified sample with urban and rural strata. The rural sample could be under-represented in the sample, but weighted up appropriately in the analysis to compensate.
The idea of random sampling by the use of lots is an old one, mentioned several times in the Bible. In 1786 Pierre Simon
Laplace estimated the population of France by using a sample, along with
ratio estimator. He also computed probabilistic estimates of the error. These were not expressed as modern
confidence intervals but as the sample size that would be needed to achieve a particular upper bound on the sampling error with probability 1000/1001. His estimates used
Bayes' theorem with a uniform
prior probability and it assumed his sample was random.The theory of small-sample statistics developed by
William Sealy Gossett put the subject on a more rigorous basis in the 20th century. However, the importance of random sampling was not universally appreciated and in the USA the 1936
Literary Digest prediction of a Republican win in the
presidential election went badly awry, due to severe
bias. A sample size of one million was obtained through magazine subscription lists and telephone directories. It was not appreciated that these lists were heavily biased towards Republicans and the resulting sample, though very large, was deeply flawed.
Doctoral and Masters Degrees
*
Program in Social Statistics (Survey Methodology) - University of Southampton*
Joint Program in Survey Methodology (JPSM) - University of Maryland-College Park and University of Michigan-Ann Arbor*
Survey Research and Methodology - University of Nebraska-Lincoln*
Program in Survey Methodology - University of Michigan-Ann Arbor*
Advanced Research and Methodology - New Bauhaus ChicagoMasters Degrees only
*
Magister in Official Statistics - University of Southampton, UK*
Graduate Program in Survey Research - University of Connecticut*
Diploma in Official Statistics - Hebrew University, Israel*
Methodology and Statistics for the Social and Behavioral Sciences - Utrecht University, the Netherlands*
Bartlett, J. E., II, Kotrlik, J. W., & Higgins, C. (2001). Organizational research: Determining appropriate sample size for survey research. Information Technology, Learning, and Performance Journal, 19(1) 43-50.* Chambers, R L, and Skinner, C J (editors) (2003),
Analysis of Survey Data, Wiley, ISBN 0471899879
* Cochran, W G (1977)
Sampling Techniques, Wiley, ISBN 047116240X
* Deming, W E (1975) On probability as a basis for action,
The American Statistician, 29(4), pp146-152.
* Flyvbjerg, B (2006) "Five Misunderstandings About Case Study Research." Qualitative Inquiry, vol. 12, no. 2, April 2006, pp. 219-245. [
1]
* Gy, P (1992)
Sampling of Heterogeneous and Dynamic Material Systems: Theories of Heterogeneity, Sampling and Homogenizing* Kish, L (1995)
Survey Sampling, Wiley, ISBN 0471109495
* Korn, E L, and Graubard, B I (1999)
Analysis of Health Surveys, Wiley, ISBN 0471137731
* Sarndal, Swenson, and Wretman (1992), Model Assisted Survey Sampling, Springer-Verlag, ISBN 0387406204
* Stuart, Alan (1962)
Basic Ideas of Scientific Sampling, Hafner Publishing Company, New York
