Sampling is, or should be, embedded in a project’s research design, written before fieldwork is initiated to spell out such topics as research objectives and questions, existing knowledge, data requirements, investigator bias, sampling plan, strategic methods, and tactical techniques. Generally, the research design is written at least for the fieldwork phase of a project. In the best of all possible worlds, the research design should also include pre-fieldwork, library/archival research, laboratory investigations, and report writing, three aspects that are occasionally given short shrift in written research designs. Library/archival research to accumulate existing knowledge and previous archaeological investigations is frequently done hand-in-hand with the writing of the research design, particularly in American Cultural Resource Management (CRM) type archaeology. Project proposals required in the worlds of American CRM and English Planning Policy
Guidance, no. 16 (PPG 16) and funding grants in the academic world force archaeologists to write research designs. This is a good thing since all of the above topics are crystallized on paper; the competitive nature of winning proposals and grants forces archaeologists to think through the entire process from library through laboratory and possibly back to library again for writing the report.
The general steps in a research design/sampling plan have been crystallized by Cochran and imported into the archaeological literature by Orton:
1. assimilation of existing knowledge,
2. objectives,
3. population to be sampled,
4. data to be collected,
5. degree of precision required,
6. method of measurement,
7. the frame,
8. sample selection,
9. the pretest,
10. fieldwork organization,
11. summary and analysis of data, and
12. information gained for future surveys.
These steps reflect the accepted worldview that sampling is part of statistical theory and that certain statistical decisions need to be made, especially at the fifth, sixth and tenth steps above. Concerning the fifth step, precision is the average size of the difference between estimates obtained from a series of samples... and the true value of the parameter. Precision is not to be confused with accuracy. Efficiency, another statistical descriptor of samples, is based on precision and states that the more precise sample scheme (the one with the lower calculated value of precision) is the more efficient, given that sample sizes are the same. Orton makes the valid practical point concerning the fourth step that perhaps budget restraints prevent sufficiently precise results about the target population and that some projects should not be undertaken if they cannot attain the needed level of statistical accuracy, precision, and bias. In the real world however, such statistical considerations rarely stop archaeologists who many times take nonstatistical, backup approaches.
An important statistical point involves the sixth and the tenth steps. The statistics selected to summarize and analyze the data must be compatible with the method of measurement. Measurement refers to the type of variable used to describe something in the archaeological record (artifacts per grid unit, a feature’s dimensions, contents, and location, distance between in situ artifacts, etc.) and come in four types from simple to complex - nominal, ordinal, interval, and ratio. Certain statistical measures are compatible with only nominal variables while other statistics are compatible with interval variables. Statistical power is maximized by making this match, while a mismatch results in the under-use or over-use of statistical measures and tests.
One overarching, dominating principle governing sampling at all levels is samples should be representative and this is heard repeatedly among archaeologists. Another way of saying this very important principle is that a sample should include all the known diversity in the target population. The negative way of saying this is samples should not be biased; bias elimination is a goal of any scientist using sampling procedures. Bias, as a statistical concept, measures the difference between the mean (or any other parameter) of a single population and the mean of a very large number of means calculated for that sampling scheme. The mean (or any other parameter) of a single population is that which is customarily calculated by an archaeologist after a sampling-based field data collection. For instance, if he/she uses stratified sampling to collect data, but then uses formulas based on SRS, the bias would be large. Procedurally, bias results when the frame consists of sampling units that are beyond the target population or when sampling units within the sampled population are omitted from the frame. Repeated rounding errors or nonstandard observational abilities by field workers are other sources of bias. Bias cannot be eliminated by the sample size or by taking more samples.
A second overarching, dominating principle governing sampling at all levels is that maximizing prior information known before fieldwork starts will improve the sampling plan and hence the interpretive results. Step #9, the pretest (or pilot project), is one way of gathering a little more information that will exponentially enhance the sampling plan; multistage research designs and adaptive sampling are other strategies for pretests. (Bayesian statistics is another method, albeit more theoretical than sampling approaches.) The research planning decisions that are traditionally associated with sampling (sample size, scheme, etc.) are made at the third, seventh and eighth steps above.
Formal, probabilistic sampling derives its analytical advantages over intuitive sampling because it is a subset of probabilistic theory and applied mathematics. The likelihood (probability) of finding and observing any archaeological characteristic from an unknown (and perhaps unknowable) larger class of observations can be measured and quantified due to the interconnection between formal sampling and probabilistic theory. ‘‘Therein lies the power and utility of probabilistic sampling’’. Expressed differently, probabilistic sampling allows a researcher to extend generalizations from the collected sample to estimates of the larger target population through a process known as statistical inference. In some situations, probabilistic sampling may not be needed. These situations include initial, pilot studies in which an academic archaeologist wishes to gain familiarity with a certain research area or in which the archaeologist wants to find some suggestive data that may help him establish working hypotheses that can be tested by a rigorous data collection effort. These exploratory studies frequently involve small sample sizes where drawing the sample is not justified in cost or in precision gained. Adaptive sampling, situated conceptually in the gray zone between probabilistic and nonprobabilistic methods, may help the archaeologist. Another situation occurs also in the world of CRM and contracted public archaeology where there is no need to generalize to the larger, target population; interpretation of the archaeological record of the sampled population is the only goal. In these cases, nonprobabilistic sampling is the necessary and sufficient method to determine what spatial areas to examine. In all these nonprobabilistic situations, the archaeologist is generally limited to descriptive, instead of the more powerful inferential statistics.
World History
