![[logo] US EPA](http://www.epa.gov/epafiles/images/logo_epaseal.gif){kind=logo}
Chapter 7 - Hypothesis Testing
Biological criteria are applied in the standards program by testing hypotheses about the biological integrity of impacted surface waters. These hypotheses include the null hypothesis-the designated use of the waterbody is not impaired-and alternative hypotheses such as the designated use of the waterbody is impaired (more specific hypotheses can also be generated that predict the type(s) of impairment). Under these hypotheses specific predictions are generated concerning the kinds and numbers of organisms representing community structure and function expected or found in unimpaired habitats. The kinds and numbers of organisms surveyed in unimpaired waters are used to establish the biological criteria. To test the alternative hypotheses, data collection and analysis procedures are used to compare the criteria to comparable measures of community structure and function in impacted waters.
Hypothesis
Testing
To detect differences of biological and regula-tory concern between
biological criteria and ambient biological integrity at a test site, it
is important to establish the sensitivity of the evaluation. A 10 percent
difference in condition is more difficult to detect than a 50 percent
difference. For the experimental/survey design to be effective, the level
of detection should be predetermined to establish sample size for data
collection (Sokal and Rohlf 1981). Knowledge of expected natural variation,
experi-mental error, and the kinds of detectable differences that can
be expected will help determine sample size and location. This forms the
basis for defining data quality objectives, standardizing data collection
procedures, and developing quality assurance/ quality control standards.
Once data are collected and analyzed, they are used to test the hypotheses to determine if characteristics of the resident biota at a test site are significantly different from established criteria values for a comparable habitat. There are three possible outcomes:
- The use is impaired when survey design and data analyses are sensitive enough to detect differences of regulatory importance, and significant differences were detected. The next step is to diagnose the cause(s) and source(s) of impairment.
- The biological criteria are met when survey design and data analyses are sensitive enough to detect differences of regulatory significance, but no differences were found. In this case, no action is required by States based on these measures. However, other evidence may indicate impairment (e.g., chemical criteria are violated; see below).
- The outcome is indeterminate when survey design and data analyses are not sensitive enough to detect differences of regulatory significance, and no differences were detected. If a State or Region determines that this is occurring, the development of study design and evaluation for biological criteria was incomplete. States must then determine whether they will accept the sensitivity of the survey or conduct additional surveys to increase the power of their analyses. If the sensitivity of the original survey is accepted, the State should determine what magnitude of difference the survey is capable of detecting. This will aid in re-evaluating research design and desired detection limits. An indeterminate outcome may also occur if the test site and the reference conditions were not comparable. This variable may also require re-evaluation.
As with all scientific studies, when implementing biological criteria, the purpose of hypothesis testing is to determine if the data support the conclusion that the null hypothesis is false (i.e., the designated use is not impaired in a particular waterbody). Biological criteria cannot prove attainment. This reasoning provides the basis for emphasizing independent application of different assessment methods (e.g., chemical verses biological criteria). No type of criteria can "prove" attainment; each type of criteria can disprove attainment.
Although this discussion is limited to the null and one alternative hypothesis, it is possible to generate multiple working hypotheses (Popper, 1968) that promote the diagnosis of water quality problems when they exist. For example, if physical habitat limitations are believed to be causing impairment (e.g., sedimentation) one alternative hypothesis could specify the loss of community components sensitive to this impact. Using multiple hypotheses can maximize the information gained from each study. See the Diagnosis section for additional discussion.
Diagnosis
When impairment of the designated use is found using biological criteria,
a diagnosis of probable cause of impairment is the next step for implementation.
Since biological criteria are primarily designed to detect water quality
impairment, problems are likely to be identified without a known cause.
Fortunately the process of evaluating test sites for biological impairment
provides significant information to aid in determining cause.
During diagnostic evaluations, three main impact categories should be considered: chemical, physical, and biological. To begin the diagnostic process two questions are posed:
- What are the obvious causes of impairment?
- If no obvious causes are apparent, what possible causes do the biological data suggest?
Obvious causes such as habitat degradation, point source discharges, or introduced species are often identified during the course of a normal field biological assessment. Biomonitoring programs normally provide knowledge of potential sources of impact and characteristics of the habitat. As such, diagnosis is partly incorporated into many existing State field-oriented bioassessment programs. If more than one impact source is obvious, diagnosis will require determining which impact(s) is the cause of impairment or the extent to which each impact contributes to impairment. The nature of the biological impairment can guide evaluation (e.g., chemical contamination may lead to the loss of sensitive species, habitat degradation may result in loss of breeding habitat for certain species).
Case studies illustrate the effectiveness of biological criteria in identifying impairments and possible sources. For example, in Kansas three sites on Little Mill Creek were assessed using Rapid Bioassessment Protocols (Plafkin et al. 1989; see Fig. 4). Based on the results of a comparative analysis, habitats at the three sites were comparable and of high quality. Biological impairment, however, was identified at two of the three sites and directly related to proximity to a point source discharge from a sewage treatment plant. The severely impaired Site (STA 2) was located approximately 100 meters downstream from the plant. The slightly impaired Site (STA 3) was located between one and two miles downstream from the plant. However, the unimpaired Site (STA 1(R)) was approximately 150 meters upstream from the plant (Plafkin et al. 1989). This simple example illustrates the basic principles of diagnosis. In this case the treatment plant appears responsible for impairment of the resident biota and the discharge needs to be evaluated. Based on the biological survey the results are clear. However, impairment in resident populations of macroinvertebrates probably would not have been recognized using more traditional methods.
In Maine, a more complex problem arose when effluents from a textile plant met chemical-specific and effluent toxicity criteria, yet a biological survey of downstream biota revealed up to 80 percent reduction in invertebrate richness below plant outfalls. Although the source of impairment seemed clear, the cause of impairment was more difficult to determine. By engaging in a diagnostic evaluation, Maine was able to determine that the discharge contained chemicals not regulated under current programs and that part of the toxicity effect was due to the sequential discharge of unique effluents (tested individually these effluents were not toxic; when exposure was in a particular sequence, toxicity occurred). Use of biological criteria resulted in the detection and diagnosis of this toxicity problem, which allowed Maine to develop workable alternative operating procedures for the textile industry to correct the problem (Courtemanch 1989, and pers. comm.).
During diagnosis it is important to consider and discriminate among multiple sources of impairment. In a North Carolina stream (see Figure 5) four sites were evaluated using rapid bioassessment techniques. An ecoregional reference site (R) established the highest level of biological integrity for that stream type. Site (1), well upstream from a local town, was used as the upstream reference condition. Degraded conditions at Site (2) suggested nonpoint source problems and habitat degradation because of proximity to residential areas on the upstream edge of town. At Site (3) habitat alterations, nonpoint runoff, and point source discharges combined to severely degrade resident biota. At this site, sedimentation and toxicity from municipal sewage treatment effluent appeared responsible for a major portion of this degradation. Site (4), although several miles downstream from town, was still impaired despite significant improvement in habitat quality. This suggests that toxicity from upstream discharges may still be occurring (Barbour, 1990 pers. comm.). Using these kinds of comparisons, through a diagnostic procedure and by using available chemical and biological assessment tools, the relative effects of impacts can be determined so that solutions can be formulated to improve water quality.
When point and nonpoint impact and physical habitat degradation occur simultaneously, diagnosis may require the combined use of biological, physical, and chemical evaluations to discriminate between these impacts. For example, sedimentation of a stream caused by logging practices is likely to result in a decrease in species that require loose gravel for spawning but increase species naturally adapted to fine sediments. This shift in community components correlates well with the observed impact. However, if the impact is a point source discharge or nonpoint runoff of toxicants, both species types are likely to be impaired whether sedimentation occurs or not (although gravel breeding species can be expected to show greater impairment if sedimentation occurs). Part of the diagnostic process is derived from an understanding of organism sensitivities to different kinds of impacts and their habitat requirements. When habitat is good but water quality is poor, aquatic community components sensitive to toxicity will be impaired. However, if both habitat and water quality degrade, the resident community is likely to be composed of tolerant and opportunistic species.
When an impaired use cannot be easily related to an obvious cause, the diagnostic process becomes investigative and iterative. The iterative diagnostic process as shown in Figure 6 may require additional time and resources to verify cause and source. Initially, potential sources of impact are identified and mapped to determine location relative to the area suffering from biological impairment. An analysis of the physical, chemical, and biological characteristics of the study area will help identify the most likely sources and determine which data will be most valuable. Hypotheses that distinguish between possible causes of impairment should be generated. Study design and appropriate data collection procedures need to be developed to test the hypotheses. The severity of the impairment, the difficulty of diagnosis, and the costs involved will determine how many iterative loops will be completed in the diagnostic process.
Normally, diagnoses of biological impairment are relatively straightforward. States may use biological criteria as a method to confirm impairment from a known source of impact. However, the diagnostic process provides an effective way to identify unknown impacts and diagnose their cause so that corrective action can be devised and implemented.