Jump to main content or area navigation.

Contact Us

Extramural Research

Bibliometrics

Research Project Search

Extramural Research Search

Bibliometric Analysis for Papers on Topics Related to Water Quality Research (Papers Published 2000 to 2005)
January 5, 2006

This is a bibliometric analysis of the papers prepared by intramural and extramural researchers of the U.S. Environmental Protection Agency (EPA) on topics related to water quality research that were published from 2000 to 2005. For this analysis, 506 papers were reviewed. These 506 papers were cited 2,547 times in the journals covered by Thomson’s Web of Science.1 Of these 506 papers, 379 (75%) have been cited at least once in a journal.

The analysis was completed using Thomson’s Essential Science Indicators (ESI) and Journal Citation Reports (JCR) as benchmarks. ESI provides access to a unique and comprehensive compilation of essential science performance statistics and science trends data derived from Thomson’s databases. The chief indicators of output, or productivity, are journal article publication counts. For influence and impact measures, ESI employs both total citation counts and cites per paper scores. The former reveals gross influence while the latter shows weighted influence, also called impact. JCR presents quantifiable statistical data that provide a systematic, objective way to evaluate the world’s leading journals and their impact and influence in the global research community.

Summary of Analysis

Nearly one-twelfth of the water quality publications are highly cited papers. A review of the citations indicates that 41 (8.1%) of the water quality papers qualify as highly cited when using the ESI criteria for the top 10% of highly cited publications. One (0.2%) of the water quality papers qualify as highly cited when using the criteria for the top 1%. None of the papers meet the 0.1% and 0.01% thresholds for very highly cited papers.

The water quality papers are more highly cited than the average paper. Using the ESI average citation rates for papers published by field as the benchmark, in 10 of the 12 fields in which the EPA water quality papers were published, the ratio of actual to expected cites is greater than 1, indicating that the water quality papers are more highly cited than the average papers in those fields.

More than one-eighth of the water quality papers are published in very high impact journals. Sixty-seven (67)of 506 papers were published in the top 10% of journals ranked by JCR Impact Factor, representing 13.2% of EPA’s water quality papers. Approximately one-eighth of the water quality papers are published in the top 10% of journals ranked by JCR Immediacy Factor. Sixty-two (62) of the 506 papers appear in the top 10% of journals, representing 12.2% of EPA’s water quality papers.

Two of the water quality publications qualified as hot papers. ESI establishes citation thresholds for hot papers, which are selected from the highly cited papers in different fields, but the time frame for citing and cited papers is much shorter—papers must be cited within 2 years of publication and the citations must occur in a 2-month time period. Using the current hot paper thresholds established by ESI as a benchmark, two of the water quality papers, representing 0.4% of the water quality publications, were identified as hot papers in the analysis.

The authors of the water quality papers cite themselves less than the average self-citation rate. Onehundred ninety-eight (198) of the 2,547 cites are author self-cites. This 7.8% author self-citation rate is below the accepted range of 10-30% author self-citation rate.

Highly Cited Water Quality Publications

The 506 water quality papers reviewed for this analysis covered 12 of the 22 ESI fields of research. The distribution of the papers among these 12 fields and the number of citations by field are presented in Table 1.

Table 1. Water Quality Papers by ESI Fields

No. of Citations

ESI Field

No. of EPA Water Quality Papers

Average Cites/Paper

1,580

Environment/Ecology

343

4.61

202

Pharmacology & Toxicology

26

7.77

151

Chemistry

28

5.39

127

Microbiology

15

8.47

108

Engineering

30

3.60

99

Multidisciplinary

4

24.75

98

Plant & Animal Science

24

4.08

86

Biology & Biochemistry

25

3.44

47

Geosciences

5

9.40

36

Agricultural Sciences

3

12.00

8

Clinical Medicine

2

4.00

5

Physics

1

5.00

Total =

2,547

 

Total =

506

5.03

There were 41 (8.1% of the papers analyzed) highly cited EPA water quality papers in 9 of the 12 fields—Microbiology, Multidisciplinary, Engineering, Pharmacology & Toxicology, Environment/Ecology, Chemistry, Plant & Animal Science, Geosciences, and Agricultural Sciences—when using the ESI criteria for the top 10% of papers. Table 2 shows the number of EPA papers in those nine fields that met the top 10% threshold in ESI.

One (0.2%) of the papers analyzed qualified as highly cited when using the ESI criteria for the top 1% of papers (see Table 3). This paper was categorized in the field of Environment/Ecology. The citation for this paper is presented in Table 4. There were no papers that met the 0.1% and 0.01% thresholds for very highly cited papers.

Table 2. Number of Highly Cited Water Quality Papers by Field (top 10%)

No. of Citations

ESI Field

No. of Papers

Average Cites/Paper

% of EPA Papers in Field

95

Microbiology

2

47.50

13.33%

85

Multidisciplinary

3

28.33

75.00%

79

Engineering

9

8.78

30.00%

71

Pharmacology & Toxicology

2

35.50

7.69%

67

Environment/Ecology

14

4.78

4.08%

51

Chemistry

5

10.20

17.86%

31

Plant & Animal Science

4

7.75

16.67%

30

Geosciences

1

30.00

20.00%

10

Agricultural Sciences

1

10.00

33.33%

Total =

519

 

Total =

41

12.66

 

Table 3. Number of Highly Cited Water Quality Papers by Field (top 1%)

No. of Citations

ESI Field

No. of Papers

Average Cites/Paper

% of EPA Papers in Field

3

Environment/Ecology

1

3.00

0.29%

Total =

3

 

Total =

1

 

3.00

 

Table 4. Highly Cited Water Quality Paper in the Field of Environment/Ecology (top 1%)

No. of Cites

First Author

Paper

3

Lackey RT

Economic growth and salmon recovery: an irreconcilable conflict? Fisheries 2005;30(3):30-32.

Ratio of Actual Cites to Expected Citation Rates

The expected citation rate is the average number of cites that a paper published in the same journal in the same year and of the same document type (article, review, editorial, etc.) has received from the year of publication to the present. Using the ESI average citation rates for papers published by field as the benchmark, in 10 of the 12 fields in which the EPA water quality papers were published, the ratio of actual to expected cites is greater than 1, indicating that the EPA papers are more highly cited than the average papers in those fields (see Table 5).

Table 5. Ratio of Average Cites to Expected Cites for Water Quality Papers by Field

ESI Field

Total Cites

Expected Cite Rate

Ratio

Environment/Ecology

1,580

1,521.89

1.04

Pharmacology & Toxicology

202

170.6

1.18

Chemistry

151

127.81

1.18

Microbiology

127

117.73

1.08

Engineering

108

38.47

2.81

Multidisciplinary

99

16.67

5.94

Plant & Animal Science

98

74.84

1.31

Biology & Biochemistry

86

232.19

0.37

Geosciences

47

26.78

1.76

Agricultural Sciences

36

12.51

2.88

Clinical Medicine

8

8.86

0.90

Physics

5

3.32

1.51

JCR Benchmarks

The Impact Factor is a well known metric in citation analysis. It is a measure of the frequency with which the average article in a journal has been cited in a particular year. The Impact Factor helps evaluate a journal’s relative importance, especially when compared to others in the same field. The Impact Factor is calculated by dividing the number of citations in the current year to articles published in the 2 previous years by the total number of articles published in the 2 previous years.

Table 6 indicates the number of water quality papers published in the top 10% of journals, based on the JCR Impact Factor. Sixty-seven (67) of 506 papers were published in the top 10% of journals, representing 13.2% of EPA’s water quality papers. This exceeds the expected number of 51 papers (10%) published in the top 10% of high impact journals.

Table 6. Water Quality Papers in Top 10% of Journals by JCR Impact Factor

EPA Water Quality Papers in that Journal

Journal

Impact Factor (IF)

JCR IF Rank

9

Environmental Science & Technology

3.557

540

7

Environmental Health Perspectives

3.929

439

5

Journal of Chromatography A

3.359

602

4

Analytical Chemistry

5.450

243

4

Ecology

4.104

394

4

Applied and Environmental Microbiology

3.810

470

4

Toxicological Sciences

3.391

591

4

Ecological Applications

3.287

623

4

Limnology and Oceanography

3.024

737

3

TRAC-Trends in Analytical Chemistry

3.888

452

2

Nature

32.182

9

2

Proceedings of the National Academy of Sciences of the United States of America

10.452

88

2

Electrophoresis

3.743

482

2

Ecosystems

3.283

624

2

Remote Sensing of Environment

3.185

666

2

Bioscience

3.041

730

1

Lancet

21.713

20

1

Progress in Nuclear Magnetic Resonance Spectroscopy

6.885

175

1

Molecular Ecology

4.375

351

1

Drug Metabolism and Disposition

3.836

461

1

Journal of the American Society for Mass Spectrometry

3.760

479

1

Proceedings of the Royal Society of London Series B-Biological Sciences

3.653

509

1

Frontiers in Ecology and the Environment

3.362

600

Total = 67

 

Immediacy Index

The journal Immediacy Index is a measure of how quickly the average article in a journal is cited. It indicates how often articles published in a journal are cited within the year they are published. The Immediacy Index is calculated by dividing the number of citations to articles published in a given year by the number of articles published in that year.

Table 7 indicates the number of EPA water quality papers published in the top 10% of journals, based on the JCR Immediacy Index. Sixty-two (62) of the 506 papers appear in the top 10% of journals, representing 12.2% of EPA’s water quality papers. This exceeds the expected number of 51 papers (10%) published in the top 10% of high impact journals.

Table 7. Water Quality Papers in Top 10% of Journals by JCR Immediacy Index

EPA Water Quality Papers in that Journal

Journal

Immediacy Index (II)

JCR II Rank

9

Environmental Science & Technology

0.623

617

7

Environmental Health Perspectives

1.202

202

5

Freshwater Biology

0.664

558

4

Analytical Chemistry

0.885

346

4

Ecological Applications

0.747

466

4

Ecology

0.590

676

3

Ecotoxicology

1.450

151

3

Hydrobiologia

0.681

532

3

Journal of Geophysical Research

0.617

630

3

TRAC-Trends in Analytical Chemistry

0.583

681

2

Nature

6.089

5

2

Ecosystems

2.048

76

2

Proceedings of the National Academy of Sciences of the United States of America

1.923

89

2

Bioscience

0.863

356

2

Electrophoresis

0.575

697

1

Lancet

5.017

12

1

Journal of Paleolimnology

1.581

132

1

Journal of Oceanography

0.692

519

1

Molecular Ecology

0.674

545

1

Progress in Nuclear Magnetic Resonance Spectroscopy

0.667

551

1

Drug Metabolism and Disposition

0.590

676

1

Journal of the American Society for Mass Spectrometry

0.575

697

Total = 62

 

Hot Papers

ESI establishes citation thresholds for hot papers, which are selected from the highly cited papers in different fields, but the time frame for citing and cited papers is much shorter—papers must be cited within 2 years of publication and the citations must occur in a 2-month time period. Papers are assigned to 2-month periods and thresholds are set for each period and field to select 0.1% of papers. There were no hot papers identified for the most recently completed 2-month period (i.e., September-October 2005), but there were two hot papers identified from previous periods.

Using the current hot paper thresholds established by ESI as a benchmark, two of the water quality papers, representing 0.4% of the water quality publications, were identified in the fields of Environment/Ecology and Multidisciplinary. The hot papers are listed in Table 8.

Table 8. Hot Papers Identified Using Current ESI Thresholds

Field

ESI Hot Papers Threshold

No. of Cites in 2-Month Period

Paper

Environment/ Ecology

7

9 cites in March-April 2003

Angradi TR, et al. Vegetation type and the intertidal macroinvertebrate fauna of a brackish marsh: Phragmites vs. Spartina. Wetlands 2001;21(1):75-92.

Multidisciplinary

6

7 cites in October-November 2001

Steidinger KA, Rublee PA. Heteroduplex mobility assay-guided sequence discovery: elucidation of the small subunit (18S) rDNA sequences of Pfiesteria piscicida and related dinoflagellates from complex algal culture and environmental sample DNA pools. Proceedings of the National Academy of Sciences of the United States of America 2000; 97(8):4303-4308.

Author Self-Citation

Self-citations are journal article references to articles from that same author (i.e., the first author). Because higher author self-citation rates can inflate the number of citations, the author self-citation rate was calculated for the water quality papers. Of the 2,547 total cites, 198 are author self-cites—a 7.8% author self-citation rate. Garfield and Sher2 found that authors working in research-based disciplines tend to cite themselves on the average of 20% of the time. MacRoberts and MacRoberts3 claim that approximately 10% to 30% of all the citations listed fall into the category of author self-citation. Therefore, the 7.8% self-cite rate for the water quality papers is below the expected range for author self-citation.

1 Thomson's Web of Science provides access to current and retrospective multidisciplinary information from approximately 8,500 of the most prestigious, high impact research journals in the world. Web of Science also provides cited reference searching.

2 Garfield E, Sher IH. New factors in the evaluation of scientific literature through citation indexing. American Documentation 1963;18(July):195-201.

3 MacRoberts MH, MacRoberts BR. Problems of citation analysis: a critical review. Journal of the American Society of Information Science 1989;40(5):342-349.

You will need Adobe Reader to view some of the files on this page. See EPA's PDF page to learn more.

Top of page

Jump to main content.