Data Quality—Quality Assurance and Quality Control Issues and Practice
J. Kuo, Freshwater Water Quality and Surveillance, Environment Canada; Graham van Aggelen, Pacific Environmental Science Centre, Environment Canada
QA/QC is essential to producing sound, credible and meaningful scientific results. It is needed every step of the way: study planning, sampling or experimentation, laboratory analysis, data storage, data interpretation, presentation of results. QA/QC can be challenging even in well established disciplines; emerging fields, such as computational biology, bioinformatics associated with “OMIC” based testing regimes, and near real time data acquisition and assessment, pose new QA/QC challenges. This session included an interesting set of presentations covering QA/QC techniques in both established and emerging environmental subdisciplines.
Dr. Malcolm Clark discussed several variance myths and misunderstandings regarding environmental monitoring data. He recommended that relative analytical variance, sampling variance and environmental heterogeneity should all be estimated for environmental data sets. He explained that the '<' flag is NOT a guarantee that the flagged values fall below the reported detection limits, but is a guarantee that there is so much noise that the analyzed sample cannot be reliably distinguished from blank samples. He used real examples to demonstrate that using detection limits at face value, substituting half the method detection limit, or substituting zero are not wise decisions. Dr. Clark emphasized that reported values below the Limit of Quantitation (LOQ) contain large random error that can obscure significant non-random error. He recommends that where data sets contain large numbers of data below detection or below LOQ that better alternative methods should be employed (editor’s note: the interested reader may wish to review Dennis Helsel’s recent IEAM paper on statistical methods for incorporating nondetect data in sums (IEAM 6(3):361-366)).
Helbing et al. cautioned that poor experimental design and execution can lead to erroneous results when using the quantitative real time polymerase chain reaction (QPCR), a very sensitive molecular tool, for risk assessments and monitoring programs. She spoke about how to design, execute, and interpret a good QPCR experiment.
Raymond Ng presented the importance of data cleansing and pre-processing for genomics data and gave an overview of quality control techniques that his group has developed for microarrays.
Lorraine Brown presented the pipeline in microarray data quality. The pipeline includes several array visualization methods to detect artefacts, anomalies and outliers, normalization, clustering and statistical analysis. She also spoke about the Pacific Environmental Science Centre, Environment Canada’s pipeline. The pipeline uses a three-tier validation system through which biomarker expression, as determined by microarray experiments, is further validated by two independent methods: QuantiGene Plex Assay (QGP) and quantitative-PCR (QPCR). These methods provide greater detection sensitivity and accuracy and thus greatly complement microarray data analysis.
Dr. Lisa Taylor talked about several QA/QC tools that can help ensure the quality of toxicological data, including utilization of standardized testing methodologies, laboratory accreditation, test-specific checklists, and statistical forethought. She also spoke about tools that should be used to evaluate the quality of data after they have been generated, including reality checks, re-calculations, peer review and internal audits. She presented three cases that highlighted some present day QA/QC issues.
Jordana Van Geest presented the important factors that can influence data quality and how they were considered in the development of an Ontario Ministry of the Environment laboratory protocol for measuring the bioaccumulation of sediment-associated contaminants in freshwater organisms. The selection of exposure techniques, method design, performance-based validity criteria, analytical requirements, traceability of measurements to international standards, and assessing the suitability of environmental samples for bioaccumulation testing were discussed.
Haffey et al. presented a real/near real-time data acquisition and assessment system that was developed to support remedial activities at the Hudson River Superfund Site, where the U.S. Environmental Protection Agency (USEPA) established performance standards designed to protect the community and the environment during dredging related activities. The quality control features included detection of real-time monitoring system malfunctions, centralized management of station IDs, chain-of-custody tracking, and automated analytical chemistry validation. The data assessments and web-based user interfaces enabled data inspection, graphical output, and automated notifications to project personnel.
Marcum et al. presented a screening process to identify inorganic chemicals above background requiring further evaluation in risk assessments. Hypothesis testing or screening against a background threshold value (BTV) is used for statistical comparisons. The USEPA’s ProUCL Version 4.00.04 software may be used for both parametric and non-parametric hypothesis tests for datasets including those with non-detect values. When a single site concentration, instead of a dataset, is compared to a background concentration, a BTV is often used as a screening value. For each inorganic, both hypothesis tests and comparison to BTVs are performed in a weight-of-evidence approach. Results from both statistical comparisons allow risk managers to decide if a chemical is of potential concern and should be selected for further evaluation in the risk assessment.
The highlights for the poster sessions included Xianming Zhang’s evaluating the performance of four partitioning priority prediction techniques (EPI Suite, ABSOLV, SPARC, and COSMOtherm) using iterative fragment selection. Also, Jen-ni Kuo presented the data quality assurance tools for water quality monitoring. A data quality assurance plan encompasses two components: data quality control to identify and quantify errors associated with sampling activities and laboratory analysis, and data quality assessment to ensure quality of data, data management, data evaluation, interpretation and reporting. In addition, a 2009 inter-laboratory study was presented.
Authors’ contact information: email@example.com, firstname.lastname@example.org
Clark, M.J.R. 2000. Quality Assurance in Environmental Analysis. Encyclopedia of Analytical Chemistry.
Clark, M.J.R., Laidlaw, M.C.A., Ryneveld, S.C. and Ward, M.I. 1996. Estimating sampling variance and local environmental heterogeneity for both known and estimated analytical variance. Chemosphere, Vol. 32, pp. 1133-1151.
Clark, M.J.R. and Whitfiled, P.H. 1994. Conflicting Perspectives about Detection Limits and about the Censoring of Environmental Data. Journal of the American Water Resources Association, Vol. 30, pages 1063-1079. http://onlinelibrary.wiley.com/doi/10.1111/j.1752-1688.1994.tb03353.x/abstract
Environment Canada. 1990 ~ 2008. Biological test method documents.
Ng, Raymond, Various years. List of publications on “On the importance of data cleansing and pre-processing for Genomic studies. http://www.cs.ubc.ca/~rng/publications.html.
Veldhoen, N., Ikonomou, M.G., Dubetz, C., MacPherson, N., Sampson, T., Kelly, B.C. and Helbing, C.C. 2010. Gene expression profiling and environmental contaminant assessment of migrating Pacific salmon in the Fraser River watershed of British Columbia. Aquatic Toxicology, 97: 212-225. http://www.biomedcentral.com/1471-2199/11/74
Zhang, X., Brown, T.N., Warina, F, Heimstad, E.S., and Goss, K-U. 2010. Assessment of chemical screening outcomes based on different partitioning property estimation methods. Environment International, Vol. 36 (6): 514 – 520.
Zhang X., Diamond, M.L., Ibarra, C., Harrad, S. 2009. Multimedia modeling of polybrominated diphenyl ether emissions and fate indoors. Environ. Sci. Technol., 2009, 43 (8) : 2845 – 2850. http://pubs.acs.org/doi/abs/10.1021/es802172a
Return to the Globe