Recommendations: Are yours defensible?
What we know is important, but being able to prove recommendations is critical in an increasingly regulated and litigious society. As agronomy, crop, soil, and environmental advisers and scientists, it is essential that our data and recommendations are defensible.
This article was prepared as a contribution of the Western Region Nutrient Management Coordinating Committee (WERA-103).
I once heard a line from a movie (Law Abiding Citizen) that really stuck with me: “It’s not what you know, it’s what you can prove.” At the risk of appearing to advocate for the conflicting heartbreak portrayed in this vigilante justice tale, I think about this statement often in my work. What we know is important—no doubt about that. But, being able to prove recommendations is critical in an increasingly regulated and litigious society. As agronomy, crop, soil, and environmental advisers and scientists, it is essential that our data and recommendations are defensible.
Case Study 1
An ongoing legal battle involving a farmer and his CCA (to remain anonymous, as it is ongoing) is pitting expert witness agronomists/environmental experts and their associated laboratories against each other. A farmer and his CCA are accused of practices that harm food safety and the environment. Important to their case were laboratory records of soil, irrigation water, plant tissue, and manure/compost analytical results. However, opposing council and their experts successfully had this critical laboratory data deemed inadmissible. Why? Because of poor sampling documentation by the CCA and inadequate laboratory quality assurance/quality control (QA/QC) procedures.
Case Study 2
Another ongoing legal battle involves an adviser and the company that employs him being accused by a client of causing the failure and bankruptcy of a moderate-sized farm operation due to excessive fertilizer recommendations. There is scrutiny of the adviser’s business practices with the accusation of sampling improperly, using a laboratory with inadequate QA/QC practices, and making fertilizer recommendations based on conjecture rather than science.
Case Study 3
An urban landscape business was being assessed massive fines by a government agency. A mediation resulted in the fines being temporarily ceased to allow the accused time to collect data. The data were collected and submitted, but the agency rejected it. The rejection was based on poor sampling documentation, analytical QA/QC, and inadequate conclusions that were not fully supported by scientific documentation.
What is QA/QC?
I teach an Environmental Analytical Chemistry class in which I explain to students the three phases of assessment: sampling, analysis, and interpretation. I also emphasize there are proper QA/QC for each.
We often associate QA/QC with manufacturing, but it also applies to services provided, including data generation and the associated recommendations we give as agronomy, crop, soil, and environmental professionals. Ethically, it is crucial to understand and employ these principles for the sake of those we serve as well as to protect ourselves and our companies/employers.
There are many and, unfortunately, often circular definitions of “quality control” and “quality assurance.” A practical definition for QC in our disciplines is the operational techniques used to achieve high quality data generation and associated recommendations. The QA is the part of quality management that provides confidence that quality data and recommendations are reasonably accurate and precise. Or, in other more succinct terms, QC is what we do to achieve quality results, and QA is how we assure ourselves and others that the quality is provable. Obviously, these overlap, but QC is largely an internal activity of what we do, and QA is providing confidence both internally to administrators, etc., and externally to customers, agencies, regulators, certifiers, and other third parties that the data and recommendations have a reasonable level of accuracy and precision.
Best Practices
It is not possible to outline every QA/QC plan component utilized by the wide variety of individuals and organizations represented in our professions. There is a plethora of published information, with a few listed below, that can provide more details. In general, the following are best practices to consider, especially aimed at CCAs and other practicing professionals:
- Organizations (such as testing laboratories, companies selling fertilizers and other products, and those providing consulting) should have a QA/QC officer that has some level of autonomy with management to avoid conflicts of interest. A sole proprietor should hire a consultant and/or serve as his or her own QA/QC officer.
- This officer should lead in the creation and maintenance of a QA/QC plan with an associated manual.
- Part of the QA/QC plan should involve regular assessment and, when needed, corrective action guidelines.
- Those providing data/recommendations should have proper educational qualifications and be certified by an organization with appropriate credentials, such as CCA, CPSS, etc. These lend great credibility, especially when it comes to governmental oversight, legal issues, etc.
- These professionals should be involved in continuing education focused on reviewing tried-and-true principles as well as new scientific discoveries pertinent to the data and recommendations being generated.
- Training and credentials need to be carefully and accurately recorded.
- It is imperative to keep excellent records and a “paper trail” for data and recommendations to avoid misunderstandings. Verbal-only communication has greater risk of misunderstanding and isn’t easy to verify.
- Organizations should participate in third-party evaluations for QA/QC when available.
- Recommendations should cite references when available. When scientific backing of references is weak or absent, qualify recommendations as “opinion based on best available information at the time.”
- State these approaches and assumptions in a contract with clients, so there is full transparency with regard to the level of QA/QC that is or is not possible.
This is only a partial list of possible practices. Each organization, individual, and situation has unique QA/QC needs and, as such, unique practices to be followed. The key is to think defensibly. Ask, “If my data and/or recommendations were challenged, in a legal setting or otherwise, can I defend them adequately?” In most instances, we can’t truly “prove” most things, and an educated opinion is an important part of many interactions. But, we must do as much as is reasonably possible to be QA/QC minded in our professional activities.
Example: Fertilizer Recommendations
Providing fertilizer recommendations to a farmer is a very common example of what most CCAs are involved in professionally. How do these QA/QC principles apply in this situation? Again, we need to think in terms of the three phases of assessment. Fertilizer recommendations are generally built upon the foundation of soil, water, and/or tissue sampling and analysis.
Phase 1: Sampling
The first step in QA/QC for fertilizer recommendations is with regard to taking samples. The QC portion of this is identifying/creating a sampling plan based on a proven scientific approach with appropriate references to provide credibility. Training the sampler properly is imperative. Handling and shipping the samples properly is also essential.
The QA portion of this process involves the records of this training, the sampling, and the handling/shipping. Sampling depth, plant part, etc. should be considered in light of scientifically based guidelines with the specifics recorded. Consider identifying precise sampling locations using global positioning systems (GPS), especially for high-scrutiny samples involved in lawsuits, regulatory compliance, etc. If samples are handled in terms of being washed, dried, cooled, frozen, acid treated, etc., these should be done following recommended procedures, and these activities recorded. Shipping and delivery information is also important as time and temperature can impact the integrity of the samples. Some high-profile samples, such as regulatory situations, require formal chain-of-custody documentation. The bottom line is that, if asked, you can say, “Yes, we followed these recommended procedures for sampling and handling, and here are our records for training and for doing so.”
Phase 2: Analysis
The next step is with regard to the analysis. It is essential to utilize a laboratory with a written QA/QC plan and with a proven track record of performance. The laboratory should have QC procedures in place with regard to sample receiving, preparation, analysis, and data management. Each of the regional soil test working groups have written specifications for laboratory QA/QC located at: www.naptprogram.org/methods. They should have a battery of QC samples of each matrix being tested (soil, plant tissue, water, etc.). The analytical results for each are evaluated to determine the quality of the data. These include standards used to calibrate instruments, process and instrument blanks (negative controls), check samples of the same matrix and with known values, and, when appropriate, solutions with known values. Additionally, some percentage of the samples should be done in duplicate or triplicate (typically 5% of samples should be duplicated, but up to 100% for samples with a relatively high need for accuracy and QA). These QA/QC samples are evaluated with each batch of client samples with the client samples being bracketed on both ends by QC samples in a similar range and matrix.
In addition to internal QC evaluation, laboratories should participate with third-party evaluation, such as the North American Proficiency Testing (NAPT) Program (www.naptprogram.org/about/participants). This program furnishes laboratories with QA/QC tools through quarterly blind statistical evaluation of soil, plant, and water samples. These tools assist laboratories in generating accurate and precise analyses as well as leveraging their participation in assuring clientele and other consumers that their data meet high standards. Participating laboratories benefit from the scientific resources of the Soil Science Society of America, which sponsors this program, and the vast expertise of its many member scientists—with specific oversight by a committee comprised of representatives from regional workgroups, government, scientific societies, and laboratories. In addition to the regular proficiency testing samples, many laboratories participate in the Performance Assessment Program (NAPT-PAP) required by the USDA-NRCS, which provides a minimum level of performance to be certified as a PAP laboratory (www.naptprogram.org/pap). Additionally, some states have specific performance criteria. These programs provide credibility for a participating laboratory.
Additionally, laboratory clients can take the further step of submitting “double-blind” samples. Blind QA/QC samples, including those from proficiency testing programs as described above, are those where the analyst is aware that the samples being tested are QA/QC samples but does not know the correct analytical values until after the evaluation is completed. The difference with a double-blind sample is that the analyst is not aware the sample is a QA/QC evaluation. This removes any possible bias from the analysis.
Laboratory clients can obtain samples of known value at www.naptprogram.org/samples or www.nist.gov/srm. A possibly less costly alternative is to collect, dry, and carefully homogenize soil and/or plant tissue samples and submit them to a lab with some or all batches of their submitted samples. Water samples are similar, and while they are obviously not dried or homogenized, they do need acid or other preservation steps.
Phase 3: Recommendations
The final step relates to making fertilizer and other soil management recommendations. The phrase “garbage in equals garbage out” is true—with recommendations of no value or even harmful if the sampling and/or analysis is faulty or has poor QA/QC backing. These recommendations need to be based on the best science available and be documented in some way with full transparency to those receiving the recommendation.
Ideally, recommendations are based on calibration studies performed on the exact soil, environment, and variety/hybrid being grown. As the possible combinations of these are infinite, this standard is not reasonable. But, rather, the closest matched data set available should be used and disclosed. For example, we know scientifically that the Bray P1 soil test works well on non-calcareous soils but fails and should not be used for determining plant-available estimates of phosphorus on calcareous soils. If a calcareous soil from Idaho was being submitted for a fertilizer recommendation for potato, it would be appropriate to use the Olsen bicarbonate extraction as this method is endorsed by the state’s land grant institution with responsibility in soil-testing guidelines as well as by regional committees with a similar mandate. It is possibly acceptable to use a different extractant if there is enough data to justify it in terms of QA. Many private laboratories do so ethically and responsibly. In order for the recommendation to be accurate, the fertilizer recommendation has to be based on calibration work done on the appropriate soil types with the correct soil test method.
Continuing with this example, if the client were planting the most commonly grown variety in the USA, ‘Russet Burbank’, there is ample data available to make the recommendation (Hopkins et al., 2020). But, if the client were planting a less studied variety, such as ‘Frontier Russet’, there would be less confidence in the recommendation. Nevertheless, the grower needs a recommendation, and we can’t just throw up our hands and say, “Sorry, there isn’t enough data.” At the same time, we need to be cautious in what level of confidence is portrayed in the recommendation. In this case, it would be appropriate to state that the recommendation is based primarily on data from ‘Russet Burbank’, giving a citation(s), and if appropriate, adjustments made based on tentative observations with ‘Frontier Russet’. In this way, the quality of the information is not overstated. The client can be assured in the level of information that is available.
Summary
As a research scientist and a former director of agricultural/environmental laboratories, my employees/students and I have taken thousands of soil, plant, and water samples. We’ve conducted tens of thousands of analyses and generated associated fertilizer recommendations. The data is our product, and the recommendations given are paid for opinions. It is an ethical responsibility to do as much as is reasonably possible to have a QC plan that allows for excellent results and to be able to have a QA program that provides confidence to those utilizing the data. This is just one example of QA/QC. Each organization/individual needs to apply these principles in general and spend the needed efforts to customize a QA/QC plan appropriate for their unique circumstances. Again, it isn’t completely about what you know, but about what you can prove.
Dig deeper
Barth, D., & Mason, B. (2002). Soil sampling quality assurance user’s guide (EPA/600/4-84/043). Washington, DC: USEPA.
Hopkins, B.G., & Hansen, N.C. (2019). Phosphorus management in high-yield systems. Journal of Environmental Quality. 48, 1265–1280. https://doi.org/10.2134/jeq2019.03.0130
Hopkins B.G., Stark, J.C., & Kelling, K.A. (2020). Nutrient management. In J. Stark, M. Thornton, & P. Nolte (Eds.) Potato production systems (pp. 155–202). New York, NY: Springer. https://doi.org/10.1007/978-3-030-39157-7_8
Konieczka, P., & Namiesnik, J. (2018). Quality assurance and quality control in the analytical chemical laboratory: A practical approach ( 2nd ed). Boca Raton, FL: CRC Press, Taylor & Francis Group.
Text © . The authors. CC BY-NC-ND 4.0. Except where otherwise noted, images are subject to copyright. Any reuse without express permission from the copyright owner is prohibited.
