Statement from the Soil Health PartnershipBy SHP Staff on Wednesday, 03 July, 2019
On Thursday, June 27th, the Soil Health Partnership (SHP) was contacted by the Cornell Soil Health Lab to alert the organization of a data quality issue that affects the 2018 and 2019 Soil Organic Matter test results conducted on farms working with SHP. This data quality issue affects previous results that were presented publicly by SHP describing increases in Soil Organic Matter (SOM) on farms in the SHP network of during the first 2-3 years in the program.
When SHP first publicly presented aggregated results of SOM increases over time across our network at our Soil Health Summit in 2019, the magnitude of these increases was met with skepticism by some collaborators and members of the SHP Science Advisory Committee. At that time, our Lead Scientist Maria Bowman called the Cornell Soil Health Lab to discuss the increases SHP was seeing in the data, and to inquire about whether there were any changes in methods or analysis at the Lab that might be responsible for an inflated result. Bowman was assured that this was unlikely due to the Lab’s quality control procedures.
Given Cornell’s reputation as a leader in the space, SHP was confident that Cornell was sharing what they knew and also that they instituted sufficient quality checks such that SHP could trust the results we had received.
How does this affect SHP’s results?
The data quality issue occurred because Cornell began outsourcing the nutrient analysis portion of their soil health test, including OM analysis, to another lab in March 2018. Although the Lab used the Cornell analysis methods, the Lab made a data processing error that resulted in overestimated percent SOM values starting in March of 2018. Cornell submitted corrected SOM results to the Soil Health Partnership on June 27, the same day we were alerted of the issue.
As we re-analyze the data, it does not change the overall conclusion, which is that SHP sees a statistically significant increase in SOM, on average, on the treatment fields participating in the Soil Health Partnership trials; 0.17%-0.21% over the first 2-3 years in the program. The estimated increases are not as high as the initial analyses indicated. However, the results still clearly show the impact of soil health management systems across the SHP network of sites and at scale across diverse geographies, soil types and trial types.
Moving forward, SHP commits to further review of all quality assurance and quality control protocols of the labs we work with, including requesting that we be notified in writing of any changes to lab protocols or procedures. We also see this as an opportunity to continue working with our several lab partners as they continue to grow and evolve and hope we can support them as they continue to build their capacity as the soil health arena develops. This also shines a light on the need for continued support and funding of critical capacity at land grant and related entities that are meet these needs for farmers.
We feel it is important to share this issue directly. The Soil Health Partnership understands our great responsibility to collect and analyze data leading to the release of accurate insights from our research. Farmers make economic decisions based on our insights and we do not take that role lightly. The interest in, and enthusiasm for, continued learning about the science and economics of soil health means we must all communicate clearly about the strengths and limitations of our work. These matters of scientific integrity should be dealt with quickly and clearly and recognize our partner, the Cornell Soil Health Lab, for their clear communication and correction of this issue.