For suggestions on how to rememdy these issues, please email firstname.lastname@example.org
Is a 0-1% Failure Rate Safe?
Researched & Written By: Austin Kang
Since the citizens of Oklahoma voted to approve the use of marijuana for medical use, product safety has been a primary concern. To ensure patient safety and provide regulation, the Oklahoma legislature put forth regulations which established the Oklahoma Medical Marijuana Authority (OMMA) to regulate the medical marijuana industry and ensure product safety. The regulations require growers and processors to have laboratory testing performed on their products for “microbials, mycotoxins, residual solvents, pesticides, THC and cannabinoid potency, terpenoid potency, heavy metals, and contaminants and filth.” (OS 310:681-8-1 (a)).
In first quarter 2020 the Oklahoma Medical Marijuana Authority began its first steps towards enforcing safety regulations. To begin enforcement, OMMA required all licensed testing laboratories to report the total number of tests performed in first quarter 2020 and to send in all certificate of analysis for all failed tests. OMMA has been taking other steps such as ordering the recall of a product (Moon Mix Recall) and tracking down batches that failed at one lab only to be retested at other labs.
OMMA shared the results of their data collection and analysis with all the laboratories which yielded surprising results. In first quarter 2020, there were approximately 10,000 tests performed. There are currently 7,143 active grower and processor licenses in Oklahoma. This indicates that each active license holder averaged a little over one test every three months. This seems to be a very low number when the OMMA regulations mandate that every ten pounds of product produced by a grower is to be tested. This would indicate that testing is not being performed as required by law, and this lack of testing puts patients at risk.
Another surprising result from the OMMA data was that out of 17 licensed labs, the overall failure rate for the other 16 labs (except Express Toxicology Services) was 1.27% while Express had an overall failure rate of 20.04%. Due to the discrepancy Express realized either we had bad science, or the other 16 licensed labs did. Express Toxicology began an extensive internal review of its testing procedures as well as testing data from outside of Oklahoma. Over the past two months, Express Toxicology contacted national experts in Cannabis Testing and reviewed testing data from mature markets to ensure we were providing our customers a quality service. What we found is the overall failure rate in mature cannabis markets falls on a range between 10-20%. The following sections provide an analysis of each issue individually.
In California, the Cannabis Business Times said that “The Bureau of Cannabis Control received 10,695 certificates of analysis; 1,904 batches had failed” (Sandy & Schiller, 2018) . This equates to approximately 17.8% of tests in the state of California failing tests. This is more in line with the testing failure rate observed in Express Toxicology’s testing results and well above the 1.27% testing failure rate reported for the other laboratory’s in Oklahoma. But to better understand the testing failures observed and their relationship to those reported for laboratories in Oklahoma, Express Toxicology went further and analyzed the testing failure data for each type of testing required by statute in the following sections:
The biggest difference in fail rates in Oklahoma between Express Toxicology Services and the 16 other labs was in the microbial testing failure rates. The flower failure rate for Express Toxicology Services in first quarter 2020 was 21.17%. The combined overall failure rate for all microbials by the 16 other labs was 1.70%. For perspective, Colorado reported a little under 15% microbial failure rate on flower (2019) . Of course Colorado does have its own set of regulations. (See excerpt below from Colorado regulatory materials.) Because of this difference, it can be expected that if products sold in Colorado were analyzed under the regulations in Oklahoma, the testing failure rates for microbials would be much higher in Colorado and closer to those reported by Express Toxicology.
The state of Oklahoma requires marijuana products to be analyzed for E. Coli and Salmonella species. Express Toxicology reported a 14.94% and a 15.35% testing failure rate for E. Coli and Salmonella, respectively. The other 16 licensed labs fail rates were 0% and 0.12%. Oklahoma has the most stringent failure thresholds for E. Coli and Salmonella of any state studied, so the expectation is that testing failures for these microorganisms would be much higher than 0% and 0.12.
For the state of Colorado, their labs reported an average fail rate of 1.3% for pesticides, which was only collected from flower and trim. Oregon, which is another state heavily invested in the cannabis industry reported an average just over 2% from 2017-2018 (Crombie & Harbarger, 2019) . Leafly stated that since March of 2019 California has seen a fail rate of 2.84% for pesticides (Leafly 2019) . Express Toxicology reported an average fail rate of 1.28%.
While Express Toxicology has a rate lower than the state average and lower than California we still have confidence in our pesticide analysis. The confidence comes from two factors: 1) Oklahoma failure threshold on pesticides allows more types of pesticides and at higher levels than California regulations, 2) Express Toxicology sent two analysts to a school in Golden, Colorado specific to the testing of pesticide residue in cannabis. In addition, we hired a PHD chemist who specializes in pesticide analysis in cannabis to come to our lab in Oklahoma and set up our methods for pesticide analysis.
California’s heavy metals were calculated based off of statistics from the state. In quarter 1 of 2020, California labs performed 1,254,887 tests and 7,690 failed for heavy metals. This equates to .61% (BCC).
After a review of the data, Express Toxicology has a high confidence in our heavy metals testing for the following reasons: 1) we purchased a brand new instrument for heavy metal analysis in Fall 2019, 2) the instrument method was installed on site by application scientists from ThermoFisher and 3) we have been able to work with our clients on most every failure and trace the source of the metal contamination which was either use of well water and/or contaminated packaging from China.
Is this an OK problem or industry wide?
After the data review showed a huge discrepancy in fail rates, Express sought out to interview national experts. The first expert interviewed was Dr. Zac Hildenbrand. Express Toxicology hired Dr. Hildenbrand as a consultant to set up Express’s lab. Dr. Hildenbrand has a PHD in biochemistry, has operated a cannabis lab in Arizona, and helped set up approximately 10 labs in 7 different states.
When Dr. Hildenbrand set up the protocols for Express’s lab he cautioned our lab that a 1 CFU threshold was unrealistic for Oklahoma. As an environmental scientist, Dr. Hildenbrand said Oklahoma should expect a higher incidence of E. Coli and Salmonella due to the agricultural nature of the state. The bovine, swine and poultry operations in Oklahoma make for a higher naturally occurring rate of E. Coli and Salmonella. Coupled with the fact Oklahoma has the most stringent threshold, Dr. Hildenbrand predicted it would not be unreasonable to see failure rates for E. Coli approaching 25%.
After the first quarter 2020 statistics were made available, we sent them to Dr. Hildenbrand to review. Dr. Hildenbrand viewed the results of the other labs as impossible, pointing directly to the fact there were 0 failures for E. Coli. When asked what would cause labs to report results that were statistically impossible Dr. Hildenbrand pointed out several possible issues being:
Corruption: A lack of responsible government regulations has allowed unethical scientists to manipulate scientific methods to produce the results manufacturers want for commercial purposes rather than the true science-based results patients need for proper dosing. Dr. Hildenbrand relayed when he operated a commercial lab in Arizona he was routinely asked how much more money it would cost to get the desired results.
Lack of Standardization: Until there is standardization of methods across the industry, unscrupulous lab owners will always be able to thrive by changing parameters to “find” the desired result.
Lack of Understanding about Testing: Instead of viewing a failure as a chance to learn and improve and use the scientific knowledge of the lab to help identify and remediate the source of the contamination, license holders view failed tests as a potential death sentence for their business. Manufacturers and growers are extremely worried about surviving in a tough business environment caused by high taxation, lack of professional resources (banking, accounting, attorney), and the lack of agreement between state and federal laws which causes many manufacturers and growers to feel the need to cut corners to compete. This includes taking their analysis from a lab providing ethical testing to a less ethical laboratory to avoid failing any analysis and subsequently prevents the manufacturer or grower from improving their production processes and providing the best and safest products for end consumers.
After learning of the issues Dr. Hildenbrand saw while working in multiple markets, we researched potential lab issues in other states. We emailed lab owners in every mature market and heard back from laboratories in the following states.
An article was published on leafly.com about the ongoing cannabis laboratory issue in the Washington State cannabis testing market. A lab in Washington lost its lab license for having a 0% microbial failure rate. The investigation was based on a complaint after a study was conducted by Confidence Analytics.
Chief executive officer of Confidence Analytics, Nick Mosely conducted a study that took place in Washington, and evaluated the top 11 labs’ failure rates for microbials. The top labs were determined by market share, and out of those 11 labs, 2 labs reported failure rates of 0%. Mr. Mosely stated, “The average failure rate for the industry as a whole is 6.5%, making a failure rate of zero over a thousand samples a near impossibility” (Mosely, 2016) .
For this project, we spoke with Pat Reynolds, the Operations Director for Confidence Analytics. Mr. Reynolds talked about Washington state going through the same growing pains of trying to regulate a market with so many black-market business practices coming through to the legal industry. Mr. Reynolds said the approach they have chosen to take as a lab is to not work with clients who do not share a common commitment to helping provide patients and consumers safe medicine. Mr. Reynolds relayed in the first years of operation, they saw microbial failure rates approaching 20% until the industry adjusted practices learned through testing and provided a safer product. Mr. Reynolds was not surprised by the numbers from Oklahoma and relayed it as a problem in every emerging market which takes time to stabilize.
California is a perfect example of the failure of a marijuana market due to pervasive black-market tactics and a lack of responsible legislation. If legislatures do not embrace the people’s will and get ahead of it you end up with archaic regulators models that protect neither patients nor commercial license holders. Ineffective regulation has caused up to 80% of the states’ cannabis market to remain illicit despite legal outlets (McGreevy, 2019) .
A large part of this is through unrealistic testing procedures. Most patients are willing to pay more for “medicine” in a legal market but when trust in the regulatory and testing scheme erode to where a patient is no more assured the medicine they are inhaling is no safer than the ditch weed smeared in axle grease they smoked before legalization, they will never pay a higher price for a “legal” market product.
We interviewed Kari Kelly who is the Vice President of Marketing at CannaSafe Labs, a cannabis testing facility in California. Ms. Kelly talked about the huge issue they have experienced with what is known in the cannabis industry as “lab shopping”. In the cannabis industry prices paid for product are often based on the percentage of cannabinoids. Labs showing higher percentages of Delta 9 THC (will often command higher prices) are more desirable. This opens the door for unethical operations to go to underfunded labs that have older equipment or methods where they get numbers that can be adjusted to meet the clients wishes instead of a scientific answer. Ms. Kelly says lab operators are trying to form industry standards to eliminate inconsistencies in results.
Express Toxicology was very sensitive to the issue of needing the best equipment. We realized earlier this year our residual solvent rate did not match what we expected to see by national statistical modeling. This led Express Toxicology Services to invest over $200,000 in the middle of Covid to purchase brand new instrumentation for terpenes and residual solvents. Since switching to a newer instrument our residual solvent fail rate went from 0% in first quarter to 5.32% for the month of May. While we understand an increased number of testing failures may not be popular with all manufacturers and growers, responsible members of this community will recognize that continued improvement in testing will expose problems that, once fixed, will produce superior and safer products for the end patient.
Food for Thought
The research we conducted leads to more questions than it answers. Such as: How can the state make sure patients receive safe medicine until the state surveillance lab and seed to sale is operational? How do we cure the system when patients need “essential” medicine immediately and the State needs the tax revenue from the industry with the most potential for growth? What does the state do when by its own numbers knows only a small percentage of product is being tested? How do we keep patients safe?
We do not pretend to know the answers, but we are willing to have those honest conversations, if the first question is “how do we protect our patients?” While the industry answers the bigger questions we at Express Toxicology have hired an expert in scientific testing and quality assurance based on 20 plus years of experience to continue to improve our microbial program and improve all other forms of analysis we perform. Express also supports the work of national scientific organizations like the Association of Official Analytical Chemists (AOAC) and the American Society for Testing and Materials (ASTM) who are bringing together scientists nationwide to develop standardized methods for the analysis of cannabis. We are open to ideas on how to fix the industry because we know it will take all of us to fix this problem together.
i. | Schiller, Melissa. “Most of Colorado’s Failed Cannabis Tests Stem from Microbials: Here’s What Cultivators and Dispensaries Can Do About It.” Cannabis Business Times, Cannabis Business Times, 13 May 2020, https://www.cannabisdispensarymag.com/article/california-testing-failures-pesticides-labeling/
ii. | Colorado Department of Revenue.(2019) 2019 Annual Report. Available at: https://drive.google.com/file/d/1rCWw9AquV9Pr1STMbv8dySrU6L2wJHfI/view (Accessed 28: May 2020).
Harbarger, Molly, and Noelle Crombie. “Oregon Marijuana Regulators Fail to Meet Even Basic Standards, State Audit Finds.” Oregon live, 1 Feb. 2019, www.oregonlive.com/news/2019/01/oregon-marijuana-regulators-fail-to-meet-even-basic-standards-state-audit-finds.html
iii. | Adlin, Ben. “Can Washington Fix Its Broken Cannabis Lab Testing System?” Leafly, 1 Oct. 2018, www.leafly.com/news/industry/can-washington-fix-its-broken-cannabis-lab-testing-system.
iv. | Mosely, Nick & Confidence Analytics. (2016). Call to Action. Memorandum: Available at: https://www.conflabs.com/wp-content/uploads/2020/06/Call-to-Action.pdf
v. | McGreevy, Patrick. “Two Years in, California’s Legal Marijuana Industry Is Stuck. Should Voters Step in?” Los Angeles Times, Los Angeles Times, 24 Dec. 2019, www.latimes.com/california/story/2019-12-24/two-years-california-legal-marijuana-industry-is-stuck