You are here

P T. 2017;42(7): 446-448,472

Drug Industry Groups Oppose FDA “Quality Metrics” Plan

The Agency Wants to Use Manufacturing Data to Attack Drug Shortages
Stephen Barlas

Pharmaceutical manufacturers and their suppliers have mounted a campaign to dissuade the Food and Drug Administration (FDA) from inaugurating a voluntary program aimed at reducing drug shortages. The agency wants to start collecting “quality metrics” about manufacturing systems starting in January 2018, and has, it thinks, provided an incentive for players up and down the drug chain to participate by promising to publish a list of voluntary participants. That list would in essence be something of a “responsible manufacturers” roll, though lacking that title.

Manufacturing shortcomings were one cause of drug shortages cited by the Government Accountability Office (GAO) in a July 2016 report, “Drug shortages: Certain factors are strongly associated with this persistent public health challenge.”1 The GAO report focused on warning letters citing non-compliance with manufacturing standards issued to the makers of sterile injectable anti-infective and cardiovascular drugs in 2012, 2013, and 2014. Other factors leading to shortages were supply disruptions and, where generics are concerned, relatively low profit margins. The GAO found that the number of new shortages decreased between 2010 and 2015, but the total number has increased as existing shortages for some drugs continued. For example, in 2015, 68% of the shortages (291 of 427) were ongoing shortages that began in a prior year.

Manufacturers of generic injectables suffering from quality issues—and receiving warning letters as a result—are a particular problem because many of those facilities produce more than one drug. For example, 69% of the 118 drugs in the GAO study were manufactured by at least one of nine establishments. If one of these nine establishments failed to comply with manufacturing standards, many drugs could be affected. In 2012, for instance, one establishment that failed to comply with manufacturing standards and received a warning letter produced 22 drugs.

The GAO report cited a study by Woodcock and Wosinska that alluded to quality issues in shortages of generic sterile injectable drugs.2 The GAO said quality problems stem from various sources, including insufficient maintenance, outdated or inadequate design of sterile manufacturing processes, and poor oversight that does not test for or respond adequately to indicators of potential quality problems.

The FDA does not mention the GAO study in its November 2016 revised draft guidance document announcing plans for its voluntary quality metrics program.3 “We believe that there is a benefit to publicly sharing the names of establishments that voluntarily choose to submit these quality data to FDA because, through their participation, these establishments demonstrate a willingness to proactively engage with the agency in pursuit of the goals described in this guidance,” the agency states. “Participation in this voluntary reporting phase of the program also demonstrates a commitment to increasing transparency between industry and FDA and a contribution to improving quality monitoring throughout the industry.

The agency would not disclose proprietary information submitted by manufacturers, compounders, processors of finished dosage forms, and active pharmaceutical ingredient (API) suppliers.

Industry Reaction to the Proposal Is Negative

The prospect of this voluntary reporting program has elicited nearly unanimous opposition from every corner of the drug manufacturing industry, despite the fact that the FDA has made numerous changes requested by manufacturers since the concept was first broached, as a mandatory program, in 2015. The Pharmaceutical Research and Manufacturers of America (PhRMA), the drug manufacturers’ trade group, concedes that the FDA has made many positive changes over the past few years and that an acceptable program, once established, would offer potential benefits for pharmaceutical companies, such as the possibility of reduced inspection frequency and reduced post-approval change reporting. According to William W. Chin, MD, PhRMA’s Executive Vice President for Science and Regulatory Advocacy:

However, our current belief is that the burden of industry and FDA collecting company-submitted quality metrics data outweighs the stated potential benefits to the extent that we do not support the program moving forward as currently proposed without further dialogue. To achieve these benefits through future utilization of the concept of quality metrics in risk-based regulatory decision-making, we recommend continued ongoing dialogue with all stakeholders, including through public consultation and other opportunities for shared learnings. We continue to believe this dialogue would help to further the concept of quality metrics in risk-based regulatory decision-making in tandem with other related regulatory policy initiatives.

Suppliers to brand-name and generics companies are equally opposed to the program in its current form. “We, like many other industry groups, feel that there are still too many unanswered questions and ambiguities in the current draft guidance to proceed forward at this time and request that the agency pause their pursuit of this program until further dialogue and alignment with industry can occur,” says John DiLoreto, Executive Director of the Bulk Pharmaceuticals Task Force.

The Association for Accessible Medications (AAM), which represents the generic drugs industry, says it supports the FDA’s goal of acting to reduce drug shortages. But David R. Gaugh, RPh, Senior Vice President for Sciences and Regulatory Affairs at AAM, states:

We are concerned that the actions FDA is taking with its quality metrics program are more likely to increase drug shortages than reduce them. Specifically, the significant burden to report the data requested in the draft guidance must be factored into a company’s decisions about continuation of products. This may lead to the discontinuation of products that give poor metric results that increase the risk of more frequent inspections. It may also lead to the discontinuation of contract manufacturers and other suppliers that cannot, or will not, meet metrics reporting requests. Firms may eventually move production capacity outside the U.S., especially scarce injectable capacity, in order to avoid FDA metrics reporting burdens.

But one academic who has written about drug shortages thinks the FDA program would be “without question a move in the right direction.” Benjamin Davies, MD, Associate Professor of Urology at the University of Pittsburgh School of Medicine and Chief of the Urology Section at the Shadyside/Hillman Cancer Center, was coauthor of a recent article in the New England Journal of Medicine (NEJM).4 Dr. Davies and colleagues found that a shortage of the bladder-cancer drug BCG in 2014 and 2015—due to manufacturing problems—led to sharp price increases for a less effective alternative treatment, mitomycin. He says the data the FDA wants companies to report are usually seen at the time of an FDA inspection, so seeing them in some fashion prior to that would be ideal.

But while it would be a useful step, Dr. Davies questions the impact the voluntary program may have. “How that actually stops drug shortages without some type of new legislation is not clear to me,” he adds. “Without regulatory change, just having the data on a potential drug shortage is kind of like a gun without a bullet.”

The FDA’s Rationale

In a sense, this list of “responsible manufacturers” that voluntarily submit data is a little like the new FDA 503B outsourcing facility program. Compounders selling across state lines can voluntarily register for the 503B program. In doing so, they get a leg up on marketing to hospitals that want to buy from compounders that have been vetted by the FDA. So in developing the “quality metrics” submission program, the FDA wants manufacturers, for example, to find the responsible manufacturers list useful in selecting contract manufacturers and component suppliers. In the same vein, the list may also be useful for health care purchasing organizations, health care providers, patients, and consumers in sourcing drugs when used in conjunction with other information (e.g., inspection histories).

Drug shortages have been a big problem for a decade, especially for hospitals. There are many causes, but a major one is that FDA inspections of manufacturers often find quality issues leading to enforcement actions, which can halt manufacturing of a particular drug. In its announcement of the voluntary program, the FDA said: “The agency has found that the majority of drug shortages stem from quality issues—the discovery of substandard manufacturing facilities or processes, or identification of significant quality defects in finished products, necessitating remediation efforts, which in turn, may interrupt production ...”

Drug shortages don’t just mean that some patients can’t get the drugs they need. Shortages also mean that alternative drugs for the same condition that do remain available jump in price. The NEJM report by Dr. Davies and his coauthors found that the shortage of BCG in 2014 and 2015 led to the average wholesale price of a vial of mitomycin, a generic drug, nearly doubling to $869.59 in August 2014, not long after the BCG shortage developed. The cost of a lower-dose vial of mitomycin rose 146% to $165.60. As the BCG shortage continued due to manufacturing problems at drug makers Sanofi SA and Merck and Company, prices for mitomycin increased again in 2015, to $1,415 for the higher-dose vial and $272.46 for the lower dose, according to the research, which cites data from Truven Health Analytics. The increases contributed to higher spending by Medicare on mitomycin, which rose to $15.8 million in 2015 from $4.3 million in 2012, according to the NEJM report.

Roots of the Quality Metrics Program

As with many FDA initiatives, this one has been germinating for quite a while. In 2002, the FDA launched an initiative entitled “Pharmaceutical cGMPs [current good manufacturing practices] for the 21st Century: a risk-based approach,” to encourage the implementation of a modern pharmaceutical quality assessment system.5 The initiative was published with several goals, among them ensuring that regulatory review, compliance, and inspection policies support continuous improvement and innovation in the pharmaceutical manufacturing industry. The idea was to develop “a maximally efficient, agile, flexible pharmaceutical manufacturing sector that reliably produces high-quality drug products without extensive regulatory oversight.”

The FDA says significant progress has been made over the years. “Despite these achievements, however, we have not fully realized our 21st century vision for manufacturing and quality, and indicators of serious product quality defects persist,” the agency says. The Food and Drug Administration Safety and Innovation Act of 2012 enhanced the FDA’s authority to seek quality data by amending the Food, Drug, and Cosmetic Act to define the term “current good manufacturing practice” as including the implementation of oversight and controls over the manufacture of drugs to ensure quality, including managing the risk of establishing the safety of raw materials, materials used in the manufacturing of drugs, and finished drug products.

A draft guidance document outlining the voluntary “quality metrics” system was published in 2015.6 Then, as now, the industry erupted in boos. The FDA plowed ahead nonetheless and published the revised draft guidance in November 2016. In its initial July 2015 proposed guidance, the FDA required reporting of 10 data elements from which the agency would calculate four metrics. In the current revised draft guidance, the FDA requires reporting of 11 data elements from which the agency will calculate three metrics. While the FDA has reduced the number of metrics it will calculate, it has increased the number of data elements that establishments will have to report. However, the agency bowed to industry concerns by making what PhRMA has applauded as positive changes, including the shift to a phased-in approach by implementing the quality metrics program with an initial voluntary phase, the option for reporting quality metrics by site in addition to reporting by product, a reduction in the number of quality metrics to be calculated, clarified metric definitions, the inclusion of example scenarios, and the addition of comment fields.

PhRMA appears to be slightly more mollified—though still critical—of the November 2016 draft guidance than the generics industry, which would probably be affected most, given that shortages are a particular problem in the generic injectables sector. “The agency’s explanation that it is reducing the burden on industry runs contrary to the evidence,” the AAM’s Gaugh argues.

Reporting Elements

The guidance describes two types of quality metric data reports: product reports and site reports. For example, a company that manufactured product A in six facilities would submit a product report. That same company that manufactured Products A, B, and C in Facility X would submit a facility report. The FDA intends to calculate quality metrics for each product and covered establishment. The metrics are:

  • Lot acceptance rate (LAR) as an indicator of manufacturing process performance. LAR = the number of accepted lots in a time frame divided by the number of lots started by the same covered establishment in the current reporting time frame.
  • Product quality complaint rate (PQCR) as an indicator of patient or customer feedback. PQCR = the number of product quality complaints received for the product divided by the total number of dosage units distributed in the current reporting time frame.
  • Invalidated out-of-specification (OOS) rate (IOOSR) as an indicator of the operation of a laboratory. IOOSR = the number of OOS test results for lot release and long-term stability testing invalidated by the covered establishment due to an aberration of the measurement process divided by the total number of lot release and long-term stability OOS test results in the current reporting time frame.

In both the product reporter and site reporter categories, the FDA would rate the submitter in one of three categories, based on the comprehensiveness of its submission. For example, the product reporter top tier would consist of companies that submitted complete data supporting all metrics for each covered establishment in the manufacturing supply chain for all covered drug products (or APIs used in the manufacture of a covered drug product) for the full-year reporting period. There would be a lower mid-tier category and a lowest category.

One reason for the industry’s opposition to the collection of the data underlying calculation of the three metrics is that the data are not the type typically collected. “The proposed requirements are complex and preclude standardization due to challenges with unclear definitions, which are different to those commonly applied in industry and in the ISPE Pilots programs,” explains John E. Bournas, CEO and President of the International Society for Pharmaceutical Engineering (ISPE). “Lack of clear and standardized quality metrics data elements will confound attempts at data analysis and will likely lead to unusable data. This will limit the ability to draw conclusions and achieve the desired benefits.”

Bournas believes total costs for all participants would be a minimum of $285 million, not including information technology and system construction costs. But it is hard to credit that estimate given that the number of participants is unknown at this stage. “We also believe there is a high likelihood that requirements will change in the future and that further significant expenditure will be required,” Bournas adds. “There are also opportunity costs for implementing this program as resources would be applied to a low-value program and be diverted from working on other company existing key performance indicator and continual improvement programs. We are also concerned about the high levels of management attention that may be given to a relatively low-value program, since data will be submitted to FDA rather than just used for internal review.”

Cost aside, the guts of the FDA program seem to be well thought out based on the results of two quality metric pilot programs conducted by the ISPE in cooperation with consultant McKinsey and Company. The pilot had participation from 28 companies and 83 sites. These companies and sites represented a wide range of technologies; they included contract manufacturing organizations and laboratories and drug-substance manufacturing sites. The pilot studies concluded that LAR, PQCR, and IOOSR have potential value using the ISPE definitions and consequently should be studied further.7

Will Voluntary Become Mandatory?

The FDA doesn’t hide its intention to make the voluntary program mandatory after a shakeout period of undefined length. The agency promises: “After evaluating the results of the voluntary phase of the quality metrics program in 2018, FDA intends to initiate notice and comment rulemaking under existing statutory authority to develop a mandatory quality metrics reporting program.”

That doesn’t mean the FDA will definitely institute a mandatory program. The agency may not be able to accomplish the overall goals of an FDA quality metrics reporting program from voluntary reporting alone. If the FDA does not receive a large body of data from reporting establishments, the ways in which the agency can use the information may be limited. For example, data received may not constitute a representative sample of the industry. Further, a self-selection bias may increase the risk of signaling an outlier where none exists. For these reasons, the FDA expects to use the information collected to specifically focus on: 1) working with establishments toward early resolution of potential quality problems and to reduce the likelihood that the establishment’s operations will be disrupted and impact the drug supply; 2) helping to prepare for and direct its inspections; and 3) using the calculated metrics as an element of the post-approval manufacturing change reporting program with an emphasis on encouraging lifecycle manufacturing improvement.

Given its unanimous opposition, it is a good bet the pharmaceutical manufacturing sector will try to persuade both the Trump administration and Congress to forestall any new FDA quality metric reporting program, whether voluntary or mandatory—because of its anticipated cost, about which opinions differ, and because of doubts that the agency has the legal authority to mandate reporting.

Author bio: 
Mr. Barlas is a freelance writer in Washington, D.C., who covers issues inside the Beltway. Send ideas for topics and your comments to


  1. Government Accountability Office. Drug shortages: Certain factors are strongly associated with this persistent public health challenge July 7, 2016; Available at: Accessed May 23, 2017
  2. Woodcock J, Wosinska M. Economic and technological drivers of generic sterile injectable drug shortages. Clin Pharmacol Ther 2013;93;(2):170–176.
  3. Food and Drug Administration. Submission of quality metrics data. Guidance for industry. Draft guidance [revision 1] November 2016; Available at: Accessed May 23, 2017
  4. Davies BJ, Hwang TJ, Kesselheim AS. Ensuring access to injectable generic drugs—the case of intravesical BCG for bladder cancer. N Engl J Med 2017;376;(15):1401–1403.
  5. Food and Drug Administration. Pharmaceutical cGMPs for the 21st century—a risk-based approach. Final report September 2004; Available at: Accessed May 23, 2017
  6. Food and Drug Administration. Request for quality metrics [PowerPoint presentation] July 2015; Available at: Accessed May 23, 2017
  7. International Society for Pharmaceutical Engineering. ISPE Quality Metrics Initiative: Wave 2 Report June 2016; Available at: Accessed May 23, 2017