Quality Metrics: More than an FDA Initiative

In 2015 the US Food and Drug Administration (FDA) had started an initiative in order to use so-called quality metrics for the planning of their risk-based inspections. The FDA wishes that, after it has come into force, manufacturers will convey defined quality scores to the FDA via an electronic portal. The FDA plans to use these to calculate specific statistics which are supposed to allow for risk-based management and planning by the FDA. The Quality Metrics Initiative then started with a voluntary phase and a revised draft guidance was published in November 2016.  However after this it became a little quite. Since the initiative started, a lot of feedback was given by industry; they feared additional efforts and expenses.

In 2018 FDA announced two updated programmes: the Quality Metrics Feedback Program and the Quality Metrics Site Visit Program. FDA now also wants to focus on the benefits for industry, saying that with implementing a quality measurement program, "companies improve their overall quality systems" and finally product quality.

And FDA is right: Quality Metrics are useful and beneficial!

Quality Metrics have already been used in pharmaceutical industry for years - mainly internally to measure operational performance. But quality can be measured on different levels and for many processes. Done in the right way, Quality Metrics can enable companies to reach a high quality performance. They will benefit from a continuous improvement in both operational performance and GMP compliance. And both are important for the continuity of business and product supply.

Quality Metrics Definitions

Let's start with some definitions:

• Metric: a measure against a standard (standards of measurement by which efficiency, performance, progress, or quality of a plan, process, or product can be assessed)
• Indicator: provides indication of performance (qualitative or quantitative), e.g., to evaluate success of an organisation or activity
• KPI: key performance indicator; target critical areas of performance (not all indicators are key)

They all should help to develop, collate and analyse information that provides current performance feedback, anticipates future needs and enables actions. For this, data must be accurate, timely appropriate, correctly analysed and meaningful.

Points to be considered in Metrics and KPIs Development:

So the following points should be considered when developing Metrics and KPIs:

• Quantitative vs. qualitative data
• Normally data is required to be SMART (Specific Measurable Accepted Realistic Time Bound) although data "for information" can be useful, especially when starting, to establish a baseline
• Measure product specific/operational quality performance, the performance of the overall quality system and Quality Culture
• Adequate indicators to show compliance with regulations
• Only a few selected KPIs for senior management, more for local management
• Measure performance(s) in all areas with resources invested for quality activities
• Quality metrics and KPIs which should be common across global sites and offices to allow visual information and data sharing

Possible Pitfalls

And where are the pitfalls?

• Too many metrics, complex and time-consuming
• Driving wrong behaviours and actions can lead to unintended consequences in order to achieve metrics target
• Actions without understanding the context surrounding the results
• Not all have same value - determine the crucial few!    
• Metrics alone will not deliver. Metrics not meeting targets should lead to actions

The outputs of process performance and product quality monitoring are the key for continual improvement. The necessary changes can be identified, introduced, monitored and controlled to continually improve processes and systems,

In the end Quality Metrics will enable companies and regulators to benefit from a continuous investment in GMP to guarantee a high quality performance and the continuity of quality product supply.

Source: presentation by Ann McGee, PharmaLex Ireland, at the ECA event KPIs and Quality Metrics