TwitterLinkedin

Contact Us

  • Biopharma Excellence
  • News & Events
    • Industry News Articles
    • Press Releases
    • Webinars
    • Events
    • Blog
    • Podcasts
  • Resource Library
    • Thought Leadership
    • Fact Flyers
    • Case Studies
    • White Papers
    • Webinars
    • Infographics
    • Video & Animation
  • Careers
    • Life at PharmaLex
    • Current Opportunities
  • Training
Top Bar
Search
PharmaLex Logo
MENUMENU
  • PharmaLex Logo
  • PharmaLex Logo
  • About UsAbout Us
    • About Us
    • Management Team
    • Corporate Social Responsibility
    • What Our Clients Say About Us
  • Our Services
    • Discovery / Non-clinical
          • Go to Discovery / Non-clinical section >>

          • Strategy and Consulting

            • Integrated Product Development
            • Market Access
            • Scientific Advice
            • Statistics and Data Sciences
            • Toxicology Services
    • Clinical Development
          • Go to Clinical Development section >>

          • Strategy and Consulting

            • Clinical Program Development
            • Scientific Advice
            • Statistics and Data Sciences
          • Regulatory Affairs

            • Clinical Trial Applications
            • Global Procedure Management
            • Health Authority / Agency Interaction
            • Regulatory Operations
          • Pharmacovigilance

            • Clinical Trial Safety Support
            • Pharmacovigilance Consulting
          • Quality

            • GxP Services
            • Interim / Contract QA
            • Quality Management Systems
            • Tech Transfer / Scale-up
    • Authorization / Approval
          • Go to Authorization / Approval section >>

          • Strategy and Consulting

            • Market Access
          • Regulatory Affairs

            • CMC Services
            • Global Procedure Management
            • Health Authority / Agency Interaction
            • Marketing and Labeling Activities
            • Regulatory Operations
            • Scientific, Regulatory and Technical Writing
            • Statistics and Data Sciences
          • Pharmacovigilance

            • EU QPPV / National QPPV
            • Pharmacovigilance Consulting
          • Quality

            • Commercialization Readiness
            • Commissioning, Qualification, and Validation (CQV)
            • GxP Services
            • Interim / Contract QA
            • PAI Readiness
    • Post-approval / Maintenance
          • Go to Post-approval / Maintenance section >>

          • Strategy and Consulting

            • Market Access
          • Regulatory Affairs

            • CMC Services
            • Global End-to-End Outsourcing
            • Global Procedure Management
            • Health Authority / Agency Interaction
            • Marketing and Labeling Activities
            • Mergers and Acquisitions (M&A) Transfers
            • Regulatory Operations
            • Toxicology Services
          • Pharmacovigilance

            • EU QPPV / National QPPV
            • ICSR (Individual Safety Case Report) Management
            • Literature Monitoring & Screening
            • Pharmacovigilance Quality & Compliance
            • Signal Management
          • Quality

            • Commissioning, Qualification, and Validation (CQV)
            • GxP Services
            • Interim / Contract QA
            • Quality Management Systems
          • Medical Affairs

            • Healthcare Compliance and Medical Approval
    • Program Management
          • Go to Program Management section >>

          • Program Management

            • Global End-to-End Outsourcing
            • Global Procedure Management
            • Integrated Product Development
            • Mergers and Acquisitions (M&A) Transfers
    • Featured Expertise
          • Industry Expertise

            • Biopharmaceuticals
            • MedTech Services
          • Service Expertise

            • ATMP / Cell and Gene Therapy
            • COVID-19 Support
            • GxP Services
            • Market Access
            • Pharmacovigilance Consulting
            • Post-Brexit Regulatory Support
            • Statistics and Data Sciences
            • SMARTPHLEX - technology-enabled services
  • Global ReachGlobal Reach
  • Contact usContact us
  • News & Events
    • Industry News Articles
    • Press Releases
    • Webinars
    • Events
    • Blog
    • Podcasts
  • Resource Library
    • Thought Leadership
    • Fact Flyers
    • Case Studies
    • White Papers
    • Webinars
    • Infographics
    • Video & Animation
  • Careers
    • Life at PharmaLex
    • Career Opportunities
  • Training
Home > News & Events > Industry News Articles > Are my (bio)pharmaceutical assay performances reliable? Only probability of success counts!

Are my (bio)pharmaceutical assay performances reliable? Only probability of success counts!

Alternative to traditional Gage R&R metrics for the pharmaceutical industry

Author(s): Thomas de Marchin (Senior Manager Statistics and Data Sciences at Pharmalex), Laurent Natalis (Associate Director Statistics and Data Sciences at Pharmalex), Tatsiana Khamiakova (Associate Director Manufacturing and Applied Statistics at Janssen), Eric Rozet (Director Statistics and Data Sciences at Pharmalex) and Hans Coppenolle (Director Manufacturing and Applied Statistics at Janssen). This article was originally presented at the conference NCB 2021.

Introduction

Gage R&R studies are often conducted in the industry to determine the operating performance of a measurement system and determine if it is capable to monitor a manufacturing process. Several metrics are commonly associated with Gage R&R studies, such as the precision-to-tolerance ratio (P/T), the precision-to-total-variation ratio (%RR), the Signal to noise ratio (SNR), the %Reproducibility and the %Repeatability. While these metrics may suit well the overall industry, they could be problematic once applied in drug manufacturing sector for several reasons, (1) (bio)pharmaceutical assays are often more variable than common physico-chemical measurement systems and the usual criteria are too restrictive for the pharma industry, (2) analytical methods cannot always be improved once qualified, and (3) measurements are usually costly and time consuming, which makes difficult to have enough data to estimate all sources of variance with high precision.

All the above-mentioned difficulties often lead to blame the measurement system and trigger alarms, while the quality of the measurements might be acceptable for its intended purpose. Moreover, it is often difficult to figure out how to interpret the most used Gage R&R metrics, which lack strong mathematical rationale[1] and because they typically do not consider uncertainty of the estimates.

The objective of this work is to propose an alternative way adapted to drug manufacturing to investigate whether the measurement system is acceptable for its intended purpose. Our approach focuses on understanding the properties of the measurement system and aims to answer the only relevant questions: (1) Is my measurement system able to discriminate good and bad batches? (2) Will future batches meet the specifications, given the process and measurement variabilities investigated in the Gage R&R experiment?

Data

GAGE R&R DATA

In a typical Gage R&R context, several batches are repeatedly measured by different analysts on different days using different equipment and, in some cases, in different laboratories. Gage R&R data were simulated:

Figure 1: Variability charts as a function of laboratory, day, analyst, device and batch. The three colors refer to three different batches.

CPV DATA

While Gage R&R data is usually used to provide information about the measurement system and the process variabilities in the overall industry, process variability cannot be estimated correctly from this data in the pharmaceutical context. Indeed, only a limited number of batches (3–5) are usually measured. Moreover, extreme batches are usually selected to span the specification range and are therefore not representative of the process effective batch-to-bath variability.

On the other hand, process variability might be estimated from other sources such as is Continued Process Verification (CPV). In CPV, each CQA is measured for each batch and monitored across time to verify if the process is remaining under control. A typical CPV dataset was simulated and graphed with its specifications ([90–110%]:

Figure 2: Measured value of batches produced. Red horizontal lines: specifications. Green dotted lines: control limits (mean ± 3SD).

Results

From the Gage R&R dataset, it is possible to estimate the measurement system variability and calculate different metrics. The following model has been fitted to the Gage R&R data using a Bayesian framework with the R package brms:

The main advantages of using a Bayesian methodology are (1) it takes into account the uncertainties related to models and parameters values, (2) it is predictive and make it easy to simulate future measurements by simulation and (3) it is possible to incorporates relevant prior probabilities, which is useful when data available are limited.

The following metrics can be computed from the Gage R&R dataset:

a) Relative contribution of each component (variance component analysis)

b) Single and Multiple laboratory precisions (uncertainty of measurement). A result of X±Y means that there is 95% chance that a future measured value will be ±Y around the true value X.

c) Probability of success to be within specification as a function of the batch true value. This plot help to determine a “safe region” where the true batch values should lie to ensure a high probability to meet the specifications, given the measurement error.

Figure 3: Probability of success to be within specification as a function of the batch true value. Vertical red dotted lines: specifications.

While these metrics are useful to assess the measurement system, it does not incorporate the actual process variability but rather let the evaluator decide if the process variability is small enough to operate in the “safe region”. Is it possible to combine both measurement and process variability to determine if future batches will meet the specifications?

To answer this, we can leverage CPV data to determine the process variability. CPV data usually consists in only one measurement per batch, making it difficult to discriminate the process and measurement variability.

To discriminate both, we used a prior on the measurement error calculated from the Gage R&R fit. This prior has been determined using all random effects of the Gage R&R fit, except the laboratory-to-laboratory one as the CPV data comes only from a single laboratory. The following model has been fitted to the CPV data using a Bayesian framework with the R package stan:

From there, results were predicted for the different laboratories using (1) The batch-to-batch variance and batch intercept posterior distributions from the CPV model and (2) The run-to-run variance, residual variance and laboratories effect posteriors distributions from the Gage R&R model

The figure below shows the posterior predictive distribution of future measured batches in the different laboratories. We see that most laboratories will likely produce measurements within specification, except Lab E which measures higher than other laboratories. We can then compute the probability of success to be within specification. Lab.A=0.98; Lab.B=0.98; Lab.C=0.99; Lab.D=0.98; Lab.E=0.93. This allows to determine which laboratories are able be perform measurements with a good probability of success and to potentially investigate the problematic laboratories.

Figure 4: Posterior predictive distribution of future measurement as a function of the measuring laboratory. Vertical red dotted lines: specifications.

Conclusions

We used a Bayesian framework to combine knowledge of the measurement system properties with knowledge of the process variability. The metrics proposed here facilitate the interpretation of the results, compared to the traditional Gage R&R metrics. This example shows the advantage and flexibility of using a Bayesian framework to determine if the measurement system and the process are capable.

References

  1. Wheeler, Donald J. “An honest gauge R&R study.” Manuscript 189 (2009).
  2. https://www.r-bloggers.com/
Related posts
Why collaboration and an Integrated Product Development program are key to successful drug development
Why collaboration and an Integrated Product Development program are key to successful drug development
14th June 2022
Key Takeaways from the PDA Europe Annex 1 Workshop
Key Takeaways from the PDA Europe Annex 1 Workshop
7th June 2022
NLS DAYS 2022
NLS DAYS 2022
September 28 - May 29th, 2022
Addressing Limitations of Sterility Testing
Addressing Limitations of Sterility Testing
9th May 2022
Are virtual audits sustainable post-pandemic?
Are virtual audits sustainable post-pandemic?
6th May 2022
Orphan Drug Designation: Securing the Significant Benefits
Orphan Drug Designation: Securing the Significant Benefits
28th March 2022
Search
Upcoming Webinars

June 30th, 2022

Transforming the drug development process with an integrated strategy

03:00 PM CET

Categories
  • All News
  • Webinars
  • Events
Archive
Biopharma Excellence Website Image

PharmaLex Brings You Biopharma Excellence

Biopharma Excellence is a fusion of three scientific powerhouses, PharmaLex, ERA Consulting and Biopharma Excellence – all under the PharmaLex brand. This global team of scientific, regulatory and commercial professionals provide strategic product development and proactive regulatory services to developers of biopharmaceuticals, cell and gene therapies, monoclonal antibodies (MABs), vaccines and biosimilars.

Visit Website
USEFUL LINKS
  • Home
  • Contact us
  • Imprint
  • Data Protection
  • Terms and Conditions
RECENT TWEETS
PharmaLexGLOBALPharmaLex@PharmaLexGLOBAL·
9h

June 30, 3:00-3:30 PM CET Last chance to register https://bit.ly/3GwUxdX for our #webinar where you’ll hear the latest findings from a recent PharmaLex #IntegratedProductDevelopment survey.

💡 Webinar registrants receive a complimentary copy of the full survey report.

COVID-19 NEWSLETTER SIGN UP

SIGN UP NOW

WHAT OUR CLIENTS SAY
  • Highly competent, knowledgeable, absolutely reliable and dedicated to the projects

    Europe based medium-sized biotech development company
    Senior Director
PharmaLex
©2022 PharmaLex GmbH. All rights reserved.

ISO9001:2015 LogoEnergie Audit LogoSGS Logo

Suspicious Emails

We are aware of a number of suspicious emails about recruitment in circulation purporting to be from PharmaLex. Emails sent by PharmaLex will originate from @pharmalex.com. Should you receive an email and are unsure as to its validity, please report it to contact@pharmalex.com.

Many thanks for your understanding.

The PharmaLex Team

    Please DO NOT send us event/conference information.
    We will not respond and these will be deleted immediately.






    Select your state:

    .
    If you do not wish to receive any communication from us, you may unsubscribe at any time.

    #AskTheExpert #TogetherBEYONDCOVID19





      Select your state:

      .
      You can unsubscribe at any time at data.protection@pharmalex.com

      If you do not wish to receive any communication from us, you can unsubscribe at any time at. Click here to view our Privacy Policy.

      #TogetherBEYONDCOVID19

      We closely monitor developments and updates surrounding the current outbreak of Coronavirus Disease 2019 (COVID-19) from official sources including the World Health Organisation and are following the guidance and direction of the governments and the local authorities. We are taking this situation very seriously and our number one priority is the safety and business continuity for our customers and colleagues. As ever, we will do everything we can to ensure that services are delivered within the required time and the trusted quality PharmaLex is known for