Written by Bill Finger, Managing Director, Kineticos
With the advent of Next Generation Sequencing (NGS), researchers can now analyze an entire genome in less than 24 hours. NGS applications help to identify rare genetic mutations, analyze gene expressions, and discover disease specific genetic mutations. Regulatory agencies categorize NGS assays as Lab Developed Tests (LDTs), and consequently there is no regulatory pathway to develop and validate these tests.
In July, the FDA issued a guidance document to provide recommendations for the development and validation of Next Generation Sequencing based tests – Use of Standards in FDA Regulatory Oversight of Next Generation Sequencing (NGS) – Based In Vitro Diagnostics (IVDs) Used for Diagnosing Germline Disease.
With a goal to develop an approach to ensure the safety and effectiveness of NGS based testing, the FDA held two workshops with key opinion leaders. Below is an overview of the guidance.
Test development starts with the test design. The design of a new test includes many variables such as indications for use, user requirements, specimen type, interrogated regions of the genome, as well as the components and methods required. Given that NGS is a new and highly complex technology, developing standards to comply with and a design framework provides a gateway to enter the next stage – Test Performance and Characteristics.
Test Performance and Characteristics includes development of a validation plan that defines the analytical performance characteristics used to assess the assay. The guidance document recommends a set of performance metrics that the FDA recommends during validation. Below is a snapshot of the performance metrics to be analyzed:
- Accuracy – closeness of agreement between a measured value and a true value
- Percent positive agreement
- Negative percent agreement
- Technical positive predictive value
- No call rate
- Reproducibility – variability under different conditions
- Repeatability – variability under similar conditions
- Limit of Detection – minimum and maximum amount of DNA that enables the test to provide expected results
- Analytical Specificity
- Read depth
- Test Run Metrics and Performance Thresholds
- Specimen quality
- DNA quality
- Sequence generation / base calling
- Mapping or assembly metrics
- Variant calling metrics
- General Recommendations for Performance Evaluation Studies
- Supplemental Procedures
- Variant Annotation and Filtering
- Presentation of Test Performance
- Test Reports
Defining the key needs of a test ensures the consistency of development and analytical performance across all labs that perform NGS based tests; therefore, clinicians and researchers can have more confidence in the data they are receiving to make patient treatment decisions. This draft guidance provides clarity to laboratories and diagnostic companies on how to evaluate the performance characteristics of the tests they are developing.
Given that NGS based testing is a key analytical technique for advancing personalized medicine, I believe that this will increase the comfort level of pharmaceutical companies including this type of testing in clinical trials.
If you would like to receive emails containing insights on life sciences topics relevant to you, please subscribe
Bill Finger, Managing Director of Kineticos’ Diagnostics Practice, brings 20 years of diagnostics and laboratory experience to the team. His team is focused on helping diagnostic companies maximize their commercial potential at the corporate and product levels while ensuring companies operate in an efficient manner.