How to gather clinical evidence for IVD performance evaluation under EU IVDR

Under the In Vitro Diagnostic Regulation (EU) 2017/746 (IVDR), clinical evidence for IVD devices is gathered through the performance evaluation process. Manufacturers must gather comprehensive clinical evidence for their devices comprising of scientific validity, analytical performance, and clinical performance, and identify, appraise, and document this data in a robust and ongoing manner from scientific literature, performance studies or post-market surveillance.

The transition from the In Vitro Diagnostic Directive 98/79/EC (IVDD) to the IVDR has significantly increased the regulatory requirements for IVD devices in Europe. While the IVDD provided a general framework, the IVDR introduces more stringent and detailed expectations, particularly regarding performance evaluation.

Understanding these requirements is crucial, particularly during the ongoing IVDR transition period, which has set deadlines for compliance that are rapidly approaching for many products. Without a solid grasp of performance evaluation obligations and the potential need for additional studies, manufacturers risk delays in placing their devices on the EU market, or even device withdrawal.

As IVDR demands thorough, methodologically sound performance evaluations backed by up-to-date clinical evidence, it’s essential for manufacturers to proactively assess potential gaps and plan accordingly. This is not only a regulatory necessity but also an opportunity to reinforce patient safety and product reliability in a changing landscape.

In this blog post, we explain the IVDR performance evaluation process and the different elements of clinical evidence. Through a clear breakdown of expectations, processes, and planning, this blog aims to help manufacturers understand IVDR requirements, prepare robust clinical evidence, and navigate the transition period to maintain market access and uphold patient safety. In later blog posts, we offer more detailed practical guidance about the performance evaluation documentation and study requirements necessary for compliance.

See also blog posts related to the topic: When and how to conduct IVD performance studies under EU IVDR and How to document IVD performance evaluations under EU IVDR.

What is a performance evaluation as described in the IVDR?

Performance evaluation of a device is a continuous process by which data is assessed and analysed to demonstrate the scientific validity, analytical performance and clinical performance of that device for its intended purpose. The aim of performance evaluation is to demonstrate conformity with the relevant general safety and performance requirements (GSPRs) set out in IVDR Annex I, in particular those concerning the performance characteristics, when the device is used according to the manufacturer’s instructions for use (IFU). Additionally, the purpose is to evaluate the interferences and cross-reactions. Finally, the aim is to conclude an acceptable benefit-risk ratio for using the device.

The performance evaluation process must be a part of the manufacturer’s quality management system as described in IVDR Article 10 (8). The data elements and the aims of clinical performance are described briefly in Figure 1 below.

The performance evaluation must follow a defined and methodologically sound procedure that is interlinked with the risk management process as described in Figure 2 below. As a general methodological principle, manufacturers should:

• identify through a systematic scientific literature review the available data relevant to the device and its intended purpose and identify any remaining unaddressed issues or gaps in the data,
• appraise all relevant data by evaluating its suitability for establishing the safety and performance of the device, and
• generate any new or additional data necessary to address outstanding issues.

How does device risk class affect the performance evaluation process?

The performance evaluation needs to be thorough and objective, consider both favourable and unfavourable data, and include a sufficient level of clinical evidence (constituting of scientific validity, analytical performance, and clinical performance data and conclusions drawn from the assessment of these elements) to demonstrate conformity with the relevant GSPRs. The level of clinical evidence needs to be such as to scientifically demonstrate, by reference to the state of the art in medicine, that the intended clinical benefits will be achieved and that the device is safe. To that end, manufacturers need to plan, conduct and document a performance evaluation in accordance with the IVDR, in particular Article 56 and Part A of Annex XIII.

The depth and extent of the performance evaluation should be proportionate and appropriate to the characteristics of the device including the risks, risk class, performance and its intended purpose. In essence, the higher risk devices need more robust clinical evidence, and the requirement for performance studies increases, as greater scrutiny is placed on demonstrating their safety and effectiveness under the IVDR framework.

What should be included in the performance evaluation plan?

The performance evaluation process begins with the establishment of the performance evaluation plan (PEP). To generate the necessary clinical evidence, the PEP should specify certain characteristics, performance of the device, the process and the criteria to be applied. According to the IVDR, the PEP must include at least:

• a specification of the intended purpose of the device,
• a specification of the performance characteristics of the device and all characteristics encompassing the intended purpose,
• a specification of the analyte or marker to be determined by the device,
• a specification of the intended use of the device,
• identification of certified reference materials or reference measurement procedures to allow for metrological traceability,
• a clear identification of specified target patient groups with clear indications, limitations and contra-indications,
• an identification of the relevant GSPRs that require support from relevant scientific validity and analytical and clinical performance data,
• a specification of methods, including the appropriate statistical tools, used for the examination of the analytical and clinical performance of the device and of the limitations of the device and information provided by it,
• a description of the state of the art, including an identification of existing relevant standards, common specifications, guidance or best practices documents,
• an indication and specification of parameters to be used to determine, based on the state of the art in medicine, the acceptability of the benefit-risk ratio for the intended purpose or purposes and for the analytical and clinical performance of the device,
• for software qualified as a device, an identification and specification of reference databases and other sources of data used as the basis for its decision making,
• an outline of the different development phases including the sequence and means of determination of the scientific validity, the analytical and clinical performance, including an indication of milestones and a description of potential acceptance criteria, and
• the post-market performance follow-up (PMPF) plan.

A justification needs to be included in the PEP if any of the above-mentioned elements are not deemed appropriate to include in the PEP due to the specific device characteristics.

What is scientific validity and how is it demonstrated?

‘Scientific validity of an analyte’ means the association of an analyte with a clinical condition or a physiological state. Thus, demonstrating scientific validity means showing that there is reliable scientific evidence to support that the analyte is truly linked to the medical condition or disease it is intended to detect or monitor.

The following sources can be used to demonstrate scientific validity individually or as a combination:

• relevant information on the scientific validity of devices measuring the same analyte or marker,
• scientific (peer-reviewed) literature,
• consensus expert opinions/positions from relevant professional associations,
• results from proof-of-concept studies, and
• results from clinical performance studies.

In most cases, when the association of the analyte to a clinical condition or physiological state is well established, scientific validity can be demonstrated through the use of existing data, as long as sufficient quality and quantity of data is available, and the generally acknowledged state of the art is taken into account. However, if existing data is not available or it is considered insufficient, or in cases of new analytes and/or new intended purposes, generation of new or additional data will be necessary. In these cases, scientific validity could be strengthened during the clinical performance study for example by comparing to existing well-established analytes.

Manufacturers should employ a systematic approach when conducting literature reviews or other data retrieval methods, including:

1. Defining a search protocol. The searching strategy must be thorough and objective, i.e. it should identify all relevant favourable and unfavourable data.
2. Conducting several searches with consolidated and relevant criteria or focus in order to obtain all relevant and necessary data.
3. Ensuring appropriate documentation is produced such that the methods can be appraised critically, the results can be verified, and searches reproduced if necessary.

The scientific validity report (SVR) is used for documenting the data identification, appraisal, analysis, and conclusion of the scientific validity of the analyte.

What is analytical performance and how is it demonstrated?

‘Analytical performance’ means the ability of a device to correctly detect or measure a particular analyte. Thus, demonstrating analytical performance means showing that there is reliable scientific evidence to support that the device can accurately and reliably measure the analyte in question, under defined conditions, according to its intended purpose.

As a general rule, the analytical performance must always be demonstrated based on analytical performance studies. For novel markers or other markers without available certified reference materials or reference measurement procedures, it may not be possible to demonstrate trueness. If there are no comparative methods, different approaches may be used if demonstrated to be appropriate, such as comparison to some other well-documented methods or the composite reference standard. In the absence of such approaches, a clinical performance study comparing performance of the novel device to the current clinical standard practice is required.

The analytical performance report (APR) is used for documenting the data identification, appraisal, analysis, and conclusion of the analytical performance of the device.

Which elements need to be included in the analytical performance report?

The analytical performance data must include the following aspects, as applicable:

1. Specimen type

• The different specimen types that can be analysed.
• Specific storage, and transport conditions of each specimen type including duration of storage, temperature limits and freeze/thaw cycles.

2. Accuracy of measurement

• Trueness of measurement
  • Information on the trueness of the measurement procedure on a level that allows an assessment of the adequacy of the means selected to establish the trueness.
  • Trueness measures apply to both quantitative and qualitative assays only when a certified reference material or certified reference method is available.
• Precision of measurement
  • Repeatability
  • Reproducibility

3. Analytical sensitivity

• Limit of blank (LoB)
• Limit of detection (LoD)
• Limit of quantitation (LoQ)
• Description of specimen type and preparation including matrix, analyte levels, and how levels were established. The number of replicates tested at each concentration must also be provided as well as a description of the calculation used to determine assay sensitivity.

4. Analytical specificity

• Interference and cross reactivity studies performed to determine the analytical specificity in the presence of other substances/agents in the specimen.
• Evaluation of potentially interfering and cross-reacting substances or agents on the assay, on the tested substance or agent type and its concentration, specimen type, analyte test concentration, and results.
• Interferents and cross-reacting substances or agents, which vary greatly depending on the assay type and design, could derive from exogenous or endogenous sources such as:
  • substances used for patient treatment such as medicinal products,
  • substances ingested by the patient such as alcohol, foods,
  • substances added during specimen preparation such as preservatives, stabilisers,
  • substances encountered in specific specimen types such as haemoglobin, lipids, bilirubin, proteins, or
  • analytes of similar structure such as precursors, metabolites or medical conditions unrelated to the test condition including specimens negative for the assay but positive for a condition that can mimic the test condition.

5. Metrological traceability of calibrator and control material values

6. Measuring range of the assay

• Measuring range regardless of whether the measuring systems are linear or non-linear, including the limit of detection and how the range and detection limit were established.
• A description of specimen type, number of specimens, number of replicates, and specimen preparation including information on the matrix, analyte levels and how levels were established.
• If applicable, a description of any high dose hook effect and the data supporting the mitigation such as dilution steps must be added.

7. Definition of assay cut-off

• A summary of analytical data with a description of the study design including methods for determining the assay cut-off, such as:
  • the population(s) studied: demographics, selection, inclusion and exclusion criteria, number of individuals included,
  • method or mode of characterisation of specimens, and
  • statistical methods such as Receiver Operator Characteristic (ROC) to generate results and if applicable, defining a grey zone/equivocal zone.

What is clinical performance and how is it demonstrated?

‘Clinical performance’ means the ability of a device to yield results that are correlated with a particular clinical condition or a physiological or pathological process or state in accordance with the target population and intended user. In other words, demonstrating clinical performance means providing robust clinical data that confirms the device can accurately and reliably produce results that correspond to the clinical condition or characteristic being tested for, under defined conditions and for its intended purpose.

The clinical performance needs to be demonstrated in relation to all the following parameters as applicable: diagnostic sensitivity, diagnostic specificity, positive predictive value, negative predictive value, likelihood ratio, expected values in normal and affected populations

The following sources can be used to demonstrate clinical performance individually or as a combination:

• clinical performance studies,
• scientific peer-reviewed literature, and
• published experience gained by routine diagnostic testing.

Clinical performance studies need to be performed unless a due justification can be provided for relying on other sources of clinical performance data.

When demonstrating clinical performance, the intended users and use environment need to be taken into account as they might affect the performance, for example if the devices is for self-testing, thus will be used by laypersons, or if the device is for devices for near-patient testing and thus might be used for example in patients homes, emergency units, or ambulances.

The clinical performance report (CPR) is used for documenting the data identification, appraisal, analysis, and conclusion of the clinical performance of the device.

Where can I get more information or help for gathering clinical evidence?

The performance evaluation process is described in detail in IVDR Article 56 and Annexes XIII. Additionally, the Medical Device Coordination Group (MDCG) guidance documents MDCG 2022-2 Guidance on general principles of clinical evidence for In Vitro Diagnostic medical devices (IVDs) gives additional guidance on the performance evaluation and study requirements. Commission Implementing Regulation (EU) 2022/1107 lays down common specifications for certain class D IVDs that are a mandatory requirement for those certain class D devices and can be used as guidance when conducting performance evaluations for other devices. The CLSI guidelines are also important documents to be considered when gathering clinical evidence.

Medfiles offers support for IVD performance evaluations, covering everything from writing performance evaluation plans to planning, monitoring, and documenting clinical performance studies, as well as writing comprehensive performance evaluation reports. Working with Medfiles ensures that your performance evaluation process is well-structured from the outset and seamlessly integrated with post-market surveillance and risk management activities.

Our experts ensure your performance evaluation is tailored to the device’s characteristics, the performance evaluation documents meet IVDR requirements and all necessary clinical evidence has been identified, appraised, and analysed. If you lack the time or internal resources/expertise to conduct a performance evaluation, Medfiles can step in to support your project efficiently, offering expert assistance throughout the process.

Author: Emma Talvitie 
Regulatory Affairs Expert, Medical Devices and IVDs  

Emma Talvitie is a medical device and in vitro diagnostic (IVD) regulatory affairs professional with a Master of Science (Tech.) degree in biotechnology, who has over 4 years of professional experience in MD and IVD regulatory affairs. Emma has a strong background in IVD regulatory affairs and assay development. During her studies she focused on assay development, especially in molecular diagnostics and immunoassays. She started her career by updating the IVDD compliant technical documentation of an IVD analyser to be compliant with the IVDR. She has also been responsible for planning, executing, data analysis and reporting of IVD assay verification and validation studies and contributed to writing and reviewing IVDR and FDA 510(k) documentation for IVD assays.

Since April 2023, as a Regulatory Affairs Expert at Medfiles, Emma has worked in varying projects helping clients with regulatory issues and compiling technical documentation for multiple medical devices and IVDs. Emma is exited about contributing to the development of innovative, high-quality devices that meet both regulatory requirements and user needs.  

See also:

• How to document IVD performance evaluations under EU IVDR
• How to conduct IVD performance studies under EU IVDR
• EU MDR and IVDR notified body processes and transition periods
• EU IVDR: Compliance with technical documentation, notified body processes and transition periods
• What to know about medical device and IVD class changes and transition periods