Eurosurveillance

Euro Surveill. 25(24): 2001057

Multicentre comparison of quantitative PCR-based assays to detect SARS-CoV-2, Germany, March 2020

Maximilian Muenchhoff ¹ ² , Helga Mairhofer ¹ ² , Hans Nitschko ¹ ² , Natascha Grzimek-Koschewa ¹ ² , Dieter Hoffmann ² ³ , Annemarie Berger ² ⁴ , Holger Rabenau ² ⁴ , Marek Widera ² ⁴ , Nikolaus Ackermann ⁵ , Regina Konrad ⁵ , Sabine Zange ² ⁶ , Alexander Graf ⁷ , Stefan Krebs ⁷ , Helmut Blum ⁷ , Andreas Sing ⁵ , Bernhard Liebl ⁵ , Roman Wölfel ² ⁶ , Sandra Ciesek ² ⁴ , Christian Drosten ² ⁸ , Ulrike Protzer ² ³ , Stephan Boehm ¹ ² , Oliver T Keppler ¹ ²

1. Max von Pettenkofer Institute and Gene Center, Virology, National Reference Center for Retroviruses, Ludwig Maximilian University, Munich, Germany

2. German Center for Infection Research, Partner Site Munich and Associated Partner Site Charité, Berlin and Associated Partner Site Frankfurt, Germany

3. Institute of Virology, School of Medicine, Technical University Munich/Helmholtz Zentrum München, Munich, Germany

4. Institute of Medical Virology, University Hospital, Goethe University Frankfurt am Main, Frankfurt, Germany

5. Bavarian Health and Food Safety Authority, Oberschleißheim, Germany

6. Bundeswehr Institute of Microbiology, Munich, Germany

7. Laboratory for Functional Genome Analysis (LAFUGA), Gene Center, Ludwig Maximilian University, Munich, Germany

8. Institute of Virology, Charité University Medicine, Berlin, Germany

Correspondence: Oliver T Keppler (keppler@mvp.lmu.de)

This article is copyright of the authors or their affiliated institutions, 2020.

DOI: 10.2807/1560-7917.ES.2020.25.24.2001057

Published in print: 18 June 2020

Abstract

Containment strategies and clinical management of coronavirus disease (COVID-19) patients during the current pandemic depend on reliable diagnostic PCR assays for the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Here, we compare 11 different RT-PCR test systems used in seven diagnostic laboratories in Germany in March 2020. While most assays performed well, we identified detection problems in a commonly used assay that may have resulted in false-negative test results during the first weeks of the pandemic.

Strategies to limit the severe pandemic and to manage coronavirus disease (COVID-19) patients strongly depend on readily available, accurate and reliable RT-PCR assays to detect the genome of the causative agent acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in biosamples. The first full-length SARS-CoV-2 genome sequence was made publicly available in early January 2020 [1] and, soon after, various RT-PCR assays were reported by academic laboratories, public health agencies and diagnostics companies [2-6]. Their overall performance and relative sensitivity are largely unclear.

The aim of this study was to compare the inter-laboratory and inter-method sensitivity of different RT-PCR assays by providing a blinded, frozen dilution series of a nucleic acid extract of a highly positive biosample to seven different diagnostic laboratories in Germany in March 2020.

Sample preparation and study design

Nucleic acids were pooled from multiple extractions of one SARS-CoV-2-positive stool sample using the QIAsymphony DSP Virus/Pathogen Kit (Qiagen, Hilden, Germany). This stool sample was from a 5-year-old child with COVID-19 [7] and was chosen because of high initial PCR signals and sufficient sample availability to generate large quantities of eluate for further distribution. Of note, no PCR inhibition was observed for detection of the spiked-in extraction RNA control (QuantiNova IC Probe Assays Red 650, Qiagen). A 1:10 dilution series was prepared and aliquots were labelled in a blinded fashion to be shipped on dry ice to participating laboratories in March 2020. Participants were instructed to perform the diagnostic assays used at their centre for SARS-CoV-2 detection in quadruplicate using 5 µL of the aliquot per reaction. All results were reported back to the initiating laboratory (Laboratory 1) before the results were unblinded. The details of all these PCR-based assays are summarised in the Table.

See full table Table. Specifications of different molecular assays used for detection of SARS-CoV-2, Germany, March 2020 (n =11 test systems with 34 different reaction–lab combinations)

In parallel, samples were quantified using the One-Step RT-digital droplet (dd)PCR Advanced Kit for Probes (BioRad, Feldkirchen, Germany) on the BioRad QX200 platform. Primer and probe sequences were used for detection of the SARS-CoV-2 nucleocapsid gene (N) as published by the Centers for Disease Control and Prevention (CDC) [2] and the ﻿envelope gene (E), the ﻿RNA-dependent RNA polymerase (RdRp) gene and the N gene as published by Corman et al. (referred to as Charité protocol) [3] (Figure 1).

Figure 1View larger versionFigure 1. Digital droplet PCR quantification of the distributed dilution series of nucleic acid eluate of SARS-CoV-2-positive clinical material, Germany, March 2020

The undiluted sample showed between 4,325 and 5,015 SARS-CoV-2 RNA copies per reaction using 5 µL of eluate for the CDC N1, N2, N3 and Charité E protocols, but only 850 and 1,951 RNA copies for the Charité N and P primer/probe combinations (Figure 1A), respectively, indicating a lower sensitivity of the latter. The 1:10 dilution series displayed good linearity down to a calculated concentration of 0.4 RNA copies per reaction at the 10⁻⁴ dilution for both the CDC N1 and N2 primer/probe combinations (Figure 1B).

Multicentre and multi-assay comparison

Result interpretations from the seven participating laboratories are summarised in Figure 2 displaying the number of replicates scored positive by the respective laboratory for each method and dilution. Most methods reliably detected the sample at the 10⁻³ dilution, which is equivalent to ca 5 RNA copies for the CDC N1, N2, N3 and Charité E reactions based on the absolute quantification by ddPCR. Of note, the Seegene Allplex 2019-nCoV Assay gave negative results for all four replicates in the E gene at the 10⁻³ dilution, while reporting positive results for N and RdRp (Laboratory 1). According to the manufacturer’s instructions at the time of analysis, this would have been interpreted as an inconclusive result. Of note, the RdRp primer/probe did not show any positive result at the 10⁻⁴ dilution.

Figure 2View larger versionFigure 2. Dilution series comparing various RT-PCR assays for the detection of SARS-CoV-2 at different laboratories, Germany, March 2020 (n =11 test systems with 34 reaction–lab combinations)

Sequence analysis of primer pairs

Driven by false-negative results for samples with low PCR-positivity using the original Charité RdRp reaction (see below and others [8,9]), we compared the primer/probe sequences with currently available SARS-CoV-2 genomes. When compared with all genomes available on GISAID (9,184 SARS-CoV-2 genomes on 15 April 2020, Supplement), the regions used for amplification in the CDC and Charité protocol are highly conserved: Only 1.55%, 0.45% and 2.4% of genome sequences contain any kind of mismatch within the primer/probe regions of the CDC N1, N2 and N3 protocols, respectively, and 0.25%, 0.29% and 0.67% in the primer/probe regions of the Charité E, RdRp and N protocols, respectively.

The Charité RdRp reverse primer contains an ambiguity base at position 15,519 that does not match the reference sequence (Wuhan-Hu-1/2019), with an S (i.e. G or C) instead of T for the reverse complement (Supplementary Figure S1). The other ambiguity base at 15,528 showing Y (i.e. C or T) should be changed to T because the currently circulating viruses have a T at this position and no polymorphisms were detected in any of the 9,184 sequences submitted to date (accession date: 15 April 2020). Based on computation using Primer Express v3.0 (Applied Biosystems, Dreieich, Germany) annealing temperatures were predicted to be 64 °C for the RdRp forward and 51 °C for the RdRp reverse primer of the Charité protocol. This temperature difference may result in reduced PCR efficiency. To address this issue, modified RdRp primers were synthesised as shown in Supplementary Figure S1 and tested in comparison with the original primers.

Differential detection of respiratory samples with low PCR positivity

Testing the dilution series with these modified RdRp primers (see above and Supplementary Figure S1) yielded positive results for two additional dilution steps (10⁻³ and 10⁻⁴) compared with the original Charité RdRp primers (Figure 1). To further compare the sensitivity of these modified RdRp primers with the original version of the Charité RdRp primers and the Charité E and the CDC N1 reaction, we retested 28 eluates of clinical respiratory specimens from the diagnostic unit at Laboratory 1 that had shown crossing point (Cp) values > 35 in the initial CDC N1 reaction. Using the original version of the confirmatory Charité RdRp primers, 16 of 28 samples tested negative, but 11 of these showed positive results using the modified primers (Figure 3). Overall, the detection by the Charité E, modified Charité RdRp, and CDC N1 reactions were robust. Notably, six and seven of these 28 respiratory samples scored negative or at the limit of detection (Cp = 40) in the Charité E and modified Charité RdRp reactions, while only one sample came up negative in retesting in the CDC N1 reaction (p = 0.04 and p = 0.02, chi-squared-test comparing Charité E and modified RdRp to CDC N1, respectively). Of note, in a routine clinical setting, the CDC N1 reaction also detected SARS-CoV-2 RNA in nucleic acid extracts from 37 of 83 sera (45%) from COVID-19 patients in intensive care units, with a positive correlation of their Cp values with those of the corresponding respiratory material (Spearman Rank correlation co-efficient r=0.4285, p (two-tailed) < 0.0001 (data not shown)).

Figure 3View larger versionFigure 3. RT-PCR results of respiratory samples with low positivity, SARS-CoV-2 detection, Germany, March 2020 (n = 28 samples)

Conclusion

The majority of RT-PCR assays for SARS-CoV-2 examined in this study detected ca 5 RNA copies per reaction, reflecting a high sensitivity and their suitability for screening purposes world-wide. A reduced sensitivity was noted for the original Charité RdRp gene confirmatory protocol, which may have impacted the confirmation of some COVID-19 cases in the early weeks of the pandemic. The protocol needs to be amended to improve the sensitivity of the RdRp reaction. The CDC N1 primer/probe set was sensitive and robust for detection of SARS-CoV-2 in nucleic acid extracts from respiratory material, stool and serum from COVID-19 patients.

Acknowledgements

We acknowledge the authors, originating and submitting laboratories of the sequences from GISAID’s EpiCov database on which this research is based. All submitters of data may be contacted directly via the GISAID website (www.gisaid.org). This work was supported by LMUexcellent funding of LMU München. We thank Mikrogen for participating in this investigator-initiated study.

Notes

Conflict of interest: None declared.

Authors’ contributions: MM, HM, HN, NG-K, SB and OTK designed the study. MM, HM, DH, NA, RK, MW, SZ performed experiments. RW, DH, AB, HR, AS, BL, SC, CD, UP analysed data. HB, AG and SK assembled SARS-CoV-2 genome sequences and extracted variants for the analysis of the primer binding regions. MM and OTK wrote the manuscript.

All authors discussed the results and commented on the final manuscript.

Supplementary Data

Supplementary Data

Articles from Eurosurveillance are provided here courtesy of European Centre for Disease Prevention and Control

PMC Copyright Notice

The articles available from the PMC site are protected by copyright, even though access is free. Copyright is held by the respective authors or publishers who provide these articles to PMC. Users of PMC are responsible for complying with the terms and conditions defined by the copyright holder.

Users should assume that standard copyright protection applies to articles in PMC, unless an article contains an explicit license statement that gives a user additional reuse or redistribution rights. PMC does not allow automated/bulk downloading of articles that have standard copyright protection.

See the copyright notice on the PMC site, https://www.ncbi.nlm.nih.gov/pmc/about/copyright/, for further details and specific exceptions.

References

F WuS ZhaoB YuYM ChenW WangZG Song A new coronavirus associated with human respiratory disease in China. Nature. 2020;579(7798):265-9. 10.1038/s41586-020-2008-3 32015508

Centers for Disease Control and Prevention (CDC). Information for laboratories about coronavirus (COVID-19). Atlanta: CDC. [Accessed: 16 April 2020]. Available from: https://www.cdc.gov/coronavirus/2019-ncov/lab/index.html

VM CormanO LandtM KaiserR MolenkampA MeijerDK Chu Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR. Euro Surveill. 2020;25(3):2000045. 10.2807/1560-7917.ES.2020.25.3.2000045 31992387

World Health Organization (WHO). Coronavirus disease (COVID-19) technical guidance: Laboratory testing for 2019-nCoV in humans. Geneva: WHO. [Accessed: 16 April 2020]. Available from: https://www.who.int/emergencies/diseases/novel-coronavirus-2019/technical-guidance/laboratory-guidance

PB van KasterenB van der VeerS van den BrinkL WijsmanJ de JongeA van den Brandt Comparison of seven commercial RT-PCR diagnostic kits for COVID-19. J Clin Virol. 2020;128:104412. 10.1016/j.jcv.2020.104412 32416600

R KonradU EberleA DangelB TreisA BergerK Bengs Rapid establishment of laboratory diagnostics for the novel coronavirus SARS-CoV-2 in Bavaria, Germany, February 2020. Euro Surveill. 2020;25(9):2000173. 10.2807/1560-7917.ES.2020.25.9.2000173 32156330

GK WolfT GlueckJ HuebnerM MuenchhoffD HoffmannLE French Clinical and epidemiological features of a family cluster of symptomatic and asymptomatic SARS-CoV-2 infection. J Pediatric Infect Dis Soc. 2020;piaa060. 10.1093/jpids/piaa060 32441753

Vogels CBF, Brito AF, Wyllie AL, Fauver JR, Ott IM, Kalinich CC, et al. Analytical sensitivity and efficiency comparisons of SARS-COV-2 qRT-PCR primer-probe sets. medRxiv. 2020:2020.03.30.20048108.

AK NallaAM CastoMW HuangGA PerchettiR SampoleoL Shrestha Comparative performance of SARS-CoV-2 detection assays using seven different primer-probe sets and one assay kit. J Clin Microbiol. 2020;58(6):e00557-20. 10.1128/JCM.00557-20 32269100

[Back]

Table. Specifications of different molecular assays used for detection of SARS-CoV-2, Germany, March 2020 (n =11 test systems with 34 different reaction–lab combinations)

Laboratory | Protocol | Target | Primer/probe | Supermix | Instrument
Laboratory 1 | CDC [2] | N1, N2, N3 | Ella Biotech | QuantiNova Multiplex RT-PCR Kit | Roche LightCycler 480 II
Charité [3,4] | E, N, RdRp | Tib-Molbiol | QuantiNova Multiplex RT-PCR Kit | Roche LightCycler 480 II
Modified Charité RdRp primers | RdRp | Ella Biotech | QuantiNova Multiplex RT-PCR Kit | Roche LightCycler 480 II
Applied Biosystems TaqMan 2019-nCoV Assay Kit v1 | S, N, RdRp | Commercial kit | TaqMan Fast Virus 1-Step Master Mix | Applied Biosystems 7500 fast
Seegene Allplex 2019-nCoV Assay | E, N, RdRp | Commercial kit | Commercial kit | Biorad CFX 96 Real-Time System
Digital droplet PCR using CDC primer and probe sequences | N1, N2, N3 | Ella primers/IDT ZEN Double-Quenched Probe | BioRad 1-Step RT-ddPCR Advanced Kit for Probes | Biorad QX200 droplet digital PCR
Digital droplet PCR using Charité primer and probe sequences | E, N, RdRp | Ella primers/IDT ZEN Double-Quenched Probe | BioRad 1-Step RT-ddPCR Advanced Kit for Probes | Biorad QX200 droplet digital PCR
Laboratory 2 | Charité [3,4] | E, RdRp | Tib-Molbiol | Superscript III One-Step RT-PCR System With Platinum Taq Polymerase | Roche LightCcycler 480 II
Laboratory 3 | Charité [3,4] | E, RdRp (2 or 1 and 2) | Tib-Molbiol | Superscript III One-Step RT-PCR System With Platinum Taq Polymerase | Biorad CFX 96 Real-Time System
Altona diagnostics RealSstar SARS-CoV-2 RT-PCR | Beta-CoV, SARS-CoV-2 | Commercial kit | Commercial kit | Biorad CFX 96 Real-Time System
Laboratory 4 | Charité [3,4] | E, RdRp | Tib-Molbiol | RNA to CT 1-step | Applied Biosystems 7500 fast
Laboratory developed test | M, S | Tib-Molbiol | Roche Multiplex RNA Virusmaster | Roche LightCycler 480 II
Laboratory 5 | Charité [3,4] | E, N, RdRp | Tib-Molbiol | Quantitect Virus +ROX Vial Kit | Applied Biosystems 7500 fast
CDC [2] | N1, N2, N3 | Microsynth | Quantitect Virus +ROX Vial Kit | Applied Biosystems 7500 fast
Laboratory 6 | Charité [3,4] | E, N, RdRp | Tib-Molbiol | Qiagen one step RT-PCR Kit | Bio Molecular Systems MIC Cycler
Laboratory 7 | Mikrogen ampliCube Coronavirus Panel | Various coronaviruses | Commercial kit | Commercial kit | Roche LightCycler 480 II
Mikrogen ampliCube Coronavirus SARS-CoV-2 | E, Orf1a | Commercial kit | Commercial kit | Roche LightCycler 480 II

Beta-CoV: Betacoronavirus; CDC: Centers for Disease Control and Prevention; E: envelope gene; N: nucleocapsid gene; Orf: open reading frame; RdRp: RNA-dependent RNA-polymerase gene; SARS-CoV-2: severe acute respiratory syndrome coronavirus 2; WHO: World Health Organization.

Performing laboratory, assay protocol, target, manufacturer of primer/probe, PCR chemistry and instrument are indicated.

[Back]

[Back]

Figure 1.

Digital droplet PCR quantification of the distributed dilution series of nucleic acid eluate of SARS-CoV-2-positive clinical material, Germany, March 2020

CDC: Centers for Disease Control and Prevention; E: envelope gene; N: nucleocapsid gene; RdRp: RNA-dependent RNA-polymerase gene; SARS-CoV-2: severe acute respiratory syndrome coronavirus 2.

The RNA copy numbers are indicated for each PCR reaction using either the CDC and the Charité primer/probe combinations and were measured from 5 µL nucleic acid eluate. Mean values of quadruplicates are indicated by the horizontal lines (A) or symbols (B). Error bars represent the 95% Poisson confidence interval (B).

[Back]

[Back]

Figure 2.

Dilution series comparing various RT-PCR assays for the detection of SARS-CoV-2 at different laboratories, Germany, March 2020 (n =11 test systems with 34 reaction–lab combinations)

Beta-CoV: Betacoronavirus; CDC: Centers for Disease Control and Prevention; E: envelope gene; N: nucleocapsid gene; ORF: open reading frame; RdRp: RNA-dependent RNA-polymerase gene; S: spike gene; SARS-CoV-2: severe acute respiratory syndrome coronavirus 2.

a Copy numbers are estimated based on ddPCR results shown in Figure 1.

A 1:10 dilution series was prepared from pooled eluates of one SARS-CoV-2 stool sample and sent to participating laboratories to be tested in quadruplicate. The number of positive replicates is indicated and colour-coded as a heatmap with four positive results shown in green and four negative results shown in red.

[Back]

[Back]

Figure 3.

RT-PCR results of respiratory samples with low positivity, SARS-CoV-2 detection, Germany, March 2020 (n = 28 samples)

CDC: Centers for Disease Control and Prevention; Cp: crossing point; E: envelope gene; N: nucleocapsid gene; RdRp: RNA-dependent RNA-polymerase gene; SARS-CoV-2: severe acute respiratory syndrome coronavirus 2.

Nucleic acid eluates of clinical respiratory specimens that initially showed low positive results in the CDC N1 reaction (Cp value > 35) were retested side by side in the CDC N1, the Charité E and original RdRp reaction and using the modified RdRp primers on the Roche LightCycler 480 using the QuantiNova Multiplex RT-PCR kit. Cp values are shown with positive amplifications beyond cycle 40 shown as > 40 (dotted line).

[Back]