View Article PDF

The gold standard for the detection of SARS-CoV-2 is real-time reverse transcription polymerase chain reaction (RT-PCR). However, this method is subject to the quality of collection and changes in viral dynamics over time.1

Serological assays can diagnose SARS-CoV-2 infections where exposure was several weeks earlier and missed by RT-PCR alone.2 A serological response is generally reliably detected two weeks after symptom onset.3,4 Serology could be utilised in contact tracing and testing high-risk groups, eg, border workers, but must not be used to infer immunity against reinfection as lasting protective immunity is yet to be clearly demonstrated.2,5,6

Post market studies of several commercial assays show variable sensitivity between reports, suggesting heterogeneity in studied populations.4 In low prevalence settings, like New Zealand (1,607 RT-PCR confirmed cases, attack rate 0.03%), ensuring high specificity is more consequential than high sensitivity.5,7,8

Therefore, we present a comparison of nine SARS-CoV-2 serological assays evaluated in the Auckland region, to determine assay performance in the relevant epidemiological context.

Assays evaluated were the Abbott Architect (Abbott Diagnostics, US), Roche Elecsys (Roche Diagnostics, Switzerland), Euroimmun (Euroimmun Medizinische Labordiagnostika AG, Germany), EDI (Epitope Diagnostics, Inc, US), in-house (University of Auckland) two-step enzyme-linked immunoassay (ELISA) that includes a screen on the receptor binding domain followed by serial dilution on the Spike protein, and a surrogate viral neutralisation assay (cPASS GenScript, Singapore). The methodology of the latter two is described in detail in McGregor et al.9 The RT-PCR and serological assays were validated at each participating laboratory according to standard practice and following manufacturer’s instructions. The four laboratories were LabPLUS (Auckland City Hospital), the Faculty of Medical and Health Sciences (University of Auckland), Labtests Auckland and Middlemore Hospital laboratory.

Three categories of samples were used: pre-pandemic, pandemic-RT-PCR-negative and pandemic-RT-PCR-positive, described in McGregor et al.9 Pre-pandemic sera tested on the Abbott, in-house ELISA, cPASS and Euroimmun S1 protein assays were from healthy volunteers and patients with respiratory infections and/or symptoms (n=113). Pre-pandemic sera tested on the Euroimmun N protein, EDI IgG and Roche assays were archived blood bank sera (n=40).

Pandemic samples from patients at Auckland City Hospital and Middlemore Hospital, March to May 2020, were tested on all platforms (n=103), with the exception of the Euroimmun N protein and EDI assays on which only Auckland City Hospital samples were tested (n=90). Due to volume constraints, all pandemic samples were not tested on the Abbott assay.

Pre-pandemic sera are by definition true negatives. Pandemic sera were categorised as positive if the corresponding RT-PCR result was positive, or if RT-PCR negative and more than two serological assays resulted positive for that sample. This applied to one sample from a returning traveller.

Equivocal serology results were categorised as positive. The assays with equivocal ranges are Euroimmun (0.8 to <1.1) and EDI (10% of the cut-off).

Table 1: A summary of the methodology, the number of sera that tested positive and negative in each category, and the calculated sensitivity and specificity of each serological assay for SARS-CoV-2 evaluated in this study.

Pandemic RT-PCR positive samples include 21 RT-PCR positive samples and one RT-PCR-negative sample included in this group because >2 serological assays returned positive results—see methods for details; N is the nucleocapsid protein of SARS-CoV-2; S is the Spike protein of SARS-CoV-2; S1 is the S1 subunit of the Spike protein; RBD is the receptor binding domain of the Spike protein; IgG is immunoglobulin G; IgA is immunoglobulin A; IgM is immunoglobulin M; CI is confidence interval; n is the total number of samples within that category.

This evaluation demonstrates that the IgG, total antibody (IgG plus IgM) and surrogate viral neutralisation assays are broadly comparable with sensitivities between 77.8–100% and specificities of 94.2–100%. In contrast, the IgA and IgM assays demonstrated poor specificity and sensitivity, respectively.

Three RT-PCR-positive samples tested on the Abbott, Roche, in-house ELISA, cPASS and Euroimmun S1 protein assays were collected within seven days of symptom onset. With these samples excluded, the sensitivity of those assays increased (86.7–100%). This reflects the time to seroconversion (seven days would include IgM seroconversion), and concurs with prior reports.3,10

With samples within seven days of symptom onset excluded, the in-house ELISA and cPASS assays demonstrate 100% sensitivity and specificity, meeting the UK acceptability criteria (>98% sensitivity and specificity) for serological assays.11 A detailed comparison of these two assays was recently published.9 Despite excellent technical performance, utility in diagnostic laboratories may be limited by workflow and throughput considerations.

The Abbott and Roche assays (targeting N protein antigens) demonstrated moderate sensitivity (86.7 and 89.5% respectively) with samples within seven days of symptom onset excluded, and high specificity (100% and 99.2% respectively). They are deployable on high-throughput analysers; however, this utility must be balanced against their imperfect sensitivity.

The immunokinetics of the response to N and S proteins are reported to differ, with more persistent responses seen with the latter,12 which may contribute to the differences in assay sensitivity observed.

New Zealand has a low prevalence of SARS-CoV-2: an effective serological assay for widespread use would need to have high specificity, to minimise the risk of false positive results.5 Most assays in this study were highly specific, though combining assays with different targets in a two-stage protocol (all positives re-tested using an assay targeting a different antigen) would further increase specificity. We recommend this approach for New Zealand currently.

The compromise for highly specific assays is lower sensitivity, as found for most assays in a recent Cochrane review.3 Therefore, in select circumstances (eg, clear high risk of exposure >14 days previously), it may be prudent to test samples that are initially seronegative on a second platform, in case the shortfall in sensitivity results in a falsely negative result.

This study was limited by the small sample size, particularly with only 22 positive samples included, hence the wide confidence intervals on the sensitivity and specificity calculations. Nonetheless, given current difficulties in sourcing positive and negative samples, this evaluation provides valuable initial data for other New Zealand laboratories when considering which assays to validate for their own use and a basis for future studies. Unfortunately we were not able to test exactly the same panel of sera on all assays; this represented the operational constraints laboratories were under during the first stages of the pandemic.

In summary, this study presents a local evaluation of nine serological assays, with results specific to the epidemiological context of New Zealand. The IgG, the total antibody and surrogate viral neutralisation assays demonstrated high specificity and reasonable sensitivity. The application of these assays needs to incorporate local laboratory workflow and logistics, with respect to anticipated testing volume. These data support deployment of serological assays in New Zealand for testing high-risk groups to assist contact tracing and public health investigations.

Summary

Abstract

Aim

Method

Results

Conclusion

Author Information

Shivani Fox-Lewis, Clinical Virology Registrar, Department of Virology-Immunology, LabPLUS, Auckland City Hospital, Auckland; Alana Whitcombe, PhD Student, Faculty of Medical and Health Sciences, University of Auckland, Auckland; Maurice Wilkins Centre, University of Auckland, Auckland; Reuben McGregor, Post-Doctoral Research Fellow, Faculty of Medical and Health Sciences, University of Auckland, Auckland; Maurice Wilkins Centre, University of Auckland, Auckland; Lauren Carlton, Research Technician, Faculty of Medical and Health Sciences, University of Auckland, Auckland; Yulia Hwang, Registered Medical Laboratory Scientist, Department of Virology-Immunology, LabPLUS, Auckland City Hospital, Auckland; Paul Austin, Section Leader For Serology, Department of Virology-Immunology, LabPLUS, Auckland City Hospital, Auckland; Lisa Aspin, Head of the Department of Biochemistry and Immunology, Department of Biochemistry and Immunology, Labtests, Auckland; Bryan Raill, Section Head for Toxicology and Immunoassay, Department of Biochemistry, Middlemore Hospital, Auckland; Rachel Webb, Consultant Paediatrician, Paediatric Infectious Diseases, Faculty of Medical and Health Sciences, University of Auckland, Auckland; Starship Children’s Hospital, Auckland; KidzFirst Children’s Hospital, Middlemore Hospital, Auckland; Susan Taylor, Clinical Microbiologist, Department of Microbiology, Middlemore Hospital, Auckland; Susan Morpeth, Clinical Microbiologist, Department of Microbiology, Middlemore Hospital, Auckland; Sally Roberts, Clinical Microbiologist, Head of Department of Microbiology Maurice Wilkins Centre, University of Auckland, Auckland; Department of Microbiology, LabPLUS, Auckland City Hospital, Auckland; Nicole J Moreland, Senior Lecturer, Faculty of Medical and Health Sciences, University of Auckland, Auckland; Maurice Wilkins Centre, University of Auckland, Auckland; Gary McAuliffe, Clinical Microbiologist, Head of Department of Virology, Department of Virology-Immunology, LabPLUS, Auckland City Hospital, Auckland; Department of Microbiology, Labtests, Auckland.

Acknowledgements

This work was funded in part by the School of Medicine Foundation (University of Auckland) and the COVID-19 Innovation Acceleration Fund (Ministry of Business, Innovation and Employment).

Correspondence

Shivani Fox-Lewis, Virology department, LabPlus, PO Box 110031, Auckland City Hospital, Auckland 1148.

Correspondence Email

sfoxlewis@adhb.govt.nz

Competing Interests

Nil.

1. Miller TE, Garcia Beltran WF, Bard AZ, et al. Clinical sensitivity and interpretation of PCR and serological COVID-19 diagnostics for patients presenting to the hospital. FASEB J 2020 doi: 10.1096/fj.202001700RR

2. Winter AK, Hegde ST. The important role of serology for COVID-19 control. Lancet Infect Dis 2020; 20(7):758–759.

3. Deeks JJ, Dinnes J, Takwoingi Y, et al. Antibody tests for identification of current and past infection with SARS-CoV-2. Cochrane Database Syst Rev 2020; 6(6):CD013652

4. Peeling RW, Wedderburn CJ, Garcia PJ, et al. Serology testing in the COVID-19 pandemic response. Lancet Infect Dis 2020; 20(9):e245–e249.

5. Krammer F, Simon V. Serology assays to manage COVID-19. Science 2020; 368(6495):1060–1061.

6. Clapham H, Hay J, Routledge I, et al. Seroepidemiologic Study Designs for Determining SARS-COV-2 Transmission and Immunity. Emerg Infect Dis. 2020; 26(9):1978–1986.

7. Ministry of Health. COVID-19: Current cases. http://www.health.govt.nz/our-work/diseases-and-conditions/covid-19-novel-coronavirus/covid-19-current-situation/covid-19-current-cases [accessed on 03/11/2020].

8. Stats NZ. Estimated population of NZ. http://www.stats.govt.nz/indicators/population-of-nz [accessed on 03/11/2020].

9. McGregor R, Whitcombe AL, Sheen CR, et al. Collaborative networks enable the rapid establishment of serological assays for SARS-CoV-2 during nationwide lockdown in New Zealand. PeerJ 2020; 8:e9863.

10. Public Health England. Evaluation of sensitivity and specificity of four commercially available SARS-CoV-2 antibody immunoassays. Public Health England 2020.

11. Medicines & Healthcare products Regulatory Agency. Guidance Target Product Profile: antibody tests to help determine if people have recent infection to SARS-CoV-2: Version 2 http://www.gov.uk/government/publications/how-tests-and-testing-kits-for-coronavirus-covid-19-work/target-product-profile-antibody-tests-to-help-determine-if-people-have-recent-infection-to-sars-cov-2-version-2 [accessed on 10/09/2020].

12. Ripperger TJ, Uhrlaub JL, Watanabe M, et al. Detection, prevalence, and duration of humoral responses to SARS-CoV-2 under conditions of limited population exposure. Preprint at http://www.medrxiv.org/content/10.1101/2020.08.14.20174490v1.full.pdf

Contact diana@nzma.org.nz
for the PDF of this article

View Article PDF

The gold standard for the detection of SARS-CoV-2 is real-time reverse transcription polymerase chain reaction (RT-PCR). However, this method is subject to the quality of collection and changes in viral dynamics over time.1

Serological assays can diagnose SARS-CoV-2 infections where exposure was several weeks earlier and missed by RT-PCR alone.2 A serological response is generally reliably detected two weeks after symptom onset.3,4 Serology could be utilised in contact tracing and testing high-risk groups, eg, border workers, but must not be used to infer immunity against reinfection as lasting protective immunity is yet to be clearly demonstrated.2,5,6

Post market studies of several commercial assays show variable sensitivity between reports, suggesting heterogeneity in studied populations.4 In low prevalence settings, like New Zealand (1,607 RT-PCR confirmed cases, attack rate 0.03%), ensuring high specificity is more consequential than high sensitivity.5,7,8

Therefore, we present a comparison of nine SARS-CoV-2 serological assays evaluated in the Auckland region, to determine assay performance in the relevant epidemiological context.

Assays evaluated were the Abbott Architect (Abbott Diagnostics, US), Roche Elecsys (Roche Diagnostics, Switzerland), Euroimmun (Euroimmun Medizinische Labordiagnostika AG, Germany), EDI (Epitope Diagnostics, Inc, US), in-house (University of Auckland) two-step enzyme-linked immunoassay (ELISA) that includes a screen on the receptor binding domain followed by serial dilution on the Spike protein, and a surrogate viral neutralisation assay (cPASS GenScript, Singapore). The methodology of the latter two is described in detail in McGregor et al.9 The RT-PCR and serological assays were validated at each participating laboratory according to standard practice and following manufacturer’s instructions. The four laboratories were LabPLUS (Auckland City Hospital), the Faculty of Medical and Health Sciences (University of Auckland), Labtests Auckland and Middlemore Hospital laboratory.

Three categories of samples were used: pre-pandemic, pandemic-RT-PCR-negative and pandemic-RT-PCR-positive, described in McGregor et al.9 Pre-pandemic sera tested on the Abbott, in-house ELISA, cPASS and Euroimmun S1 protein assays were from healthy volunteers and patients with respiratory infections and/or symptoms (n=113). Pre-pandemic sera tested on the Euroimmun N protein, EDI IgG and Roche assays were archived blood bank sera (n=40).

Pandemic samples from patients at Auckland City Hospital and Middlemore Hospital, March to May 2020, were tested on all platforms (n=103), with the exception of the Euroimmun N protein and EDI assays on which only Auckland City Hospital samples were tested (n=90). Due to volume constraints, all pandemic samples were not tested on the Abbott assay.

Pre-pandemic sera are by definition true negatives. Pandemic sera were categorised as positive if the corresponding RT-PCR result was positive, or if RT-PCR negative and more than two serological assays resulted positive for that sample. This applied to one sample from a returning traveller.

Equivocal serology results were categorised as positive. The assays with equivocal ranges are Euroimmun (0.8 to <1.1) and EDI (10% of the cut-off).

Table 1: A summary of the methodology, the number of sera that tested positive and negative in each category, and the calculated sensitivity and specificity of each serological assay for SARS-CoV-2 evaluated in this study.

Pandemic RT-PCR positive samples include 21 RT-PCR positive samples and one RT-PCR-negative sample included in this group because >2 serological assays returned positive results—see methods for details; N is the nucleocapsid protein of SARS-CoV-2; S is the Spike protein of SARS-CoV-2; S1 is the S1 subunit of the Spike protein; RBD is the receptor binding domain of the Spike protein; IgG is immunoglobulin G; IgA is immunoglobulin A; IgM is immunoglobulin M; CI is confidence interval; n is the total number of samples within that category.

This evaluation demonstrates that the IgG, total antibody (IgG plus IgM) and surrogate viral neutralisation assays are broadly comparable with sensitivities between 77.8–100% and specificities of 94.2–100%. In contrast, the IgA and IgM assays demonstrated poor specificity and sensitivity, respectively.

Three RT-PCR-positive samples tested on the Abbott, Roche, in-house ELISA, cPASS and Euroimmun S1 protein assays were collected within seven days of symptom onset. With these samples excluded, the sensitivity of those assays increased (86.7–100%). This reflects the time to seroconversion (seven days would include IgM seroconversion), and concurs with prior reports.3,10

With samples within seven days of symptom onset excluded, the in-house ELISA and cPASS assays demonstrate 100% sensitivity and specificity, meeting the UK acceptability criteria (>98% sensitivity and specificity) for serological assays.11 A detailed comparison of these two assays was recently published.9 Despite excellent technical performance, utility in diagnostic laboratories may be limited by workflow and throughput considerations.

The Abbott and Roche assays (targeting N protein antigens) demonstrated moderate sensitivity (86.7 and 89.5% respectively) with samples within seven days of symptom onset excluded, and high specificity (100% and 99.2% respectively). They are deployable on high-throughput analysers; however, this utility must be balanced against their imperfect sensitivity.

The immunokinetics of the response to N and S proteins are reported to differ, with more persistent responses seen with the latter,12 which may contribute to the differences in assay sensitivity observed.

New Zealand has a low prevalence of SARS-CoV-2: an effective serological assay for widespread use would need to have high specificity, to minimise the risk of false positive results.5 Most assays in this study were highly specific, though combining assays with different targets in a two-stage protocol (all positives re-tested using an assay targeting a different antigen) would further increase specificity. We recommend this approach for New Zealand currently.

The compromise for highly specific assays is lower sensitivity, as found for most assays in a recent Cochrane review.3 Therefore, in select circumstances (eg, clear high risk of exposure >14 days previously), it may be prudent to test samples that are initially seronegative on a second platform, in case the shortfall in sensitivity results in a falsely negative result.

This study was limited by the small sample size, particularly with only 22 positive samples included, hence the wide confidence intervals on the sensitivity and specificity calculations. Nonetheless, given current difficulties in sourcing positive and negative samples, this evaluation provides valuable initial data for other New Zealand laboratories when considering which assays to validate for their own use and a basis for future studies. Unfortunately we were not able to test exactly the same panel of sera on all assays; this represented the operational constraints laboratories were under during the first stages of the pandemic.

In summary, this study presents a local evaluation of nine serological assays, with results specific to the epidemiological context of New Zealand. The IgG, the total antibody and surrogate viral neutralisation assays demonstrated high specificity and reasonable sensitivity. The application of these assays needs to incorporate local laboratory workflow and logistics, with respect to anticipated testing volume. These data support deployment of serological assays in New Zealand for testing high-risk groups to assist contact tracing and public health investigations.

Summary

Abstract

Aim

Method

Results

Conclusion

Author Information

Shivani Fox-Lewis, Clinical Virology Registrar, Department of Virology-Immunology, LabPLUS, Auckland City Hospital, Auckland; Alana Whitcombe, PhD Student, Faculty of Medical and Health Sciences, University of Auckland, Auckland; Maurice Wilkins Centre, University of Auckland, Auckland; Reuben McGregor, Post-Doctoral Research Fellow, Faculty of Medical and Health Sciences, University of Auckland, Auckland; Maurice Wilkins Centre, University of Auckland, Auckland; Lauren Carlton, Research Technician, Faculty of Medical and Health Sciences, University of Auckland, Auckland; Yulia Hwang, Registered Medical Laboratory Scientist, Department of Virology-Immunology, LabPLUS, Auckland City Hospital, Auckland; Paul Austin, Section Leader For Serology, Department of Virology-Immunology, LabPLUS, Auckland City Hospital, Auckland; Lisa Aspin, Head of the Department of Biochemistry and Immunology, Department of Biochemistry and Immunology, Labtests, Auckland; Bryan Raill, Section Head for Toxicology and Immunoassay, Department of Biochemistry, Middlemore Hospital, Auckland; Rachel Webb, Consultant Paediatrician, Paediatric Infectious Diseases, Faculty of Medical and Health Sciences, University of Auckland, Auckland; Starship Children’s Hospital, Auckland; KidzFirst Children’s Hospital, Middlemore Hospital, Auckland; Susan Taylor, Clinical Microbiologist, Department of Microbiology, Middlemore Hospital, Auckland; Susan Morpeth, Clinical Microbiologist, Department of Microbiology, Middlemore Hospital, Auckland; Sally Roberts, Clinical Microbiologist, Head of Department of Microbiology Maurice Wilkins Centre, University of Auckland, Auckland; Department of Microbiology, LabPLUS, Auckland City Hospital, Auckland; Nicole J Moreland, Senior Lecturer, Faculty of Medical and Health Sciences, University of Auckland, Auckland; Maurice Wilkins Centre, University of Auckland, Auckland; Gary McAuliffe, Clinical Microbiologist, Head of Department of Virology, Department of Virology-Immunology, LabPLUS, Auckland City Hospital, Auckland; Department of Microbiology, Labtests, Auckland.

Acknowledgements

This work was funded in part by the School of Medicine Foundation (University of Auckland) and the COVID-19 Innovation Acceleration Fund (Ministry of Business, Innovation and Employment).

Correspondence

Shivani Fox-Lewis, Virology department, LabPlus, PO Box 110031, Auckland City Hospital, Auckland 1148.

Correspondence Email

sfoxlewis@adhb.govt.nz

Competing Interests

Nil.

1. Miller TE, Garcia Beltran WF, Bard AZ, et al. Clinical sensitivity and interpretation of PCR and serological COVID-19 diagnostics for patients presenting to the hospital. FASEB J 2020 doi: 10.1096/fj.202001700RR

2. Winter AK, Hegde ST. The important role of serology for COVID-19 control. Lancet Infect Dis 2020; 20(7):758–759.

3. Deeks JJ, Dinnes J, Takwoingi Y, et al. Antibody tests for identification of current and past infection with SARS-CoV-2. Cochrane Database Syst Rev 2020; 6(6):CD013652

4. Peeling RW, Wedderburn CJ, Garcia PJ, et al. Serology testing in the COVID-19 pandemic response. Lancet Infect Dis 2020; 20(9):e245–e249.

5. Krammer F, Simon V. Serology assays to manage COVID-19. Science 2020; 368(6495):1060–1061.

6. Clapham H, Hay J, Routledge I, et al. Seroepidemiologic Study Designs for Determining SARS-COV-2 Transmission and Immunity. Emerg Infect Dis. 2020; 26(9):1978–1986.

7. Ministry of Health. COVID-19: Current cases. http://www.health.govt.nz/our-work/diseases-and-conditions/covid-19-novel-coronavirus/covid-19-current-situation/covid-19-current-cases [accessed on 03/11/2020].

8. Stats NZ. Estimated population of NZ. http://www.stats.govt.nz/indicators/population-of-nz [accessed on 03/11/2020].

9. McGregor R, Whitcombe AL, Sheen CR, et al. Collaborative networks enable the rapid establishment of serological assays for SARS-CoV-2 during nationwide lockdown in New Zealand. PeerJ 2020; 8:e9863.

10. Public Health England. Evaluation of sensitivity and specificity of four commercially available SARS-CoV-2 antibody immunoassays. Public Health England 2020.

11. Medicines & Healthcare products Regulatory Agency. Guidance Target Product Profile: antibody tests to help determine if people have recent infection to SARS-CoV-2: Version 2 http://www.gov.uk/government/publications/how-tests-and-testing-kits-for-coronavirus-covid-19-work/target-product-profile-antibody-tests-to-help-determine-if-people-have-recent-infection-to-sars-cov-2-version-2 [accessed on 10/09/2020].

12. Ripperger TJ, Uhrlaub JL, Watanabe M, et al. Detection, prevalence, and duration of humoral responses to SARS-CoV-2 under conditions of limited population exposure. Preprint at http://www.medrxiv.org/content/10.1101/2020.08.14.20174490v1.full.pdf

Contact diana@nzma.org.nz
for the PDF of this article

View Article PDF

The gold standard for the detection of SARS-CoV-2 is real-time reverse transcription polymerase chain reaction (RT-PCR). However, this method is subject to the quality of collection and changes in viral dynamics over time.1

Serological assays can diagnose SARS-CoV-2 infections where exposure was several weeks earlier and missed by RT-PCR alone.2 A serological response is generally reliably detected two weeks after symptom onset.3,4 Serology could be utilised in contact tracing and testing high-risk groups, eg, border workers, but must not be used to infer immunity against reinfection as lasting protective immunity is yet to be clearly demonstrated.2,5,6

Post market studies of several commercial assays show variable sensitivity between reports, suggesting heterogeneity in studied populations.4 In low prevalence settings, like New Zealand (1,607 RT-PCR confirmed cases, attack rate 0.03%), ensuring high specificity is more consequential than high sensitivity.5,7,8

Therefore, we present a comparison of nine SARS-CoV-2 serological assays evaluated in the Auckland region, to determine assay performance in the relevant epidemiological context.

Assays evaluated were the Abbott Architect (Abbott Diagnostics, US), Roche Elecsys (Roche Diagnostics, Switzerland), Euroimmun (Euroimmun Medizinische Labordiagnostika AG, Germany), EDI (Epitope Diagnostics, Inc, US), in-house (University of Auckland) two-step enzyme-linked immunoassay (ELISA) that includes a screen on the receptor binding domain followed by serial dilution on the Spike protein, and a surrogate viral neutralisation assay (cPASS GenScript, Singapore). The methodology of the latter two is described in detail in McGregor et al.9 The RT-PCR and serological assays were validated at each participating laboratory according to standard practice and following manufacturer’s instructions. The four laboratories were LabPLUS (Auckland City Hospital), the Faculty of Medical and Health Sciences (University of Auckland), Labtests Auckland and Middlemore Hospital laboratory.

Three categories of samples were used: pre-pandemic, pandemic-RT-PCR-negative and pandemic-RT-PCR-positive, described in McGregor et al.9 Pre-pandemic sera tested on the Abbott, in-house ELISA, cPASS and Euroimmun S1 protein assays were from healthy volunteers and patients with respiratory infections and/or symptoms (n=113). Pre-pandemic sera tested on the Euroimmun N protein, EDI IgG and Roche assays were archived blood bank sera (n=40).

Pandemic samples from patients at Auckland City Hospital and Middlemore Hospital, March to May 2020, were tested on all platforms (n=103), with the exception of the Euroimmun N protein and EDI assays on which only Auckland City Hospital samples were tested (n=90). Due to volume constraints, all pandemic samples were not tested on the Abbott assay.

Pre-pandemic sera are by definition true negatives. Pandemic sera were categorised as positive if the corresponding RT-PCR result was positive, or if RT-PCR negative and more than two serological assays resulted positive for that sample. This applied to one sample from a returning traveller.

Equivocal serology results were categorised as positive. The assays with equivocal ranges are Euroimmun (0.8 to <1.1) and EDI (10% of the cut-off).

Table 1: A summary of the methodology, the number of sera that tested positive and negative in each category, and the calculated sensitivity and specificity of each serological assay for SARS-CoV-2 evaluated in this study.

Pandemic RT-PCR positive samples include 21 RT-PCR positive samples and one RT-PCR-negative sample included in this group because >2 serological assays returned positive results—see methods for details; N is the nucleocapsid protein of SARS-CoV-2; S is the Spike protein of SARS-CoV-2; S1 is the S1 subunit of the Spike protein; RBD is the receptor binding domain of the Spike protein; IgG is immunoglobulin G; IgA is immunoglobulin A; IgM is immunoglobulin M; CI is confidence interval; n is the total number of samples within that category.

This evaluation demonstrates that the IgG, total antibody (IgG plus IgM) and surrogate viral neutralisation assays are broadly comparable with sensitivities between 77.8–100% and specificities of 94.2–100%. In contrast, the IgA and IgM assays demonstrated poor specificity and sensitivity, respectively.

Three RT-PCR-positive samples tested on the Abbott, Roche, in-house ELISA, cPASS and Euroimmun S1 protein assays were collected within seven days of symptom onset. With these samples excluded, the sensitivity of those assays increased (86.7–100%). This reflects the time to seroconversion (seven days would include IgM seroconversion), and concurs with prior reports.3,10

With samples within seven days of symptom onset excluded, the in-house ELISA and cPASS assays demonstrate 100% sensitivity and specificity, meeting the UK acceptability criteria (>98% sensitivity and specificity) for serological assays.11 A detailed comparison of these two assays was recently published.9 Despite excellent technical performance, utility in diagnostic laboratories may be limited by workflow and throughput considerations.

The Abbott and Roche assays (targeting N protein antigens) demonstrated moderate sensitivity (86.7 and 89.5% respectively) with samples within seven days of symptom onset excluded, and high specificity (100% and 99.2% respectively). They are deployable on high-throughput analysers; however, this utility must be balanced against their imperfect sensitivity.

The immunokinetics of the response to N and S proteins are reported to differ, with more persistent responses seen with the latter,12 which may contribute to the differences in assay sensitivity observed.

New Zealand has a low prevalence of SARS-CoV-2: an effective serological assay for widespread use would need to have high specificity, to minimise the risk of false positive results.5 Most assays in this study were highly specific, though combining assays with different targets in a two-stage protocol (all positives re-tested using an assay targeting a different antigen) would further increase specificity. We recommend this approach for New Zealand currently.

The compromise for highly specific assays is lower sensitivity, as found for most assays in a recent Cochrane review.3 Therefore, in select circumstances (eg, clear high risk of exposure >14 days previously), it may be prudent to test samples that are initially seronegative on a second platform, in case the shortfall in sensitivity results in a falsely negative result.

This study was limited by the small sample size, particularly with only 22 positive samples included, hence the wide confidence intervals on the sensitivity and specificity calculations. Nonetheless, given current difficulties in sourcing positive and negative samples, this evaluation provides valuable initial data for other New Zealand laboratories when considering which assays to validate for their own use and a basis for future studies. Unfortunately we were not able to test exactly the same panel of sera on all assays; this represented the operational constraints laboratories were under during the first stages of the pandemic.

In summary, this study presents a local evaluation of nine serological assays, with results specific to the epidemiological context of New Zealand. The IgG, the total antibody and surrogate viral neutralisation assays demonstrated high specificity and reasonable sensitivity. The application of these assays needs to incorporate local laboratory workflow and logistics, with respect to anticipated testing volume. These data support deployment of serological assays in New Zealand for testing high-risk groups to assist contact tracing and public health investigations.

Summary

Abstract

Aim

Method

Results

Conclusion

Author Information

Shivani Fox-Lewis, Clinical Virology Registrar, Department of Virology-Immunology, LabPLUS, Auckland City Hospital, Auckland; Alana Whitcombe, PhD Student, Faculty of Medical and Health Sciences, University of Auckland, Auckland; Maurice Wilkins Centre, University of Auckland, Auckland; Reuben McGregor, Post-Doctoral Research Fellow, Faculty of Medical and Health Sciences, University of Auckland, Auckland; Maurice Wilkins Centre, University of Auckland, Auckland; Lauren Carlton, Research Technician, Faculty of Medical and Health Sciences, University of Auckland, Auckland; Yulia Hwang, Registered Medical Laboratory Scientist, Department of Virology-Immunology, LabPLUS, Auckland City Hospital, Auckland; Paul Austin, Section Leader For Serology, Department of Virology-Immunology, LabPLUS, Auckland City Hospital, Auckland; Lisa Aspin, Head of the Department of Biochemistry and Immunology, Department of Biochemistry and Immunology, Labtests, Auckland; Bryan Raill, Section Head for Toxicology and Immunoassay, Department of Biochemistry, Middlemore Hospital, Auckland; Rachel Webb, Consultant Paediatrician, Paediatric Infectious Diseases, Faculty of Medical and Health Sciences, University of Auckland, Auckland; Starship Children’s Hospital, Auckland; KidzFirst Children’s Hospital, Middlemore Hospital, Auckland; Susan Taylor, Clinical Microbiologist, Department of Microbiology, Middlemore Hospital, Auckland; Susan Morpeth, Clinical Microbiologist, Department of Microbiology, Middlemore Hospital, Auckland; Sally Roberts, Clinical Microbiologist, Head of Department of Microbiology Maurice Wilkins Centre, University of Auckland, Auckland; Department of Microbiology, LabPLUS, Auckland City Hospital, Auckland; Nicole J Moreland, Senior Lecturer, Faculty of Medical and Health Sciences, University of Auckland, Auckland; Maurice Wilkins Centre, University of Auckland, Auckland; Gary McAuliffe, Clinical Microbiologist, Head of Department of Virology, Department of Virology-Immunology, LabPLUS, Auckland City Hospital, Auckland; Department of Microbiology, Labtests, Auckland.

Acknowledgements

This work was funded in part by the School of Medicine Foundation (University of Auckland) and the COVID-19 Innovation Acceleration Fund (Ministry of Business, Innovation and Employment).

Correspondence

Shivani Fox-Lewis, Virology department, LabPlus, PO Box 110031, Auckland City Hospital, Auckland 1148.

Correspondence Email

sfoxlewis@adhb.govt.nz

Competing Interests

Nil.

1. Miller TE, Garcia Beltran WF, Bard AZ, et al. Clinical sensitivity and interpretation of PCR and serological COVID-19 diagnostics for patients presenting to the hospital. FASEB J 2020 doi: 10.1096/fj.202001700RR

2. Winter AK, Hegde ST. The important role of serology for COVID-19 control. Lancet Infect Dis 2020; 20(7):758–759.

3. Deeks JJ, Dinnes J, Takwoingi Y, et al. Antibody tests for identification of current and past infection with SARS-CoV-2. Cochrane Database Syst Rev 2020; 6(6):CD013652

4. Peeling RW, Wedderburn CJ, Garcia PJ, et al. Serology testing in the COVID-19 pandemic response. Lancet Infect Dis 2020; 20(9):e245–e249.

5. Krammer F, Simon V. Serology assays to manage COVID-19. Science 2020; 368(6495):1060–1061.

6. Clapham H, Hay J, Routledge I, et al. Seroepidemiologic Study Designs for Determining SARS-COV-2 Transmission and Immunity. Emerg Infect Dis. 2020; 26(9):1978–1986.

7. Ministry of Health. COVID-19: Current cases. http://www.health.govt.nz/our-work/diseases-and-conditions/covid-19-novel-coronavirus/covid-19-current-situation/covid-19-current-cases [accessed on 03/11/2020].

8. Stats NZ. Estimated population of NZ. http://www.stats.govt.nz/indicators/population-of-nz [accessed on 03/11/2020].

9. McGregor R, Whitcombe AL, Sheen CR, et al. Collaborative networks enable the rapid establishment of serological assays for SARS-CoV-2 during nationwide lockdown in New Zealand. PeerJ 2020; 8:e9863.

10. Public Health England. Evaluation of sensitivity and specificity of four commercially available SARS-CoV-2 antibody immunoassays. Public Health England 2020.

11. Medicines & Healthcare products Regulatory Agency. Guidance Target Product Profile: antibody tests to help determine if people have recent infection to SARS-CoV-2: Version 2 http://www.gov.uk/government/publications/how-tests-and-testing-kits-for-coronavirus-covid-19-work/target-product-profile-antibody-tests-to-help-determine-if-people-have-recent-infection-to-sars-cov-2-version-2 [accessed on 10/09/2020].

12. Ripperger TJ, Uhrlaub JL, Watanabe M, et al. Detection, prevalence, and duration of humoral responses to SARS-CoV-2 under conditions of limited population exposure. Preprint at http://www.medrxiv.org/content/10.1101/2020.08.14.20174490v1.full.pdf

Contact diana@nzma.org.nz
for the PDF of this article

Subscriber Content

The full contents of this pages only available to subscribers.

LOGINSUBSCRIBE