Background
COVID-19 is an infectious disease caused by the SARS-CoV-2 virus that spreads easily between people in a similar way to the common cold or ‘flu’. Most people with COVID-19 have a mild-to-moderate respiratory illness, and some may have no symptoms (asymptomatic infection). Others experience severe symptoms and need specialist treatment and intensive care.
In response to COVID-19 infection, the immune system develops proteins called antibodies that can attack the virus as it circulates in their blood. People who have been vaccinated against COVID-19 also produce these antibodies against the virus. Tests are available to detect antibodies in peoples' blood, which may indicate that they currently have COVID-19 or have had it previously, or it may indicate that they have been vaccinated (although this group was not the focus of this review).
Why are accurate tests important?
Accurate testing allows identification of people who need to isolate themselves to prevent the spread of infection, or who might need treatment for their infection. Failure of diagnostic tests to detect infection with COVID-19 when it is present (a false negative result) may delay treatment and risk further spread of infection to others. Incorrect diagnosis of COVID-19 when it is not present (a false positive result) may lead to unnecessary further testing, treatment, and isolation of the person and close contacts. Accurate identification of people who have previously had COVID-19 is important in measuring disease spread and assessing the success of public health interventions.
To determine the accuracy of an antibody test in identifying COVID-19, test results are compared in people known to have (or have had) COVID-19 and in people known not to have (or have had) COVID-19. The criteria used to determine whether people are known or not known to have COVID-19 is called the ‘reference standard’. Many studies use a test called reverse transcriptase polymerase chain reaction (RT-PCR) as the reference standard, with samples taken from the nose and throat. Additional tests that can be used include measuring symptoms, like coughing or high temperature, or ‘imaging’ tests like chest X-rays. People known not to have COVID-19 are sometimes identified from stored blood samples taken before COVID-19 existed, or from patients with symptoms confirmed to be caused by other diseases.
What did the review study?
We wanted to find out whether antibody tests:
- are able to diagnose infection in people with or without symptoms of COVID-19, and
- can be used to find out if someone has already had COVID-19.
The studies we included in our review looked at three types of antibodies. Most commonly, antibody tests measure two types known as IgG and IgM, but some tests only measure a single type of antibody or different combinations of the three types of antibodies (IgA, IgG, IgM).
What did we do?
We looked for studies that measured the diagnostic accuracy of antibody tests to detect current or past COVID-19 infection and compared them with reference standard criteria. Since there are many antibody tests available, we included studies assessing any antibody test compared with any reference standard. People could be tested in hospital or in the community. The people tested may have been confirmed to have, or not to have, COVID-19 infection, or they may be suspected of having COVID-19.
Study characteristics
We found 178 relevant studies. Studies took place in Europe (94), Asia (45), North America (35), Australia (2), and South America (2).
Seventy-eight studies included people who were in hospital with suspected or confirmed COVID-19 infection and 14 studies included people in community settings. Several studies included people from multiple settings (35) or did not report where the participants were recruited from (39).
One hundred and forty-one studies included recent infection cases (mainly week 1 to week 3 after onset of symptoms), and many also included people tested later (from day 21 onwards after infection) (117).
Main results
In participants that had COVID-19 and were tested one week after symptoms developed, antibody tests detected only 27% to 41% of infections. In week 2 after first symptoms, 64% to 79% of infections were detected, rising to 78% to 88% in week 3. Tests that specifically detected IgG or IgM antibodies were the most accurate and, when testing people from 21 days after first symptoms, they detected 93% of people with COVID-19. Tests gave false positive results for 1% of those without COVID-19.
Below we illustrate results for two different scenarios.
If 1000 people were tested for IgG or IgM antibodies during the third week after onset of symptoms and only 20 (2%) of them actually had COVID-19:
- 26 people would test positive. Of these, 8 people (31%) would not have COVID-19 (false positive result).
- 974 people would test negative. Of these, 2 people (0.2%) would actually have COVID-19 (false negative result).
If 1000 people with no symptoms for COVID-19 were tested for IgG antibodies and 500 (50%) of them had previously had COVID-19 infection more than 21 days previously:
- 455 people would test positive. Of these, 6 people (1%) would not have been infected (false positive result).
- 545 people would test negative. Of these, 51 (9%) would actually have had a prior COVID-19 infection (false negative result).
How reliable were the results of the studies of this review?
We have limited confidence in the evidence for several reasons. The number of samples contributed by studies for each week post-symptom onset was often small, and there were sometimes problems with how studies were conducted. Participants included in the studies were often hospital patients who were more likely to have experienced severe symptoms of COVID-19. The accuracy of antibody tests for detecting COVID-19 in these patients may be different from the accuracy of the tests in people with mild or moderate symptoms. It is not possible to identify by how much the test results would differ in other populations.
Who do the results of this review apply to?
A high percentage of participants were in hospital with COVID-19, so were likely to have more severe disease than people with COVID-19 who were not hospitalised. Only a small number of studies assessed these tests in people with no symptoms. The results of the review may therefore be more applicable to those with severe disease than people with mild symptoms.
Studies frequently did not report whether participants had symptoms at the time samples were taken for testing making it difficult to fully separate test results for early-phase infection as opposed to later-phase infections.
The studies in our review assessed several test methods across a global population, therefore it is likely that test results would be similar in most areas of the world.
What are the implications of this review?
The review shows that antibody tests could have a useful role in detecting if someone has had COVID-19, but the timing of test use is important. Some antibody tests may help to confirm COVID-19 infection in people who have had symptoms for more than two weeks but who have been unable to confirm their infection using other methods. This is particularly useful if they are experiencing potentially serious symptoms that may be due to COVID-19 as they may require specific treatment. Antibody tests may also be useful to determine how many people have had a previous COVID-19 infection. We could not be certain about how well the tests work for people who have milder disease or no symptoms, or for detecting antibodies resulting from vaccination.
How up-to-date is this review?
This review updates our previous review. The evidence is up-to-date to September 2020.
Some antibody tests could be a useful diagnostic tool for those in whom molecular- or antigen-based tests have failed to detect the SARS-CoV-2 virus, including in those with ongoing symptoms of acute infection (from week three onwards) or those presenting with post-acute sequelae of COVID-19. However, antibody tests have an increasing likelihood of detecting an immune response to infection as time since onset of infection progresses and have demonstrated adequate performance for detection of prior infection for sero-epidemiological purposes. The applicability of results for detection of vaccination-induced antibodies is uncertain.
The diagnostic challenges associated with the COVID-19 pandemic resulted in rapid development of diagnostic test methods for detecting SARS-CoV-2 infection. Serology tests to detect the presence of antibodies to SARS-CoV-2 enable detection of past infection and may detect cases of SARS-CoV-2 infection that were missed by earlier diagnostic tests. Understanding the diagnostic accuracy of serology tests for SARS-CoV-2 infection may enable development of effective diagnostic and management pathways, inform public health management decisions and understanding of SARS-CoV-2 epidemiology.
To assess the accuracy of antibody tests, firstly, to determine if a person presenting in the community, or in primary or secondary care has current SARS-CoV-2 infection according to time after onset of infection and, secondly, to determine if a person has previously been infected with SARS-CoV-2. Sources of heterogeneity investigated included: timing of test, test method, SARS-CoV-2 antigen used, test brand, and reference standard for non-SARS-CoV-2 cases.
The COVID-19 Open Access Project living evidence database from the University of Bern (which includes daily updates from PubMed and Embase and preprints from medRxiv and bioRxiv) was searched on 30 September 2020. We included additional publications from the Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre) ‘COVID-19: Living map of the evidence’ and the Norwegian Institute of Public Health ’NIPH systematic and living map on COVID-19 evidence’. We did not apply language restrictions.
We included test accuracy studies of any design that evaluated commercially produced serology tests, targeting IgG, IgM, IgA alone, or in combination. Studies must have provided data for sensitivity, that could be allocated to a predefined time period after onset of symptoms, or after a positive RT-PCR test. Small studies with fewer than 25 SARS-CoV-2 infection cases were excluded. We included any reference standard to define the presence or absence of SARS-CoV-2 (including reverse transcription polymerase chain reaction tests (RT-PCR), clinical diagnostic criteria, and pre-pandemic samples).
We use standard screening procedures with three reviewers. Quality assessment (using the QUADAS-2 tool) and numeric study results were extracted independently by two people. Other study characteristics were extracted by one reviewer and checked by a second. We present sensitivity and specificity with 95% confidence intervals (CIs) for each test and, for meta-analysis, we fitted univariate random-effects logistic regression models for sensitivity by eligible time period and for specificity by reference standard group. Heterogeneity was investigated by including indicator variables in the random-effects logistic regression models. We tabulated results by test manufacturer and summarised results for tests that were evaluated in 200 or more samples and that met a modification of UK Medicines and Healthcare products Regulatory Agency (MHRA) target performance criteria.
We included 178 separate studies (described in 177 study reports, with 45 as pre-prints) providing 527 test evaluations. The studies included 64,688 samples including 25,724 from people with confirmed SARS-CoV-2; most compared the accuracy of two or more assays (102/178, 57%). Participants with confirmed SARS-CoV-2 infection were most commonly hospital inpatients (78/178, 44%), and pre-pandemic samples were used by 45% (81/178) to estimate specificity. Over two-thirds of studies recruited participants based on known SARS-CoV-2 infection status (123/178, 69%). All studies were conducted prior to the introduction of SARS-CoV-2 vaccines and present data for naturally acquired antibody responses. Seventy-nine percent (141/178) of studies reported sensitivity by week after symptom onset and 66% (117/178) for convalescent phase infection. Studies evaluated enzyme-linked immunosorbent assays (ELISA) (165/527; 31%), chemiluminescent assays (CLIA) (167/527; 32%) or lateral flow assays (LFA) (188/527; 36%).
Risk of bias was high because of participant selection (172, 97%); application and interpretation of the index test (35, 20%); weaknesses in the reference standard (38, 21%); and issues related to participant flow and timing (148, 82%). We judged that there were high concerns about the applicability of the evidence related to participants in 170 (96%) studies, and about the applicability of the reference standard in 162 (91%) studies.
Average sensitivities for current SARS-CoV-2 infection increased by week after onset for all target antibodies. Average sensitivity for the combination of either IgG or IgM was 41.1% in week one (95% CI 38.1 to 44.2; 103 evaluations; 3881 samples, 1593 cases), 74.9% in week two (95% CI 72.4 to 77.3; 96 evaluations, 3948 samples, 2904 cases) and 88.0% by week three after onset of symptoms (95% CI 86.3 to 89.5; 103 evaluations, 2929 samples, 2571 cases). Average sensitivity during the convalescent phase of infection (up to a maximum of 100 days since onset of symptoms, where reported) was 89.8% for IgG (95% CI 88.5 to 90.9; 253 evaluations, 16,846 samples, 14,183 cases), 92.9% for IgG or IgM combined (95% CI 91.0 to 94.4; 108 evaluations, 3571 samples, 3206 cases) and 94.3% for total antibodies (95% CI 92.8 to 95.5; 58 evaluations, 7063 samples, 6652 cases). Average sensitivities for IgM alone followed a similar pattern but were of a lower test accuracy in every time slot.
Average specificities were consistently high and precise, particularly for pre-pandemic samples which provide the least biased estimates of specificity (ranging from 98.6% for IgM to 99.8% for total antibodies).
Subgroup analyses suggested small differences in sensitivity and specificity by test technology however heterogeneity in study results, timing of sample collection, and smaller sample numbers in some groups made comparisons difficult. For IgG, CLIAs were the most sensitive (convalescent-phase infection) and specific (pre-pandemic samples) compared to both ELISAs and LFAs (P < 0.001 for differences across test methods). The antigen(s) used (whether from the Spike-protein or nucleocapsid) appeared to have some effect on average sensitivity in the first weeks after onset but there was no clear evidence of an effect during convalescent-phase infection.
Investigations of test performance by brand showed considerable variation in sensitivity between tests, and in results between studies evaluating the same test. For tests that were evaluated in 200 or more samples, the lower bound of the 95% CI for sensitivity was 90% or more for only a small number of tests (IgG, n = 5; IgG or IgM, n = 1; total antibodies, n = 4). More test brands met the MHRA minimum criteria for specificity of 98% or above (IgG, n = 16; IgG or IgM, n = 5; total antibodies, n = 7). Seven assays met the specified criteria for both sensitivity and specificity.
In a low-prevalence (2%) setting, where antibody testing is used to diagnose COVID-19 in people with symptoms but who have had a negative PCR test, we would anticipate that 1 (1 to 2) case would be missed and 8 (5 to 15) would be falsely positive in 1000 people undergoing IgG or IgM testing in week three after onset of SARS-CoV-2 infection.
In a seroprevalence survey, where prevalence of prior infection is 50%, we would anticipate that 51 (46 to 58) cases would be missed and 6 (5 to 7) would be falsely positive in 1000 people having IgG tests during the convalescent phase (21 to 100 days post-symptom onset or post-positive PCR) of SARS-CoV-2 infection.