Clicky

Cookies policy

Accuracy of the TB test for cattle

Most recent
additions and updates
 The political problem
Added or updated 3 weeks ago
 Accuracy of the TB test for cattle
Added or updated 4 weeks ago
 Prevalence of bovine TB in Wales, England, Scotland and Ireland
Added or updated one month ago
 TB restricted herds in the Irish Republic by county
Added or updated one month ago
 TB restricted herds in England by county
Added or updated one month ago
 TB reactor cattle in the Irish Republic by county
Added or updated one month ago
 Level of spend on culling needed to reduce TB
Added or updated one month ago
 Bovine TB in Ireland
Added or updated one month ago
 Bovine TB in the UK, England, Ireland, Wales and New Zealand
Added or updated one month ago
 How politics has trumped science in the current Welsh bovine TB policy
Added or updated one month ago
PDF icon Summary of badger control monitoring during 2016. Updating of minimum and maximum numbers during the cull. Annex A2. DEFRA. December 2016.
Added or updated 4 months ago
PDF icon Summary of badger control monitoring during 2016. DEFRA. December 2016.
Added or updated 4 months ago
PDF icon Bovine Tuberculosis in Wales. A Record of Proceedings in the Senedd. The Climate Change, Environment and Rural Affairs Committee. National Assembly for Wales. Presenters: Fenwick, Wright, James, Howells, Byrne, Wilberforce, Hovi, Livingstone. 8th Dec 2016
Added or updated 4 months ago
PDF icon Setting the minimum and maximum numbers in West Gloucestershire and West Somerset for Year 3 of the badger cull. DEFRA. Advice to Natural England. August 2015.
Added or updated 4 months ago
PDF icon Unknown impact of reactive culling in the RBCT trial. Bovine Tuberculosis in Wales. The Climate Change, Environment and Rural Affairs Committee. National Assembly for Wales. The Record of Proceedings in the Welsh Senedd on 08/12/2016.
Added or updated 4 months ago
PDF icon Funding arrangements for TB control in New Zealand. Bovine Tuberculosis in Wales. The Climate Change, Environment and Rural Affairs Committee. National Assembly for Wales. The Record of Proceedings in the Welsh Senedd on 08/12/2016.
Added or updated 4 months ago
PDF icon Bovine Tuberculosis in Wales. A Record of Proceedings in the Senedd. The Climate Change, Environment and Rural Affairs Committee. National Assembly for Wales. Presenters: Rosie Woodroffe, Gareth Enticott, Neil Paton. 10th Nov 2016.
Added or updated 4 months ago
 Bovine TB in GB. Latest SAM data, maps and older VETNET data.
Added or updated 5 months ago
PDF icon An update on methods for finding TB infection in badger faeces. TB Hub. September 2016.
Added or updated 5 months ago
PDF icon Badger Control Model for Wales - Trap-Test-Cull-Vaccinate. Supplemental Report - 4th March 2009. D. Wilkinson, G.C. Smith. Central Science Laboratory, Sand Hutton, York, YO41 1LZ. Last modified: 20/03/2009.
Added or updated 5 months ago

Under reporting of TB reactors

A study carried out in Northern Ireland found that in-house staff were 1.5 and 1.8 times more likely than Private Veterinary Practitioners to classify a herd as a breakdown herd.5 In the Irish Republic, equipment is monitored annually and the performance and results for each testing veterinary surgeon is tested.6 However this monitoring may be too simplistic as stated in the following extract from Reference 7.

The government has been concerned about the quality of bTB testing for some time. Countries like Ireland already use a system of performance indicators to audit testers. But these results show that method to be too simplistic and could lead to false comparisons between vets.

Nevertheless such monitoring is probably better than no monitoring at all as in England for which Reference 8 reports the following.

There is a lack of supervision and monitoring of Local Veterinary Inspector performance by the State Veterinary Service. Not all divisions carry out supervised tests before they are given a permanent appointment and there is no systematic monitoring of subsequent performance.

Indeed Reference 36 reports that AHVLA, as of 25th February 2013, have not audited all veterinaries who are in active service. In fact it has only been since April 2012 that a process has been put in place to ensure that all vets have a supervised visit within six months of their training.

In February 2012 there was no agreement on quality control and DEFRA had evidence that performance was variable and in some cases so inadequate that it prejudiced disease control.57

As announced in Reference 17, which was published on 20th March 2012, Wales have started to pilot an audit of TB testing carried out by official veterinarians. Indeed Reference 18 on 18 April 2012 reported that a veterinary practice on Anglesey had been suspended from testing pending the completion of retraining for its staff.

On 4th June 2013 DEFRA added details of an enhanced audit to their website.40 This was stated to build on experience gained during the pilot in Wales referred to above. Reference 41 describes the audit as follows.

AHVLA field audits will spot-check the testing performance of the OVs employed by those practices using a risk-based approach over a period of seven years...

Individual OVs may be selected for audit on the basis of intelligence and/or data that raises concerns about their TB testing performance.


When a veterinary performs a test on a herd which fails, it is not only unpleasant for the farmer but also for the veterinary when the veterinary has to inform the farmer that an animal has reacted to the test. This is because the repercussions of a failed test cause extreme hardship to the farmer13. In addition to this making the right decision can be very difficult because the results of many herd tests are close to borderline.

Regarding the consequencies of a wrong judgement, according to Reference 9, one infected animal will on average infect five others over a year. Although housing conditions and ventilation would be expected to be somewhat different in a controlled study than in a typical farm setup, this transmission ratio is very different to what was found in the carefully controlled cattle vaccination study carried out at Weybridge, Surrey and reported in Reference 38 in 2012. In this study it was found that when 6 healthy cattle were penned with 4 skin-test-positive animals in each of 10 pens for a year, infection in 8 out of the originally 60 healthy cattle was found. Another words the detected transmission rate was considerably less than one to one let alone one to five. Nevertheless if a large number of TB reactors are being under-reported, this may be increasing the incidence of cattle-to-cattle transmission. Although typical housing arrangements today are vastly better than in previous decades the conditions are still unlikely to match those in a carefully controlled study.

The following shows an extract taken from Reference 42.

I have attended a number of tests with different vets. It is likely that the majority of vets follow very high standards. It is also quite likely that my experience has been with one or two vets whose standards are lower than average. This should be borne in mind when reading the following account.

When I have attended tests, I started to notice that all vets, apart from one, were not refilling their syringes after injecting 20 cattle. They seemed to be refilling their syringes about every 40 shots (i.e. at about half the recommended frequency). During our tests we are well staffed so I am able to closely monitor what the vets are doing. After I started to realize what the vets may be doing, in the last 2 tests I have really concentrated on how often the vet refilled. In fact during the last 2 tests the vet was not only failing to refill his syringes but when his syringes were discharging there were occasions when he was inadvertently squirting the tuberculin over the animal's coat. During the last test I not only asked how often vets have to refill their syringes (in the hope that this would prompt the vet to start refilling his syringes more often) but later in the test I asked if air locks in syringes are a problem. At this point the vet showed me his syringes. I was appalled by their dilapidated state.

All this makes me wonder what proportion of animals are getting through tests in England without having any tuberculin injected into them at all. I understand that cattle do not have to have the full dose to react but obviously they need to have something injected into them. In fact in the last test which I attended one cow was diagnosed as inconclusive which due to severe interpretation went away to be slaughtered and postmortemed. Even though the cow was only diagnosed as inconclusive, lesions were reported in the PM.

This may be cause for concern for farmers on 2 counts. Firstly if infectious cows are being left in the herd these cows may be infecting other animals. Secondly the vast majority of dairy farmers drink the unpasteurised milk which their herd produces.

Regarding infection to other herd mates, although the following may be an overestimate, a report published in New Zealand in 2010 said the following.

"Bovine Tb is an ancient disease that used to be endemic in cattle and dairy herds. It causes a wasting condition which results in a 5% production loss and it is infectious, with approximately one affected animal infecting five others over a year, in a typical dairy herd."

I am not sure which period in time that account was taken. I know that today cattle sheds are a lot better ventilated then they use to be and this will reduce the risk of cattle-to-cattle transmission.

Regarding transfer to the farmer's family this perhaps should be more of a concern to dairy farmers because I suspect the vast majority of dairy farmers drink the milk straight from their bulk tanks and this includes during periods when their herds are diagnosed with TB. Although the chances of TB spreading to the udder are quite small if this was to occur in a missed cow then the farmer and his family drinking this infected milk would be at high risk.

Regarding the monitoring carried out by Public Health England, I understand that PHE only screen farmers who are at particularly high risk. These tests may involve a skin test which tests for a lump a few days later (as in cattle but no avian) and may involve a more specific blood (IGRA) test. I understand that PHE do not collect the results of these tests for latent TB (not even the more specific blood test) so it follows that they have very limited awareness of whether or not the incidence of tests which come back positive are increasing. I question whether this is wise particularly if animals in tests are getting through the test without any tuberculin being injected into them.

The following table describes the quality of testing on a farm where over £9,000 was spent in 2008 erecting steel barriers to stop badgers entering buildings.

DateActionResultNotesPost-mortem results
Jul08Annual testFailConfirmed.-
Sep0860-day testFail2 inconclusives.-
Sep08--Buildings were sealed at a cost of £9403.81 to the farmer and each night badgers were excluded from all buildings containing any cattle, feed and troughs.-
Nov0860-day testPass--
Feb0960-day testPassThis was the second, consecutive, clear test so the farmer is now free to trade.-
Jul096-month testPass--
Jun10Annual testPass--
Jun11Annual testPass--
Mar12--A brand-new slurry spreader was purchased for exclusive use on the farm to avoid the risk of using shared, contaminated equipment.-
Apr12Annual testPass--
Apr13Annual testFail1 reactor. Confirmed by culture only.-
Jun1360-day testFail1 inconclusive. Confirmed by finding lesions.-
Aug1360-day testPass--
Oct1360-day testFailDifferent vet did the test from last time. 2 reactors. 1 confirmed by finding lesions.-
Dec1360-day testPassUsual vet did the test.-
Feb1460-day testPassUsual vet did the test. This was the second, consecutive, clear test so the farmer is now free to trade.-
Jul146-month testFailUsual vet did the test. 1 reactor. Large lump. Lesions found in intestines and lungs. Culture results not back yet.-
Aug14Abattoir inspectionFailOut of 7 cattle slaughtered, 1 was found to have TB in multiple organs and the meat was condemned. 2 others had suspected lesions and culture tests are being done.-
Sep1460-day testFailThis was a thorough test performed by the head vet at the practice who did the test in Sep08. (This vet actually stopped testing to refill his 20-shot syringe when part way through testing cattle in the cattle race.) 13 reactors were found in 5 different groups of cattle. Size of lumps ranged from that of walnuts to that of dinner plates. Two cattle had lumps the size of dinner plates.-
Nov1460-day testFailThis was a thorough test performed by the head vet at the practice. 4 reactors. The size of lumps were that of a satsuma citrus fruit, so smaller than in the last test but clearly reactors.-
Jan1560-day testFail25 reactors, many of which were young heifers about to come into the herd. These heifers were grazing in the field on the day the moribund badger was found wandering on 7th November 2014. See below. The test was thorough and performed by a vet who has never tested on the farm before.17 out of the 25 were found to have visible lesions. Lesions were found in the chest, liver, head, legs and intestines.

To see a copy of the report, please click here.
Apr1560-day testPassThis was another thorough test performed by another new tester. Indeed the young tester refilled her syringe whenever needed and this sometimes included when part way down the cattle race.

Regarding the results - no reactors, not even an inconclusive! All stock had been housed throughout the winter and only the milking herd had been recently allowed out to grass during the day.
-
Jun1560-day testPassThis was another thorough test performed by the tester who found 25 reactors in Jan 2015 i.e. 6 months earlier. No reactors. The farmer is now free to trade.-
Jan166-month testFailNew vet performed this test. This was the first test that this vet had performed on the farm. Test was followed to the book. Calipers were even used to measure skin thickness on the day of injection. 2 reactors in 2 Belgian Blue cattle.Both reactors were found to be lesioned; one in the chest and the other in the head.
Mar1660-day testPassThe usual vet did this test and it was poorly done. The main concern was the amount of tuberculin which was squirted over the cattles' coats. Reporting this to the vet as he was performing the test made no difference to how he was injecting.
According to the vet, the vet had just performed a test at another farm which was audited by XLVets. According to the XLVets web site, XLVets are a group of independently owned, progressive veterinary practices that work together to achieve the highest standards of veterinary care. After that test the XLVet auditor reported to the vet that no fault was found in how he had performed that earlier test.
Result of this test: No reactors. No inconclusives.
-
May1660-day testPassThe usual vet did this test. Although I was not present at the start of the test because the vet arrived 45 minutes earlier than arranged, I saw no tuberculin squirted over the cattles' coats whilst I was present.
Result of this test: No reactors. No inconclusives.
-
Dec166-month testFailThis was a thorough test performed to the book by the vet who first tested on the farm in Jan 2016.
Result of this test: One inconclusive. Prior to tightening up of the rules this result would have led to a pass. The herd will now be restricted. If the animal tests inconclusive or reacts in the test due in 60 days time, it will be slaughtered.
-
Feb1760-day testPassThis was a thorough test performed to the book by the vet who first tested on the farm in Jan 2016.
Result of this test: Pass. The animal which was inconclusive 60 days ago passed the test. This was not a herd test because only the animal which was inconclusive 60-days earlier had to be retested. However some calves were also tested and these all passed too.
-

Unfortunately the UK appears to place more reliance on routine skin testing than New Zealand due to different protocols used in slaughter houses during post-mortem inspection. The following is an extract from Reference 9 which was published in 2010.

We observe the difference in cattle detection rates between the UK and New Zealand, which is presumably due to different post-mortem protocols. In New Zealand slaughter houses specifically examine lung tissue for Tb, with an 85% accuracy rate (AHB pers. comm.), whereas a different protocol appears to be used in the UK.

If on-farm tests are failing to detect infected herds, this would be expected to lead to an increase in the proportion of new herd incidents being detected through slaughterhouse surveillance. Fig 96 below shows that from 2002 to 2012 the number of confirmed slaughterhouse cases divided by new herd incidents has increased more than 4 times from 5 to 23%. This would suggest that during these 10 years the under-reporting of herd incidence by on-farm tests has vastly increased. However it should be noted that slaughterhouse staff have recently (in 2011) undergone re-training and submissions have been re-organised21. Reference 37 also lists slaughterhouse surveillance milestones between 1984 and 2012 which DEFRA attribute to an increasing proportion of new bovine TB herd breakdowns initiated through slaughter surveillance. If however there has NOT been continual improvement in slaughterhouse surveillance over the last 10 years, the steady increase in slaughterhouse detection over this timescale implies that there may have been a substantial deterioration in on-farm testing performance.

New bovine TB cattle incidents 2008 to 2012.

In Fig 93, the number of new herd incidents peak in about March each year. This is thought to be due to the fact that intensity of herd tests peak in about February each year.

Slaughterhouse bovine TB cases 2008 to 2012.


Confirmed slaughterhouse cases divided by new bovine TB herd incidents 2008 to 2012.


Confirmed slaughterhouse cases divided by new bovine TB herd incidents 2000 to 2012.

Data for Figs 93 to 95 was taken from Reference 33. In Fig 96 data extracted from Chief Veterinary Officer reports was extracted from References 24 to 32 and data extracted from the DEFRA spreadsheet was taken from the GB monthly spreadsheet in Reference 33.

Low levels of testing in Great Britain

The following graph shows that in Great Britain the number of herd tests to each new incident has reduced substantially since 1996 and has remained low since 2003.

Number of herd tests divided by the number of new incidents disclosed each month in Great Britain

Data for the above graph was extracted from Incidence of TB in cattle in Great Britain - GB dataset. Table 1 Herds.

The following graph shows that for every herd found infected, both Northern Ireland and the Irish Republic have in the past 5 years (2008 to 2012) carried out at least twice as many herd tests as Great Britain.

Number of bovine TB herd tests divided by the number of herds found infected

Data in the above plot for Great Britain was taken from the detailed TB statistics supplied by the AHVLA, county herd statistics supplied by DEFRA, and AHVLA Chief Veterinary Officer's reports from 1967 to 1994. Data for Northern Ireland was taken from DARDNI statistics. Data for the Irish Republic was taken from a reply and spreadsheets received from the Department of Agriculture, Food and the Marine.

Data plotted on the animals tested and slaughtered page in Figs 7,10 and 24 reveals that in 2009 the number of tests to every animal tested positive in Great Britain was overall about one half of that in the Irish Republic and about one twentieth of that in New Zealand.

An alternative explanation for the 4-fold increase in the proportion of new cases picked up by abattoir inspection (Fig 96) is that more new cases are appearing in areas where on-farm testing is not being performed frequently enough. There would be a higher tendency to identify these cases in abattoirs. Examination of how the proportion of cases identified in abbatoirs has changed in the last 10 years split up according to 1-year, 2-year, 3-year and 4-year testing areas would reveal whether or not on-farm testing was being performed frequently enough in each area.

Liver fluke

The incidence of fluke disease has been increasing over the past ten years, due to wetter summers which increase snail populations22. The summer of 2012 has been one of the wettest summers on record and it is held responsible for the extraordinary levels of disease and death in sheep due to liver fluke.34 In the autumn and winter of 2012, the occurrence of death due to liver fluke had increased 10-fold compared to 2011 values, based on submissions to veterinary diagnostic centres.34 Tests to diagnose bovine TB rely on inflammation of the skin in response to injected TB proteins, but if the animal also has liver fluke infection, this inflammation is suppressed, reducing the detection of bovine TB.23 Fig 95 shows on a monthly basis how the proportion of new bovine TB herd incidences detected by slaughterhouse surveillance changed. In 2012 ground conditions were very dry up until April when conditions became very wet and accommodating for liver fluke. However the increase in the percentage of slaughterhouse detection for bovine TB after April in Fig 95 is not as big as might be expected in view of the substantial increase in liver fluke incidence. As such, up to January 2013 liver fluke has not been having an obvious impact on the extent to which herd incidence is being under reported due to reduced sensitivity of the bovine TB skin test.

Influence of Weybridge and Lelystad tuberculins

Tuberculin is the name given to extracts of Mycobacterium tuberculosis, M. bovis, or M. avium that is used in skin testing in cattle (and humans) to identify a tuberculosis infection. The skin test involves injecting a small amount of the tuberculin into the skin of the animal. In most cattle infected with M. bovis, this will cause the animal's immune system to react to the tuberculin and cause a localised allergic reaction (swelling) of the skin a few days after the injection.

In the summer of 2005, after difficulties with the production of tuberculin at VLA Weybridge, Defra began to source paired stocks of bovine and avian tuberculins from ID-Lelystad in The Netherlands. These stocks started to be used in herds across GB from October 2005 and were alternated with Weybridge tuberculins for release to veterinarians on a strict temporal basis, dependent on the shelf life of the available stocks from each manufacturer. The alternate use of both tuberculins continued in GB until the production of Weybridge tuberculin at VLA ceased and stocks eventually ran out in September 2009. Since then (this was written in 19th January 2011), Dutch tuberculin from ID-Lelystad (now owned by Prionics) has been the only antigen used in the UK bovine TB testing programme (and it had also been in use in Ireland for many years before it started to replace Weybridge tuberculin in the UK).11

This section looks at how the number of cattle slaughtered has changed over these years to examine if there may be any dependence on which tuberculin was used.

Although cattle incidence each year probably depends on a number of factors, such as changes to TB testing regulations19 and a succession of cold winters20, cattle incidence in Great Britain went through large fluctuations between 2005 and 2010 as shown in the graph below. (Data for this graph were sourced here)

Cattle incidence to bovine TB in Great Britain from 1956

A Veterinary Laboratories Agency report12 reports that between 1st Jan 2005 and 30th Jun 2009, out of 26,363,877 cattle tests, tests using the Weybridge tuberculin found on average that 497 animals tested positive in every 100,000 tests whereas the Lelystad tuberculin only found 391.12 The ratio given by 497 to 391 closely matches the ratio 0.47 to 0.38 which, as can be seen in the above graph, are the percentage prevalences of TB in cattle in 2008 and 2010 respectively. Although the Weybridge tuberculin was not used exclusively in 2008 (in fact between 60 and 65% of tests in 2008 were performed using it - deduced from Figure 1 in Reference 12), if no Weybridge tuberculin was used in 2010, the switch over to using the Lelystad tuberculin may have contributed to the drop in the reported TB prevalence between 2008 and 2010. In fact in March 2012 in Reference 21 it was reported to the EU that during the last 12-month-period there were 4800 new TB incidents and that 24.7% of these were triggered by slaughterhouse surveillance. It was discussed whether a change of tuberculin has led to reduced sensitivity of the test so that a higher number of reactors are missed and detected at slaughter instead.

Note: A paper on tuberculin manufacturing source and breakdown incidence rate of bovine tuberculosis in British cattle, 2005-2009 was published in Nov 201256.

The influence of switching to the Lelystad tuberculin on the risk of an animal testing positive in each month from January 2004 to August 2011 is illustrated in the graph below. Risk was calculated by dividing the number of TB reactors slaughtered14 by the number of cattle tested14. The highlighting shows the months in which the Lelystad tuberculin was predominantly used. Reference 12 only shows usage up to 31st June 2009 and it is assumed that the Lelystad tuberculin was predominantly used in each month from July 2009 onwards.

Cattle incidence to bovine TB in Great Britain from 1956

It should be noted that unlike Great Britain, Northern Ireland abruptly switched from using the Weybridge tuberculin to the Lelystad tuberculin. In fact they recalled and disposed of any remaining Weybridge tuberculin and then started to use Lelystad, so there was not a period when both tuberculins were in use at the same time. As reported in Reference 35, they made the switch in April 2007. Fig 97 below shows that the risk of an animal testing positive did not drop when this switch occurred. However the risk, after smoothing, for 2006 and 2008 in mid summer were both greater than that for 2007. This would imply that use of the Lelystad tuberculin in Great Britain may have only caused a small increase in the extent to which reactors are under reported.

Influence of the Lelystad tuberculin in Northern Ireland

Is the skin test at best only 80% accurate?

On the One Show broadcasted on the 30th September 2011 on BBC1, Brain May was featured presenting an initiative to bring Jan Rowe who is a spokesman for the National Farmers Union into talks with the Badger Trust. Both Jan Rowe and Jeff Hayden, who is the Finance Director of the Badger Trust, were shown being interviewed by Brian May. The following shows the course of the dialogue (as broadcasted on the One Show) between Brian May and Jeff Hayden.

Brian May said:
Animal welfare group the Badger Trust reject the theory that badgers are the main cause of the transmission of the disease. They believe infected cattle flick?/flip?/flit? through the TB testing and infect the rest of the herd.

Jeff Hayden said:
The so called skin test that they use to test cattle at the moment is at best only 80% accurate so we hear this figure from the farmers about 25,000 cattle being slaughtered last year in England alone and we feel as sorry about that as we do about the death of badgers but what this means is that there are over 6,000 infected cattle left in the national herd. That's the single biggest problem.

If animals were slaughtered based on a method which was only 80% accurate at best the situation would be very dire. In theory, this is not the case because interpretation rules which give 80% are usually only applied to detect an infected herd. If a herd is confirmed to be infected by postmortem examination, another set of stricter rules are then applied to identify infected cattle. It is this stricter, more severe, set of rules which are used to remove infected cattle from the herd, if the herd is confirmed to be infected. Very roughly half of herd breakdowns are confirmed. All herds have to pass 2 consecutive tests at 60 day intervals before restrictions are removed.

The sensitivity of the cattle skin test when using severe interpretation

Reference 4 states the following.

Several studies from various countries have reported estimates of sensitivity for the comparative and other variants of the tuberculin skin test. Test sensitivity (and specificity) is independent of the prevalence of infection in the population and is frequently assumed to be constant across different populations. In practice, however, it can be influenced by a host of other factors including the test procedure, cut-off point for a positive result, tuberculin potency, the stage of infection in the host, other inter-current infections and prevalence of cross-reacting organisms in the locality. It is thus very difficult to quote a single sensitivity estimate for the comparative skin test that would apply to all herds in GB at all times.

Studies evaluating the sensitivity of the test suggest that its sensitivity lies between 52% and 100%, with median values of 80% and 93.5% for standard and severe interpretations, respectively.


It would appear from this that the 'accuracy' (sensitivity) of the skin test when removing infected cattle from the herd using severe interpretation is about 93.5%. Although this is better than 80% 'accuracy' (sensitivity) which would result in one in every 5 tests failing to identify an infected animal, 93.5% still means that one in about every 15 tests fail to detect infection and this is not good. In addition to this, replacement of the tuberculin with a less potent tuberculin, which has been used in the last 5 years since 2005, implies that sensitivities have been reduced to less than those reported above. However the tuberculin(s) used to arrive at the 80% sensititivity is not currently known by the author of this page and enquiries to DEFRA are currently in progress.

The risk of false negatives and positives when testing cattle

In the extract below quoted specificities imply that in Great Britain the blood test is on average about 40 times more likely than the skin test to indicate that a healthy animal is infected. Average specificity of the skin test is quoted to be 99.9% (1 in 1000 wrongly classified as a reactor) whereas that of the blood test is quoted to be 96% (4 in 100 wrongly classified as a reactor).

The extract below is taken from a reply received on 27th March 2009 sent by DEFRA in response to the FOI request shown in Reference 1.

" There are two immunologically based diagnostic tests used in Great Britain (GB) for bTB screening of cattle herds. The primary screening test is the single intradermal comparative cervical tuberculin (SICCT) test, which is commonly known as the comparative tuberculin skin test. Additionally, the ancillary gamma-interferon (γ-IFN) diagnostic blood test has been used since 2002 alongside the skin test in certain prescribed circumstances.

The diagnostic sensitivity and specificity ranges for these tests have been evaluated in several articles and reports published in the veterinary scientific literature over the years. These are in the public domain and for more information you are kindly referred to those on our website at

Editor note
Links to the following 2 documents have been removed as they no longer work after the DEFRA web site was redesigned in October 2009. Unfortunately the webmaster was unable to provide me with new links when I made enquiries on 17Oct09. I was however directed to Reference 2 in a later reply.

http://www.defra.gov.uk/animalh/tb/pdf/gifn_trialfinalreport.pdf
http://www.defra.gov.uk/animalh/tb/pdf/gifn_specificityreport.pdf

To summarise, the published estimates for the animal-level sensitivity of the SICCT test in various countries, if correctly performed, range from 52 to 100%, with a median value of 80% at standard interpretation. In other words, this test can be expected to miss about 2 in every 10 infected cattle on a single round of testing (a 20% false negative probability).

Other studies carried out in TB-free cattle populations have found the animal-level specificity of this test to lie between 78.8% and 100%, with a median value of 99.5%. In particular, the SICCT test applied to cattle in bTB-free herds in GB is believed to have a specificity of 99.9%, which is equivalent to a 0.1% probability of false positives, or a one in 1,000 chance that a non-infected animal will be wrongly classified as a reactor.

With regard to the γ-IFN blood test, performance evaluation carried out in a number of countries shows that at the laboratory cut offs used in GB, it has a sensitivity of between 73 and 100%, with a median value of about 87% (i.e. a false negative probability of 10 to 15%, which is slightly lower than that of the SICCT test). Because the two tests detect slightly different sub-groups of infected cattle, by combining the two tests a higher overall sensitivity can be achieved.

A trial conducted in GB to evaluate the specificity of the γ-IFN blood test confirmed the findings of previous studies in other countries, in that it estimated its specificity to be about 96% (i.e. a 4% probability of false positive reactors). This is higher than the false positive probability for the SICCT test and it is one of the reasons why the γ-IFN blood test cannot be used for routine bTB surveillance.

Nevertheless, there is no consensus with regard to the most appropriate values of these classical test characteristics that currently apply to GB. It is important to appreciate that the sensitivity and specificity estimates cited above are indicative averages. The actual performance of a screening test in a particular herd under field conditions is, of course, dependent on a range of variables, such as the diligence of the tester, in adhering to the correct testing procedure, the within-herd prevalence of cattle sensitised to other non-TB environmental mycobacteria, and factors that may alter the immune response to tuberculin of individual animals (e.g. nutritional status, pregnancy, stress levels, concurrent infections, etc). Additionally, for all diagnostic tests there is a trade-off between sensitivity and specificity, so that different interpretations of the test can be used under different disease situations. Sensitivity is enhanced in herds with post-mortem or cultural evidence of TB infection by application of the so-called severe interpretation.
"

Is specificity of the skin test as high as DEFRA claims?

Specificity measures the proportion of negatives which are correctly identified as such. In context, this is the percentage of healthy bovines which are correctly identified as not having TB. This is sometimes called the true negative rate.48

A "lower bound" to specificity may be obtained by looking at the number of reactors disclosed in an area where TB prevalence is very low. This involves making 2 assumptions. Firstly, all positive results are assumed to be false. If the area is not totally free of infection, true positives will occur and this will mean that any derived specificity will be too low. Hence in the absence of any other assumption this would give a lower bound estimate to actual specificity. However in addition to this assumption is the assumption that disclosure by veterinaries will be the same in an area which is considered to be free of infection and in an area where infection is expected to be found. Fig 131a below shows specificities derived from test results in Scotland.

TB test specificity derived from test results in Scotland


Data in the above graphs were downloaded from References 53 and 54.

If extremely different levels of prevalences of infection in 2 different areas can be considered to have insignificant impact on veterinary performance, the derived specificities shown in Fig131a do appear to support a specificity as high as 99.9%. Unfortunately the resultant impact of true positives and variation in disclosure performance (should this be occurring) is hard to judge.

Regarding disclosure, Fig 132a below shows that the proportion of herd tests in England which leads to the herd being restricted is about 40% greater when performed by AHVLA vets than by local vets. However it should be noted that the AHVLA vet sample size (as shown in Fig 132b) was small so this figure should be treated as approximate. It should also be noted that the difference between discolsure rates between local and AHVLA vets may be somewhat different if AHVLA vets selected herds to test at random. A small proportion of herds were tested by AHVLA vets and the basis on which this small proportion of herds were selected was not disclosed by DEFRA when supplying the data.

Proportion of TB herds tested in England which fail when broken down by test performer


Data shown in the above graph was extracted from Reference 55. This data extends up to 31st July 2013 so does not show all tests for 2013. Lower sample sizes, as obtained for years 2012 and 2013, would tend to result in larger sampling error.

Returning to skin test specificity, there is a piece of work which on first impressions does not tally with a specificity as high as 99.9%. This was the RBCT anlysis which was confined to high incidence areas in England and looked at data collected between 1998 and 2005. In this analysis the ISG favoured use of confirmed herd breakdowns over total herd breakdowns where total herd breakdowns is given by the number of confirmed plus unconfirmed breakdowns. The ISG must have restricted analysis to confirmed breakdowns only for good reason because doing so significantly reduced sample size which would have increased sampling error and confidence intervals. The following explains why they did this.

When unconfirmed breakdowns in the RBCT were analysed in isolation as reported in Reference 39 and presented in Table S6 of that reference, the impact of badger culling became indiscernible. (According to DEFRA statistics,28 about 38% of all new herd incidents in the South West of England in 2005 were unconfirmed.)

If
  • the proportion of breakdowns due to badgers is the same in the unconfirmed populations as it is in the confirmed populations, and
  • from the start of the RBCT to 25 February 2011, incidence of confirmed breakdowns in culled areas was 27.4% lower than in untreated areas, as is reported in Reference 44
no discernible difference in the number of unconfirmed breakdowns in the culled and untreated areas is an unexpected result. If no false positives were occurring and all breakdowns were due to TB infection, then culling would have reduced incidence by 27.4% as in confirmed breakdowns.

If the existence of false positives does not explain the lack of impact in these results, it is interesting to note that confirmation rate plummets with age from 50-70% to 10-20% as shown in Fig 4 of Ref 51. However this observation on its own falls short of explaining why badger culling appeared to have no visible impact.

The following is an extract taken from Reference 39 which explains why only confirmed cases were used in the RBCT analysis.

Analyses of all (confirmed and unconfirmed) breakdowns revealed attenuated estimates of the impacts of proactive culling, in comparison with analyses considering confirmed breakdowns only (compare Table S5 with Table 1 in the main text). To investigate this, we examined analyses of unconfirmed breakdowns (Table S6) only and found considerable overdispersion inside trial areas, estimated effects that were all consistent with no effect of proactive culling on unconfirmed breakdowns and many estimates that were in the opposite direction to the significant effects found on confirmed breakdowns. For these reasons we conclude that there is no evidence of an impact of proactive culling on unconfirmed breakdowns and focus our attention on the analyses based on confirmed breakdowns only.

The reason why it is necessary to slaughter TB reactors which are not confirmed to be infected in post-mortem inspections and tests

Even though there may be grounds to doubt the specificity of the skin test claimed by DEFRA, there are still strong grounds to slaughter all animals which test positive to the current skin test. An explanation is given below.

5-10% of latently infected humans develop clinical tuberculosis during their lifetime through re-activation of the latent infection (re-activation tuberculosis). The argument that latently infected individuals (culturenegative NVL, skin test reactors for example) constitute a continuous and unpredictable source of reinfection, is equally valid for cattle as it is for human TB. At the early stages of infection, or in latently infected cattle, a period will occur when M. bovis appears to be absent because the bacillary load is not large enough to be detected by culture. In addition, the pathological changes caused by the bacilli are not yet profound enough to be detected during routine abattoir inspection. However, cellular immune responses will be detectable in these animals at an earlier stage of infection than the pathological changes caused by the disease (e.g.visible lesions), or before the bacterial loads exceed the numbers necessary to be able to culture M. bovis from tissue samples. To designate these animals as 'false-positive' is inappropriate as they can harbour bacilli, and become infectious to other cattle, specifically when the disease has progressed further.10

The number of confirmed and unconfirmed herd incidences in the 16 years between 1994 and 2009 is shown in Table 9 of Reference 32. Evidently just less than half of these incidents are unconfirmed. However, if account is taken of the average number of animals which are slaughtered in one incident, more than half of cattle which test positive to the skin test have not developed symptoms which are sufficiently far advanced to be detectable by either growing culture or finding lesions.

References
  1. Information request: False negatives and positives associated with bovine TB tests
  2. Review of the gamma interferon testing policy
  3. Review of TB testing procedures: Report for Defra and the Welsh Assembly Government report no.: 22115204-02 Final Report rev 3, 12th June 2006
  4. Bovine tuberculosis in England: Towards eradication. Final Report of the Bovine TB Advisory Group. Presented to Defra on 8 April 2009.
  5. The Control of Bovine Tuberculosis in Northern Ireland
  6. Bovine tuberculosis eradication in Ireland
  7. Bovine TB testing under scrutiny
  8. Review of TB testing procedures: Report for Defra and the Welsh Assembly Government report no.: 22115204-02 Final Report rev 3, 12th June 2006
  9. Bovine Tb Strategy - Review of Costs
  10. Government Veterinary Journal
  11. Usage of Weybridge and Lelystad tuberculins in bovine TB testing in Great Britain
  12. Tuberculin Usage for testing for bovine TB in Great Britain. Report for 1st January 2005 to 30th June 2009.
  13. A farmer's utter despair over bovine TB
  14. TB in Cattle in Great Britain
  15. Tuberculin usage from 1956 to 2011 - Request
  16. Tuberculin usage from 1956 to 2011
  17. Statement: A Programme for Eradicating Bovine Tuberculosis in Wales. The Record of Proceedings. 20 March 2012.
  18. Anglesey vets pracice banned from testing for Bovine TB. 18 April 2012.
  19. Bovine TB: Restrictions after Foot and Mouth in 2001
  20. Central England Temperatures
  21. REPORT OF THE BOVINE TUBERCULOSIS SUB-GROUP, EUROPEAN COMMISSION HEALTH & CONSUMERS DIRECTORATE-GENERAL, Veterinary and international affaires, Meeting held in The UK 27-28 March 2012.
  22. Better Returns from Controlling Liver Fluke. Eblex
  23. University of Liverpool News - Improving bovine TB diagnosis
  24. Animal health: the report of the Chief Veterinary Officer 2001
  25. Animal health: the report of the Chief Veterinary Officer 2002
  26. Animal health: the report of the Chief Veterinary Officer 2003
  27. Animal health: the report of the Chief Veterinary Officer 2004
  28. Animal health: the report of the Chief Veterinary Officer 2005
  29. Animal health: the report of the Chief Veterinary Officer 2006
  30. Animal health: the report of the Chief Veterinary Officer 2007
  31. Animal health: the report of the Chief Veterinary Officer 2008
  32. Animal health: the report of the Chief Veterinary Officer 2009
  33. Zip archive of DEFRA stats foodfarm landuse livestock TB regional. 13th March 2013
  34. Liver fluke in sheep and cattle - a major animal health concern following one of the wettest UK summers on record. Published on 24 Jan 2013.
  35. Bovine TB: Weybridge to Lelystad tuberculin switch in Northern Ireland in 2007. Email request to DARDNI dated 4th April 2013.
  36. Bovine TB: Proportion of TB-testing vets who have been audited. Freedom of Information request to AHVLA. 11 February 2013.
  37. Bovine TB: Slaughterhouse surveillance milestones. RFI 5416. helpline@defra.gsi.gov.uk. 1st May 2013.
  38. Results: DEFRA SE3227 bovine tb cattle vaccination study. AHVLA, Weybridge, Surrey. 2005 to 2012.
  39. Impacts of widespread badger culling on cattle tuberculosis: concluding analyses from a large-scale field trial. SUPPLEMENTARY MATERIAL. CA. Donnelly et al. 2007.
  40. Introduction of quality assurance programme for tuberculin testing by Official Veterinarians. DEFRA. Page published: 4 June 2013.
  41. OV TB Testing Audit. DEFRA. 2013.
  42. Tb lumps. British Farming Forum.
  43. BADGERS AND BOVINE TUBERCULOSIS - REVIEW OF POLICY. MINISTRY OF AGRICULTURE, FISHERIES AND FOOD. G M Dunnet, D M Jones, J P Mclnerney. March 1986.
  44. Bovine TB - Key conclusions from the meeting of scientific experts1, held at Defra on 4th April 2011
  45. Trend towards larger cattle operations continues. AHDB Market Intelligence. 2013.
  46. Bovine TB prevalence in Scotland and prospects for this to increase
  47. Dairy industry in the UK: statistics. House of Commons Library. Standard Note: SN/SG/2721. Last updated: 30 June 2011. Author: Oliver Hawkins.
  48. Sensitivity and specificity. Wikipedia. Last modified on 16 November 2013.
  49. Cattle and calves in the UK from 1866 to 2012
  50. TB testing intervals 2013. AHVLA. Page last modified: 19 November 2013.
  51. Age-dependent patterns of bovine tuberculosis in cattle. E Brooks-Pollock et al. Veterinary Research. 2013. 44:97.
  52. Final Report of the Independent Scientific Group on Cattle TB
  53. County cattle statistics downloaded from the old AHVLA database (VETNET) on 7th March 2011.
  54. Bovine TB statistics downloaded from the AHVLA database (SAM) on 11th December 2013.
  55. Proportion of TB herds tested in England which fail when broken down by test performer
  56. Tuberculin manufacturing source and breakdown incidence rate of bovine tuberculosis in British cattle, 2005-2009. S. H. Downs et al. Journal of the British Veterinary Association. Accepted 10 Oct 2012. Published Online 16 Nov 2012.
  57. The rationalisation of the Animal Health and Veterinary Laboratories Agency (AHVLA). Written evidence submitted by the Department for Environment, Food and Rural Affairs (Defra) (AHVLA 05). February 2012.
If you would like to send to me your comments, please contact me
Last Modified 24 Feb 2017 07:52
Javascript is disabled