Quote Mining and Misrepresentation: Poor Ways to Claim Clinical Validation or Sound Science
I've addressed the matter of Patrick Holford's endorsement of IgG food intolerance tests in a previous post. Holford is a nutritionist for whom recommending these tests is an expansion of his own commercial offerings so his endorsement in understandable although his claims of "sound science" behind IgG testing are not. I'm rather more troubled by Allergy UK's endorsement of IgG testing; particularly because they presented evidence to a Select Committee on Health that they deal with 60,000 requests for assistance in a year, and say that "the number of people seeking advice had grown on average by 21% in each of the last three years". It is obvious that people turn to them in expectation of authoritative information. Allergy UK is a medical charity; people who request assistance might expect their endorsements to be thoroughly researched and a reflection of current bio-medical research and opinion.
Muriel Simmons of Allergy UK is liberally quoted by a commercial laboratory that offers direct-to-consumer testing (YorkTest): she is quoted as endorsing a number of their allergy (e.g., the multi-allergy screening test) and intolerance tests. Surprisingly, the Allergy UK website does not provide a summary of the research to support this position on their website. There is a buried reference to Whorwell's paper that investigated IgG testing and an elimination diet in the treatment of irritable bowel syndrome (IBS). Although others may have quote-mined or misrepresented the extent to which their work can be generalised, Whorwell and his co-authors responded to comments about their reported findings and were punctilious in defining the limits of a reasonable interpetation of their results:
it is entirely possible that IgG antibodies may be important in IBS, where we now know that there is an inflammatory component in some cases, whereas they may not be relevant in food intolerance in general. Furthermore, it is likely that only a subset of patients are likely to have an immuno-inflammatory basis to their condition and these might be the very individuals who respond to dietary exlusion based on IgG antibodies. This would fit with our results where only a proportion of patients responded despite all having antibodies. This, of course, limits the specificity and usefulness of the test unless such subgroups can be indentified beforehand. [Emphases added.]For several years, Allergy UK and YorkTest have made references to research that is in progress. Some of this research was published recently and addresses Dietary advice based on food-specific IgG results.
The PR release for the Allergy UK-sponsored study claims the following:
The new study...is the largest ever food intolerance study undertaken in the UK and reveals for the first time that over 60% of patients involved in the study had to endure over 3 years of NHS appointments, suffering and misdiagnosis before obtaining improvement through the food intolerance testing.Other PR-reported findings in the study reveal:
5200 people took part in the study and were treated for a wide range of mild and chronic illnesses. In addition, nearly a third of all patients received NHS treatment for over a decade without success. 32.4% of the sample showed they had suffered with their condition for over ten years before taking up a food intolerance health solution.
At the other end of the scale just 5% of patients had illnesses lasting from 1-6 months before using food intolerance and 22% were upto 35 months before seeking a non-NHS resolve via food intolerance testing.
- Over 3 out of 4 patients get better from their original symptoms.
- 68.2% of patients benefited within the first three weeks.
- 9 out 10 patients had a return of symptoms when introducing offending foods back into their diet.
- Many patients would rather have a dietary solution than taking medication.
- Patients who reported more than one condition were most likely to report improvement.
- On average, patients had symptoms for at least 10 years before taking up a food intolerance option.
I located a copy on Gillian McKeith's site (it is possibly a pre-publication copy). I was hopeful that the paper would answer some of my questions, particularly as it has been bruited about as having significant findings about IgG food intolerance testing, dietary modification and chronic illness.
I have been disappointed on a number of fronts. Other issues aside, it is bewildering that Hardman and Hart refer to the survey participants as patients: neither of the authors is medically qualified and there is no indication within the paper that those who responded to the survey are under medical care for their self-reported symptoms or illnesses. By referring to patients the authors imply that all of the participants are patients and that there may be some form of validation for their symptoms and illnesses. It is understandable that the PR release lards its text with references to the NHS, treatment, and patients but it is potentially misleading if it leads people to believe that this work is NHS endorsed or was being offered as a treatment programme.
There is no explanation as to why there are two questionnaires, nor do the authors provide a clear list of the differences between them. Participants received one survey, 3 months post-test: there was no recorded baseline of symptoms and diet. Any claims about either symptoms or diet must be retrospective. Further, we don't see the questions so it is difficult to evaluate some of the claims and findings and how the response may have been manipulated by the form of the question.
The postal survey does not tell us what the respondents were eating before their intolerance testing. If people have been speculating about food problems in relation to their illness for some time, it is possible that they may have experimented with an elimination diet that may have skewed the results of their IgG testing. E.g., if somebody had been avoiding soya for some time they would probably have a negative IgG test at that time whereas they might have a positive result with recent exposure.
Likewise, it would not be too difficult to imagine that people who are chronically tired or experiencing 'brain fog' might not prepare food from scratch on a regular basis. It is possible that the striking improvements in well-being might occur in anyone who shifts from a diet of processed food to home-prepared 'healthy' meals. The best way to test this would be to find a group of people who are chronically tired etc. but switch them to a more wholefood way of eating without any food intolerance tests. Actually, that sounds like a number of diet makeover T.V. programmes which show startling results when people change their way of eating (although, to be fair, they usually start exercising as well).
The participants in this survey had previously purchased a direct-to-consumer 113 foodSCAN test from YorkTest. They sent a small blood sample to the laboratories for testing. The testing evaluates the level of IgG in response to the panel of foodstuffs. The consumers receive a report in their levels of food-specific IgG are listed, and they are advised to avoid those foods with increased antibody levels (the red, yello and green system). The lab results are accompanied by helpful food lists and food rotation instructions. As part of the service package, the consumers have limited phone time with a nutritional adviser.
Hardman and Hart do not discuss the IgG results: it would have been interesting to know if the positive IgG results clustered around the usual suspects of wheat, milk, shellfish, citrus fruits etc. or if they were evenly spread throughout the 113 test foods. An analysis like this would be valuable in designing a placebo diet for use in any future research involving IgG-guided elimination diets.
A high number of survey participants claimed to have "rigorously followed their elimination diet" however, this depends on their subjective assessment and recollection. There is no reference to any request that participants should keep food diaries so these recollections may be unreliable. Even those participants who made a "reasonable attempt at the diet" (N.B., there is no explanation of what this means) reported "noticeable improvement". So, these findings may suggest that making a few changes, which may not necessarily reflect the IgG testing results, is effective enough to question whether it is worth the added inconvenience of implementing a rigorous elimination diet.
There is remarkably little discussion of the finding about the relationship of response to the food elimination diet and symptoms:
The information obtained from asking which was the primary condition that concerned patients was grouped into diagnostic categories. As previously mentioned this question was not asked of all patients as it was only part of the first questionnaire. Of the 2221 replies 38.0% were gastro-intestinal, 13.7% were dermatological, 10.7% were neurological, 10.1% were respiratory, 9.4% were psychological, and 6.2% were musculo-skeletal. 11.9% were categorised as ‘other’.It does have to be said that some of these results would be comparable to a placebo condition: in an IBS study, the researchers "estimated that approximately 40% of the placebo arm would report a significant improvement in symptoms". Hardman and Hart summarise their audit of the survey:
The distribution of benefit reported varied according to the medical condition of most concern...For example, 40.6% of patients reporting psychological problems as their main concern report high benefit from dieting rigorously, whereas only 21.0% of those reporting respiratory or musculo-skeletal problems as the main concern reported high benefit. [Emphasis added.]
All the measures considered were categorical and based on self reported perceptions so quantification of comparisons was not possible. However, there was consistent evidence that noticeable benefit was gained from removing offending foods from the diet. 75.8% of those that rigorously followed the recommended diet had a noticeable improvement in their condition. 68.2% of those that benefited from following the recommendations felt benefit within 3 weeks of following the diet. The survey covered subjects with a wide range of medical conditions, and it was clear that those who reported more than one condition were more likely to report noticeable improvement. 81.5% of those that dieted rigorously and reported three or more co-morbidities showed noticeable improvement in their overall condition.The authors reported some widely different findings about challenges with the 'offending' foodstuffs.
Subjects were asked specifically to say whether the result of reintroducing foods was a strong return of symptoms, a slight return of symptoms, or no change. Of the 3026 subjects that responded to the second questionnaire, 2275 (75.2%) said they had reintroduced offending foods either on purpose or by accident. 2219 of these patients also answered the question regarding the return of symptoms. 824 (37.1%) reported a strong return of symptoms, 902 (40.6%) reported a slight return of symptoms, and 493 (22.2%) reported no change. That is 77.7% reported the return of symptoms after the reintroduction of offending foods...Obviously, the participants were typically aware that they had eaten "offending foods" and it is probable that this is reflected in the results. It would have been interesting to have had a breakdown of the symptoms or conditions that were most likely to recur upon re-introduction of "offending foods" and what these foods were.
Those reporting more benefit were more likely to feel a return of symptoms after reintroducing offending foods. For those who dieted rigorously and reported high benefit, 92.3% felt a return of symptoms after reintroducing offending foods.
When people follow elimination diets, it is not unusual for them to adopt a de facto low carbohydrate way of eating. This may be particularly true when consumers are guided by the results of a food intolerance test because if they are advised to eliminate foods such as wheat, they may find it difficult to replace this with another (safe) carbohydrate source. They may be unwilling to replace (say) wheat with quinoa if they were not tested for quinoa. So, it is possible that a number of people in this survey who showed the strongest reaction to the reintroduction of foodstuffs might have been exhibiting a reaction that reflects a disturbed glucose metabolism that is related to carbohydrate restriction. Bethune and colleagues have previously reported that this phenomenon can be interpreted as food allergy.
All three of the patients described developed symptoms several hours after meals and attributed these to food allergy. Further restriction of carbohydrate intake exacerbated their problem. Symptoms continued to occur after meals and were erroneously interpreted as further evidence of their carbohydrate allergy.The researchers discuss 3 case studies and report that
[o]nce patients have a fixed belief about a cause for their symptoms, it may be difficult to persuade them to entertain an alternative diagnosis. In case 1, negative results of blinded food challenges did not dissuade the patient from her belief that food allergy was the cause of her symptoms.Because Hardman and Hart report on their audit of the YorkTest 113 foodSCAN but do not provide sufficient detail in the paper, the reader is reduced to speculating about plausible mechanisms for the results. There is no reminder that these results only cover a 3 month period, we do not know if the reported improvements persisted. The authors conclude:
The observation of a clear relationship between adherence to the diet and outcome is critical in showing that the diet is an ‘active treatment’. Similarly the fact that over three-quarters of subjects who reintroduced offending foods back into their diet, whether on purpose or by accident, showed reoccurrence of their symptoms. These two criteria are the basis for the diagnosis of ‘food intolerance’ by the laborious elimination diet process which, it appears, can be largely ‘bypassed’ by following a diet based on the results of food-specific IgG testing. The percentage of patients reporting noticeable improvement suggests that such specified elimination diets are a valid intervention in the relief of certain symptoms. The degree of success varies with the type of problem being experienced.I can not agree that these survey results support these conclusions. Without examining well-maintained food diaries, it is impossible to quantify the number of people who did alter their diet rigorously: the results indicate that even those who followed an elimination diet of some sort (that does not necessarily adhere to the IgG results) will produce results and therefore qualify as an "active treatment".
Similarly, Hardman and Hart are not entitled to claim support the diagnosis of food intolerance based on the reaction to the reintroduction of "offending foods" because the participants did not attempt a blinded food challenge. Young and colleagues published a classic population study of food intolerance where there was a perceived prevalence of food intolerance of almost 20%, but the clinically definitive double blind placebo food challenge test indicated a rate of less than 2%.
Hardman and Hart acknowledge the equivocal status of IgG as a marker for food intolerance in the introduction:
the exact role of IgG antibodies as markers of food intolerance in general is not clear. IgG antibodies to food antigens are often present in healthy individuals and are generally considered to be part of the normal immune response to food allergens [refs].However, they do not comment any further on this in their discussion of the findings of this audit. I hope that this reticence is continued by others and that there is no attempt to misrepresent this audit as supportive of the "sound science" of the IgG diagnosis of food intolerance or the "clinically validated" virtues of an IgG-guided elimination for the treatment of a variety of symptoms and chronic medical conditions.
In the UK we have very poor provision of allergy diagnosis and management. People report that GPs are wary of the validity of allergy and intolerance and that it is very difficult to obtain a referral to the limited NHS services that do exist. The cause of appropriate allergy provision may be damaged by its association with the dubious science of food intolerance testing.
A diagnosis of food allergy should be confirmed by a clinician with an understanding of the multisystem, polysymptomatic patterns of illness involved. A careful history will usually reveal these patterns and suggest a diagnosis that can be made on clinical grounds. Clinical allergists rely upon an interpretation of the history and the tests; however, there is no wholly definitive laboratory test because an interpretation of the results relies upon the clinician's understanding of somebody's complex allergic responses. E.g., a clinician may need to interpret the individual and relative levels of several antibody series, such as IgE, IgG, IgA and IgM. There may be complex shifts in the distribution of IgE, IgG, IgA and IgM that indicates immune activity in response to antigen loading. Patients who have severe or prolonged food allergy may have depressed levels of IgM and IgG; they may also have lower white cell counts. If IgE levels are low, this may compromise the clinical value of both skin testing and IgE RAST. Overall, a high IgG might be associated with an immune-mediated diseases, and reflect increased antibody production, possibly against unknown antigens.
The status of IgG testing in diagnosing food intolerance is controversial. It is premature at best and boarding on deceptive to claim that IgG testing for food intolerance is "clinically validated" or has "sound science" behind it. There is so much mis-information on the internet that it would be helpful if the British Society for Allergy and Clinical Immunology (BSACI) would issue a position statement on the matter (however, Prof. Kay, a former president of BSACI has made some lack of scientific support for this use of IgG testing). Both BSACI and Allergy UK are core members of the National Allergy Strategy Group. Membership of such bodies adds to the status and authority of Allergy UK and their endorsements. It is unfortunate that Allergy UK gives the imprimatur of respectability to IgG testing in the diagnosis of food intolerance.
*Allergy UK and YorkTest have been flagging up this audit for some time. I've seen references to more than 7000 participants, 5000 participants, and for the relevant claim about unsuccessful treatment by the NHS, this is now modified further:
The findings, conducted by York University, suggest a growing dissatisfaction among patients who wasted time and money on treatment on the NHS. The study using 3219 patients and published in the February issue of Nutrition and Food Science looked at a range of mild and chronic illnesses including migraine / headache, skin symptoms, IBS, and digestive problems.Why have neither Allergy UK nor YorkTest made the full detail of this study available. According to the published paper, this was a questionnaire with scaled category responses. However, from the current claims, it now seems as if the participants were invited to complain, or spontaneously complained about their NHS treatment. This study is very poor; the claims being made for it are over-blown and disproportionate. Readers can not possibly assess these claims of NHS mis-treatment or mis-guided treatment if we have no way of discovering which treatment modalities were attempted.
YorkTest also has the chutzpah to promote an obscure petition to provide free food intolerance tests on the NHS. Their pious hope is that the petition:
re-enforces the view that the NHS should put peoples health at the forefront of its health service strategy. If the petition takes off, then it could make the health minister sit up and take action in saving the health service thousands of pounds whilst freeing up doctors valuable time.There is no decent science behind IgG food intolerance testing. Leaving that issue aside, YorkTest is now claiming that an IgG-guided elimination diet is a "credible and fast alternative"? Unusually enough, words fail me. [Edited: It is possible that somebody associated with YorkTest instigated this e-petition.]
At best this petition might make the stakeholders of the health service to look at what is best for the people it aims to help and accept it as a credible and fast alternative to dishing out a diet of pills and potions. [I have not sic'd the errors.]
It is time to publish the questionnaires and the full data from this audit. Without the full information it is impossible to evaluate the claims that are being made and that is an unsatisfactory situation that may well lead to many mis-understandings.
Labels: allergen avoidance, allergy testing, Allergy UK, BSACI, chronic illness, elimination diet, food allergies, food allergy, food intolerance, Hardman, Hart, Holford, IgG, intolerance testing, YorkTest