First posted on Friday, April 10, 2009 at northerndoctor.com
Hill PB, Hoare J, Lau-Gillard P, Rybnicek J, Mathie RT. Pilot study of the effect of individualised
homeopathy on the pruritus associated with atopic dermatitis in dogs. Vet Rec. 2009. 164(12):364-70
I am not a vet but I enjoy sniffing around The Veterinary Record on a weekly basis. After reading this article I was found wandering around the house guffawing loudly in pleasure at the insanity of the world. This is probably not the professional approach to critical appraisal so I turned to Trish for help.
Trisha Greenhalgh on ‘the science of “trashing” papers’ suggests that :
‘Strictly speaking if you are going to trash a paper, you should do so before you even look at the
This seems to be one of the most singularly useful and regularly ignored pieces of advice in the whole of critical appraisal. Ultimately, if the methodology of the trial is rubbish then stop reading. Stick your fingers in your ears and go nahnahnah. So given Trisha G’s comments let us consider the methodology for a moment. They took a load of itchy scratchy dogs and gave ‘em homeopathy. There were two phases:
Open phase. Owners with itchy pooches were invited to take part. There were 20 dogs in this phase and they were given individualised homeopathic remedies by a veterinary homeopath. They had their pruritus scores measured on a validated scale. If the owners felt there had been a substantial response to homeopathy they were invited to take part in Phase 2. Only 5 dogs showed any response but dog 15 was euthanased for an unrelated illness and dog 6 got better.
Blinded, randomised, placebo-controlled phase. There were 3 dogs left in this phase. This was actually a cross-over study where dogs received both a placebo and a homeopathic remedy in random order. They had their pruritus scores measured again during placebo and homeopathy phases. Why could they not just put dogs straight into Phase 2 and what purpose does the open phase possibly have? The main issue that screams at me from this is bias. BIAS! Volunteers are never an ideal sample for any trial but they have ratcheted it up another notch by further sub-selecting those that think there is an effect. There is also an issue with a cross-over design in a condition that is not stable. My understanding of atopic canine dermatitis is that is a disease that will wax and wane.
However, one of the few advantages of only having 20 subjects is that it is possible to give details of the doggie themselves. Whatever the concerns with the methodology this paper might be worth reading for these vital scientific details alone. Some highlights: dog 4 had an aversion to onions, dog 11 was very sensitive but did not console his owner, dog 15 has loud flatus when excited (who doesn’t) and dog 16 fears thunderstorms, desires chicken and is clairvoyant (I’m not making this up).Oh, one other thing. This study received funding from the British Homeopathic Society. Poor sampling, massive bias introduced systematically into the methodology, inadequate numbers and conflict of interest issues. Amusing doggie vignettes aside; my fingers are now in my ears.
The Veterinary Record has been publishing weekly for vets since 1888. The British Veterinary Association and the Vet Record are the vet equivalent of the BMA and the BMJ. One has to wonder what on earth happened to the peer-review process here.
Interestingly, the provision of homeopathy to animals is very well-regulated – it can only be provided by gen-u-ine vets. This is both reassuring and alarming in equal measure. It might ensure that animals do not receive ineffective therapies for serious illnesses but it does professionalise one of the most scientifically ludicrous alternative treatments in history. Homeopaths for humans could claim (although inexplicably they tend not to) that they are providing some kind of psychosocial intervention for the benefit of patients. After all, as Bob Hoskins once said: it’s good to talk. Of course, this potential benefit is somewhat diluted for vets who are not actually Dr Dolittle himself. What is left is a sugar pill coated with an active ingredient diluted to 200C. My advice is to cut out the middle dog and give the psychological intervention directly to the owner.
The British Homeopathic Association couldn’t resist trumpeting this trial and released a press release.
…a small, rigorously designed, research study at the University of Bristol’s Department of Clinical Veterinary Science.
The other four dogs that responded well in this first phase were then put forward into a blinded randomised trial in which they received their homeopathic prescription at some times and placebo atother times. The three dogs that completed this phase of the study improved more with the active remedy than with placebo, and owners were able to distinguish correctly which pill was which. As mentioned in the methodology there was just three dogs in Phase 2. If there was a 50/50 chance of the owners guessing correctly then there would be a one in eight chance (12.5%) of getting the results by chance alone. Statistical significance it ain’t. It is not too difficult to imagine tossing a coin three times and getting three heads in a row.
Dr Peter Hill, who was lead clinician on the study, said “These preliminary data indicate the need for a large randomised controlled trial of homeopathy in canine atopic dermatitis.”
In no way does this study constitute any evidence that there is some effect of homeopathy on pruritus in dogs. It certainly does not justify the “need for a large randomised controlled trial of homeopathy in canine atopic dermatitis.”
The discussion is a masterpiece of obfuscation. The authors trot out some miserable canards as they grasp for explanations as to why the homeopathy didn’t reach the success rate of ’60-70 per cent predicted’. The dogs remained on the immunosuppressants corticosteroids and ciclosporin and this interferes with the action of the homeopathic remedy. Because the cases were volunteers these were cases which were more difficult to treat. These cases were from a referral population so were more severe than usual.
They finally suggest:
“If the beneficial responses were due to spontaneous recoveries and chance occurrence, the failure rate would merely reflect the fact that homeopathy is not an effective treatment for canine atopic dermatitis.”
Bingo. The authors also seem to be confused about their motivations for the study. On one hand: The authors’ interpretation of the results is that they provide data to justify a larger study to determine whether the findings are repeatable.
Two paragraphs later and they are suggesting:
…the authors consider that the overall success or failure rate in this study is somewhat irrelevant. The
objective of this study was to determine whether homeopathic remedies appeared to be beneficial in
enough cases of canine atopic dermatitis to justify a larger trial. Even with a cautious interpretation,
the preliminary data appear to support that view…
This is all a touch circular. Surely the way to establish benefit is to conduct a proper randomised trial in the first instance. We are right back where we started. Labelling this study as a ‘pilot’ is a ruse to throw the scent off the appraisal hounds. If I may paraphrase Blackadder, this study has about as much power as an asthmatic ant with some heavy shopping. It is methodologically flawed with no reasonable conclusions.
The British Veterinary Voodoo Society have the measure of those that would pervert the evidence process.
Clinical trials are still in their early stages, however we are confident that by performing a sufficient
number of small, poorly-controlled investigations we will easily generate enough p<0.05 outcomes
to be able to claim with absolute assurance that the method is well proven by properly conducted
So why publish it? You could drive a coach and horses through the holes in the methodology. I usually treat with deep suspicion any published ‘pilot’ trials. Pilots might be essential pieces of work to facilitate research projects but rarely justify independent publication. Imagine for one minute if this was a Big Pharma sponsored ‘pilot’ study. It wouldn’t get past the editor’s shredder. Yet alternative medicine seems to have a special dispensation to exert influence over journals of which Big Pharma can but dream.Read More