A special course for learning and research.
Erik Stolper, Lex Rutten, Roland Lugten, Rob Barthels
A Dutch homeopathic research group, the Committee of Methods and Validation, has developed a post-graduate course - the Materia Medica Validation - where successful cases can be compared. The group made a protocol to get reliable data and to build up validated pictures of homeopathic medicines. In this article we describe the procedure of collecting data. Our Materia Medica Validation can be considered as a special kind of meta-analysis. Therefore we describe first the procedure of meta-analysing.
The word "meta-analysis", as used in medicine, has a special significance. It means the pooling process of data collected from a systematic review. In fact it is all statistics, it is the quantitative part of a systematic review. The qualitative part of the review is the first and the most important step. In that phase the inquiry focuses on the gathering of articles related to a special topic and on the criteria one needs to judge the scientific quality of the research. At the end you have a number of articles, complying with the conditions you have set.
The next step is the question: are the data comparable? Is it right when we consider all the data as the total result of one single experiment? Again there are special criteria useful to get an idea about the answer.
The third and final step is the application of statistical methods aimed to assess the probability that a special intervention has more effects than could be expected from chance or after a placebo. This final step is called meta-analysis.
There is an aid to memorise the steps in this process: PICO. Which Patients, what kind of Intervention, what is the Comparison we make and what is the Outcome, what do we want to know at the end of the process? It all serves the final question: how likely is the relation between an intervention - a medicine, an operation or another therapeutic action - and a change afterwards in a special group of patients?
With this knowledge we consider the possibility to do the same in case of a homeopathic intervention. We concentrate on the question: when a number of cases is cured after the same homeopathic treatment, is it right to compare the single cases to draw conclusions about the picture of that special medicine? Can we learn from the individual stories of patients cured after taking the same medicine?
In Holland we have some experience with this topic. We collect the data of cured cases and consider them from two viewpoints. The first one we called the horizontal level, to arrange all symptoms together. The other one is the more-dimensional level, to formulate a common opinion about the heart, the nucleus of the medicine.
Since 1996 we organise a kind of a post-graduate course, called Materia Medica validation. It takes two days, one in spring and one in autumn. By mailing we invite our colleagues to select one or more successful cases cured by one medicine. During the day we follow an elaborate procedure to make sure that only cured cases will be accepted and to optimise the possibilities to learn from each case. We create small groups where each participant presents his cases. After clarifying the case and how the medicine was perceived, the group judges the effect of the medicine. It is not our purpose to criticize the colleagues but it is only to exclude the possibility that other interventions could also be responsible for the positive effects. In 1996 eighty colleagues agreed on consensus about criteria to consider a case successful, called the VHAN-consensus. It is derived from the Glasgow homeopathic scale but is somewhat more detailed. This consensus is not validated till now.
The next step in the process of the group is to identify the symptoms responsible for the choice of the medicine and other clear symptoms. We developed a special form to fill in the symptoms and other important data. We collect the forms at a central point and put the data in a spreadsheet program. After the course two members of our committee work up the data in special tables. We put symptoms, words with semantically comparable meanings together in the same category. We try to discover patterns, typical for the medicine. There’s a lot of bias possible during the process of collecting data but this phase is critical. Which words, which kind of symptoms belong together? Then we count the symptoms. In the past we have thought and read a lot about the significance of these figures. After all we have chosen for the statistical notion Likelihood Ratio, LR. In other articles on this site you will find more information about the significance of LR.
In the third step we try to get a sort of overall idea about the medicine, the background, the depth, the essence. With that purpose we have a central meeting to discuss the common experience. Only the members with cured and validated cases can talk about their ideas. The question is then: "what was your first impression from the patient, you brought in today?" After discussing the answers we invite the colleagues to tell more about key ideas of the patients mind, and the somatic problems. Next we listen to some very clear cases and ask each other whether some point may have been forgotten. And are there important differences between today’s findings and what we already know from our books? In a second meeting there is the possibility to criticize this consensus and to adjust. After we agree with the text, we make it ready for publication.
In fact this whole process is a form of meta-analysis, looking from a distance, on a higher level. Hovering all together in the same helicopter above our patients we get a more complete picture of the medicine. By means of deliberation, dealing with our experiences, weighing our ideas, recognition of observations we try to find a reliable picture of the medicine. What kind of people belongs to this medicine, what personalities, what backgrounds, fixed ideas and typical qualities.
The chairman of the meeting is responsible for the process; everybody must have the chance to tell his or her experience with and ideas about the medicine on the basis of the validated case. After the meeting the chairman makes a sort of consensus of opinion from what was brought up. He sends it to all colleagues attending the meeting.
Up till now this procedure is in our opinion useful to get reliable data, but there is a lot of bias, risk possible. The most important is the confirmation bias: we only bring successful cases that correspond with the MM of the books or only select symptoms we already know. And we ignore all the failed cases with almost the same symptoms!
The second type of bias is the number of successful cases brought in. It seems very difficult for most doctors to identify good cases in their archives, in addition to the low percentage success we have in reality. The average number is 10 and the highest is 16 at our meetings, apart from 6 to 12 participants without a case. Maybe it is enough for making a consensus but this number is too small for counting symptoms and drawing inferences from it. It is also impossible to collect all symptoms from the cases. So we have to limit.
Another important source of bias is the semantic problem. Each patient uses his own words, his own expressions. The doctor translates it automatically in synonyms with sometimes a different meaning, for instance in words of Kent’s repertory, a language used two centuries ago.
Unfortunately there are more problems. His ideas about the patient could be incorrect. The doctor is listening to his patient and considers the patient as a loquacious lady. But the patient sitting in front of the doctor has her own thoughts: "The doctor seems to know nothing by himself. I must talk the whole consultation. Everything I said is apparently important, he writes down, looks up in big books and on a flat screen, observes me in silence and says nothing". No wonder the patient talks and talks, but in fact she is not loquacious.
There can be differences in judging what’s normal or not, e.g. cravings and aversions. Dutch people drink a lot of coffee. How can we determine whether there is a craving for coffee? Must we count the cups patient drink or would like to drink? There can be great differences between countries. What in Holland is considered as a slight preference, can be an addiction in England.
In Holland we have a lot of experience with this combination of learning and research. It stimulated the course-attendants to share knowledge and experience. The active participation is the best guarantee for a successful course. And the data we obtained stimulated us also to develop scientific instruments, useful for improving the homeopathic method.