Tag Archives: Soybean

On self experimentation, soy, and memory

24 Mar

When I ask my students to read a research paper, they often argue that “the sample size was too small.” Typically, students who are new to statistical analysis do not understand the mathematics of sample size, and this becomes a reflexive criticism. If they don’t like the result of a study they argue that a larger sample was needed.

This is why students are often shocked when I talk about the value of single subject research and argue that it is possible to gain information when the sample size is N = 1. One of the advantages of single subject research is that you can control for background characteristics, such as genetics, because these traits remain constant over the individual.

One variant of single subject design is self experimentation, as advocated by the always interesting Seth Roberts. Roberts recently ran an experiment on himself where he found, in his words, that tofu made him “stupid.” To his credit Roberts recently linked to a post by  Alex Chernjavsky which reached the opposite conclusion:

 “The results were not consistent with the hypothesis that eating soy is harmful to brain function.  Surprisingly, my scores became significantly faster during the study.”

One possibility for the different results is that self-experimental trials are often not blinded, that is the subjects frequently know which treatment they are receiving and unconscious bias might play a role in the results. For example,  Roberts is an advocate of a meat based diet while Chernjavsky is a vegan. Chernjavsky, himself suggests:

“I don’t know why my results are inconsistent with prior work.  Perhaps people differ in their sensitivity to soy.  Or perhaps I’ve been eating so much soy for so long that I’ve made myself resistant to any changes that might result from relatively short-term fluctuations in level of soy consumption.”

I have three suggestions:

1. While obviously difficult to implement, self experimentation would be more persuasive if the the experimenter was blinded to the different treatments.

2. We should develop better methods for combining the results of many single subject and self experimental designs. Perhaps some form of Bayesian analysis.

3. The results of single subject and self experimental designs should be replicated with larger sample sizes (although they don’t have to be very large) using within subject designs. This will help guarantee the reliability, validity, and  generalizability of the results.

Here is Dr. Greger on the brain effects of tofu:

Enhanced by Zemanta
%d bloggers like this: