Note: This website, including this article, is a work in progress.
Summary
Creatine is a naturally occurring compound found in the human body. As a supplement, it’s widely considered to be effective at boosting athletic performance, and there is some evidence that creatine can also act as a cognitive enhancer. I conducted a two-month-long self-blinded, placebo-controlled experiment to see if creatine supplementation (5 grams daily, taken in the morning) would improve my reaction time (RT). I chose creatine because it’s known to be safe, and particularly because I’m probably deficient in it due to my longstanding vegan diet. Someone starting from a deficient state might be expected to experience a larger effect from supplementation. I chose RT as a metric because multiple studies have demonstrated a correlation between RT and cognitive ability. I found that my RT was slightly faster when I was taking creatine vs. placebo, though I haven’t yet performed the necessary analysis to determine whether the effect is significant. This research project is part of my long-term goal to refine and validate reaction-time testing as a tool for assessing the cognitive benefits of various interventions.
Study design and results
My study was divided into six periods of ten days each, beginning with a baseline phase and concluding with a washout period. Between these bookends, the design consisted of a placebo period followed by creatine supplementation, and then another placebo/creatine pair. The capsules I took each morning were disguised so that I didn’t know whether I was taking the creatine or the placebo. The creatine was in the form of ten 500mg capsules (for a total of 5 grams of creatine monohydrate) sourced from Life Extension. The placebo capsules contained allulose and were made to be indistinguishable from the creatine capsules. I measured my reaction time three times each day, roughly in the late morning, mid-afternoon, and evening.
The following graphs show top-level results and include means from all 176 testing sessions (I missed four sessions out of a planned 180). As described in more detail further below, each testing session took about three minutes and consisted of 35 individual trials that measured my reaction time in response to a stimulus presented on the computer.
In the two graphs below, each data point represents the mean RT from a testing session. The shorter lines represent linear regression fits to the data in each phase, and the longer green line is a linear regression fit to all the data points. Note that lower values are better in the sense that they represent faster reaction times.
I’ve made the data available as a CSV file, aggregated by testing session. The raw data (including each individual trial, but without incorrect responses) is here.
The graph below depicts the same data as above but without the regression lines. Instead, the solid green line represents a locally estimated scatterplot smoothing (loess) curve fitted to all the data points:
The following table summarizes the mean RTs. The left side shows the data for all the testing sessions, including morning, afternoon, and evening. The right side excludes the data from the evening sessions. I took the creatine supplements first thing in the morning, and it’s possible that the effect wore off by evening time. In this context, the number of observations corresponds to the number of testing sessions using my reaction-time app.
| Study Phase | All Testing Sessions Included (Morning, Afternoon, and Evening) |
Morning and Afternoon Sessions Only (No Evening Sessions) |
||
|---|---|---|---|---|
| Mean RT | # of Observations | Mean RT | # of Observations | |
| Baseline | 313.4 | 28 | 313.0 | 19 |
| Placebo 1 | 313.4 | 30 | 314.3 | 19 |
| Creatine 1 | 311.8 | 30 | 310.8 | 20 |
| Placebo 2 | 313.9 | 29 | 314.0 | 19 |
| Creatine 2 | 312.9 | 29 | 311.1 | 19 |
| Washout | 313.4 | 30 | 312.6 | 20 |
| All phases | 313.1 | 176 | 312.6 | 116 |
The following two graphs show the RT data without including the evening sessions (so morning and afternoon sessions only). The effect seems clearer here, though I don’t know if it’s significant:
The graph below depicts the same data as the one above, but the regression lines have been replaced by a loess curve:
So… did the creatine actually have an effect?
Good question, and one that still needs to be decided. The graphs above depict results that seem consistent with a small effect of creatine. I’m not proficient in statistics, and I don’t know what type of statistical analysis should be done to determine whether the effect is larger than what one might expect from chance. Also, in retrospect, the study phases should probably have continued longer than ten days. If I do a follow-up study on creatine, I will run it longer (though how much longer is another question that should be considered).
How was reaction time measured?
I measured my reaction time using a Windows app that I wrote. The specific testing protocol was developed by the late psychologist (and noted self-experimenter) Seth Roberts. Each testing session consists of 35 separate trials, where the app displays a random digit between 2 and 8, inclusive. The subject’s task is to hit the corresponding key as quickly as possible. Incorrect responses are not counted, and each digit is presented five times during a session. This type of protocol is known as a “choice reaction time test”, meaning that the experimental subject is required to select and execute the correct response from multiple options when presented with various stimuli. This type of test is often used to evaluate decision-making speed and cognitive processing abilities. Each testing session takes approximately three minutes to perform.
At some point soon, I will post the code on this site and release it under a creative commons license. I just need to make some minor changes to the script first.
Why measure RT?
There is a well-established correlation between RT and measures of cognitive ability, including notably IQ. These correlations stem from studies where the RT and cognitive measures are averaged across populations of subjects. It’s perhaps less established that RT can be used to track changes in cognitive performance within a single individual over time, but it seems plausible – and Roberts theorized that this was a valid approach. His own “n=1” experiments further reinforced his view.
Before his untimely death in 2014, Seth Roberts was promoting the idea that reaction time could be used by self-experimenters to track their brain health and to test the effects of various interventions. His plan was to help establish a distributed community of people who would all conduct self-experiments using RT as a metric. I collaborated with him on studies of caffeine, soy, and flaxseed oil, but I stopped working on it after he died. However, after a long hiatus, I was prompted to revisit this project after I read an interesting post on Scott Alexander’s Substack page.
Alexander posted a long list of putative nootropics ranked by their effectiveness, as gauged by a survey of users. Asking people, “Did substance X improve your cognitive performance?” may produce some useful leads on what works and doesn’t work, but my goal was to find an assessment method that is more objective, where changes in reaction time can be used to estimate differences in effectiveness.
Have other self-experimenters done studies on RT?
Yes, they have. Juraj Karpis, Richard Sprague, and Eri Gentry are three other self-experimenters who worked with the late Seth Roberts and performed studies using his RT metric.
Juraj tested the effects of eating two cans of sardines in the morning, and he found a small improvement in his RT. He also investigated the effect of a vegan diet and observed a worsening in his RT. An earlier, similar diet experiment also demonstrated the same effect (the linked page is written in Slovak, but your browser can probably translate it automatically). Note that the RT data is presented in Section 5, about halfway down the page. Juraj also used his RT test in a blinded experiment of caffeinated coffee as compared to decaffeinated. He found that that the caffeinated coffee made him faster.
Richard found an intriguing effect of simvastatin (a drug to lower cholesterol). And, similar to Juraj’s findings, Richard determined that fish oil makes him faster on the RT test. Richard also has a video where he talks about his RT testing.
Eri Gentry, Greg Biggers, and their collaborators organized a crowdsourced study (titled “Butter Mind”) to examine the effects of butter consumption on reaction time. A total of 65 participants completed the study. Gentry discusses the origin of the study in this video, and the results are outlined on the Quantified Self website (retrieved from the Internet Archive). Briefly, the consumption of butter was associated with a significant improvement in RT scores, as compared to no intervention or consuming coconut oil.
I performed earlier self-experiments on caffeine (made me faster), soy consumption (no effect) and flaxseed oil (also no effect).
Instead of measuring RT, why not track productivity, ability to concentrate, or other more direct metrics of cognitive performance?
I actually tried doing this, but I found it surprisingly difficult. For a period of about three weeks, I would sit down at the end of the day and rate myself on my ability to concentrate and on my productivity that day. I used a scale of one to ten. I discovered that it was hard to come up with a number. I’d reflect on various events from the day, and my thinking would go something like this: “Well, I did complete that one project at work, but then I also forgot (again) to make a dentist appointment. And I got caught daydreaming in a boring meeting. On the other hand, I finally figured out the subtle bug in the Python script I’ve been developing”. In short, most days were about the same overall (with some positive achievements as well as some areas where I was deficient), and the process of assigning ratings seemed arbitrary and highly subjective. I gave up.
How was the self-blinding done?
I bought some large, empty, opaque capsules from Amazon, and I placed the creatine capsules inside the larger ones. The outer capsules were packed with a small amount of inert microcrystalline cellulose to immobilize the inner creatine capsule (otherwise, it would bounce around inside).
The placebo capsules were filled with allulose, a low-calorie sweetener that I use in my morning coffee. I originally tried to use microcrystalline cellulose, but it was very lightweight, and I could easily tell the difference between an active capsule and the placebo. The capsules filled with allulose were roughly the same weight as the creatine-containing ones. The placebos and the active capsules were placed in separate containers with folded-up notes inside that identified the contents. My wife assisted in randomizing the containers so that I didn’t know which one was which.
Why study creatine?
[This part still remains to be written.]
More about self-experimentation and personal science
- The late experimental psychologist Seth Roberts was one of the pioneers and popularizers of self-experimentation. Here is a video of him giving a talk on how he learned to improve his sleep and his mood through self-experimentation.
- Seth Roberts died in 2014. His blog isn’t available online anymore, but you can view it through the Wayback Machine. Also, this large PDF file contains a complete record of all his blog posts and the comments on those posts.
- The Slime Mold Time Mold bloggers discuss n=1 experiments
- Adam Mastroianni: “An invitation to a secret society“
- Richard Sprague’s Personal Science Substack site
- Gary Wolf is working on a book titled, Personal Science: Learning to Observe . It doesn’t seem to be available yet, but Richard Sprague reviewed a draft of it.
- The Personal Science Wiki
More links coming soon.
Seeking collaborators
If any of this sounds interesting to you, I’d love to collaborate! Please contact me.