-
Notifications
You must be signed in to change notification settings - Fork 21
/
Copy pathLLLT.page
331 lines (233 loc) · 19.8 KB
/
LLLT.page
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
---
title: 2013 LLLT self-experiment
description: LLLT user's blind trial: positive results
tags: psychology, experiments, statistics
created: 20 Dec 2013
status: finished
belief: unlikely
...
> A short randomized & blinded self-experiment on near-infrared LED light stimulation of one's brain yields statistically-significant dose-related improvements to 4 measures of cognitive & motor performance. Concerns include whether the blinding succeeded and why the results are *so* good.
[Low level laser therapy](!Wikipedia) (LLLT) is the medical practice of shining infrared/visible light of particular wavelengths on body parts for potential benefits ranging from reduction of inflammation to pain-relief to faster healing. Despite the name, it's generally done with arrays of LEDs since they are vastly cheaper and as good. LLLT seems to deliver real benefits in some applications to the body, but it remains an open question why exactly it works, since there is no obvious reason that shining some light on body parts would do anything at all much less help, and whether it would have any effects on one's brain. (One theory is that light of the specific frequency is absorbed by an enzyme involved in synthesizing ATP, [cytochrome c oxidase](!Wikipedia), and the extra ATP is responsible for the broad benefits.)
There have been some small human neurological studies (many with severe methodological limitations) with generally positive results; they are reviewed in [Chung et al 2012](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3288797/ "The Nuts and Bolts of Low-level Laser (Light) Therapy"), [Rojas & Gonzalez-Lima 2013](/docs/nootropics/2013-rojas.pdf "Neurological and psychological applications of transcranial lasers and LEDs"), [Gonzalez-Lima & Barrett 2014](http://journal.frontiersin.org/Journal/10.3389/fnsys.2014.00036/full "Augmentation of cognitive brain functions with transcranial lasers"). On the plus side, the non-brain studies indicate minimal risk of harm or negative side-effects (as do the studies in Rojas & Gonzalez-Lima 2013), and LED arrays emitting infrared light near the appropriate wavelengths are available for as low as $15 since they are manufactured in bulk to illuminate outdoor scenes for infrared cameras. So one can try out LLLT safely & cheaply, and some people have done so.
At the time of this analysis, I knew of no reported studies examining LLLT's effect on reaction-time.
In March 2014, I learned of the small experiment [Barrett & Gonzalez-Lima 2013](http://filosofi.ca/wordpress/wp-content/uploads/2013/10/2013-transcranial-infrared-laser-stimulaiton-produces-beneficial-cognitive-and-emotional-effects-in-humans.pdf "Transcranial infrared laser stimulation produces beneficial cognitive and emotional effects in humans") which reports improvement in reaction-time on the [PVT](!Wikipedia "Psychomotor vigilance task") & another task, and an improvement in mood.
EnLilaSko did not record moods, but his reaction-time data is consistent with Barrett & Gonzalez-Lima 2013.
# Experiment
The Longecity user [Nattzor](http://www.longecity.org/forum/user/20393-nattzor/) (Reddit: [EnLilaSko](http://www.reddit.com/user/EnLilaSko)), a male Swedish Caucasian college student, attracted by the discussion in [Lostfalco's Longecity thread on LLLT & other topics](http://www.longecity.org/forum/topic/63228-lostfalcos-extensive-nootropic-experiments/ "Lostfalco's Extensive Nootropic Experiments"), purchased a ["Details about 48 LED illuminator light CCTV IR Infrared Night Vision"](http://www.ebay.co.uk/itm/48-LED-illuminator-light-CCTV-IR-Infrared-Night-Vision-/180410200537) (~$13; 850nm^[808nm is more common in the research literature, but 850nm IR LEDs are easier to get.]) to run his own self-experiment testing reaction-time.
Specifically, he did a _n_=40 with two pairs of randomized blocks (result: ABBA) from 16 September 2013 - 17 December 2013 (with occasional breaks).
His blinding procedure:
> I covered my eyes (to not see the lamp), ears (to not hear if it's plugged in or not), hands (to not feel heat from the lamp) and used a water bag between the lamp and skin (to not feel heat). I asked my dad to walk into the room when I had prepared everything and to turn it on or not. The first 2 stages were done for about 12 minutes with about 1 minute per spot (I counted in my head, obviously not optimal), the last two stages were for 2 minutes (24 min total).
Randomization was done with the assistance of a second party:
> What I do: Sit in a room with the lamp, literally blinded, headphones on, etc, then he comes in and either turns it on or doesn't (I don't know which he does), then he comes back and turn it off, does the same for the 10 day periods, then change (at least how we do now).
> The tests were a battery on Quantified-Mind consisting of Choice Reaction Time (testing reaction time), visual matching (testing visual perception), sorting (testing executive function) and finger tapping (testing motor skills). Something obviously dumb from my part was not to check what areas of the brain that are related to those parts. If I have used LLLT on the front of my head and the function is related to an area at the back of the brain it's obviously useless. I mainly did at the forehead and 2 spots back on the head.
Varying dose:
> Some factors that are probably making the results fucked up is that the first two blocks were done with about 3 days rest between. The third phase was done maybe a month (probably more) after that (with double time, still placebo though) and then the fourth phase was done about a month after that, with no school at all (more focused, still double time). So it's either because the long wait or that I respond waaaay better to LLLT with 2 minutes / place rather than 1 minute / place. I think that fucked up things hard, but can't fix that now (if I don't re-do the experiment).
>
> ... [applied to:] F3, F4, along the hairline, on the forehead and P3 and P4^[See [this chart of skull positions](http://www.intechopen.com/source/html/17370/media/image3.jpeg) for an idea of the rough locations.]
Measurements:
> The tests were a battery on [Quantified-Mind](http://www.quantified-mind.com/) consisting of Choice Reaction Time (testing reaction time)^["Choice Reaction Time" is not, as it sounds like, measuring number of milliseconds, but rather some sort of video-game-like score.], visual matching (testing visual perception), sorting (testing executive function) and finger tapping (testing motor skills). Something obviously dumb from my part was not to check what areas of the brain that are related to those parts. If I have used LLLT on the front of my head and the function is related to an area at the back of the brain it's obviously useless. I mainly did at the forehead and 2 spots back on the head.
# Analysis
## Descriptive
He provided the data prior to his analysis, and I did my own. The basics:
~~~{.R}
lllt <- read.csv("http://www.gwern.net/docs/nootropics/2013-nattzor-lllt.csv")
lllt$LLLT <- as.logical(lllt$LLLT)
summary(lllt)
LLLT Choice.Reaction.Time Visual.Matching Sorting Finger.Tapping
Mode :logical Min. :506 Min. :554 Min. :592 Min. :504
FALSE:20 1st Qu.:543 1st Qu.:581 1st Qu.:606 1st Qu.:542
TRUE :20 Median :566 Median :584 Median :614 Median :560
NA's :0 Mean :564 Mean :586 Mean :616 Mean :560
3rd Qu.:583 3rd Qu.:593 3rd Qu.:622 3rd Qu.:583
Max. :609 Max. :612 Max. :645 Max. :610
cor(lllt[-1])
Choice.Reaction.Time Visual.Matching Sorting Finger.Tapping
Choice.Reaction.Time
Visual.Matching 0.4266
Sorting 0.6576 0.7173
Finger.Tapping 0.7982 0.5185 0.7070
~~~
As one would expect from the descriptions, the _r_ correlations are all high and the same sign, indicating that they vary together a lot. (This also means it may be dangerous to use a set of independent _t_-tests since the _p_-values and standard errors could be all wrong, so one should use [multivariate linear model](!Wikipedia) + [MANOVA](!Wikipedia).)
~~~{.R}
lllt$time <- 1:40
library(reshape2)
df.melt <- melt(lllt, id.vars=c('time', 'LLLT'))
~~~
All data, colored by test type:
All data, colored by LLLT-affected:
Combined (LLLT dose against smoothed performance curves; code courtesy of [iGotMyPhdInThis](http://www.reddit.com/r/Nootropics/comments/1tibnk/an_lllt_selfexperiment/ce8og3f?context=3)):
The third (the second "A") group of data looks *very* different from the other two groups, as not just are the scores all high, but they're also very narrowly bunched in an ascending line compared to the really spread out second group or even the first group. What's going on there? Pretty anomalous. This is at least [partially related to the increased dose](#continuous-dose) Nattzor used, but I feel that still doesn't explain everything like why it's steeply increasing over time or why the variance seems to narrow drastically.
## Modeling
### Binary dose
At first I assumed that the LLLT doses were the same in all time periods, so I did a straight multivariate regression on a binary variable:
~~~{.R}
summary(lm(cbind(Choice.Reaction.Time, Visual.Matching, Sorting, Finger.Tapping) ~ LLLT, data=lllt))
...
Response Choice.Reaction.Time :
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 544.45 4.03 135.15 < 2e-16
LLLT 39.20 5.70 6.88 3.6e-08
Response Visual.Matching :
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 580.75 2.69 216.22 <2e-16
LLLT 9.65 3.80 2.54 0.015
Response Sorting :
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 606.65 2.50 242.2 < 2e-16
LLLT 18.05 3.54 5.1 9.9e-06
Response Finger.Tapping :
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 537.40 3.67 146.29 < 2e-16
LLLT 46.10 5.20 8.87 8.5e-11
p.adjust(c(3.6e-08, 0.015, 9.9e-06, 8.5e-11), method="BH") < 0.05
[1] TRUE TRUE TRUE TRUE
summary(manova(lm(cbind(Choice.Reaction.Time, Visual.Matching, Sorting, Finger.Tapping) ~ LLLT, data=lllt)))
Df Pillai approx F num Df den Df Pr(>F)
LLLT 1 0.71 21.4 4 35 5.3e-09
Residuals 38
~~~
For all 4 tests, higher=better; since all the coefficients are positive, this suggests LLLT helped. The MANOVA agrees that LLLT made an overall difference. All the coefficients are statistically-significant and pass multiple-correction too.
Generally, we're not talking *huge* absolute differences here: like <10% of the raw scores (eg `Visual.Matching`: $\frac{9.65}{580.75} = 0.01662$). But the scores don't vary vmuch over time, so the LLLT influence sticks out with large effect-sizes (eg `9.65 / sd(lllt$Visual.Matching)` ~> _d_=0.75).
Since the variables were so highly intercorrelated, I was curious if a single _z_-score combination would show different results, but it didn't:
~~~{.R}
lllt$All <- with(lllt, scale(Choice.Reaction.Time) + scale(Visual.Matching) +
scale(Sorting) + scale(Finger.Tapping))
summary(lm(All ~ LLLT, data=lllt))
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -2.552 0.505 -5.05 1.1e-05
LLLTTRUE 5.103 0.714 7.14 1.6e-08
...
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -2.552 0.505 -5.05 1.1e-05
LLLTTRUE 5.103 0.714 7.14 1.6e-08
~~~
### Continuous dose
Then I learned Nattzor had actually *doubled* the time spent on LLLT in the second group. That means the right analysis is going to be different, since I need to take into account the dose size in case that matters, which it turns out, it does (as one would expect since Nattzor doubled the time for the same group I was wondering why it was so high in the graphs). So I redid the analysis by regressing on a continuous dose variable measured in minutes, rather than a binary dose/no-dose:
~~~{.R}
lllt$Dose <- c(rep(12, 10), rep(0, 20), rep(20, 10))
l1 <- lm(cbind(Choice.Reaction.Time, Visual.Matching, Sorting, Finger.Tapping) ~ LLLT, data=lllt)
l2 <- lm(cbind(Choice.Reaction.Time, Visual.Matching, Sorting, Finger.Tapping) ~ Dose, data=lllt)
summary(l2)
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 544.306 3.553 153.2 < 2e-16
Dose 2.468 0.305 8.1 8.4e-10
Response Visual.Matching :
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 580.219 2.523 229.96 <2e-16
Dose 0.669 0.216 3.09 0.0037
Response Sorting :
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 606.16 2.22 273.39 < 2e-16
Dose 1.19 0.19 6.25 2.6e-07
Response Finger.Tapping :
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 537.561 3.140 171.2 < 2e-16
Dose 2.861 0.269 10.6 6.1e-13
~~~
And compared it to the prior regression to see which fit better:
~~~{.R}
anova(l1,l2)
Res.Df Df Gen.var. Pillai approx F num Df den Df Pr(>F)
1 38 151
2 38 0 138 0 0 0
~~~
The second dose model fits far better.
### Robustness
One might ask based on the graph: is this all being driven by that anomalous third-group? Where the dose was increased? No, because even if we ignore the third group, the results are very similar:
~~~{.R}
summary(lm(cbind(Choice.Reaction.Time, Visual.Matching, Sorting, Finger.Tapping) ~ Dose, data=lllt[1:30,]))
Response Choice.Reaction.Time :
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 544.450 3.986 136.58 < 2e-16
Dose 2.396 0.575 4.16 0.00027
Response Visual.Matching :
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 580.750 2.605 222.94 <2e-16
Dose 0.404 0.376 1.07 0.29
Response Sorting :
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 606.650 2.028 299.18 <2e-16
Dose 0.946 0.293 3.23 0.0031
Response Finger.Tapping :
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 537.400 3.413 157.44 <2e-16
Dose 2.942 0.493 5.97 2e-06
~~~
The `Visual.Matching` response variable loses a lot of its strength, but in general, the results look the same as before: positive coefficients with statistically-significant effects of LLLT.
### Training effects
The anomalous third group prompts me to wonder if maybe it reflects a practice effect where subjects slowly get better at tasks over time. A quick cheap gesture towards time-series analysis is to just insert the index of each set of results and use that in the regression. But there seems to be only a small and statistically-insignificant result of all scores increasing with time:
~~~{.R}
lllt$Time <- 1:40
l1 <- lm(cbind(Choice.Reaction.Time, Visual.Matching, Sorting, Finger.Tapping) ~ Dose, data=lllt)
l2 <- lm(cbind(Choice.Reaction.Time, Visual.Matching, Sorting, Finger.Tapping) ~ Dose + Time, data=lllt)
anova(l1, l2)
Res.Df Df Gen.var. Pillai approx F num Df den Df Pr(>F)
1 38 138
2 37 -1 137 0.131 1.28 4 34 0.3
~~~
Curiously, if I delete the anomalous third group and rerun, the `Time` variable becomes much more significant:
~~~{.R}
lllt <- lllt[1:30,]
...
anova(l1, l2)
Res.Df Df Gen.var. Pillai approx F num Df den Df Pr(>F)
1 28 153
2 27 -1 142 0.367 3.47 4 24 0.023
~~~
But the gain is being driven by `Choice.Reaction.Time` and 2 of the coefficients are now even negative:
~~~{.R}
summary(l2)
Response Choice.Reaction.Time :
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 511.143 12.531 40.79 < 2e-16
Dose 4.427 0.896 4.94 3.6e-05
Time 1.625 0.586 2.77 0.0099
Response Visual.Matching :
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 570.534 9.053 63.02 <2e-16
Dose 1.027 0.648 1.59 0.12
Time 0.498 0.423 1.18 0.25
Response Sorting :
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 608.885 7.212 84.43 <2e-16
Dose 0.810 0.516 1.57 0.13
Time -0.109 0.337 -0.32 0.75
Response Finger.Tapping :
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 537.7977 12.1631 44.22 <2e-16
Dose 2.9174 0.8700 3.35 0.0024
Time -0.0194 0.5686 -0.03 0.9730
~~~
So it seems that the third group is driving the apparent training effect.
# Discussion
The methodology was not the usual worthless self-report: Nattzor systematically recorded objective metrics in a randomized intervention with even an attempt at blinding; the effect sizes are large, the _p_-values small. Overall, Nattzor has conducted an excellent self-experiment which is a model for others to emulate.
Still, Nattzor is just one man, so the problem of external validity remains, and I am troubled by the anomaly in the third group (even if the overall results are robust to excluding that data entirely). And in part, I find his results *too* good to be true - usually self-experiments just don't yield results this powerful. In particular, I'm concerned that despite his best efforts, the blinding may not have succeeded: perhaps some residual heat let him subconsciously figure out which block he was in (they were long and permitted time for guessing), or perhaps LLLT has some subjective effects which allow guessing even if it has no other benefits^[For example, it is widely reported among people trying out LLLT that after the first application of the LEDs to the head, one feels weirdly tired for around an hour. I felt this myself upon trying, several people report it in the Lostfalco thread, and an acquaintance of mine who had never seen the Lostfalco thread and had tried out LLLT a year before I first heard of it mentioned he had felt the same exact thing. This feeling *seems* to go away after the first time, but perhaps it just becomes weaker?] Nattzor didn't record any data during the self-experiment about whether he had been able to guess whether he was being treated or not.
# Followup experiment
How I would modify Nattzor's self-experiment to deal with my concerns, in roughly descending order of importance:
- make some sort of blinding index: for example, each day you could write down after the testing what you think you got, and then when it's done, check to see if you outperformed a coin flip. If you did, then the blinding failed and it's just randomized
- switch to much shorter blocks: closer to 3, maybe even just randomize daily; this helps minimize any learning/guessing of condition
- omit any breaks and intervals, and do the experiment steadily to eliminate selection concerns
- use a wider range of randomized doses: for example, 0.5 minutes, 1 minute, 2 minutes / place, or maybe 1/2/3 to see where the benefits being to break down
- run the measurements on each day, even days without LLLT. I'm interested in the fadeout/washout - in the first experiment's data, it looks like the effects of LLLT are almost instantaneous, which isn't very consistent with a theory of increased repair and neural growth, which should take longer
- upgrade to 808nm-wavelength LEDs for greater comparability with the research literature