Research methods | My Eyes Are Up Here

I’ve read a lot about “chemo brain” in breast cancer blogs. A lot of breast cancer patients observe a decline in their attention and memory during and after chemotherapy. They complain to their physicians and many of them feel invalidated by the responses they receive. This is in part, because the evidence of chemo brain is sparse. (Before you throw your shoe at your computer, bear with me.)

I have a Ph.D. in psychology and in addition to my seven years of graduate school during which I conducted research, I worked as a researcher for 10 years after I graduated. One of the things that researchers are trained to do is to test hypotheses and sets of hypotheses. In clinical research, there is also the testing of treatments in the context of clinical trials. We are trained to interpret hypotheses in terms of whether they are empirically supported or not. If they are supported in multiple studies, we accept them as “truths” (there are no absolute truths), especially if findings are replicated by another lab. If they are not supported, we conclude that (1) the hypothesis was incorrect or (2) the hypothesis was tested incorrectly.

However, there is a third reason why a hypothesis has inadequate research support and that is when the hypothesis has not been adequately tested. Now as far as I can see, this is the case for the chemobrain hypothesis. So does the fact that it does not have adequate experimental support mean that it doesn’t exist? No, it means that it could exist but we don’t know because we haven’t thoroughly looked at the question. A downside of our careful and methodical ways is that we take our sweet time assessing potential “truths.” This is also a source of frustration for most of the rest of the world. Further, sometimes as researchers or as clinicians who do evidence-based practice, we lump all of the hypotheses that have not yet been deemed empirically-supported into the same group. For example, when asked, “does treatment x work”, they might answer “no” even for a treatment that has never been tested. The correct answer in this case is “We don’t know.” As a clinician who is supposed to have the answers, it is hard to say this to people. But it’s part of our job. To the great credit of my oncologists, they are both extremely knowledgeable but honest about the limitations of their respective fields. My breast surgeon actually discussed the concern about over treating breast cancer because they do not yet know how to distinguish between tumors that will spread verses those that will not. (In breast cancer, an estimated 25% of tumors never spread. If you want this article, let me know. I have the .pdf and it was given to me by a childhood friend who is a professor at Rutgers and does cancer research on polarity in cancer cells.)

Then there is the confusion provided by some (not all) of the folks in the media who go around spreading rumors and making generalizations based on one small result from a single study. Or who totally misrepresent the findings of a research study. Unfortunately, most of us do not have university library privileges that allow us to check out the primary source material on which the story is based. Also, even if we could, we might not have the necessary background knowledge to interpret the study. As an example, there was a news story that made it rounds in the blogisphere recently. The investigators used used neuropsych measures and found that breast cancer patients who underwent chemo showed declines in executive functions such as memory. The comparison group were “healthy controls.” So a group of women who have been subjected to a variable onslaught of chemo, surgery, radiation, endocrine therapy, not to mention the stress of having a serious illness were compared to women with no known medical problems. The story was presented as evidence of chemo brain. This is not specific evidence of chemo brain because chemotherapy was one of many variables that could explained the findings. Does this make the study useless? No, I think it shows that the cancer assessment and treatment experience is associated with a decline in cognitive functioning. It is a little bite out of a much larger question. Further, the use of neuropsych measures was really smart. They can be more sensitive to subtle real world changes than other measures plus they are safer and probably less expensive (I’m guessing that they did not do a full neuropsych battery, which is kind of expensive) than using an MRI.

Boy, I wish I could remember where I saw that article but I can’t remember where I saw it. That’s because my attention and memory have been impaired since I was diagnosed with breast cancer nearly a year ago! As a psychologist, I validated this for myself as the stress alone of having cancer is enough to impair executive functions. And also as a psychologist, I don’t dismiss the real impact that stress can have on a brain. In simple words, psychological stuff is real. People might say, “It’s all in your head.” Guess what? Your head is part of your body. Also, your brain is in there and it’s kind of an important organ.

Stress impacts cortisol regulation. Cortisol is a hormone that is triggered by stress and it’s purpose is to help us function better during those “fight or flight” times of our life. A problem with this is that too much stress or chronic stress can break down this regulatory system and lead to a break down in attention and memory. Speaking of hormones, there is another hormone that is thought to be important in memory functioning and that hormone is estrogen. Most, but not all breast cancer is estrogen responsive. How many of us have our cancer treated with hormone blockers? So at least some of that fog could be due to reduced estrogen. I am not an endocrinologist and concede that I am oversimplifying the role of these hormones to make my point and also because I don’t understand endocrinology terribly well. But my larger point is that there is evidence that (1) cortisol and estrogen functioning impact attention and memory and (2) cortisol and estrogen functioning is impacted by having and being treated for cancer. And theses are just examples. There are a lot of potential mediated relationships (indirect effects) as well. Interrupted sleep impairs attention and memory, too. How many of you haven’t had difficulty sleeping due to the stress of cancer or due to increased hot flashes, for those of you who receive endocrine therapy?

But what about chemotherapy? All of my chemical warfare has been in the form of anesthesia, pain meds, tamoxifen, and Lupron. I did not receive I.V. chemotherapy. My heart is with all of you who are enduring or have endured this. It’s not unreasonable to hypothesize that chemotherapy drugs might have a direct negative effect on memory and attention. The blood brain barrier does not exactly work like Fort Knox in keeping chemo drugs from entering it. There is some permeability. So at least some of those nasty chemicals might get in and do damage. (I’m not a neuro-pharmacologist or neurobiologist but I believe that my general point is true.) And perhaps some of that damage might be to parts of the brain that impact attention and memory. And I do see some research in my Google Scholar searching that supports these hypotheses. But one job of a researcher is to interpret findings from a single study into the larger context of multiple studies. They also use the level of rigor of the particular journal in which the article is published in their interpretations. I can do this in my own field but this is after many years and having read thousands of research studies.

One question that is buzzing around my head like a gadfly is “Why isn’t this question rigorously tested?” If any or all of the chemotherapy drugs cause cognitive decline, shouldn’t we investigate it so that patients can be informed of the potential treatment side effects? Isn’t it important to know whether the potential effects vary in duration, frequency, or intensity as a function of the drugs chosen for treatment? And in the mean time, let’s hear it for better integration of psychology into cancer research and treatment. I think we can all agree that cancer is stressful. We also know that it increases risk of anxiety and depression. Stress, anxiety, and depression can all negatively impact attention and memory. We have tools for addressing these issues and some of them like mindfulness meditation are incredibly cheap and safe. Finally, we are trained in measurement as well as in research design and clinical trials.

As I have mentioned in the past, my initial college career goal was to be an academic researcher. My particular emphasis was on conducting controlled clinical trials. And what I mean by that is doing research to evaluate a treatment by comparing it to an untreated group. This, ladies and gentlemen, is what we call an experiment! And I can say in all sincerity, “Yay, science!” And that’s what I did for my doctoral thesis and during the ten years following my obtaining my Ph.D. in 1997. My career trajectory took a different course, which I have explained in an earlier post. To make a long story short, I didn’t know how to effectively keep chasing grant money while maintaining a healthy family life. But don’t boo hoo for me because going into private practice full time ended up being the perfect job for me.

Back when I was doing science, there was a lot of effort put into designing experiments that were as highly controlled as possible. We picked measures that actually had statistical properties that demonstrated that they actually measured what they were designed to measure. And since we were trying to show that our treatments produced desired change, we had to choose measures that were sensitive enough to detect change. And if there was a variable we wanted to measure for which there were no suitable measures already available, we had to develop our own. Measure development is no joke, people. You might think good social scientists just make up a bunch of questionnaire items and then give them out to their research participants and assume all is being measured in a reliable and valid way but it just doesn’t work that way. Well, I guess one could do it this way but it would not be good science.

We also tried to “control” or account for variables that could explain changes (or lack thereof) between the treatment and control group, other than the nifty treatment program we were testing. These little extra trouble making variables are called “confounds.” Confound it, variable, you have messed up my experiment! Sometimes these variables can be controlled for statistically but other times, they cannot. Inclusion criteria for studies are developed to screen out the latter variables. For example, when I was doing research evaluating a parenting program to support positive behavioral and emotional development in young children with behavior problems, we screened out children with Attention-Deficit/Hyperactivity Disorder because research on AD/HD shows that psychological treatment alone is not usually effective for kids with AD/HD. (I wish it were but it is not.) Young children with AD/HD often show behavioral problems but they would be unlikely to respond to treatment, unlike other youngster who did not have this additional diagnosis. So, those of you who have tried to join breast cancer trials and have been denied based on the inclusion criteria for the study, this is the reason why. The researchers are not trying to be mean. They are trying to get the clearest picture that they can about whether the treatment is helpful for its intended purpose. After a treatment has initial support and the findings are replicated, subsequent studies may shift the criteria to other groups, which may have been screened out of earlier trials. But if a researcher cannot show a positive impact in the early trials, they risk that treatment being seen as a dead end. And if the treatment looks bad, subsequent research on it will not get funded.

Okay, that’s a little about my former life as an investigator on clinical trials. As a clinician, the test tube gets a little dirtier. I am trained in using evidence-based practices and I use them. However, they do not describe every possible scenario. Often, my recommendations are based on my understanding of the principles that underlie the evidence-based treatment models that I use, rather than from a treatment manual. Also, families come in distress and I often recommend that they get a number of interventions going at once, interventions at home, at school, and with a physician. When there is improvement, from a science perspective, I don’t really know what the most critical components of the intervention were for that particular child. This is because I am not doing research on my patients. I am a clinician. I work systematically and my efforts are guided by what research is currently telling me about best practices. My work is also informed by my clinical experience. I can’t exercise the same control as I did as a scientist.

Now I am a cancer patient. My physicians, just like I do with my patients, have hit me with multiple treatments at once. To further muddy things, I have opted for an integrative approach to my treatment. In addition to my onco surgeon and my medical oncologist, I see a naturopathic oncologist and receive acupuncture from a physician trained in Chinese medicine. I also follow a special diet, take nutritional supplements, engage in mindfulness meditation, get a massage every 3 weeks, and walk 3 miles a day. My holistic treatment plan is quite variable in terms of the research evidence available to support it.

Does this mean that I threw my whole education away? I don’t think so. For one, I am mindful of the fact that there are no guarantees that my cancer won’t come back or that I won’t get sick with some other disease or that I won’t get hit by a bus tomorrow. My mindset is one of influencing rather than trying to control my outcomes. Some of my complementary or integrative practices are no-brainers. Maybe my losing nearly 40 pounds, eating healthy food, and walking 3 miles a day won’t keep cancer away. But I know that (1) I feel better now and (2) I am reducing my risk of all kinds of future health problems. Eating lots of chard and broccoli is not the same as buying snake oil. And research is still out on whether there are higher nutritional benefits for organically grown produce. I think there probably are more benefits nutritionally. But even if there is not, there are environmental sustainability benefits. So, this again, is a no brainer to me. I don’t exclusively eat organic, but I mostly eat organic. And my last example is the flaxseed meal I take every day. Maybe it will not really prevent breast cancer recurrence but it’s good fiber and Tamoxifen is a little constipating.

Some of the actions I am taking are for potential long-term benefit. I may never know if they help but they might help. (Obviously, I am omitting interventions that may cause harm unless there is evidence that the potential pros greatly outweigh the cons.) I am also not in favor of doing anything that just seems outright illogical or doesn’t have some kind of track record. I must admit that I don’t get the logic behind acupuncture but I respect that it is based on thousands of years of practice from a an amazing culture. It also has western-research validated applications, especially in pain management. Plus, I get to meditate while the needles are in and there is some research suggesting that mindfulness meditation reduces the risk of breast cancer recurrence. And even if that research doesn’t bear out, there’s ample research of the effectiveness of mindfulness meditation in stress and anxiety management. And in case you haven’t heard, having breast cancer is stressful and often causes anxiety!

But I haven’t thrown away my research training. Enter the single case study design. It’s a very simple design, often called “ABA” or “ABAB” design depending on how it’s set up. For me, the single case is me. “A” refers to baseline. “B” refers to treatment. What? Okay, it’s easier to explain with an example. As I have mentioned in the past, I have a long history of eczema going back to my early 30’s. My naturopathic oncologist suggested that it might be due to a wheat allergy and asked me to consider not eating wheat for a couple of weeks. In this case, “A” is my baseline, otherwise known as the 15 years I spent eating wheat and scratching. “B” is the time I spent off of wheat. I actually refrained from eating wheat for about three months. My skin cleared considerably. But there’s some variability in my eczema. It waxes and wanes. So the clearing could have just been part of that cycle. I decided to go back to “A” by eating a slice of pizza that one of my brother’s brought to me right after my TRAM surgery. Two days later, bam! Huge outbreak of eczema. So I promptly went back to “B”, not eating wheat and my skin improved. Usually, results are not this striking. This was a darned good use of the ABAB single case study design. I am using the AB or possibly ABA design with acupuncture. Although I am noting improvements in hot flashes and my energy level, it is hard to say whether the former is just due to the general pattern I’ve noted of improvements in hot flashes a couple of months after each Lupron shot and the latter might just be due to the natural course of my healing from my surgery. However, I got my new three month Lupron shot yesterday and another session of acupuncture today. All of these variables manipulated at once! We will see what happens.

You can try AB or ABA or ABAB designs for yourself. One trick is that it will only work for interventions for which you expect quick results. I can’t eat Swiss chard for two weeks, stop eating it for two weeks and expect to detect any changes in my health, for example. But don’t make the same mistake a coworker made years ago. She had chronic neck pain for many years (A) and decided to try acupuncture (B). She decided to stop acupuncture (back to A) to see if it was really a “cure” because unless it was a cure, it was not real in her eyes. Her neck pain came back so she decided that acupuncture didn’t work. This is like deciding to no longer take insulin injections because your Type 1 diabetes is poorly controlled when you don’t take it. Not all effective treatments are cures but that doesn’t mean they are fake or useless. I suspect she would have viewed diabetes treatment differently and her illogical reasoning had more to do with her discomfort with eastern medicine because she was quite an intelligent person.

Finally, I found an excellent article on the Anderson Cancer Center website that describes practice guidelines for integrative care for cancer. Check it out!