While my Facebook feed was flooded with shares of the Association of Massage Therapy’s research finding of massage therapy for various health issues last week, I stumbled upon an interest Tweet from PainScience’s Paul Ingraham:

This led me to probe a little deeper into AMT’s research summary. Well, a “little deeper” led to a “lot deeper” than I like.

I thought the summary would give a detailed review of the current scientific evidence for various problems, like osteoarthritis knee pain, neck pain, low back pain, etc., but to me, the summary read like a movie synopsis on IMDB that only show the “good” or “exciting” parts of the show. For example, under “HIV,” AMT wrote, “A 2010 Cochrane Review found evidence to support the use of massage therapy to improve the quality of life of people living with AIDS/HIV. A 2013 clinical trial showed massage therapy to be effective in the treatment of depression in HIV patients.”

Okay, that doesn’t tell me much about how strong the evidence for treating HIV patients or how well the studies were conducted.

Also, AMT cited a 2008 systematic review on low back pain that massage “might be beneficial for patients with subacute and chronic non-specific low-back pain, especially when combined with exercises and education” and another systematic review by the Ottawa Panel in which AMT concluded “massage interventions provide short-term improvement of sub-acute and chronic low back pain symptoms and decrease disability at immediate post-treatment.”

However, a more recent and rigorous Cochrane review (2015) stated that the authors “might be beneficial for patients with subacute and chronic non-specific low-back pain, especially when combined with exercises and education”  The 2008 review is based on 13 eligible trials, which gave a tentative and reasonable conclusion that massage might be beneficial for low back pain. With 12 additional trials added and evaluated, their conclusion contradicts their earlier review. Like most massage research the authors wrote that they “did not find any large effect size,” and “the magnitude of the effect was small to medium in all meta-analyses of continuous outcomes.” In other words, the lack of a large effect size — due to small samples — have a higher chance of getting a false positive or false negative, which is more prone to error.

The Cochrane Review also found most of the trials — old and new — were of low quality to very low quality, due to lack of large effect sizes, methodology flaws, and reported adverse effects. For the latter, the authors reported, “Only 11 trials measured adverse events. The remaining trials did not mention whether or not adverse events were measured. There were no serious adverse events in these trials, and the most common adverse event was increased pain after the massage sessions.”

Compared with the 2008 review, the authors stated, “This updated review is also different from previous versions in relation to the quality of the evidence. The current approach yielded “low” to “very low” quality evidence, which differs from the previous version of this review, (Furlan 2008), where the quality of the evidence was judged ‘moderate’ for most comparisons. The explanation for these changes could be: first, we grouped more studies in the same comparisons, therefore increasing the types of biases that were introduced in each comparison; and second, the definitions of imprecision and inconsistency were stricter in the current than in the previous review.

“In this review update we found high risk of selection, performance, attrition and measurement bias, suggesting that blinding patients, health care providers and outcomes were the most challenging methodological steps in clinical trials of massage.”

Psychologist and massage researcher Christopher Moyer, who had read and reviewed the 2015 Cochrane Review, reminded that scientific thinking and research should be oftentimes tentative and cautious when interpreting results, especially if the number of trials reviewed or sample sizes are small. “Speaking as a scientist, we are very careful to guard against declaring a finding if there is even a small risk of it being a false positive,” Moyer said in an online interview with Massage & Fitness Magazine back in September 2015 when we covered the low back pain Cochrane Review. “We never want to say ‘we’ve found something’ and later have it turn out we were wrong when more data comes in. So, I think they are hewing to scientific norms in this regard, and I do not fault them for that; it is important to be careful in science.”

Anything Else We Should Know?

A superficial review does not well-inform massage therapists, other healthcare professionals, or patients in pain. While I have not yet explored the rest of the claims and research, I have found a few more inconsistencies between what the summary says and what the research says.

The first citation on aromatherapy for pain relief among cancer patients was withdrawn from the Cochrane Database, which was replaced by a more recent Cochrane Review that contradicts the previous review: “might be beneficial for patients with subacute and chronic non-specific low-back pain, especially when combined with exercises and education”

For neck pain: While AMT is correct that massage therapy has immediate neck pain relief and any long-term benefits is unknown, there are a few problems the researchers encountered. Having read all four (not five as AMT’s summary stated) systematic reviews on neck pain and massage, I summarized the findings in more details on Massage Therapist Development Centre by Jamie Johnston.

For this systematic review, “might be beneficial for patients with subacute and chronic non-specific low-back pain, especially when combined with exercises and education”

1. “Not many studies qualified for this review because the quality of studies is poor. Out of 4,099 studies on the topic, 16 qualified for qualitative assessment and 9 qualified for quantitative assessment.”

2. “Based on those 9 and 16, there’s not a lot of conclusions on massage for cancer pain other than “might help people feel better, probably won’t hurt”.

3. “Problems that hinder finding any strong evidence/findings include small sample sizes, widely varying study designs, widely varying “outcomes/effects/endpoints” studied and lack of objective and/or validated measurement tools or techniques

4. “Lack of comparison groups for controls, i.e., sham treatment, no treatment, etc. and difficulties in blinding of study participants and researchers.”

5. “Widely varying practitioner characteristics (qualifications, affiliation, experience, etc.) and agreed terminiology for specific intervention protocols and repeatability as far down the line as lack of agreement on definitions of and styles of “massage.”

“So basically, that review says ‘there’s some evidence, not great evidence, that people with cancer pain might find that massage makes them feel a bit better,” Dr. Gregas told Massage & Fitness Magazine. “We could go off into the weeds on how medical care is handled in different countries and why and how massage would fit into those scenarios, but for the purpose of this discussion, [AMT is] overselling the findings and completely ignoring the limitations.”

As Paul Ingraham had indicated in his Tweet.

Getting Meta With Systematic Reviews

Systematic reviews are only as good as the data reviewed. It is no surprise to find that a 2016 systematic review of massage systematic reviews find the strength of the evidence for massage therapy’s effectiveness for various types of pain to be low or very low.

Out of 31 eligible systematic reviews found, 21 of them are of high quality. “Findings from high-quality systematic reviews describe potential benefits of massage for pain indications including labor, shoulder, neck, back, cancer, fibromyalgia, and temporomandibular disorder. However, no findings were rated as moderate- or high-strength.”

The authors found, “A third of included reviews did not meet our threshold for high quality. While some of these reviews overlapped in scope with higher quality reviews, others did not, and these topics need to be revisited with strong synthesis methodology before conclusions can be drawn from the findings. Systematic review authors found that primary studies often do not provide adequate details of the massage therapy provided, especially in the descriptions of provider type.”

How Can the Summary Be Better?

Can we see why critical thinking, research/science literacy, and asking questions are vital in our profession? If we just skim the summary and accept it for its face value, then we would be misinformed and oversell our services.

Given the immense amount of time and energy needed to review a single category, I presume that other massage research quality would be similar for other topics.

The five-page AMT summary should:

1. Acknowledge limitations and biases of the trials and systematic reviews;
2. Disclose how many studies were included in each systematic reviews and describe their eligibility and quality;
3. Update the citations;
4. Stick to the results and conclusions; do not cherry-pick parts of them. Both positive and negative findings should be included.

“A small, specific positive in any of these studies is not able to be extrapolated to the general population,” Monica Noy commented, who is a practicing osteopath and a continuing massage education provider in Toronto, Ontario. “A strong study with good internal validity is specific to the population under consideration, but the massage research is small, reductive, and very narrow. That’s not a bad thing but these factors but should be respected in reporting results.”

Despite the criticism with AMT’s summary, Australian massage therapist Samuel McCracken reminded that AMT is a small organization with about 3,000 members and the research work are done by volunteers during their spare time.

AMT also has a strict rules and guidelines within its membership. “As for scope of practice, if you break the rules you get thrown out of AMT and you are then free to go and join another organization and practice  dolphin healing and chakra balancing with no consequences,” McCracken told M&F Magazine online. “So AMT members join knowing that they will be bound by strict codes of practice and much higher continued education requirements. Massage in Australia is unregulated anyone can set up shop with no qualifications or organization membership​.”

As for the summary, we should take a little easy with it since it is one of the rare massage organizations that promotes science literacy and ethical practices like Canada’s RMTBC. But as a massage therapist who works with clients in pain regularly, I should be better informed at what my work could do and could not do — even if the guidelines are for Australian therapists. Despite the short-comings, I laud their effort in compiling such a large research list. It’s not easy. Been there done that.

“I have heard that the volunteers who did the research work are feeling very deflated right now with all the international criticism. They may have missed the mark but their efforts were well-intended,” McCracken added.

Massage therapy research is still in its infancy and has a lot of potential for growth and further professional development. Analytical criticism may make us feel uncomfortable, but it’s necessary for professions to change and grow. It’s a normal process that happens in most professions. We cannot always rely on anecdotes or clinical experience alone to justify what and how we do and why because we are prone to many types of cognitive biases that distorts our perceptions. Thus, this is why massage therapy needs to do better science and reporting.

I thank Dr. Keith Eric Grant, Dr. Molly Gregas, Monica Noy, and Paul Ingraham for their insights in helping me gather my thoughts and and the full papers to all citations listed and not listed on the summary.