Advertisement

Methodological flaws in meta-analysis of low-level laser therapy in knee osteoarthritis: A letter to the editor

Open ArchivePublished:November 02, 2016DOI:https://doi.org/10.1016/j.joca.2016.09.022
      We read the meta-analysis of low-level laser therapy (LLLT) in knee osteoarthritis (KOA) by Huang et al. with interest
      • Huang Z.
      • Chen J.
      • Ma J.
      • Shen B.
      • Kraus V.B.
      Effectiveness of low-level laser therapy in patients with knee osteoarthritis: a systematic review and meta-analysis.
      . The authors of the review conclude that the best available current evidence does not support the effectiveness of LLLT in patients with KOA. They underpin their conclusion by asserting that, “based on seven studies, the Standardized Mean Difference (SMD) in Visual Analog Scale (VAS) pain score right after treatment was not significantly different from control (SMD: 0.28 [95% CI: −0.10; 0.66], I2 = 66%)” and that “no significant difference was identified in four studies conforming to the World Association for Laser Therapy (WALT) recommendations”. Huang et al. also postulate that “no study has synthesized the results in a meta-analysis”, but positive results from LLLT in KOA have been reported in at least four previously published meta-analyses, including two of our own
      • Bjordal J.M.
      • Johnson M.I.
      • Lopes-Martins R.A.B.
      • Bogen B.
      • Chow R.
      • Ljunggren A.E.
      Short-term efficacy of physical interventions in osteoarthritic knee pain. A systematic review and meta-analysis of randomised placebo-controlled trials.
      • Jang H.
      • Lee H.
      Meta-analysis of pain relief effects by laser irradiation on joint areas.
      • Zhang W.
      • Moskowitz R.W.
      • Nuki G.
      • Abramson S.
      • Altman R.D.
      • Arden N.
      • et al.
      OARSI recommendations for the management of hip and knee osteoarthritis, part I: critical appraisal of existing treatment guidelines and systematic review of current research evidence.
      . The conclusion by Huang et al. stands in contrast to the positive findings in these reviews. Hence, we decided to critically appraise their review with A MeaSurement Tool to Assess Systematic Reviews (AMSTAR) and test the robustness of their meta-analysis to see how this discrepancy could be explained.

      AMSTAR validity tool

      • 1.
        Was an a priori design provided?
      No.
      • 2.
        Was there duplicate study selection and data extraction?
      No: Two assessors independently selected the studies and extracted the data, however, there was no consensus procedure for disagreements among them.
      • 3.
        Was a comprehensive literature search performed?
      Yes: The literature search satisfies the criteria for a ‘yes’. However, we question whether a comprehensive search was indeed performed: All Randomized Clinical Trials (RCTs) published before year 2000 were systematically excluded by their search criteria, thereby excluding at least three otherwise possible eligible RCTs
      • Bülow P.M.
      • Jensen H.
      • Danneskiold-Samsøe B.
      Low power Ga-Al-As laser treatment of painful osteoarthritis of the knee. A double-blind placebo-controlled study.
      • Nivbrant B.
      • Friberg S.
      Laser tycks ha effekt på knäledsartros men videnskapligt bevis saknas.
      • Stelian J.
      • Gil I.
      • Habot B.
      • Rosenthal M.
      • Abramovici I.
      • Kutok N.
      • et al.
      Improvement of pain and disability in elderly patients with degenerative osteoarthritis of the knee treated with narrow-band light therapy.
      . They also failed to identify the RCT by Hegedus et al.
      • Hegedus B.
      • Viharos L.
      • Gervain M.
      • Gálfi M.
      The effect of low-level laser in knee osteoarthritis: a double-blind, randomized, placebo-controlled trial.
      , published after year 2000. All these four RCTs demonstrated pain outcomes in favour of LLLT over placebo.
      • 4.
        Was the status of publication (i.e., grey literature) used as an inclusion criterion?
      No: Non-English literature was excluded systematically.
      • 5.
        Was a list of studies (included and excluded) provided?
      No.
      • 6.
        Were the characteristics of the included studies provided?
      Yes.
      • 7.
        Was the scientific quality of the included studies assessed and documented?
      No: The quality of the included trials was assessed, however the choice of assessment tool was not provided prior to the search.
      • 8.
        Was the scientific quality of the included studies used appropriately in formulating conclusions?
      Yes: The scientific quality of the included studies did not affect the conclusion.
      • 9.
        Were the methods used to combine the findings of the studies appropriate?
      No: The clinical appropriateness of combining the trials in the different subgroups was not considered. Huang and colleagues attempted to subgroup the trials by adherence or non-adherence to the WALT dosage recommendations. However, their subgrouping was incorrect, which was indicated by a high heterogeneity in the optimal dose subgroup in Huang and colleagues' analysis. Correcting this misclassification eliminated this heterogeneity.
      • 10.
        Was the likelihood of publication bias assessed?
      No.
      • 11.
        Was the conflict of interest stated?
      No: It was not reported for the included trials. Also, we challenge the statement by Huang et al. in which it is claimed that the authors have no competing interests to disclose, since the senior author has declared several conflicts of interests in papers related to research on pharmaceutical painkillers, which are competitors to LLLT.
      Total AMSTAR score: 3/11.

      VAS pain sensitivity analysis

      Relying solely on AMSTAR can lead to masking of strengths and weaknesses of individual systematic reviews. Therefore, we additionally tested the robustness of their meta-analysis by applying a different valid statistical analysis approach to the very same trials included by Huang et al.
      By (1) extracting pain scores from the difference in mean change from the same time points of assessment, (2) correcting the subgroups and (3) adding all eligible included trials and intervention groups (n = 83) the results changed fundamentally: The overall result significantly favours LLLT over placebo by 7.22 mm VAS ([95% CI: 1.15; 13.3]) or SMD = 0.34. The subgroup with dosages adhering to the WALT recommendations obtained a significant positive result in favour of LLLT over placebo by 12.61 mm VAS ([95% CI: 6.06; 19.16]) or SMD = 0.48. Moreover, the heterogeneity within the two subgroups dropped from I2 = 69% and 74% to I2 = 0% and 66%, respectively, indicating that our meta-analysis results are more robust. We found the same pattern emerging from a similar pain sensitivity analysis using The Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) (see forest plots in Supplementary material).
      We have demonstrated that the review by Huang et al. displays some severe methodological shortcomings, and that their meta-analysis is subject to type-II error. In our opinion, the internal validity and construct validity can be amended with a few changes. However, the statistical conclusion validity and external validity of the review are too severely compromised to be trusted.

      Author contributions

      J.M. Bjordal and R.A.B. Lopes-Martins developed the idea of the letter and sketched an initial draft, while M.B. Stausholm refined the text and performed the majority of meta-analyses under supervision by J. Joensen and J.M. Bjordal. All authors participated in discussions regarding the intellectual content and read and approved the final manuscript.

      Conflicts of interest

      We declare that we have no conflicts of interests.

      Acknowledgements

      We received no grant from any funding agency in the public, commercial or not-for-profit sectors.

      Appendix A. Supplementary material

      The following is the supplementary material related to this article:

      References

        • Huang Z.
        • Chen J.
        • Ma J.
        • Shen B.
        • Kraus V.B.
        Effectiveness of low-level laser therapy in patients with knee osteoarthritis: a systematic review and meta-analysis.
        Osteoarthritis Cartilage. 2015; 23: 1437-1444
        • Bjordal J.M.
        • Johnson M.I.
        • Lopes-Martins R.A.B.
        • Bogen B.
        • Chow R.
        • Ljunggren A.E.
        Short-term efficacy of physical interventions in osteoarthritic knee pain. A systematic review and meta-analysis of randomised placebo-controlled trials.
        BMC Musculoskelet Disord. 2007 Jun 22; 8: 51
        • Jang H.
        • Lee H.
        Meta-analysis of pain relief effects by laser irradiation on joint areas.
        Photomed Laser Surg. 2012; 30: 405-417
        • Zhang W.
        • Moskowitz R.W.
        • Nuki G.
        • Abramson S.
        • Altman R.D.
        • Arden N.
        • et al.
        OARSI recommendations for the management of hip and knee osteoarthritis, part I: critical appraisal of existing treatment guidelines and systematic review of current research evidence.
        Osteoarthritis Cartilage. 2007; 15: 981-1000
        • Bülow P.M.
        • Jensen H.
        • Danneskiold-Samsøe B.
        Low power Ga-Al-As laser treatment of painful osteoarthritis of the knee. A double-blind placebo-controlled study.
        Scand J Rehabil Med. 1994; 26: 155-159
        • Nivbrant B.
        • Friberg S.
        Laser tycks ha effekt på knäledsartros men videnskapligt bevis saknas.
        Läkartidningen. 1992; : 859-861
        • Stelian J.
        • Gil I.
        • Habot B.
        • Rosenthal M.
        • Abramovici I.
        • Kutok N.
        • et al.
        Improvement of pain and disability in elderly patients with degenerative osteoarthritis of the knee treated with narrow-band light therapy.
        J Am Geriatr Soc. 1992; : 23-26
        • Hegedus B.
        • Viharos L.
        • Gervain M.
        • Gálfi M.
        The effect of low-level laser in knee osteoarthritis: a double-blind, randomized, placebo-controlled trial.
        Photomed Laser Surg. 2006; 27: 577-584