Search JIM Advanced Search

Journal of Integrative Medicine ›› 2021, Vol. 19 ›› Issue (4): 317-326.doi: 10.1016/j.joim.2021.03.003

Special Issue: COVID-19

• Systematic Review • Previous Articles     Next Articles

Evaluating the methodology of studies conducted during the global COVID-19 pandemic: A systematic review of randomized controlled trials

Meng-zhu Zhaoa,b, Chen Zhaoc, Shuang-shuang Tud, Xu-xu Weie, Hong-cai Shange,f   

  1. a Department of Cardiology, First Teaching Hospital of Tianjin University of Traditional Chinese Medicine, Tianjin 300381, China
    b National Clinical Research Center for Chinese Medicine Acupuncture and Moxibustion, Tianjin 300381, China
    c Institute of Basic Research in Clinical Medicine, China Academy of Chinese Medical Sciences, Beijing 100700, China
    d Editorial Department of Journals, Tianjin University of Traditional Chinese Medicine, Tianjin 301617, China
    e Key Laboratory of Chinese Internal Medicine of Ministry of Education and Beijing, Dongzhimen Hospital Affiliated to Beijing University of Chinese Medicine, Beijing 100700, China
    f International Evidence-based Research Institute of Chinese Medicine, Beijing University of Chinese Medicine, Beijing 100029, China
  • Received:2020-10-09 Accepted:2021-02-22 Online:2021-08-15 Published:2021-03-24
  • Contact: Hong-cai Shang

The therapeutic evidence collected from well-designed studies is needed to help manage the global pandemic of the coronavirus disease 2019 (COVID-19). Evaluating the quality of therapeutic data collected during this most recent pandemic is important for improving future clinical research under similar circumstances.

To assess the methodological quality and variability in implementation of randomized controlled trials (RCTs) for treating COVID-19, and to analyze the support that should be provided to improve data collected during an urgent pandemic situation.

Search strategy
PubMed, Excerpta Medica Database, China National Knowledge Infrastructure, Wanfang, and Chongqing VIP, and the preprint repositories including Social Science Research Network and MedRxiv were systematically searched, up to September 30, 2020, using the keywords “coronavirus disease 2019 (COVID-19),” “2019 novel coronavirus (2019-nCoV),” “severe acute respiratory syndrome-related coronavirus-2 (SARS-CoV-2),” “novel coronavirus pneumonia (NCP),” “randomized controlled trial (RCT)” and “random.”

Inclusion criteria
RCTs studying the treatment of COVID-19 were eligible for inclusion. 

Data extraction and analysis
Screening of published RCTs for inclusion and data extraction were each conducted by two researchers. Analysis of general information on COVID-19 RCTs was done using descriptive statistics. Methodological quality was assessed using the risk-of-bias tools in the Cochrane Handbook for Systematic Reviews of Interventions (Version 5.1.0). Variability in implementation was assessed by comparing consistency between RCT reports and registration information. 

A total of 5886 COVID-19 RCTs were identified. Eighty-one RCTs were finally included, of which, 45 had registration information. Methodological quality of the RTCs was not optimal due to deficiencies in five main domains: allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, and selective reporting. Comparisons of consistency between published protocols and registration information showed that the 45 RCTs with registration information had common deviations in seven items: inclusion and exclusion criteria, sample size, outcomes, research sites of recruitment, interventions, and blinding.

The methodological quality of COVID-19 RCTs conducted in early to mid 2020 was consistently low and variability in implementation was common. More support for implementing high-quality methodology is needed to obtain the quality of therapeutic evidence needed to provide positive guidance for clinical care. We make an urgent appeal for accelerating the construction of a collaborative sharing platform and preparing multidisciplinary talent and professional teams to conduct excellent clinical research when faced with epidemic diseases of the future. Further, variability in RCT implementation should be clearly reported and interpreted to improve the utility of data resulting from those trials.

Key words: COVID-19, Randomized controlled trial, Therapeutic evidence, Methodology, Systematic review

[1] Chiung-Hui Huang, Mei-Ling Yeh, Fang-Pey Chen, Daphne Wu. Low-level laser acupuncture reduces postoperative pain and morphine consumption in older patients with total knee arthroplasty: A randomized placebo-controlled trial. Journal of Integrative Medicine, 2022, 20(4): 321-328.
[2] Ming-min Xu, Pei Guo, Qing-yu Ma, Xuan Zhou, Yu-long Wei, Lu Wang, Yue Chen, Yu Guo. Can acupuncture enhance therapeutic effectiveness of antidepressants and reduce adverse drug reactions in patients with depression? A systematic review and meta-analysis. Journal of Integrative Medicine, 2022, 20(4): 305-320.
[3] Ting-ting Lu, Cun-cun Lu, Mei-xuan Li, Li-xin Ke, Hui Cai, Ke-hu Yang. Reporting and methodological quality of meta-analyses of acupuncture for patients with migraine: A methodological investigation with evidence map. Journal of Integrative Medicine, 2022, 20(3): 213-220.
[4] Maryam Azimi, Hanieh Niayesh, Mahboobeh Raeiszadeh, Sedigheh Khodabandeh-shahraki. Efficacy of the herbal formula of Foeniculum vulgare and Rosa damascena on elderly patients with functional constipation: A double-blind randomized controlled tria. Journal of Integrative Medicine, 2022, 20(3): 230-236.
[5] Ubiratan Cardinalli Adler, Maristela Schiabel Adler, Ana Elisa Madureira Padula, Livia Mitchiguian Hotta, Amarilysde Toledo Cesar, José Nelson Martins Diniz, Helen de Freitas Santos, Edson Zangiacomi Martinez. Homeopathy for COVID-19 in primary care: A randomized, double-blind, placebo-controlled trial (COVID-Simile study). Journal of Integrative Medicine, 2022, 20(3): 221-229.
[6] Xin Wang, Wen-juan Song, Yi Ruan, Bing-chu Li, Can Lü, Nian Huang, Fan-fu Fang, Wei Gu. Core muscle functional strength training for reducing the risk of low-back pain in military recruits: An open-label randomized controlled trial. Journal of Integrative Medicine, 2022, 20(2): 145-152.
[7] Witoo Dilokthornsakul, Ramanya Kosiyaporn, Rattanamanee Wuttipongwaragon, Piyameth Dilokthornsakul. Potential effects of propolis and honey in COVID-19 prevention and treatment: A systematic review of in silico and clinical studies. Journal of Integrative Medicine, 2022, 20(2): 114-125.
[8] Mahlagha Dehghan, Alireza Ghanbari, Fatemeh Ghaedi Heidari, Parvin Mangalian, Mohammad Ali Zakerid. Use of complementary and alternative medicine in general population during COVID-19 outbreak: A survey in Iran. Journal of Integrative Medicine, 2022, 20(1): 45-51.
[9] Jun-peng Yao, Li-ping Chen, Xian-jun Xiao, Ting-hui Hou, Si-yuan Zhou, Ming-min Xu, Kai Wang, Yu-jun Hou, Lin Zhang, Ying Li. Effectiveness and safety of acupuncture for treating functional constipation: An overview of systematic reviews. Journal of Integrative Medicine, 2022, 20(1): 13-25.
[10] Jing Hong, Xiao-wan Xu, Jing Yang, Jing Zheng, Shu-mei Dai, Ju Zhou, Qing-mei Zhang, Yi Ruan, Chang-quan Ling. Knowledge about, attitude and acceptance towards, and predictors of intention to receive the COVID-19 vaccine among cancer patients in Eastern China: A cross-sectional survey. Journal of Integrative Medicine, 2022, 20(1): 34-44.
[11] Yi Yuan, Rui-Ting Wang, Jun Xia, Hui-Juan Cao. Interventions for preventing influenza: an overview of Cochrane systematic reviews and a Bayesian network meta-analysis. Journal of Integrative Medicine, 2021, 19(6): 503-514.
[12] Andréa D. Fuzimoto. An overview of the anti-SARS-CoV-2 properties of Artemisia annua, its antiviral action, protein-associated mechanisms, and repurposing for COVID-19 treatment. Journal of Integrative Medicine, 2021, 19(5): 375-388.
[13] Xin Yin, Shu-bin Cai, Lan-ting Tao, Lu-ming Chen, Zhong-de Zhang, Su-hong Xiao, Arthur Yin Fan, Xu Zou. Recovery of a patient with severe COVID-19 by acupuncture and Chinese herbal medicine adjuvant to standard care. Journal of Integrative Medicine, 2021, 19(5): 460-466.
[14] Si-chun Gu, Jie Zhou, Qing Ye, Can-xing Yuan. Pingchan granule for depressive symptoms in Parkinson’s disease: A randomized, double-blind, placebo-controlled trial. Journal of Integrative Medicine, 2021, 19(2): 120-128.
[15] Jie Wu, Shu-wei Duan, Hong-tao Yang, Yue-yi Deng, Wei Li, Ya-ni He, Zhao-hui Ni, Yong-li Zhan, Shan Lin, Zhi-yong Guo, Jun Zhu, Jing-ai Fang, Xu-sheng Liu, Li-hua Wang, Rong Wang, Nian-song Wang, Xiao-hong Cheng, Li-qun He, Ping Luo, Shi-ren Sun s, Ji-feng Sun, Ai-ping Yin, Geng-ru Jiang, Hong-yu Chen, Wen-hu Liu, Hong-li Lin, Meng Liang, Lu Ma, Ming Chen, Li-qun Song, Jian Chen, Qing Zhu, Chang-ying Xing, Yun Li, Ji-ning Gao, Rong-shan Li, Ying Li, Hao Zhang, Ying Lu, Qiao-ling Zhou, Jun-zhou Fu, Qiang He, Guang-yan Cai, Xiang-mei Chen. Efficacy and safety of Shenyankangfu Tablet, a Chinese patent medicine, for primary glomerulonephritis: A multicenter randomized controlled trial. Journal of Integrative Medicine, 2021, 19(2): 111-119.
Full text



[1] Dong Yang, Yong-ping Du, Qing Shen, Wei Chen, Yan Yu, Guang-lei Chen. Expression of alpha-smooth muscle actin in renal tubulointerstitium in patients with kidney collateral stasis. Journal of Chinese Integrative Medicine, 2008, 6(1): 41-44
[2] Hai-feng Wei, Bai-liu Ya, Ling Zhao, Cui-fei Ye, Li Zhang, Lin Li. Evaluation of tongue manifestation of blood stasis syndrome and its relationship with blood rheological disorder in a rat model of transient brain ischemia. Journal of Chinese Integrative Medicine, 2008, 6(1): 73-76
[3] Xi Lin, Jian-ping Liu. Herbal medicines for viral myocarditis. Journal of Chinese Integrative Medicine, 2008, 6(1): 76
[4] Xi Lin, Jian-ping Liu. Tai chi for treating rheumatoid arthritis. Journal of Chinese Integrative Medicine, 2008, 6(1): 82
[5] Liang-ping Hu, Hui Gao. Discrimination of errors in statistical analysis of medical papers published in the first issue of 2006 in Journal of Chinese Integrative Medicine. Journal of Chinese Integrative Medicine, 2008, 6(1): 98-106
[6] Yan-bo Zhu , Qi Wang, Cheng-yu Wu, Guo-ming Pang, Jian-xiong Zhao, Shi-lin Shen, Zhong-yuan Xia , Xue Yan . Logistic regression analysis on relationships between traditional Chinese medicine constitutional types and overweight or obesity. Journal of Chinese Integrative Medicine, 2010, 8(11): 1023-1035
[7] Wei Xu, Meng Shi, Jian-gang Liu, Cheng-long Wang . Collagen protein expressions in ischemic myocardium of rats with acute myocardial infarction and effects of qi-tonifying, yin-tonifying and blood-activating herbs and detoxifying and blood-activating herbs. Journal of Chinese Integrative Medicine, 2010, 8(11): 1041-1047
[8] Tao Wang , Feng Qin. Effects of Chinese herbal medicine Xiaoyao Powder on monoamine neurotransmitters in hippocampus of rats with postpartum depression. Journal of Chinese Integrative Medicine, 2010, 8(11): 1075-1079
[9] Ying Xu , Chang-chun Zeng , Xiu-yu Cai , Rong-ping Guo , Guang Nie , Ying Jin. Chromaticity and optical spectrum colorimetry of the tongue color in different syndromes of primary hepatic carcinoma. Journal of Chinese Integrative Medicine, 2012, 10(11): 1263-1271
[10] Xiang-ying Mao , Qin Bian , Zi-yin Shen. Analysis of the osteogenetic effects exerted on mesenchymal stem cell strain C3H10T1/2 by icariin via MAPK signaling pathway in vitro. Journal of Chinese Integrative Medicine, 2012, 10(11): 1272-1278