Search JIM Advanced Search

Journal of Integrative Medicine ›› 2021, Vol. 19 ›› Issue (4): 317-326.doi: 10.1016/j.joim.2021.03.003

• Systematic Review • Previous Articles     Next Articles

Evaluating the methodology of studies conducted during the global COVID-19 pandemic: A systematic review of randomized controlled trials

Meng-zhu Zhaoa,b, Chen Zhaoc, Shuang-shuang Tud, Xu-xu Weie, Hong-cai Shange,f   

  1. a Department of Cardiology, First Teaching Hospital of Tianjin University of Traditional Chinese Medicine, Tianjin 300381, China
    b National Clinical Research Center for Chinese Medicine Acupuncture and Moxibustion, Tianjin 300381, China
    c Institute of Basic Research in Clinical Medicine, China Academy of Chinese Medical Sciences, Beijing 100700, China
    d Editorial Department of Journals, Tianjin University of Traditional Chinese Medicine, Tianjin 301617, China
    e Key Laboratory of Chinese Internal Medicine of Ministry of Education and Beijing, Dongzhimen Hospital Affiliated to Beijing University of Chinese Medicine, Beijing 100700, China
    f International Evidence-based Research Institute of Chinese Medicine, Beijing University of Chinese Medicine, Beijing 100029, China
  • Received:2020-10-09 Accepted:2021-02-22 Online:2021-08-15 Published:2021-03-24
  • Contact: Hong-cai Shang

The therapeutic evidence collected from well-designed studies is needed to help manage the global pandemic of the coronavirus disease 2019 (COVID-19). Evaluating the quality of therapeutic data collected during this most recent pandemic is important for improving future clinical research under similar circumstances.

To assess the methodological quality and variability in implementation of randomized controlled trials (RCTs) for treating COVID-19, and to analyze the support that should be provided to improve data collected during an urgent pandemic situation.

Search strategy
PubMed, Excerpta Medica Database, China National Knowledge Infrastructure, Wanfang, and Chongqing VIP, and the preprint repositories including Social Science Research Network and MedRxiv were systematically searched, up to September 30, 2020, using the keywords “coronavirus disease 2019 (COVID-19),” “2019 novel coronavirus (2019-nCoV),” “severe acute respiratory syndrome-related coronavirus-2 (SARS-CoV-2),” “novel coronavirus pneumonia (NCP),” “randomized controlled trial (RCT)” and “random.”

Inclusion criteria
RCTs studying the treatment of COVID-19 were eligible for inclusion. 

Data extraction and analysis
Screening of published RCTs for inclusion and data extraction were each conducted by two researchers. Analysis of general information on COVID-19 RCTs was done using descriptive statistics. Methodological quality was assessed using the risk-of-bias tools in the Cochrane Handbook for Systematic Reviews of Interventions (Version 5.1.0). Variability in implementation was assessed by comparing consistency between RCT reports and registration information. 

A total of 5886 COVID-19 RCTs were identified. Eighty-one RCTs were finally included, of which, 45 had registration information. Methodological quality of the RTCs was not optimal due to deficiencies in five main domains: allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, and selective reporting. Comparisons of consistency between published protocols and registration information showed that the 45 RCTs with registration information had common deviations in seven items: inclusion and exclusion criteria, sample size, outcomes, research sites of recruitment, interventions, and blinding.

The methodological quality of COVID-19 RCTs conducted in early to mid 2020 was consistently low and variability in implementation was common. More support for implementing high-quality methodology is needed to obtain the quality of therapeutic evidence needed to provide positive guidance for clinical care. We make an urgent appeal for accelerating the construction of a collaborative sharing platform and preparing multidisciplinary talent and professional teams to conduct excellent clinical research when faced with epidemic diseases of the future. Further, variability in RCT implementation should be clearly reported and interpreted to improve the utility of data resulting from those trials.

Key words: COVID-19, Randomized controlled trial, Therapeutic evidence, Methodology, Systematic review

[1] Xin Yin, Shu-bin Cai, Lan-ting Tao, Lu-ming Chen, Zhong-de Zhang, Su-hong Xiao, Arthur Yin Fan, Xu Zou. Recovery of a patient with severe COVID-19 by acupuncture and Chinese herbal medicine adjuvant to standard care. Journal of Integrative Medicine, 2021, 19(5): 1-.
[2] Si-chun Gu, Jie Zhou, Qing Ye, Can-xing Yuan. Pingchan granule for depressive symptoms in Parkinson’s disease: A randomized, double-blind, placebo-controlled trial. Journal of Integrative Medicine, 2021, 19(2): 120-128.
[3] Jie Wu, Shu-wei Duan, Hong-tao Yang, Yue-yi Deng, Wei Li, Ya-ni He, Zhao-hui Ni, Yong-li Zhan, Shan Lin, Zhi-yong Guo, Jun Zhu, Jing-ai Fang, Xu-sheng Liu, Li-hua Wang, Rong Wang, Nian-song Wang, Xiao-hong Cheng, Li-qun He, Ping Luo, Shi-ren Sun s, Ji-feng Sun, Ai-ping Yin, Geng-ru Jiang, Hong-yu Chen, Wen-hu Liu, Hong-li Lin, Meng Liang, Lu Ma, Ming Chen, Li-qun Song, Jian Chen, Qing Zhu, Chang-ying Xing, Yun Li, Ji-ning Gao, Rong-shan Li, Ying Li, Hao Zhang, Ying Lu, Qiao-ling Zhou, Jun-zhou Fu, Qiang He, Guang-yan Cai, Xiang-mei Chen. Efficacy and safety of Shenyankangfu Tablet, a Chinese patent medicine, for primary glomerulonephritis: A multicenter randomized controlled trial. Journal of Integrative Medicine, 2021, 19(2): 111-119.
[4] Xing Zhang, YanXue, Xuan Chen, Jia-min, Wu, Zi-jian Su, Meng Sun, Lu-jiong Liu, Yi-bao Zhang, Yi-le Zhang, Gui-hua Xu, Miao-yan Shi, Xiu-ming Song, Yun-fei Lu, Xiao-rong Chen, Wei Zhang, Qi Chen. Effects of Tanreqing Capsule on the negative conversion time of nucleic acid in patients with COVID-19: A retrospective cohort study. Journal of Integrative Medicine, 2021, 19(1): 36-41.
[5] Xian Wang, Xuan Yin, Xiu-tian Guo, Yan Wang, Wen-qi Jin, Ai-jun Mao, Lixing Lao, Zhang-jin Zhang, Jie Zhang, Shi-fen Xu. Effects of the pestle needle therapy, a type of acupoint stimulation, on post-hemorrhoidectomy pain: A randomized controlled trial. Journal of Integrative Medicine, 2020, 18(6): 492-498.
[6] Liang Dai, Ka Kei Chan, Jian-chun Mao, Yu Tian, Jun-hua Gu, Jun Zhou, Linda L.D. Zhong. Modified Zhibai Dihuang pill, a traditional Chinese medicine formula, on steroid withdrawal in systemic lupus erythematosus: A systematic review and meta-analysis. Journal of Integrative Medicine, 2020, 18(6): 478-491.
[7] Jonathan M. Fields. Dangers of scientific bias against herbal drugs for coronavirus disease 2019. Journal of Integrative Medicine, 2020, 18(6): 459-461.
[8] Arthur Yin Fan, Sherman Gu, Sarah Faggert Alemi, Research Group for Evidence-based Chinese Medicine. Chinese herbal medicine for COVID-19: Current evidence with systematic review and meta-analysis. Journal of Integrative Medicine, 2020, 18(5): 385-394.
[9] Jian-zhong Dang, Gang-yan Zhu, Ying-jie Yang, Fang Zheng. Clinical characteristics of coronavirus disease 2019 in patients aged 80 years and older. Journal of Integrative Medicine, 2020, 18(5): 395-400.
[10] Dong-dong Yu, Liang-zhen You, Wan-qiu Huang, Hui Cao, Fan-jing Wang, Xiu-qin Tang, Zhao-hui Fang, Guo-ming Shen, Yu-xiang Guan. Effects of traditional Chinese exercises on blood glucose and hemoglobin A1c levels in patients with prediabetes: A systematic review and meta-analysis. Journal of Integrative Medicine, 2020, 18(4): 292-302.
[11] Xian-jun Xiao, Lei-xiao Zhang, Yun-zhou Shi, Jun-peng Yao, Wei Cao, Ying Liu, Zi-hao Zou, Si-yuan Zhou, Ming-ling Chen, Chun-xiao Li, Qian-hua Zheng, Ying Li. Cupping therapy for patients with chronic urticaria: A systematic review and meta-analysis. Journal of Integrative Medicine, 2020, 18(4): 303-312.
[12] Zi-jia Zhang, Wen-yong Wu, Jin-jun Hou, Lin-lin Zhang, Fei-fei Li, Lei Gao, Xing-dong Wu, Jing-ying Shi, Rong Zhang, Hua-li Long, Min Lei, Wan-ying Wu, De-an Guo, Kai-xian Chen, Lewis A. Hofmann, Zhonghua Ci. Active constituents and mechanisms of Respiratory Detox Shot, a traditional Chinese medicine prescription, for COVID-19 control and prevention: network-molecular docking-LC-MSE analysis. Journal of Integrative Medicine, 2020, 18(3): 229-241.
[13] Sherman Gu, Arthur Yin Fan. Controversial conclusions from two randomized controlled trials for acupuncture's effects on polycystic ovary syndrome or in vitro fertilization support. Journal of Integrative Medicine, 2020, 18(2): 89-91.
[14] Chang-quan Ling. Traditional Chinese medicine is a resource for drug discovery against 2019 novel coronavirus (SARS-CoV-2). Journal of Integrative Medicine, 2020, 18(2): 87-88.
[15] Gerli E.G. Herr, Fábio Goulart da Silva, Francisco José Cidral-Filho, Fabricia Petronilho, Lucinéia Gainski Danielski, Mariana Pereira de Souza Goldim, Afonso Shiguemi Inoue Salgado, Franciane Bobinski, Daniel Fernandes Martins, Eliane R. Winkelmann. Effects of the use of bioceramic wraps in patients with lower limb venous ulcers: A randomized double-blind placebo-controlled trial. Journal of Integrative Medicine, 2020, 18(1): 27-.
Full text



[1] Jin-zhou Tian, Jing Shi, Xin-qing Zhang, Qi Bi, Xin Ma, Zhi-liang Wang, Xiao-bin Li, Shu-li Shen, Lin Li, Zhen-yun Wu, Li-yan Fang, Xiao-dong Zhao, Ying-chun Miao, Peng-wen Wang, Ying Ren, Jun-xiang Yin, Yong-yan Wang, Beijing United Study Group on MCI of the Capital Foundation of Medical Developments. An explanation on "guiding principles of clinical research on mild cognitive impairment (protocol)". Journal of Chinese Integrative Medicine, 2008, 6(1): 15-21
[2] Yi-ting He, Qing-lin Zha, Jian-ping Yu, Yong Tan, Cheng Lu, Ai-ping Lv. Principal factor analysis of symptoms of rheumatoid arthritis and their correlations with efficacy of traditional Chinese medicine and Western medicine. Journal of Chinese Integrative Medicine, 2008, 6(1): 32-36
[3] Jun Cai, Hua Wang, Sheng Zhou, Bin Wu, Hua-rong Song, Zheng-rong Xuan. Effect of Sijunzi Decoction and enteral nutrition on T-cell subsets and nutritional status in patients with gastric cancer after operation: A randomized controlled trial. Journal of Chinese Integrative Medicine, 2008, 6(1): 37-40
[4] Wei Zhang, Xiang-feng Lu, Xiao-mei Zhang, Jian-jun Wu, Liang-duo Jiang. A rat model of pulmonary fibrosis induced by infusing bleomycin quickly through tracheal intubation. Journal of Chinese Integrative Medicine, 2008, 6(1): 60-67
[5] A-gao Zhou, Yong Zhang, Gang Kui, De-Yun Kong, Hai-liang Ge, Qiu-hua Ren, Jia-rong Dong, Sheng Hong, Xu-ming Mao, Yin Wang, Hui-zheng Zhang, Shu-jun Wang. Influence of traditional Chinese compound recipes with different efficacy on body weight, tumor weight and immune function in H22 cancer-bearing mice. Journal of Chinese Integrative Medicine, 2008, 6(1): 77-82
[6] Guo-hong Yuan, Xiao-jing Pang, He-chao Ma. Synergic effects of Danggui Buxue Decoction in reducing toxicity of cytoxan in tumor-bearing mice. Journal of Chinese Integrative Medicine, 2008, 6(1): 83-88
[7] Li Zhou, Hong-xing Zhang, Ling-guang Liu, Wen-jun Wan. Effect of electro-acupuncture at Fenglong (GV 16) on nitric oxide and endothelin in rats with hyperlipidemia. Journal of Chinese Integrative Medicine, 2008, 6(1): 89-92
[8] Jin-zhou Tian, Jing Shi, Xin-qing Zhang, Qi Bi, Xin Ma, Zhi-liang Wang, Xiao-bin Li, Shu-li Shen, Lin Li, Zhen-yun Wu, Li-yan Fang, Xiao-dong Zhao, Ying-chun Miao, Peng-wen Wang, Ying Ren, Jun-xiang Yin, Yong-yan Wang, Beijing United Study Group on MCI of the Capital Foundation of Medical Developments. Guiding principles of clinical research on mild cognitive impairment (protocol). Journal of Chinese Integrative Medicine, 2008, 6(1): 9-14
[9] Xin-jun Wang, Ling-ling Wang . A mechanism of endogenous opioid peptides for rapid onset of acupuncture effect in treatment of depression. Journal of Chinese Integrative Medicine, 2010, 8(11): 1014-1017
[10] Bo Wang , Wei Yan , Li-hui Hou, Xiao-ke Wu. Disorder of Tiangui (kidney essence) and reproductive dysfunction in patients with polycystic ovary syndrome. Journal of Chinese Integrative Medicine, 2010, 8(11): 1018-1022