Search JIM Advanced Search

Journal of Chinese Integrative Medicine ›› 2012, Vol. 10 ›› Issue (3): 271-278.doi: 10.3736/jcim20120305

• Review • Previous Articles     Next Articles

Modern testing theory and its application in the field of health measurement

Da-rong Wu()   

  1. Applied Clinical Epidemiology Research Unit, the Second Affiliated Hospital (Guangdong Provincial Hospital of Chinese Medicine), Guangzhou University of Chinese Medicine, Guangzhou 510120, Guangdong Province, China
  • Received:2011-09-28 Accepted:2011-11-19 Online:2012-03-20 Published:2018-04-15

This paper briefly introduces item response theory (IRT) as a typical representation of modern testing theory (MTT), and systematically reviews the processes and contents of the application of IRT in the area of health measurement, including, for example, item bank development, scale revision and computerized adaptive testing. The author presents the potential benefits and the notable problems during health measuring by IRT. Then, the author asserts the need for thorough assessment of feasibility when using the IRT in patient-reported outcome research. Further research based on IRT and computerized adaptive testing in health measurement will be carried out in the field of medical care including traditional Chinese medicine and integrative medicine.

Key words: modern testing theory, item response theory, health measurement, reviews

[1] Crocker L, Algina J. Introduction to classical and modern test theory. Translated by Jin Y. Shanghai: East China Normal University Press. 2004: 5, 7-8. Chinese.
Crocker L, Algina J. 经典和现代测验理论导论.金瑜译.上海: 华东师范大学出版社. 2004: 5, 7-8.
[2] DeVellis RF . Classical test theory. Med Care. 2006; 44(11 Suppl 3) : S50-S59.
doi: 10.1097/01.mlr.0000245426.10853.30
[3] Embretson SE, Reise SP . Item response theory for psychologists. London: Lawrence Erlbaum Associates. 2000: 15.
[4] Baker FB . The basics of item response theory. ERIC Clearinghouse on Assessment and Evaluation. 2nd ed. 2001: 1.
[5] Yuan CY . IRT : the theory and its applications. Nan Hua Da Xue Xue Bao She Hui Ke Xue Ban. 2001 ; 2(4) : 69-71. Chinese with abstract in English.
袁春阳 . IRT:理论与应用. 南华大学学报(社会科学版). 2001; 2(4):69-71.
[6] Hays RD, Lipscomb J . Next steps for use of item response theory in the assessment of health outcomes. Qual Life Res. 2007 ; 16(Suppl 1) : 195-199.
doi: 10.1007/s11136-007-9175-7
[7] Celia D, Yount S, Rothrock N, Gershon R, Cook K, Reeve B, Ader D, Fries JF, Bruce B, Rose M; PROMIS Cooperative Group . The Patient-Reported Outcomes Measurement Information System (PROMTS): progress of an NIH Roadmap cooperative group during its first two years. Med Care. 2007 ; 45(5 Suppl 1):S3-S11.
[8] Hay R . Promis_wl_final_20110108_executive_summary- Version 1 January 8, 2011. [ 2011-12-10]. .
[9] Hays RD, Liu H, Spritzer K, Celia D . Item response theory analyses of physical functioning items in the medical outcomes study. Med Care. 2007 ; 45(5 Suppl 1):S32-S38.
doi: 10.1097/01.mlr.0000246649.43232.82
[10] Kopec JA, Sayre EC, Davis AM, Badley EM, Abraha- mowicz M, Sherlock L, Williams JI, Anis AH, Esdaile JM . Assessment of health-related quality of life in arthritis : conceptualization and development of five item banks using item response theory. Health Qual Life Outcomes. 2006; 4:33.
doi: 10.1186/1477-7525-4-33
[11] Bjorner JB, Kosinski M, Ware JE Jr . Calibration of an item pool for assessing the burden of headaches: an application of item response theory to the headache impact test (HIT™). Qual Life Res. 2003; 12(8):913-933.
doi: 10.1023/A:1026163113446
[12] Lai JS, Celia D, Dineen K, Bode R, Von Roenn J, Gershon RC, Shevrin D . An item bank was created to improve the measurement of cancer-related fatigue. J Clin Epidemiol. 2005; 58(2):190-197.
doi: 10.1016/j.jclinepi.2003.07.016
[13] Lai JS, Dineen K, Reeve BB, Von Roenn J, Shervin D, McGuire M, Bode RK, Paice J, Celia D . An item response theory-based pain item bank can enhance measurement precision. J Pain Symptom Manage. 2005; 30(3):278-288.
doi: 10.1016/j.jpainsymman.2005.03.009
[14] Lai JS, Celia D, Chang CH, Bode RK, Heinemann AW . Item banking to improve, shorten and computerize self-reported fatigue: an illustration of steps to create a core item bank from the FACIT-Fatigue Scale. Qual Life Res. 2003; 12(5):485-501.
doi: 10.1023/A:1025014509626
[15] Teresi JA, Fleishman JA . Differential item functioning and health assessment. Qual Life Res. 2007 ; 16(Suppl 1) : 33-42.
doi: 10.1007/s11136-007-9184-6
[16] Pagano IS, Gotay CC . Ethnic differential item functioning in the assessment of quality of life in cancer patients. Health Qual Life Outcomes. 2005; 3:60.
doi: 10.1186/1477-7525-3-60
[17] Crane PK, Cetin K, Cook KF, Johnson K, Deyo R, Amtmann D . Differential item functioning impact in a modified version of the Roland-Morris Disability Questionnaire. Qual Life Res. 2007; 16(6):981-990.
doi: 10.1007/s11136-007-9200-x
[18] Noerholm V, Groenvold M, Watt T, Bjorner JB, Rasmussen NA, Bech P . Quality of life in the Danish general population — normative data and validity of WHOQOL-BREF using Rasch and item response theory models. Qual Life Res. 2004; 13(2):531-540.
doi: 10.1023/B:QURE.0000018485.05372.d6
[19] Weinstock LM, Strong D, Uebelacker LA, Miller IW . Differential item functioning of DSM-JY depressive symptoms in individuals with a history of mania versus those without: an item response theory analysis. Bipolar Disord. 2009; 11(3):289-297.
doi: 10.1111/bdi.2009.11.issue-3
[20] Tsutsumi A, Iwata N, Watanabe N, de Jonge J, Pikhart H, Fernández-López JA, Xu L, Peter R, Knutsson A, Niedhammer I, Kawakami N, Siegrist J . Application of item response theory to achieve cross- cultural comparability of occupational stress measurement. Int J Methods Psychiatr Res. 2009; 18(1):58-67.
doi: 10.1002/mpr.v18:1
[21] Uebelacker LA, Strong D, Weinstock LM, Miller IW . Use of item response theory to understand differential functioning of DSM-JY major depression symptoms by race, ethnicity and gender. Psychol Med. 2009; 39(4):591-601.
doi: 10.1017/S0033291708003875
[22] Bjorner JB, Petersen MA, Groenvold M, Aaronson N, Ahlner-Elmqvist M, Arraras JI, Brédart A, Fayers P, Jordhoy M, Sprangers M, Watson M, Young T; European Organisation for Research and Treatment of Cancer Quality of Life Group. Use of item response theory to develop a shortened version of the EORTC QLQ-C30 emotional functioning scale. Qual Life Res. 2004; 13(10) : 1683-1697.
doi: 10.1007/s11136-004-7866-x
[23] Misajon R, Hawthorne G, Richardson J, Barton J, Peacock S, Iezzi A, Keeffe J . Vision and quality of life: the development of a utility measure. Invest Ophthalmol Vis Sci. 2005; 46(11):4007-4015.
doi: 10.1167/iovs.04-1389
[24] Atroshi I, Lyrén PE, Gummesson C . The 6-item CTS symptoms scale: a brief outcomes measure for carpal tunnel syndrome. Qual Life Res. 2009 ; 18(3) : 347-358.
doi: 10.1007/s11136-009-9449-3
[25] Takegami M, Suzukamo Y, Wakita T, Noguchi H, Chin K, Kadotani H, Inoue Y, Oka Y, Nakamura T, Green J, Johns MW, Fukuhara S . Development of a Japanese version of the Epworth Sleepiness Scale (JESS) based on item response theory. Sleep Med. 2009; 10(5) : 556-565.
doi: 10.1016/j.sleep.2008.04.015
[26] Adair CE, Marcoux GC, Cram BS, Ewashen CJ, Chafe J, Cassin SE, Pinzon J, Gusella JL, Geller J, Scattolon Y, Fergusson P, Styles L, Brown KE . Development and multi-site validation of a new condition-specific quality of life measure for eating disorders. Health Qual Life Outcomes. 2007 ; 5:23.
doi: 10.1186/1477-7525-5-23
[27] Chiang KS, Green KE, Cox EO . Rasch analysis of the Geriatric Depression Scale-Short Form. Gerontologist. 2009; 49(2) : 262-275.
doi: 10.1093/geront/gnp018
[28] Wolfe F, Kong SX . Rasch analysis of the Western Ontario MacMaster questionnaire (WOMAC) in 2 205 patients with osteoarthritis, rheumatoid arthritis, and fibromyalgia. Ann Rheum Dis. 1999; 58(9):563-568.
doi: 10.1136/ard.58.9.563
[29] Metz SM, Wyrwich KW, Babu AN, Kroenke K, Tierney WM, Wolinsky FD . A comparison of traditional and Rasch cut points for assessing clinically important change in health-related quality of life among patients with asthma. Qual Life Res. 2006; 15(10):1639-1649.
doi: 10.1007/s11136-006-0036-6
[30] Van der Linden WJ, Glas CAW . Computerized adaptive testing : theory and practice. Dordrecht: Kluwer Academic Publishers. 2000: Preface.
[31] Cook KF, O'Malley KJ, Roddey TS . Dynamic assessment of health outcomes : time to let the CAT out of the bag? Health Ser Res. 2005; 40(5 Pt 2) : 1694-1711.
doi: 10.1111/hesr.2005.40.issue-5p2
[32] Ware JE Jr, Kosinski M, Bjorner JB, Bayliss MS, Batenhorst A, Dahlof CG, Tepper S, Dowson A . Applications of computerized adaptive testing (CAT) to the assessment of headache impact. Qual Life Res. 2003; 12(8):935-952.
doi: 10.1023/A:1026115230284
[33] Fliege H, Becker J, Walter OB, Bjorner JB, Klapp BF, Rose M . Development of a computer-adaptive test for depression (D-CAT). Qual Life Res. 2005; 14(10):2277-2291.
doi: 10.1007/s11136-005-6651-9
[34] Kosinski M, Bjorner JB, Ware JE Jr, Sullivan E, Straus WL . An evaluation of a patient-reported outcomes found computerized adaptive testing was efficient in assessing osteoarthritis impact. J Clin Epidemiol. 2006 ; 59(7) : 715-723.
doi: 10.1016/j.jclinepi.2005.07.019
[35] Gibbons RD, Weiss DJ, Kupfer DJ, Frank E, Fagiolini A, Grochocinski VJ, Bhaumik DK, Stover A, Bock RD, Immekus JC . Using computerized adaptive testing to reduce the burden of mental health assessment. Psychiatr Serv. 2008; 59(4):361-368.
doi: 10.1176/ps.2008.59.4.361
[36] Haley SM, Ni P, Jette AM, Tao W, Moed R, Meyers D, Ludlow LH . Replenishing a computerized adaptive test of patient-reported daily activity functioning. Qual Life Res. 2009; 18(4):461-471.
doi: 10.1007/s11136-009-9463-5
[37] Hart DL, Mioduski JE, Stratford PW . Simulated computerized adaptive tests for measuring functional status were efficient with good discriminant validity in patients with hip, knee, or foot/ankle impairments. J Clin Epidemiol. 2005; 58(6):629-638.
doi: 10.1016/j.jclinepi.2004.12.004
[38] Bjorner JB, Kosinski M, Ware JE Jr . The feasibility of applying item response theory to measures of migraine impact: a re-analysis of three clinical studies. Qual Life Res. 2003; 12(8) : 887-902.
[39] Kosinski M, Bjorner JB, Ware JE Jr, Batenhorst A, Cady RK . The responsiveness of headache impact scales scored using 4classical 5 and 4modern 5 psychometric methods: a re-analysis of three clinical trials . Qual Life Res. 2003; 12(8) : 903-912.
doi: 10.1023/A:1026111029376
[40] Bjorner JB, Kosinski M, Ware JE Jr . Using item response theory to calibrate the Headache Impact Test (HIT) to the metric of traditional headache scales. Qual Life Res. 2003; 12(8):981-1002.
doi: 10.1023/A:1026123400242
[41] Petersen MA, Groenvold M, Aaronson N, Brenne E, Fayers P, Nielsen JD, Sprangers M, Bjorner JB; for the European Organisation for Research and Treatment of Cancer Quality of Life Group . Scoring based on item response theory did not alter the measurement ability of EORTC QLQ-C30 scales. J Clin Epidemiol. 2005; 58(9):902-908.
doi: 10.1016/j.jclinepi.2005.02.008
[42] Allison KC, Engel SG, Crosby RD, De Zwaan M, O'Reardon JP, Wonderlich SA, Mitchell JE, West DS, Wadden TA, Stunkard AJ . Evaluation of diagnostic criteria for night eating syndrome using item response theory analysis . Eat Behav. 2008; 9(4):398-407.
doi: 10.1016/j.eatbeh.2008.04.004
[43] Wu LT, Pan JJ, Blazer DG, Tai B, Stitzer ML, Brooner RK, Woody GE, Patkar AA, Blaine JD . An item response theory modeling of alcohol and marijuana dependences: a National Drug Abuse Treatment Clinical Trials Network study. J Stud Alcohol Drugs. 2009; 70(3):414-425.
doi: 10.15288/jsad.2009.70.414
[44] Jiang Y, Hesser JE . Using item response theory to analyze the relationship between health-related quality of life and health risk factors. Prev Chronic Dis. 2009 ; 6(1) : A30.
[45] Chan KS, Orlando M, Ghosh-Dastidar B, Duan N, Sherbourne CD . The interview mode effect on the Center for Epidemiological Studies Depression (CES-D) scale: an item response theory analysis. Med Care. 2004; 42(3) : 281-289.
doi: 10.1097/01.mlr.0000115632.78486.1f
[46] Hahn EA, Celia D, Dobrez DG, Weiss BD, Du H, Lai JS, Victorson D, Garcia SF . The impact of literacy on health-related quality of life measurement and outcomes in cancer outpatients. Qual Life Res. 2007 ; 16(3):495-507.
doi: 10.1007/s11136-006-9128-6
[47] Birbeck GL, Kim S, Hays RD, Vickrey BG . Quality of life measures in epilepsy: how well can they detect change over time? Neurology. 2000 ; 54(9 ): 1822-1827.
doi: 10.1212/WNL.54.9.1822
[48] Hill CD, Edwards MC, Thissen D, Langer MM, Wirth RJ, Bur winkle TM, Varni JW . Practical issues in the application of item response theory : a demonstration using items from the pediatric quality of life inventory (PedsQL) 4.0 generic core scales. Med Care. 2007 ; 45(5 Suppl 1) : S39-S47.
doi: 10.1097/01.mlr.0000259879.05499.eb
[49] Masse LC, Heesch KC, Eason KE, Wilson M . Evaluating the properties of a stage-specific self-efficacy scale for physical activity using classical test theory, confirmatory factor analysis and item response modeling. Health Educ Res. 2006; 21(Suppll):i33-i46.
doi: 10.1093/her/cyl106
[50] Tao W, Haley SM, Coster WJ, Ni P, Jette AM . An exploratory analysis of functional staging using an item response theory approach. Arch Phys Med Rehabil. 2008; 89(6) : 1046-1053.
doi: 10.1016/j.apmr.2007.11.036
[51] Jenkinson C, Fitzpatrick R, Garratt A, Peto V, Stewart-Brown S . Can item response theory reduce patient burden when measuring health status in neurological disorders? Results from Rasch analysis of the SF-36 physical functioning scale (PF-10). J Neurol Neurosurg Psychiatry. 2001; 71(2):220-224.
doi: 10.1136/jnnp.71.2.220
[52] Van Nispen RM, Knol DL, Langelaan M, de Boer MR, Ter wee CB, van Rens GH . Applying multilevel item response theory to vision-related quality of life in Dutch visually impaired elderly. Optom Vis Sci. 2007 ; 84(8):710-720.
doi: 10.1097/OPX.0b013e31813375b8
[53] Hays RD, Morales LS, Reise SP . Item response theory and health outcomes measurement in the 21st century. Med Care. 2000; 38(9 Suppl):1128-1142.
[54] Cook KF, Monahan PO , McHorney CA. Delicate balance between theory and practice : health status assessment and item response theory. Med Care. 2003; 41(5) : 571-574.
[55] Zhang MQ , Liu XY. A study on the applying of item response models. Xin Li Xue Bao. 1998; 30(4): 436- 441. Chinese with abstract in English.
张敏强, 刘晓瑜 . 项目反应模型的应用问题研究.心理学报. 1998; 30(4):436-441.
[56] Cook KF, Teal CR, Bjorner JB, Celia D, Chang CH, Crane PK, Gibbons LE, Hays RD, McHorney CA, Ocepek-Welikson K, Raczek AE, Teresi JA, Reeve BB . IRT health outcomes data analysis project: an overview and summary. Qual Life Res. 2007 ; 16(Suppl 1):121-132.
doi: 10.1007/s11136-007-9177-5
[57] Chang CH . Patient-reported outcomes measurement and management with innovative methodologies and technologies. Qual Life Res. 2007; 16(Suppll):157-166.
doi: 10.1007/s11136-007-9196-2
[58] Unlii A . A note on monotone likelihood ratio of the total score variable in unidimensional item response theory. Br J Math Stat Psychol. 2008; 61(Ptl):179-187.
doi: 10.1348/000711007X173391
[59] Sinharay S . Bayesian item fit analysis for unidimensional item response theory models. Br J Math Stat Psychol. 2006; 59(Pt 2) : 429-449.
doi: 10.1348/000711005X66888
[60] Cai L, Maydeu-Olivares A, Coffman DL, Thissen D . Limited-information goodness-of-fit testing of item response theory models for sparse 2 tables. Br J Math Stat Psychol. 2006; 59(Pt 1) : 173-194.
doi: 10.1348/000711005X66419
[61] Meredith W, Teresi JA . An essay on measurement and factorial invariance. Med Care. 2006; 44(11 Suppl 3):S69-S77.
doi: 10.1097/01.mlr.0000245438.73837.89
[62] Fayers PM . Applying item response theory and computer adaptive testing : the challenges for health outcomes assessment. Qual Life Res. 2007 ; 16(Suppl 1) : 187-194.
doi: 10.1007/s11136-007-9197-1
[1] Vamsi Reddy, Arvind Sridhar, Roberto F. Machado, Jiwang Chen. High sodium causes hypertension: Evidence from clinical trials and animal experiments. Journal of Integrative Medicine, 2015, 13(1): 1-8.
[2] Atefeh Arabzadeh, Mehdi Ajdari Tafti, Mohammad M. Zarshenas. The very early textbook of pediatrics: Tadbir-Al-Sebyān. Journal of Integrative Medicine, 2014, 12(6): 531-532.
[3] Hui-ling Wang, Nan-mei Liu, Rui Li . Role of adult resident renal progenitor cells in tubular repair after acute kidney injury. Journal of Integrative Medicine, 2014, 12(6): 469-475.
[4] Chang-qing Zhao, Yang Zhou, Jian Ping, Lie-ming Xu. Traditional Chinese medicine for treatment of liver diseases: Progress, challenges and opportunities. Journal of Integrative Medicine, 2014, 12(5): 401-408.
[5] Zemin Yao​, Li Zhang​, Guang Ji​. Efficacy of polyphenolic ingredients of Chinese herbs in treating dyslipidemia of metabolic syndromes. Journal of Integrative Medicine, 2014, 12(3): 135-146.
[6] Shu Dong, Shi-bing Su​. Advances in mesenchymal stem cells combined with traditional Chinese medicine therapy for liver fibrosis. Journal of Integrative Medicine, 2014, 12(3): 147-155.
[7] Bai-xiao Zhao, Hai-Yong Chen, Xue-yong Shen, Lixing Lao. Can moxibustion, an ancient treatment modality, be evaluated in a double-blind randomized controlled trial? — A narrative review. Journal of Integrative Medicine, 2014, 12(3): 131-134.
[8] Yelaware Puttaswamy Naveen, Gunashekar Divya Rupini, Faiyaz Ahmed, Asna Urooj. Pharmacological effects and active phytoconstituents of Swietenia mahagoni: A review. Journal of Integrative Medicine, 2014, 12(2): 86-93.
[9] Cheng Huang​. Natural modulators of liver X receptors. Journal of Integrative Medicine, 2014, 12(2): 76-85.
[10] Chang-quan Ling​, Li-na Wang, Yuan Wang​, Yuan-hui Zhang​, Zi-fei Yin, Meng Wang​, Chen Ling​. The roles of traditional Chinese medicine in gene therapy. Journal of Integrative Medicine, 2014, 12(2): 67-75.
[11] Xin-lin Chen, Feng-bin Liu, Li Guo, Xiao-bin Liu. Development of patient-reported outcome scale for myasthenia gravis: A psychometric test. Journal of Chinese Integrative Medicine, 2010, 8(2): 121-125.
[12] Yong Miao Edwin. Clinical critical qualitative evaluation of the selected randomized controlled trials in current acupuncture researches for low back pain. Journal of Chinese Integrative Medicine, 2010, 8(12): 1133-1146.
[13] Jia-qing Zhang. Pondering over some blunders on scientificity in diabetic researches by integrated Chinese and western medicine. Journal of Chinese Integrative Medicine, 2004, 2(1): 3-6.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] Dong Yang, Yong-ping Du, Qing Shen, Wei Chen, Yan Yu, Guang-lei Chen. Expression of alpha-smooth muscle actin in renal tubulointerstitium in patients with kidney collateral stasis. Journal of Chinese Integrative Medicine, 2008, 6(1): 41-44
[2] Hai-feng Wei, Bai-liu Ya, Ling Zhao, Cui-fei Ye, Li Zhang, Lin Li. Evaluation of tongue manifestation of blood stasis syndrome and its relationship with blood rheological disorder in a rat model of transient brain ischemia. Journal of Chinese Integrative Medicine, 2008, 6(1): 73-76
[3] Xi Lin, Jian-ping Liu. Herbal medicines for viral myocarditis. Journal of Chinese Integrative Medicine, 2008, 6(1): 76
[4] Xi Lin, Jian-ping Liu. Tai chi for treating rheumatoid arthritis. Journal of Chinese Integrative Medicine, 2008, 6(1): 82
[5] Liang-ping Hu, Hui Gao. Discrimination of errors in statistical analysis of medical papers published in the first issue of 2006 in Journal of Chinese Integrative Medicine. Journal of Chinese Integrative Medicine, 2008, 6(1): 98-106
[6] Yan-bo Zhu , Qi Wang, Cheng-yu Wu, Guo-ming Pang, Jian-xiong Zhao, Shi-lin Shen, Zhong-yuan Xia , Xue Yan . Logistic regression analysis on relationships between traditional Chinese medicine constitutional types and overweight or obesity. Journal of Chinese Integrative Medicine, 2010, 8(11): 1023-1035
[7] Wei Xu, Meng Shi, Jian-gang Liu, Cheng-long Wang . Collagen protein expressions in ischemic myocardium of rats with acute myocardial infarction and effects of qi-tonifying, yin-tonifying and blood-activating herbs and detoxifying and blood-activating herbs. Journal of Chinese Integrative Medicine, 2010, 8(11): 1041-1047
[8] Tao Wang , Feng Qin. Effects of Chinese herbal medicine Xiaoyao Powder on monoamine neurotransmitters in hippocampus of rats with postpartum depression. Journal of Chinese Integrative Medicine, 2010, 8(11): 1075-1079
[9] Ying Xu , Chang-chun Zeng , Xiu-yu Cai , Rong-ping Guo , Guang Nie , Ying Jin. Chromaticity and optical spectrum colorimetry of the tongue color in different syndromes of primary hepatic carcinoma. Journal of Chinese Integrative Medicine, 2012, 10(11): 1263-1271
[10] Xiang-ying Mao , Qin Bian , Zi-yin Shen. Analysis of the osteogenetic effects exerted on mesenchymal stem cell strain C3H10T1/2 by icariin via MAPK signaling pathway in vitro. Journal of Chinese Integrative Medicine, 2012, 10(11): 1272-1278