Rand Ucla Appropriateness Method User's Manual

Posted : admin On 12/26/2021
Int J Health Policy Manag. 2013 Jun; 1(1): 17–21.

This manual presents step-by-step guidelines for conceptualising, designing, and carrying out a study of the appropriateness of medical or surgical procedures (for either diagnosis or treatment) using theRAND/UCLA Appropriateness Method. The manual distills the experience of many researchers in North America and Europe and presents current (as. Although the manual is selfcontained and complete, the authors do not recommend that those unfamiliar with the Rand/UCLA Appropriateness Method independently conduct an appropriateness study; instead, they suggest 'seeing one' before 'doing one.' To this end, contact information is provided to assist potential users of the method.

  1. Get this from a library! The Rand/UCLA appropriateness method user's manual. Kathryn Fitch; - Health systems should function in such a way that the amount of inappropriate care is minimized, while at the same time stinting as little as possible on appropriate and necessary care.
  2. The RAND/UCLA Appropriateness Method User's Manual The concepts of appropriate and necessary care are fundamental to the creation of an efficient and equitable health-care delivery system.
Published online 2013 Apr 10. doi: 10.15171/ijhpm.2013.04
PMID: 24596832
This article has been cited by other articles in PMC.

Associated Data

Supplementary Materials
ijhpm-1-17-s001.pdf (128K)
ijhpm-1-17-s002.pdf (80K)

Abstract

Background: MRI is a new and expensive diagnostic technology, which has been used increasingly all over the world. Low back pain is a worldwide prevalent disorder and MRI technique is one of the several ways to diagnose it. This paper aims to identify the appropriateness of lumbar spine MRI prescriptions in Shiraz teaching hospitals using standardized RAND Appropriateness Method (RAM) criteria in 2012.

User

Methods: This study consisted of two phases. The first phase involved a qualitative enquiry and the second phase had a quantitative cross-sectional nature. In the first phase RAM was used for developing lumbar spine MRI indications and scenarios. In the second phase, the finalized scenarios were compared with the history and physical examination of 300 patients with low back pain. The rate of appropriateness of lumbar spine MRI prescription was then calculated.

Results: Of 300 cases of lumbar spine MRI prescriptions, approximately 167 (56%) were considered inappropriate, 72 (24%) were uncertain, and 61 (20%) were deemed to be appropriate. The economic burden of inappropriate prescriptions was calculated at 88,009,000 Rials. In addition, the types of expertise and physical examination were considered as related factors to appropriateness of prescriptions.

Conclusion: In conclusion, a large proportion of lumbar spine MRI prescriptions, which result in financial burden on the insurance companies and the patients alike is unnecessary. This study suggests that policy makers consider this evidence while decision-making. Our findings highlight the imperative role of Health Technology Assessment (HTA) and Clinical Practice Guidelines (CPGs). As a result, developing local clinical guidelines may create the commitment needed in physicians in prescribing appropriate prescriptions within the health sector. The study further recommends that appropriate scenarios should be considered as a criterion for payment and reimbursement.

Keywords: MRI Prescription, Lumbar Spine, RAND Appropriateness Method, Evidence, Clinical Practice Guideline, Shiraz

Background

The ever changing population’s needs and increasing costs of treatment are two major challenges facing health systems worldwide (). A number of studies have shown that large volumes of provided health care may be inappropriate or unnecessary. Some studies show that the quality of and access to health care services are not necessarily related to spending more on health care systems (,).

RAND (Research AND Development) Appropriateness Method (hereafter RAM) is a transparent approach to assess the appropriateness of health care services. This method was designed in the 1980s by RAND Corporation and University of California, Los Angeles and has been used in a number of studies on appropriateness of health care services (4-).

Radiologic diagnostic methods help physicians in early diagnosis of disorders, and prevent the provision of subsequent aggressive treatments. However, studies show that during the past two decades, the radiologic diagnostic procedures have been progressively more common all over the world. According to statistics of the review conducted by the Board of Radiologic Imaging in the United States, 30% to 40% of diagnostic imaging performed in this country have been unnecessary or have failed to detect the disease (7). As a result, inappropriate order of medical imaging procedures by physicians may result in serious problems in terms of economy and quality of health care ().

Low back pain is a prevalent musculoskeletal disorder and has a huge burden to the health care systems of several countries including Iran. Around 80% of people deal with this problem during their lives. In the United States, low back pain is the second most common prevalent disorder after headaches (-11).

Analysing and understanding the pattern of medical imaging technology usage is of great significance for planning health systems, especially in low- and middle- income countries (). However, based on our literature review, no published paper has ever addressed this issue in Iran. The current study aims to fill this gap by investigating the appropriateness and necessity of lumbar spine MRI prescriptions in hospitals affiliated with Shiraz University of Medical Sciences (SUMS) in 2012.

Methods

This study consisted of two phases. The first phase involved a qualitative enquiry and the second phase had a quantitative cross-sectional nature. The two phases of study are described in details below.

Phase one: developing appropriateness criteria

In this study, RAM was used to develop the appropriate scenarios for lumbar spine MRI prescriptions. This phase consisted of two rounds; in the first round, the panel members scored the scenarios independently and there was no interaction among them, while the second round involved interaction among the panel members.

According to RAM, the available clinical guidelines on lumbar spine MRI were searched, from which the indications and scenarios were extracted. After that, in order to come to an appropriate consensus on scenarios, two rounds of expert panel were held. Each panel composed of nine specialists: two neurosurgeons, two orthopedists, two radiologists, one neurologist, one rheumatologist, and one physiatrist, all of whom were faculty members working in teaching hospitals. First round was without interaction among panel members. We asked panel members to score the scenarios. Based on the scores of panel members, scenarios were placed into three categories: appropriate, uncertain, and inappropriate. The criteria for this categorization are explained below.

For each indication, the panel members rated the benefit-to-harm ratio of the procedure on a scale of 1 to 9, where 1 meant that the expected harms greatly outweighed the expected benefits, and 9 meant that the expected benefits greatly outweighed the expected harms. Each indication was considered appropriate if the panel’s median rating was 7–9 without disagreement, inappropriate if the value was 1–3 without disagreement, or uncertain if the median rating was 4–6 or if the members of the panel disagreed (4).

The panel members further added some indications and scenarios on the list based on their experiences and published papers. In the second round, the members were invited to a meeting. In the meeting, they scientifically discussed the appropriateness of the scenarios and reached a consensus. The process of development of appropriate scenarios and its results are described elsewhere ().

Phase two: identifying the appropriateness of prescription

This part of work was a descriptive analytical and cross-sectional study performed in hospitals affiliated with SUMS in 2012. It aimed to identify the appropriateness of lumbar spine MRI prescription. Sample size was calculated to be 300 samples and the convenient sampling was used. In the data gathering phase, the first author, accompanied by a trained physician, attended in MRI centers of two hospitals affiliated with SUMS to have the questionnaires filled. They asked the patients some questions and performed physical examinations on the patients. Before selecting the samples, the research aims were explained to patients and if they were willing to participate, they would be recruited. This work continued until a sample size of 300 was reached.

A short questionnaire, including 13 questions, designed by the authors, was used as the data gathering instrument. Out of 13 questions, six questions were about the patient demographic information (Additional file 1) and seven were on MRI prescription (Additional file 2). The last question concerned the appropriateness of MRI prescription and was regarded as the key question in the questionnaire. Its response was required for comparing the exact physical examination of patients with the developed scenarios. This sensitive task was performed by a knowledgeable trained physician. The patients’ informed written consents were earned prior to the physical examination.

SPSS 15 (SPSS Inc., Chicago, IL, USA) was used for data analysis. Frequency tables and chi-square test were selected for statistical analyses.

Results

Demographic information

Of 300 recruited subjects, approximately 51% were male and 49% were female. Most of them were in 20-35 yrs age groups and 35% had primary education.In terms of job status, 53% were housekeeper. In terms of monthly income level, 56% had 2500000-5000000 Rials income per month and 97% had medical insurance (see Table 1).

Table 1

Characteristics of patients referring to hospitals affiliated with SUMS in order to perform lumbar spine MRI in 2012
VariableN%P
Sex<0.001
Male15451 %
Female14649 %
Age Group >0.001
20-358127 %
35-5010334.33 %
50-656120.33 %
65-804916.33 %
> 8062 %
Education 0.18
Primary15853 %
High School8829 %
University Degree5418 %
Job Status<0.001
Worker8127 %
Clerk6622 %
Housekeeper10635.3 %
Private Corporation3411.3 %
Student134.4 %
Income (Rials- per month)<0.001
Under 2,500,000217 %
2,500,000 -5,000,00016956 %
5,000,000 -10,000,0008127 %
10,000,000-20,000,0002910 %
Medical Insurance Status 0.27
Insured29297 %
Not insured83 %
Note: Currency Rates, Iran; 2012: 1US$~25000 Rials

Prescription information

The research results on prescriptions demonstrated that 93% of the patients were outpatient and only 7% were inpatient. In addition, 35% of the patients had only back pain as their symptoms, 17% reported only radicular leg pain, and 44% of the patients had simultaneous back and leg pain as their symptoms. About 18% of patients declared that their prescriptions were administrated without any physical examination, while physical examination had been performed for 82% of the patients. Around 9% of the patients had asked the doctors for MRI prescription. Almost 40% of patients had been referred from public hospitals, 11% from private hospitals, and 49% from private offices. About 31% of patients were referred to the hospital to conduct lumbar spine MRI by neurosurgeon, 15% by physiatrist, 10% by neurologist, 31% by orthopedists, 9% by rheumatologist, and 4% by other specialists.

Appropriateness of physicians’ orders

The research results further revealed that about 56% of prescriptions were inappropriate, 24% were uncertain, and only 20% could be considered appropriate.

Financial burden

The costs of 167 cases of inappropriate MRIs were considered as a financial burden imposed on both the insurance companies and the patients (Table 2).

Table 2

The financial burden caused by inappropriate prescribing
Financial Burden Imposed to the InsuranceFinancial Burden Imposed to the PatientsTotal Financial BurdenTotal Inappropriate Prescribing
Amount (%)Amount (%)Amount (%)167 cases out of 300
81,743,300 Rials (92 %)6,265,700 Rials (8 %)88,009,000 Rials (100 %)

Related factors

The Chi-square test shows a significant relationship between the type of expertise and appropriateness of prescriptions (P>0.001). According to finalized indications, among the physiatrists and neurosurgeons prescriptions, 64% and 60% were inappropriate respectively. Also, 48% of rheumatologists prescriptions were appropriate and 26% of orthopedics prescriptions were uncertain (Table 3). In addition, Chi-square test demonstrated a significant relationship between physical examination and appropriateness of prescriptions (P<0.001). The inappropriate prescriptions without physical examination were approximately 82% (Table 4).

Table 3

Frequency and relation between appropriateness of prescription and type of expertise
Specialist Appropriateness Total
AppropriateUncertainInappropriate
Neurosurgeon14 (15 %)24 (25 %)58 (60 %)96 (100 %)
Physiatrist7 (15 %)10 (21 %)30 (64 %)47 (100 %)
Neurologist7 (22 %)8 (25 %)17 (53 %)32 (100 %)
Orthopedics19 (20 %)25 (26 %)52 (54 %)96 (100 %)
Rheumatologist14 (48 %)5 (17 %)10 (35 %)29 (100 %)
Total61 (20 %)72 (24 %)167 (56 %)300 (100 %)

Table 4

Frequency and relation between appropriateness of prescription and physical examination
Physical examinationAppropriateTotalP
AppropriateUncertainInappropriate
with57 (23 %)66 (27 %)122 (50 %)245 (100%)<0.001
without4 (7 %)6 (11 %)45 (82 %)55 (100%)
Total61 (20 %)72 (24 %)167 (56 %)300 (100%)

Discussion

The results of this study indicate that 56% of lumbar spine MRI prescriptions were inappropriate, about 24% were uncertain and around 20% were appropriate.MRI is a very expensive diagnostic procedure and imposes huge financial and emotional burden on both the society and patients. These unnecessary healthcare procedures could impose high intangible costs on the patients such as wasting time, energy, and money. Therefore, physicians should prescribe them only when necessary.Although in the finalized scenario (the first phase of study), performing an appropriate MRI is recommended after a careful physical examination and six weeksof maintenance therapy, the results of the second phase of the study disclosed that MRI was chosen as the first diagnostic procedure for low back pain. It seems that the inappropriate rate of MRI prescriptions in Iran is higher than that of the US and Canada. According to the study ofLehnert and Bree (), the inappropriate rate of lumbar spine MRI prescription was 26% in the US; similarly Emery et al.revealed that the inappropriate rate of MRI prescription was 28.50% in Canada ().

Rand Ucla Appropriateness Method User's Manual Template

A large portion of increasing costs in the health sector is caused by unnecessary and inappropriate care.Studies show that one third of medical tests and procedures performed for patients in the world have been unnecessary orunsafe ().The current study shows a high financial burden imposed upon the insurance companies as a result of inappropriate and unnecessary care. This finding highlights the failure of insurance market in Iran. It is therefore suggested that appropriateness criteria should be considered for financing and reimbursement.

The inappropriate prescriptions were 56% (regardless of the uncertain cases). Probably we can generalize our results to the whole country; if 56% of the MRI prescriptions in Iran are considered inappropriate at least half of the reimbursement costs will be wasted. It is claimed that the number of MRI machines installed in Tehran province was enough for the whole population of the country ().Therefore we can imply that many of imported MRI machines to the country are unnecessary. The authors hypothesize the high availability of MRI machines within the country could be a possible reason for the overuse of MRI procedures. The problem is exacerbated knowing the fact that the price of a MRI machine is about one to three million dollars. In addition, among all imaging diagnostic devices, MRI had a more inappropriate distribution in the country (18). Given that the country is highly dependent on import of this technology (i.e. MRI machines) coupled with the high international sanctions posed on the country, policy makers should adopt proper HTA polices. Palesh and colleagues conducted a qualitative research in order to investigate policy makers’ view about diffusion and utilisation of MRI in Iran (). They found that the process of policy making does not seem to be based on a full understanding of HTA and the country does not follow an official plan for MRI adoption and diffusion. According to this study factors that contribute to the health technology diffusion and utilisation are posed by market forces such as advertisements, and physician and consumer demand. Besides, dual practice can increase the induced demand and also reduce the supervision of the private sector by the Ministry of Health. This study reported another major deficit in HTA process in Iran; a lack of need assessment, where the import of health technologies is not based on need assessment and structured planning ().

The following is a brief discussion of factors related to appropriateness. Firstly, prescriptions without physical examinations were more inappropriate. When a physician makes MRI prescription without any physical examination, it is more probable to be inappropriate. Secondly, those prescriptions, which were performed by rheumatologists, were more appropriate than others. Emery et al. however, reported a different finding. They reported that MRI scans ordered by neurologists and orthopedic surgeons were appropriate in less than half of the cases, while those ordered by neurosurgeons were appropriate about 76% of the time. More research is needed to identify the impact of different factors contributing to inappropriate prescriptions ().

Strengths and limitations

The present study enjoys some advantages. First, this was the first published study assessing appropriateness of MRI prescriptions using RAND in Iran. Second, this study, similar to other studies conducted in Iran (,), recommends RAM for the healthcare system in Iran, because the evidence is needed for policy making. Third, in order to avoid influence of professional biases in the scoring process, the expert panel members were selected from seven different areas of specialties. Last but not least, in contrast to the majority of RAM studies whose judgment is based on medical records, in this study physical examination was performed for all cases according to standardized clinical indications and scenarios. This process was very time-consuming and costly, but it could obtain more precise results due to the fact that medical records do not always contain a complete set of required data.

This study also has several limitations. First, in phase one, although a full-participation of expert panel members was required, this was not perfectly met due to the busy schedule of experts. As such, a great deal of time was devoted to coordinate the expert panel meetings. Second, in data gathering phase, the researchers had to wait a long time for patients with lumbar spine MRI orders in MRI centers.

Conclusion

The research findings revealed that the large proportion of lumbar spine MRI prescriptions was unnecessary which result in a financial burden on the insurance companies and the patients alike. The findings provide policy makers with a number of practical recommendations to improve evidence-based policy making. The results also highlight the imperative role of HTA and clinical practice guidelines. Developing local clinical guidelines may create the commitment needed in physicians in performing appropriate prescriptions in the health sector. The Ministry of Health should develop and completely follow the HTA programs. We need a national coordination between HTA policies.

Acknowledgments

We would like to thank members of expert panel who were Bagheri Mohammad Hadi, MD, Haghnegahdar Ali, MD, Farokhi Majid Reza, MD, Jalli Reza, MD, Pourabbas Babak, MD, Nooraee Hormoz, MD, Jazayeri Mostafa, MD, Rahimi Jaberi Abbas, MD, and HabibAgahi Zahra, MD, for their valuable contribution to this work and their scientific supports.

Notes

Citation: Salari H, Ostovar R, Esfandiari A, Keshtkaran A, Akbari Sari A, Yousefi Manesh H, et al. Evidence for policy making: clinical appropriateness study of lumbar spine MRI prescriptions using RAND appropriateness method. International Journal of Health Policy and Management 2013; 1: 17-21.

Footnotes

Ethical issues

This study was reviewed and approved by the Ethics Committee of SUMS.

Competing interests

The authors declare no competing interests.

Authors’ contributions

HS contributed to study conception, design, implementation, data analysis, interpretation, and writing of the manuscript. RO contributed to study conception, design, interpretation, and writing of the manuscript. AE contributed to the implementation, data analysis, interpretation and writing of the manuscript; she is the corresponding author. AK contributed to the study design, interpretation and writing of the manuscript. AA contributed to study design, interpretation, and writing of the manuscript. HY contributed to the implementation, interpretation and writing of the manuscript. AR contributed to the interpretation, and writing of the manuscript.

Additional files

Additional file 1

Contains the appendix 1.

Additional file 2

Contains the appendix 2.

Rand/ucla Appropriateness Method

References

1. Ostovar R, Rashidian A, Pourreza A, Rashidi BH, Hantooshzadeh S, Ardebili HE. et al. Developing criteria for cesarean section using the RAND appropriateness method. BMC Pregnancy Childbirth. 2010;10: 52.[PMC free article] [PubMed] [Google Scholar]
2. Borowitz M, Sheldon T. Controlling health care: from economic incentives to micro-clinical regulation. Health Econ. 1993;2:201–4. [PubMed] [Google Scholar]
3. Phelps CE. The methodologic foundations of studies of the appropriateness of medical care. N Engl J Med. 1993;329:1241–5. [PubMed] [Google Scholar]
4. Fitch K, Bernstein SJ, Aguilar MD, Burnand B, LaCalle JR, Lazaro P, The RAND/UCLA Appropriateness Method: Users Manual. RAND Europe; 2001. [Google Scholar]
5. Kanouse DE, Brook RH, Winkler JD, Kosecoff J, Berry SH, Carter GM, Changing medical practice through technology assessment. RAND Corporation; 1989. [Google Scholar]
6. Shekelle PG, Woolf SH, Eccles M, Grimshaw J. Clinical guidelines: developing guidelines. BMJ. 1999;318:593–6.[PMC free article] [PubMed] [Google Scholar]
7. Pennsylvania Health Care Cost Containment Council. The growth in diagnostic imaging utilization. Accessed 2007 July. Available from: http://www.phc4.org/ reports/FYI/fyi27.htm
8. Palesh M, Fredrikson S, Jamshidi H, Jonsson PM, Tomson G. Diffusion of magnetic resonance imaging in Iran. Int J Technol Assess Health Care. 2007;23:278–85. [PubMed] [Google Scholar]
9. Gilbert FJ, Grant AM, Gillan MG, Vale LD, Campbell MK, Scott NW. et al. Low back pain: influence of early MR imaging or CT on treatment and outcome--multicenter randomized trial. Radiology. 2004;231:343–51. [PubMed] [Google Scholar]
10. Hart LG, Deyo RA, Cherkin DC. Physician office visits for low back painFrequency, clinical evaluation, and treatment patterns from a USnational survey. Spine (Phila Pa 1976) 1995;20:11–9. [PubMed] [Google Scholar]
11. Lower Back Pain Fact Sheet. National Institute of Neurological Disorders and Stroke. Accessed 2011 December 5. Available from: http://www.ninds.nih.gov/disorders/backpain/detail_backpain.htm
12. Deyo RA, Mirza SK, Martin BI. Back pain prevalence and visit rates: estimates from USnational surveys, 2002. Spine (Phila Pa 1976) 2006;31:2724–7. [PubMed] [Google Scholar]
13. Keshtkaran A, Bagheri MH, Ostovar R, Salari H, Farokhi MR, Esfandiari A. et al. Developing criteria for lumbar spine magnetic resonance imaging (MRI) using RAND appropriateness method (RAM) Iran J Radiol. 2012;9:130–8.[PMC free article] [PubMed] [Google Scholar]

Rand Appropriateness Method

14. Lehnert BE, Bree RL. Analysis of appropriateness of outpatient CT and MRI referred from primary care clinics at an academic medical center: how critical is the need for improved decision support? J Am Coll Radiol. 2010;7:12–7. [PubMed] [Google Scholar]
15. Emery DJ, Shojania KG, Forster AJ, Mojaverian N, Feasby TE. Overuse of magnetic resonance imaging. JAMA Intern Med. 2013;25:1–3. [PubMed] [Google Scholar]
16. Thom DH, Kravitz RL, Kelly-Reif S, Sprinkle RV, Hopkins JR, Rubenstein LV. A new instrument to measure appropriateness of services in primary care. Int J Qual Health Care. 2004;16:133–40. [PubMed] [Google Scholar]
17. Palesh M, Tishelman C, Fredrikson S, Jamshidi H, Tomson G, Emami A. “We noticed that suddenly the country has become full of MRI” Policy makers’ views on diffusion and use of health technologies in Iran. Health Res Policy Syst. 2010 ;8:9.[PMC free article] [PubMed] [Google Scholar]
18. Salamatnews. Accessed 2011 December 5. Available from: http://www.salamatnews.com/viewNews.aspx
19. Ostovar R, Pourreza A, Rashidian A, Rashidi BH, Hantooshzadeh S, Haghollai F. et al. Appropriateness of cesarean sections using the RAND Appropriateness Method criteria. Arch Iran Med. 2012;15:8–13. [PubMed] [Google Scholar]
Articles from International Journal of Health Policy and Management are provided here courtesy of Kerman University of Medical Sciences