Editor's Note: This article was updated on February 7, 2023
Abstract
Objective: Countries worldwide, including Canada, need tools for informed decision-making on the adoption of ICD-11. The purpose of the current study is to create a cost and outcome estimation framework for the transition from ICD-10-CA to ICD-11 in Canada and anticipate the benefits/outcomes and international considerations of ICD-11 adoption.
Methods: This paper follows a cost and outcome evaluation framework. The costs of transitioning to the ICD-11 coding system are based on literature reviews and interviews adapted for the Canadian context, while the outcomes have been created using a novel methodology involving face and criterion validity.
Results: While it has been difficult and infeasible to include all possible categories and variables identified by the interviewees into the framework, this paper provides a comprehensive guideline of the outcome and cost estimation methodology.
Conclusion: The estimation technique adopted in this paper is unique and may be used as a benchmark methodology by other countries to evaluate the adoption of the ICD-11 coding system.
Keywords
International Classification of Diseases, ICD-11, ICD-10-CA, Cost and Outcome Estimation, Economic Evaluation, Framework, Qualitative Analysis, Database, Coded Data, World Health Organization
Introduction
Health data is collected from every inpatient visit to create coded administrative data. Approximately 5000 (Canadian Institute for Health Information 2019) CHIMA- Certified Health Information Management professionals across Canada use International Classification of Diseases and Related Health Problems (ICD) coding system. Through ICD, codes are assigned to medical diagnosis from patient charts, creating an internationally recognized registry to adhere with the World Health Organization (WHO) standards. Currently, Canada uses ICD-10-CA (10th version, Canadian Revision) to code, store, and analyze healthcare data that have been widely used for research, provincial and national reports on population health, and system performance. However, due the presence of multiple country-specific modifications of ICD-10-CA, changes in medicine, and it being time for a routinized update, a new and improved version of ICD was required.
In June 2018, the WHO released ICD-11, after a decade-long revision process that involved extensive consultations with international experts. Improved usability may enable easier transition for coding specialists and ensure that the system is widely used.
As Canada and other nations prepare for ICD-11 implementation, there is now a critical need for extensive preparatory work. A seamless transition to ICD-11 will require large investments of time and money; for example, all coding specialists will need to be trained to use new systems, and IT software and supports will need to be upgraded. Starting in 2001, Canadian provinces required a 5-year period to fully transition from the ICD-9 to ICD-10-CA system (this included the transition from ICD-9 to ICD-10-CCI (Canadian Classification of Health Interventions)). This five-year-long transition period resulted in inconsistent national ICD data and data usability challenges (L. walker et al. 2012).
Canadian Institute for Health Information (CIHI) anticipates adopting ICD-11 in Canada; however, the decision for adoption and timing of implementation will be made by provincial/territorial governments in Canada. In order to ensure consistent national ICD data collection during the transition to an updated system, a simultaneous adoption of ICD-11 by all provinces is ideal to optimize data comparability. Currently, there is limited research on the costs and benefits of transitioning from ICD-10-CA to ICD-11 (Education and Implementation Committee 2018; WHO 2019). Evidence for decision-making is crucial to enable an informed and a seamless provincial transition to ICD-11.
The objective of the current study is to create the framework and formulae to enable evidence-informed decisions on whether to implement the International Classification of Disease 11th Revision (ICD-11) of WHO in Canada. Specifically, we aim to 1) develop a framework for estimating the costs of ICD-10-CA to ICD-11 transition, and 2) to outline considerations to calculate potential benefits and outcomes of adopting ICD-11.
Assumptions
The framework is formulated based on the following two assumptions:
- The focus of the retrospective study (Khair 2018) on ICD-9 to ICD-10-CA transition is based on acute sector costs/transition issues. We expect ICD-11 implementation to be broader.
- The model assumes that hospital coding will follow the current (ICD-10-CA) coding procedures. It is difficult to foresee the effects and methods of computer-assisted coding (auto-coding) and the potential costs/savings which may result from it.
Methods
Scoping Review
We conducted a scoping review on studies containing cost and benefit analyses of ICD transitions following guiding methods by (Peters et al. 2015) to provide us with a foundation for our economic evaluation framework. We searched the database PubMed, MedLine, Scopus, and EconLit using the terms international classification of disease, ICD, economics, economic evaluation, cost-benefit estimation, analysis, evaluation, or projection, and combined these terms with big data, healthcare, medicine, health services, or hospital.
Since the number of current topics and knowledge regarding ICD transitions are limited, we decided to conduct a scoping literature review to identify methods and frameworks used in economic evaluations of healthcare technology transitions involving ICD. Another search was done on the Embase database by combining “ICD-10” and “ICD-11” with keywords related to economics. In total, 139 more potential articles were identified by the search strategy. The exact search strategy used keyword + textword searches with the “.mp” extension for the terms “ICD-10” and “ICD-11” using the OR Boolean operator. These results were then combined with a search for articles with “ICD-10” OR “ICD-11” in the title of the paper. Finally, these results were combined with the keywords “economic evaluation.mp” OR “cost-benefit.mp” using the AND Boolean operator. Out of these articles, 22 were found to be of potential relevance based on title and abstract screening by an independent reviewer. A common theme identified within search results was that of budget management. One study completed a cost-benefit analysis on reorganizing the diagnostic groups DSM-V and ICD-11. This strategy on the EMBASE database reveals a broad but sparse range of available literature on the economic impacts of ICD transitions in Canada.
Methods for Creating the Cost Calculation Formulae
Two reports (Libicki and Brahmakulam 2004; Calgary Health Region 2001) outline the necessary information to begin a cost projection for ICD-11. The cost estimation formulae are based primarily on the report published by the RAND Corporation (Libicki and Brahmakulam 2004), with adaptations to fit the Canadian health system. In-person interviews were conducted (between May 2017 and May 2019) in order to gain more insight into the process of ICD-10-CA implementation in the Calgary Health Region. These interviews inform the processes that will be involved in the ICD-11 implementation.
Methods for Creating the Outcome/ Benefit Calculation Formulae
Data from an ongoing ICD-11 field trial conducted by our team will be used for evaluating the outcomes of ICD-11. Data from 3000 inpatients discharged between January 1, 2015 and June 30, 2015 in three teaching hospitals in Calgary, Alberta have been abstracted. We have already validated the ICD-10-CA data and have successfully published the results of that validation study (Quan et al. 2008, 2005).
This unique dataset would be analysed to assess the outcomes of transitioning to ICD-11 using three distinct measures. Both data quantity and quality improvements will be examined.
Quantity of Codes: ICD-11 codes are more numerous in comparison with ICD-10-CA and reflect the ever-changing social values and medical content in healthcare. For example, ICD-11 includes new societally relevant codes (e.g. gaming disorder), risk factors codes, and causation codes. The enhanced coding system will enable increased surveillance of modern health issues. This is why the quantity of variables/codes in ICD-11 is used in our study as a measure of ICD-11 benefits.
- Quantity of Codes per Record. Using our ICD-11 database, we will calculate the average number of codes per record in both ICD-10-CA and ICD-11, the number of secondary diagnoses coded, and the number of unspecified codes used.
- Quantity of New Feature Codes. We will also use the number of new feature codes (which are introduced in ICD-11) in recorded ICD-11 data to measure the quantity improvement.
Quality of Codes: We propose to measure quality improvement using Validation of Commonly Used Codes. We have already developed methods to define the Charlson and Elixhauser (Charlson ME, Pompei P, Ales KL 1987; Anne Elixhauser, Claudia Steiner 1930) conditions in the ICD-10-CA data, both common conditions coded in the Canadian healthcare system. These conditions have been used by health services researchers to measure population level burden of disease (health status) or case-mix adjustment with administrative data to assess hospital level differences in care. We will calculate parameters including sensitivity, specificity, positive predictive value, and negative predictive value.
Interviews
Interviews were conducted to gain information on the process of ICD-10-CA implementation and to obtain feedback for the ICD-11 transition cost and outcome framework. Consent was obtained from each participant, and at least two research team members attended the interviews. We conducted in-person or Skype, recorded, semi-structured interviews with individuals or focus groups with Health Information Department Directors across various provinces and territories in Canada. In addition, we interviewed a wide range of stakeholders from the WHO, Canadian Institute of Health Information (CIHI), representatives of the National Coding Advisory Committee (NCAC, decision makers from provinces/territories), Canadian Health Information Management Association (CHIMA), Health Information Management (HIM) coders, provincial managers and directors, as well as software vendors across all provinces and territories in Canada.
Using a qualitative descriptive approach (Sandelowski 2010), we interviewed purposively sampled (e.g., based on role, sex, age and geographical location) stakeholders who will be affected by the transition from ICD-10-CA to ICD-11 — where multiple stakeholders in one province exist, they became a focus group. Using focus group methodology (Krueger and Casey 2014), we conducted focus groups or individual interviews until no new data were generated. In total, we spoke with 25 stakeholders with representation from nine provinces and three territories in Canada (Musselwhite et al. 2007). Representation from multiple provinces ensured that knowledge translation strategies could be tailored to the unique concerns of each province.
For the sampling strategy, we aimed to interview one representative from each Canadian province and territory. With the help of CIHI, we contacted members of NCAC (National Coding Advisory Committee). The NCAC advises CIHI on ICD code development, coding rules, and guidelines (Canadian Institute for Health Information 2003). Some members who agreed to participate in the interview invited colleagues who had experience with the ICD-9 to ICD-10-CA transition to join the interview. When we exhausted the NCAC contacts, we approached CHIMA to provide contacts for HIM managers and directors for representation from as many provinces as possible. Informed consent was obtained from all participants. An honorarium of $100 was offered to each participant for their time; some participants declined.
The interview questions were adapted to reflect the challenges specific to Canada. Participants were asked to share past experiences of transitioning from ICD-9 to ICD-10-CA (including facilitators and barriers), as well as suggest possible costs and benefits for the transition from ICD-10-CA to ICD-11. Questions involved identifying the code users in the province as well as understanding whether ICD codes were used in billing. We also asked each province to elaborate on what they think would differentiate their province from other Canadian provinces, which should be taken into account while projecting the total cost of implementing ICD-11. Table 1 provides more examples of the unique considerations for each province, which would affect the transition costs differently.
Duration for focus groups was 45-60 minutes, while individual telephone interviews took slightly less time. Data collection was iterative: information from early focus groups provided areas to probe in subsequent focus groups/interviews. A professional transcriptionist transcribed the discussions verbatim.
Qualitative Analysis
Focus group and interview transcripts were analyzed using conventional content analysis by the study team (Hsieh and Shannon 2005; Vaismoradi, Turunen, and Bondas 2013). The team members: 1) read the transcripts as they became available to acquire an overall sense of the data; 2) highlighted words or phrases that captured key concepts and patterns across participants; 3) combined codes into overarching categories; 4) reviewed categories in relation to one another; 5) synthesized final categories and content into the framework; 6) highlighted exemplary quotes. Discussion took place when need for consensus regarding categories had to be reached.
Preliminary Framework
Based on our scoping review and interviews, we created a preliminary framework using elements from two of the reports (Libicki and Brahmakulam 2004; Calgary Health Region 2001) identified in the scoping review. Three team members (SK, ML and TL) selected relevant elements from the two frameworks, and excluded calculation methods which were not applicable to the Canadian healthcare system.
Ethics Approval
Ethics was obtained through the Conjoint Health Research Ethics Board at the University of Calgary, Ethics ID #REB17-2387.
Results
Scoping Review
Our scoping review identified three reports that evaluated the costs and benefits of the ICD-9 to ICD-10-CA transition. The first is the RAND technical report (Libicki and Brahmakulam 2004), which is the only study that provided a comprehensive methodology for cost-outcome estimation of ICD-9 to ICD-10-CA transition in the United States. The second is a Canadian cost projection business case (Calgary Health Region 2001) for transitioning from ICD-9 to ICD-10-CA. The third study we identified is a transition guide (Committee 2018) describing the stages, benefits and costs of moving from ICD-10-CA to ICD-11 coding system. This paper was prepared by the WHO-FIC Education and Implementation Committee (EIC) and focuses on the Mortality and Morbidity Statistics updates in ICD-11. Since the transition guide does not include a framework to calculate the costs and outcomes of the transition to the ICD-11 coding system, we concentrate on the RAND report and the ICD Business Case to create the primary skeleton of our cost and benefit calculation framework. Our scoping review shows the lack of literature and data needed for economic analysis for ICD-10-CA to ICD-11 transition.
The variables included in the cost estimation in the RAND report (Libicki and Brahmakulam 2004) were costs of training, productivity losses and system change costs. The business case (Calgary Health Region 2001) offered an example of ICD-10-CA adoption in a Canadian context and was incorporated into our framework. The provincial and federal government provides healthcare funding in Canada. The cost categories in this framework reflect costs incurred by the hospitals, which are covered under the single-payer provincial healthcare budget. This is significantly different from the framework outlined in the RAND report, which incorporated the multi-payer health spending and insurance costs. While the benefits were not enumerated in the business case, the study estimated expected incurred costs. The calculations were based on answers to questionnaires addressing the expected training and project management costs, productivity losses and software application costs.
Interview Results
The results of the interviews ensured the framework fit the Canadian context. Furthermore, we were able to obtain differences in cost estimates for different provinces. The interviews provided robust information on the process of ICD-10-CA implementation and garnered feedback for the ICD-11 transition cost and outcome estimation framework.
Differences in estimation variables were due to the effects of a myriad of factors, including the organizational milieu, information and technological infrastructure, and whether the training sessions are in remote or urban locations. As the interviews progressed, feedback was obtained on the cost estimation framework, as well as factors within the framework categories, such as steps needed for ICD-11 implementation. Provinces were hesitant to comment on benefits of ICD-11 as very little information had been disseminated and made available prior to launching ICD-11. We refined our knowledge translation activities based on the interviews. A snapshot of the main issues raised by the interviewees is provided in Tables 1 and 2.
Table 1: Provincial Considerations for ICD-11 Implementation, Unique by Province
| British Columbia mentioned that auto-coding projects would need to be updated/linked.
Alberta is moving into Connect Care, which is a health information system allowing health providers to have access to a common system with comprehensive and consolidated patient information. If ICD-11 is implemented, it may be around the same time period of the launching of Connect Care. This significant system change will have important implication on the costs and benefits calculation of ICD-11 for the province.
New Brunswick would require re-training in both English and French. There might also be both online training and in-person training, so travel costs should be considered. They are also considering training physicians to use ICD in the future, so they can enter codes (but this has not happened yet).
Manitoba would require higher costs for system changes. Further, Manitoba has a dual abstract system which would affect productivity loss. Provincial governments provide funding support to upgrade IT systems.
Northern Territories require training centres in remote locations and would likely have to train staff together in a group (Nunavut and Yellowknife coders were trained together during the ICD-9 to ICD-10-CA transition). There may not be road access in some locations and coders may have to be flown out to the nearest training center. All of these factors contribute to additional cost variables.
Nova Scotia finds maintaining connection between coders during training is a challenge, because most coders work from home. The province is also very spread out, having small training sites in rural areas. Lastly, Nova Scotia anticipates using computer-assisted coding around the same time as the ICD-11 adoption.
Ontario anticipates challenges in transitioning to ICD-11, because coded clinical data are extensively used for hospital payment. There are many different code users (e.g. Cancer Care Ontario and QBPs). Both of these examples have their own system that would need to be updated. In addition, they use an international system and the full benefits may not be realized until the United States adopt ICD-11 as well. Ontario’s Hospital-Based Allocation Model also dictates the allocation of funds. The range in the variation of ancillary funding may be quite large. Ontario is also challenged to meet mandated quarterly deadlines in addition to the CIHI yearly deadline of May 31st annually, which may cause additional costs due to productivity losses.
|
Table 2: Summary of Interview Feedback on Framework, Benefits and Knowledge Translation Activities
| Cost of Retraining
- Code users including coding specialists, coding supervisors and managers, and coding directors would need specific training tailored to their roles
- Other code users to be trained include Federal entities (Vital Statistics Registry), provincial agencies (HQO), QBPs, eHealth (conducts quality checks), analysts, quality and safety teams, physicians in some cases, and data warehouse maintenance staff that use the data and would need to learn ICD-11 coding
- Costs of traveling to training sites is higher in rural and remote centres
- Online training and retraining costs should be considered
- Educational institutions will require materials and training
Productivity Losess Cost
- Time will be needed to integrate and maintain the new ICD-11 system (e.g. fixing broken code pathways)
- A break-in period of 6 months to 2 years will be necessary before high-quality data is produced from ICD-11
- Adoption of a new system will cause backlog of charts to code, and “there is only so much overtime available”
System Change Cost
- IT systems need to be updated
- Data collection and analytics are integrated and will be complex to update
- Licensing fees and implementation costs (e.g. purchasing new tools through the vendor) should be considered
|
Table 2 illustrates the variables identified through interviews for our framework for estimating costs of the ICD-11 transition. We have incorporated the majority of the considerations brought forward by the interviewees; however, some considerations are not feasible to calculate or measure due to insufficient data (e.g. costs associated with data conversion from ICD-10-CA to ICD-11 using crosswalk tables).
Canadian provinces still use ICD-9 codes for billing and the interviewees identified that unless the single-payer system moved to using the ICD-11 codes, re-training costs and system costs would not be affected. This was a very important insight in calculating the costs of transition as transferring the billing systems to use the new ICD-11 codes would require huge amounts of financial and human resources.
Interviewees also mentioned that if software does not require change-over, it will cost less and take less time to transition. Saskatchewan mentioned using ICD-9 for physician claims and will consider using ICD-11 once it is adopted. When interviewing the Northern Territories on how the ICD-10-CA to ICD-11 transition would affect their territory, the most specific concern was on the geographical location and the need for trainees to be able to access online materials or travel to the training location.
Foreseeable Benefits of ICD-11
Interviewees deduced potential benefits based on their experience with the ICD-9 to ICD-10-CA transition. Possible benefits mentioned by participants included relevant and up-to-date coding and enhanced digital adaptability (e.g. computer assisted coding) which may enable remote coding due to electronic and online browsing systems. The interviewees also mentioned the significance of simultaneous transition across provinces, which will lead to standardization of records and allow better comparability of data. They alluded to the benefits of having the ICD-11 language better matched to clinical terminology. Some participants expressed that pilot projects and experience using the data is needed to fully understand the benefits of ICD-11.
Recommendations for Smooth ICD-11 Transition
Based on the past transition from ICD-9 to ICD-10-CA in Canada, several participants recommended that a preparatory two-year time frame be built-in prior to ICD-11 implementation. Participants emphasized that this two-year period would allow education systems and hospitals to prepare instructional material and make necessary IT system changes. They mentioned that preparation materials should be offered across all provinces prior to the training of ICD-11 users. In addition, interviewees suggested that the transition to ICD-11 would be made smoother if all key stakeholders were involved and consulted throughout the duration of the adoption process. Key stakeholders included vendors, education providers, health system providers (e.g. AHS), and health ministries, all of which should be involved in consistent multi-disciplinary meetings across the country. Finally, some participants felt that the adoption of ICD-11 would cause impetus to streamline data and integrate sectors such as mental health facilities, long-term care, and primary care.
Preliminary Framework
From the RAND report and the ICD business case example, we have established a preliminary cost estimation framework (Figure 1). Costs of ICD transition were classified into three broad categories: cost of re-training, productivity losses, and cost of system changes (Figure 1).
Cost Estimation Calculations
Cost of re-training (See Table 3) has been divided into equivalent full-time coders, analysts, managers and trainers. Cost is calculated based on the number of people to be re-trained, the time it requires to retrain them, and hourly wage in each category. It also includes accommodation expenses, travel costs, classroom and material expenses. The people retrained are those who will record hospital data using ICD-11 codes as seen below.
Table 3: Retraining Costs
| Variable |
Formula |
| Coder retraining costs |
Coder retraining hours per coder * Number of coders * Hourly wage |
| Analyst retraining costs |
Analyst retraining hours per analyst * Number of analysts * Hourly wage |
| Manager retraining costs |
Manager retraining hours per manager * Number of Managers * Hourly wage |
| Trainer costs |
Number of trainers * Days spent for training* Daily wage of trainers |
| Educational Institution Training |
Number of institutions* Trainer costs |
| Other staff costs |
Number of other staffs * hours worked * hourly wage of staff |
| Travel and accommodation costs |
Cost of Travel + Cost of Accommodation
|
| Other Costs |
Classroom Cost + Lunch Costs + Material Costs
|
*Denotes multiplication
These calculations can be done at any time before the implementation of ICD-11 to depict true costs, adjusted with appropriate discounting to reflect the economic value of time. Discounting is further explained in subsequent sections of this paper. The total cost of re-training is calculated by adding all the components outlined in the above formula.
The productivity of coders (See Table 4) is expected to be affected during the transition phase. Implementation of ICD-11 may coincide with a new fiscal year, which might require coders to work for a longer time to complete a backlog of data during the break-in period. However, gradually, this time difference will decline as coders become more efficient and familiar with the ICD-11 system.
Table 4: Productivity Losses
| Variable |
Formula |
| Coder Productivity Loss |
Number of coders * ((Previous average coding time per chart – New average coding time per cha
|