Doctors' use of electronic medical records systems in hospitals: cross sectional survey
BMJ 2001; 323 doi: https://doi.org/10.1136/bmj.323.7325.1344 (Published 08 December 2001) Cite this as: BMJ 2001;323:1344
This article has a correction. Please see:
- Hallvard Lærum, fellow (hallvard.larum{at}medisin.ntnu.no)a,
- Gunnar Ellingsen, fellowb,
- Arild Faxvaag, associate professorc
- a Kvalis project, Department of Physiology and Biomedical Engineering, Faculty of Medicine, NTNU, Trondheim, Norway
- b Kvalis project, Department of Computer and Information Science, Faculty of Physics, Informatics, and Mathematics, NTNU, Trondheim
- c Department of Bone and Joint Disorders, Faculty of Medicine, NTNU, Trondheim
- Correspondence to: Hallvard Lærum, 5.et.kreftbygget, Regionsykehuset i Trondheim, 7006 Trondheim, Norway
- Accepted 12 October 2001
Abstract
Objectives: To compare the use of three electronic medical records systems by doctors in Norwegian hospitals for general clinical tasks.
Design: Cross sectional questionnaire survey. Semistructured telephone interviews with key staff in information technology in each hospital for details of local implementation of the systems.
Setting: 32 hospital units in 19 Norwegian hospitals with electronic medical records systems.
Participants: 227 (72%) of 314 hospital doctors responded, equally distributed between the three electronic medical records systems.
Main outcome measures: Proportion of respondents who used the electronic system, calculated for each of 23 tasks; difference in proportions of users of different systems when functionality of systems was similar.
Results: Most tasks listed in the questionnaire (15/23) were generally covered with implemented functions in the electronic medical records systems. However, the systems were used for only 2–7 of the tasks, mainly associated with reading patient data. Respondents showed significant differences in frequency of use of the different systems for four tasks for which the systems offered equivalent functionality. The respondents scored highly in computer literacy (72.2/100), and computer use showed no correlation with respondents' age, sex, or work position. User satisfaction scores were generally positive (67.2/100), with some difference between the systems.
Conclusions: Doctors used electronic medical records systems for far fewer tasks than the systems supported.
What is already known on this topic
What is already known on this topic Electronic information systems in health care have not undergone systematic evaluation, and few comparisons between electronic medical records systems have been made
Given the information intensive nature of clinical work, electronic medical records systems should be of help to doctors for most clinical tasks
What this study adds
What this study adds Doctors in Norwegian hospitals reported a low level of use of all electronic medical records systems
The systems were mainly used for reading patient data, and doctors used the systems for less than half of the tasks for which the systems were functional
Analyses of actual use of electronic medical records provide more information than user satisfaction or functionality of such records systems
Introduction
Electronic medical records systems are starting to be used in hospitals throughout Europe. However, there seem to have been few formal evaluations of them, 1 2 possibly because of a lack of established evaluation methods. 3 4 We therefore investigated the usefulness of different systems by comparing their use in general clinical tasks. Frequency of use is a possible indicator of how well such systems are adapted to clinical work in general 5 6 because a successful system ought to be used by most doctors for important tasks.7 We developed a questionnaire to investigate and compare the use of electronic medical records systems among doctors in Norwegian hospitals.
Participants and methods
Electronic medical records systems in Norwegian hospitals
Of the 72 hospitals in Norway, 53 had purchased a licence for an electronic medical records system by January 2001, covering 77% of hospital beds. In practice, there were three main electronic medical records systems—DIPS, Infomedix, and DocuLive (table). The DocuLive system is installed in the five university hospitals and hence is associated with the largest hospitals in the country. None of the largest hospitals had completed implementing the electronic medical records system in all of their departments at the time of our survey.
Distribution of electronic medical records systems in Norwegian hospitals by January 2001, and respondents in survey. Values are numbers (percentages)
Developing the questionnaire
The questionnaire consisted of eight sections (see bmj.com for details). In the section covering use of computers, we generated the list of clinical tasks on the basis of 40 hours of observations in five hospital sections at two hospitals, taking into account the information needs of doctors.8 The section asked doctors to indicate their frequency of use of computers for 23 general clinical tasks on a five point scale ranging from “Never or almost never” to “Always or almost always.” In addition, they were asked to indicate whether they were using the implemented electronic medical records system or another computer program (or both) for each task.
Reported use of computer programs for various clinical tasks by doctors from hospitals with different electronic medical records systems. Bars represent percentage of doctors who reported using computers at least half of the time for performing each task (red areas show those who used only the electronic medical records system, white areas show those who used the system and other software, and orange areas show those who did not state what program they used) and error bars show the confidence interval. Pink bars in background show percentage of respondents for whom the electronic medical records systems offered sufficient functionality for the task
We adapted existing, validated questionnaires to produce the sections covering computer literacy9 and user satisfaction. 10 11
Selection of participants, data gathering, and analysis
We randomly selected 32 hospital units (each with 4–22 doctors) in 19 of the hospitals with a licence for an electronic medical records system grouped by vendor. We excluded very small (<4 doctors) and very large units (>30 doctors) and those that had recently implemented an electronic medical records system (<3 months before). We distributed 314 questionnaires to doctors on 12 January 2001 and sent 134 reminders one month later.
The completed questionnaires were scanned with Teleform, and the data were analysed with SPSS for Windows version 10.0.8. We categorised the doctors' graded responses on their use of computers for general clinical tasks into two groups—those who used a computer for a certain task for at least half of the time normally spent on the task, and those who did not. The respondents who did use the computer for a certain task were further grouped by whether they used the electronic medical records system, another program, or both. However, some respondents (median 7%) did not state what program they used; we do not know whether these respondents overlooked the items or could not tell what software they were using.
Interviews with information technology staff
Key representatives of the 19 hospitals' information technology departments, involved in implementing the local electronic medical records, indicated through semistructured telephone interviews whether each clinical task in the questionnaire was supported locally according to certain minimal requirements (see bmj.com for details).
Results
Respondent demographics showed no effect on computer use
The response rate to our questionnaire was 72%, but we subsequently excluded two hospital units (eight respondents) because of problems with their implementing the electronic medical records system, leaving 219 respondents. Of the 208 who answered the question, 47 (23%) were less than 35 years old, 98 (47%) were aged 35-50, and 63 (30%) were aged over 50; 57/197 (29%) were women, and 140 (71%) were men; 123/205 (60%) were consultants, 74 (36%) were registrars, and eight (4%) were senior house officers. There was no significant difference between different electronic medical records systems in terms of respondents' age, sex, or work position, nor any correlation between these terms and total computer use or user satisfaction.
Respondents scored high in computer literacy
To assess respondents' computer literacy we asked them about their computer ownership, typewriting ability, prior computing experience in solving specific tasks, highest prior frequency of computer use, and self rated computing skills. The mean summed score of this section was 72.2 out of 100, with little difference between the users of the three electronic medical records systems (69.6-76.0, analysis of variance P=0.006). The correlation with total computer use was 0.39, P<0.001.
Computers were available in the respondents' work places
Most respondents (203/218 (93%)) had computers in their offices, and 209/216 (97%) had computers available to them in other rooms used for clinical work. However, 85/214 respondents (40%) were weekly or daily prevented from using these computers because others were using them, and 94/214 (44%) were monthly or weekly hindered by computer errors or problems with passwords (3% were hindered daily).
Use of the electronic medical records systems was limited
Functionality of the electronic medical records systems According to the information provided by information technology staff, most of the clinical tasks listed in our questionnaire were in some way covered by implemented functions of the electronic medical records systems. In general, 15 of the 23 tasks were covered for at least half of respondents: DIPS, Infomedix, and DocuLive supported 19, 16, and 11 of the tasks, respectively (fig 1).
The systems were mainly used for reading patient data Only two tasks (tasks 1 and 2 on fig 1) were performed with the electronic records systems by at least half of the respondents. When we included those respondents who did not indicate what type of computer program they used, the number of tasks rose to seven (tasks 1-4, 10, 22, and 23). The median proportion of respondents using programs other than the electronic medical records systems was 2% (interquartile range 1-5%); the highest proportions occurred in tasks where some of the records systems were particularly lacking in functionality (tasks 4, 7, and 10).
The number of tasks for which each respondent used an electronic records system was similar for each of the systems (mean number of tasks: DIPS 4.9, DocuLive 4.9, Infomedix 5.2; analysis of variance P=0.87). Only when we included those respondents who did not indicate what type of computer program they used did we find significant differences (DIPS 7.4, DocuLive 5.7, InfoMedix 7.8; analysis of variance P=0.002).
Considerable differences between systems in specific use We found considerable differences in doctors' use of the electronic medical records systems when we compared respondents who were offered similar functionality (fig 2). Because of some functionality not being implemented locally, the groups of respondents are smaller than in figure 1, particularly for the DocuLive system.
Clinical tasks for which significantly different percentages of doctors reported using three different electronic medical records systems that offered equivalent functionality. Bars represent percentage of doctors who reported using computers at least half of the time for performing each task (red areas show those who used only the electronic medical records system, white areas show those who used the system and other software, and orange areas show those who did not state what program they used) and error bars show the confidence interval. P values were calculated with χ2 formula (equal P values were achieved with analysis of red areas of bars only and when white and orange areas were included).
Moderate user satisfaction
The user satisfaction scale consisted of five factors: content, accuracy, format, ease of use, and timeliness.11 The mean overall score was 67.2 (SD 13.8) out of 100 (mean score for each factor: 56.9, 73.4, 70.4, 64.4, and 66.6, respectively). The DocuLive system scored significantly worse than the others (overall score 61.4 v 69.8 for DIPS and 69.7 for Infomedix; analysis of variance P=0.001), particularly in the content factor. The correlation of satisfaction with total computer use was 0.39 (P<0.001).
Discussion
Despite widespread implementation of electronic medical records systems in Norwegian hospitals, our results reveal a low level of use of all three electronic medical records systems by doctors, especially in the largest hospitals. The systems were mainly used for reading patient data, and doctors used the systems for less than half of the tasks for which the systems were functional. Among these unused functions were repetitive tasks such as writing prescriptions, which are apparently well suited for computers.
Essentially the same findings applied to all three systems, which suggests that similar results might be found in other countries. When the impact of an electronic medical records system is investigated, we suggest that its actual use should be considered rather than its claimed functionality.
Limitations of the survey
Our survey covered only doctors, but other healthcare workers probably also use the electronic medical records systems. We did not assess how frequently the various clinical tasks were performed nor how time consuming they were, making it difficult to weight them. Self reporting carries a risk of misinterpretation and bias, even when “value neutral” behaviour is investigated. Finally, the distinction between using the electronic patient records system and using a different computer application might not always have been clear to doctors.
Possible reasons for low level of use of electronic medical records systems
Access to computers and computer literacy
The low level of electronic medical records system use could be explained by a lack of available computers. This would, however, affect the use for all clinical tasks in a uniform manner. In addition, the majority of respondents reported that they had some computers available to them both in their offices and in the ward. The section covering computer literacy showed high scores, indicating at least a basic knowledge of computers. However, we cannot rule out potential unmet needs for specific training in electronic medical records system usage.
Flexibility of paper records
Paper based patient records are still in daily use in Norwegian hospitals. Thus the respondents could choose whether to use the electronic medical records systems. In some situations it might be more convenient to use paper records, such as for writing short prescriptions, spreading records on a table, or carrying documents around. Until a proper level of electronic integration is achieved, paper record will remain the most complete information source. In addition, the usefulness of an electronic records system for manipulating large amounts of data will not be apparent until historical information has accumulated for some time.
Traditional work routines
Our general findings of computer use conform to the traditional division of labour in hospitals—with writing (task 5) associated with secretaries, mediation of requests (tasks 9,11, and 13) associated with nurses, and reading associated with doctors. None of the electronic medical records systems seem to have stimulated the development of new or more advantageous ways of doing medical work,12 they have simply reinforced existing routines. This indicates that technology alone is not sufficient to achieve a well functioning electronic information system; organisational aspects must also be taken into account.
Working in new ways and performing tasks normally done by other professions often means disruption to established work roles, which may lead to local resistance.13 Staff who take on extra duties do not necessarily enjoy the benefits of more efficient work patterns, and new reward systems may be needed for acceptance of new work roles.
Differences between electronic medical records systems
We found considerable differences in the frequency of use of the three record systems for certain clinical tasks (fig 2). DocuLive was often used for checking and signing, indicating that doctors were using it, but it was used much less than the other two systems for other tasks (3, 4, and 10). A possible explanation for this is the degree of integration with other computer software. Infomedix and DIPS were predominately installed in smaller hospitals, where the same vendor often supplied any other computer modules used, simplifying integration. DocuLive was introduced in the largest hospitals, where the organisational complexity is greatest and where many independent information systems already exist, making it difficult to develop an integrated information system.14
Acknowledgments
We thank Eric Monteiro for excellent guidance, Stewart Clarke and Bernard Evans for help with translating the questionnaire, Turi Saltnes for expert help with questionnaire design and scanning, Peter Fayers and Eirik Skogvoll for experienced statistical advice, and Lars Aabakken for professional support.
Contributors: HL and GE gathered background information on implementations of electronic medical record systems, and HL gathered background information on hospitals. HL and GE performed the observational fieldwork, and HL, GE, and AF defined the content of the questionnaire. HL designed the questionnaire, coordinated its translation, programmed the database, and registered the hospital data. GE and HL performed randomisation, information gathering on each hospital unit, and follow up for successful inclusion. HL distributed, scanned, and statistically analysed the questionnaires and interviewed information technology staff by telephone. HL, GE, and AF jointly wrote the manuscript. HL is guarantor for the study.
Footnotes
-
Funding This investigation is funded by the Research Council of Norway through the Kvalis project (http://kvalis.ntnu.no/).
-
Competing interests None declared.
The study questionnaire and details of minimal requirements for electronic medical records systems appear on bmj.com