Improving Evaluation to Address the Unintended Consequences of Health Information Technology:

Journal: IMIA Yearbook
ISSN: 0943-4747

Unintended Consequences: New Problems and New Solutions

Issue: 2016: IMIA Yearbook 2016
Pages: 61-69

Improving Evaluation to Address the Unintended Consequences of Health Information Technology:

a Position Paper from the Working Group on Technology Assessment & Quality Development

Special Section: Unintended Consequences: New Problems and New Solutions

Working Group Contributions

F. Magrabi (1), E. Ammenwerth (2), H. Hyppönen (3), N. de Keizer (4), P. Nykänen (5), M. Rigby (6), P. Scott (7), J. Talmon (8), A. Georgiou (1)

(1) Australian Institute of Health Innovation, Macquarie University, Sydney, Australia; (2) UMIT, University for Health Sciences, Medical Informatics and Technology, Hall in Tyrol, Austria; (3) National Institute for Health and Welfare, Information Department, Helsinki, Finland; (4) Academic Medical Center, Department of Medical Informatics, Amsterdam, The Netherlands; (5) University of Tampere, School of Information Sciences, eHealth Research, Finland; (6) Keele University, School of Social Science and Public Policy, Keele, United Kingdom; (7) School of Computing, University of Portsmouth, Portsmouth, United Kingdom; (8) Hi-Way, Sint Odiliënberg, The Netherlands


Medical Informatics, Information Systems, evaluation studies, Evidence-based medicine, Patient safety


Background and objectives: With growing use of IT by healthcare professionals and patients, the opportunity for any unintended effects of technology to disrupt care health processes and outcomes is intensified. The objectives of this position paper by the IMIA Working Group (WG) on Technology Assessment and Quality Development are to highlight how our ongoing initiatives to enhance evaluation are also addressing the unintended consequences of health IT. Methods: Review of WG initiatives Results: We argue that an evidence-based approach underpinned by rigorous evaluation is fundamental to the safe and effective use of IT, and for detecting and addressing its unintended consequences in a timely manner. We provide an overview of our ongoing initiatives to strengthen study design, execution and reporting by using evaluation frameworks and guidelines which can enable better characterization and monitoring of unintended consequences, including the Good Evaluation Practice Guideline in Health Informatics (GEP-HI) and the Statement on Reporting of Evaluation Studies in Health Informatics (STARE-HI). Indicators to benchmark the adoption and impact of IT can similarly be used to monitor unintended effects on healthcare structures, processes and outcome. We have also developed EvalDB, a web-based database of evaluation studies to promulgate evidence about unintended effects and are developing the content for courses to improve training in health IT evaluation. Conclusion: Evaluation is an essential ingredient for the effective use of IT to improve healthcare quality and patient safety. WG resources and skills development initiatives can facilitate a proactive and evidence-based approach to detecting and addressing the unintended effects of health IT.

You may also be interested in...


For Discussion – Original Article

E. Ammenwerth (1)

Methods Inf Med 2015 54 4: 298-307


For Discussion – Original Article

N. Al-Shorbaji (1), L. Hanmer (2), R. Hussein (3), F. Magrabi (4), A. Moen (5), L. A. Moura (6), H.-A. Park (7), P. Scott (8)

Methods Inf Med 2015 54 4: 308-318


J. R. Moehr1 , C. Anglin 2 , J. Schaafsma 3 , S. Pantazi 1 , N. Grimm1

Methods Inf Med 2006 45 1: 67-72