EVAL - Assessment of Health Information Systems


Elske Ammenwerth, UMIT - University for Health Sciences, Medical Informatics and Technology, Hall in Tyrol, Austria
E-mail: elske.ammenwerth@umit.at



Pirkko Nykänen, School of Information Sciences, Tampere University, Finland, E-mail: pirkko.nykanen@uta.fi

Jytte Brender, Dept. of Health Science and Technology, Aalborg University, Denmark, E-mail: jytte@brender.dk


Website: https://iig.umit.at/efmi/

Main Objectives:

Information and communications technologies (ICTs) are being used increasingly in health care. It is evident that the use of modern ICT offers tremendous opportunities to reduce clinical errors, to support health care professionals, and to increase the efficiency of care. However, there are also hazards associated with such technologies. Therefore, rigorous, generalizable evaluations of ICT applications in health care are absolutely necessary. Therefore, the aims of the working group are:

  • to foster transdisciplinary dialogues between experts from informatics, medical informatics, economics, health care, health services research, clinical epidemiology, biometry, psychology, sociology, ethnography, organizational development, operations research and other evaluation fields, on a European level, and to encourage exchange on methodological issues between researchers from different traditions;
  • to offer an opportunity to share knowledge with the aim of obtaining profitable cross-fertilization among different fields of expertise and especially between quantitative and qualitative research;
  • to promote a combined research agenda to develop frameworks and toolkits for information systems evaluation, offering guidelines for an adequate combination of evaluation methods and tools;
  • to discuss and clarify the networking needs for long-term evaluation research in medical informatics, and to promote combined research proposals at a European level.

Relevant Activities 2016:

  • The working group organized or co-organized the following activities at HEC/MIE 2016 in Munich (27.8. – 2.9.2016):
  • Workshop on "The two-sided market perspective of e-health" (Vivian Vimarlund, Tobias Mettler, Pirkko Nykänen, Michael Rigby)
  • Presentation on "Theoretical foundations for evidence-based health informatics: Why? How?" (Philip Scott, Andrew Georgiou, Hannele Hypponen, Catherine Craven, Michael Rigby, Jytte Brender)
  • Presentation on "Mixed methods evaluations for understanding impact on patient outcomes and work practice" (Rebecca Randell, Dawn Dowding, Philip Scott, Andrew Georgiou)
  • Presentation on "Challenges on personalized medicine and dgital health literacy" (Pirkko Nykänen) at the EFMI WG summit
  • Poster presentation on "Recommendations for Health IT Evaluation Training as a Key Prerequisite to Obtaining Evidence" (Elske Ammenwerth, Nicolette de Keizer, Jytte Brender, Catherine Craven, Eric Eisenstein, Andrew Georgiou, Paula Otero, Pirkko Nykänen, Michael Rigby, Saif Khairat, Philip Scott, Farah Magrabi, Charlene Weir)
  • Participation in "HEC 2016 - Meet the Expert"
  • At HEC/MIE 2016 in Munich, Elske Ammenwerth and Michael Rigby as editors organized the official Book Launch Reception for their new book: Evidence-Based Health Informatics - Promoting Safety and Efficiency through Scientific Methods and Ethical Policy. Around 50 colleagues, friends and authors joined the celebrations and congratulated the editors and authors. The book is open access available at: http://ebooks.iospress.nl/volume/evidence-based-health-informatics-promoting-safety-and-efficiency-through-scientific-methods-and-ethical-policy and got > 3.700 downloads until January 2017.
  • The working group contributed to a position paper entitled “Evaluation considerations for secondary uses of clinical data: Principles for an evidence-based approach to policy and implementation.” This paper will be published in the IMIA Yearbook of Medical Informatics 2017.
  • The working group submitted a paper entitled “How to Teach Health IT Evaluation: Recommendations for Health IT Evaluation Courses“ to MIE 2017, Manchester
  • The working group continued its close collaboration with the IMIA Working Group on Technology Assessment and Quality Development in Health Care. Phone conferences to discuss cooperation and joint activities were held regularly.
  • The bibliography on health IT Evaluation is available at https://iig.umit.at/efmi/bibliography.htm. The list contains journal papers and books considered as milestone publications.


Activities planned for 2017:

  • Presentation of a contribution entitled “How to Teach Health IT Evaluation: Recommendations for Health IT Evaluation Courses“ at MIE 2017.
  • Development of a taxonomy for health IT systems and health IT evaluation and subsequent update of the health IT Evaluation Inventory.
  • Contributions to a book entitled “theoretical foundations of health informatics” initiated by Philip Scott.
  • Continuing the links to the sister WGs of AMIA and IMIA by joint activities


Relevant Publications 2016:

  • Magrabi F, Ammenwerth E, Hyppönen H, de Keizer N, Nykänen P, Rigby M, Scott P, Talmon J, Georgiou A. Improving Evaluation to Address the Unintended Consequences of Health Information Technology. Yearb Med Inform 2016; 61-69.
  • Rigby M, Magrabi F, Scott P, Doupi P, Hypponen H, Ammenwerth E. Steps in Moving Evidence-Based Health Informatics from Theory to Practice. Health Informatics Research 2016; 22(4): 255-60.
  • Ammenwerth E, de Keizer N, Brender J, Craven CK, Eisenstein E, Georgiou A, Khairat S, Magrabi F, Nykänen P, Otero P, Rigby M, Scott P, Weir C. Recommendations for Health IT Evaluation Training as a Key Prerequisite to Obtaining Evidence. Poster at the MIE2016/HEC2016 conference, 28.8.-2.9.2016, Munich.
  • Ammenwerth E, Rigby M. Evidence-based Health Informatics - Promoting Safety and Efficiency Through Scientific Methods and Ethical Policy. Stud Health Tech Inform. Vol 222. 2016. IOS Press. Here are selected chapters from this book that were written by core members of the working group:
    • Rigby M, Ammenwerth E. The need for Evidence in Health Informatics. pp. 3 - 13.
    • Magrabi F, Ong M-S, Coiera E. Health IT for Patient Safety and Improving the Safety of Health IT. pp. 25-36.
    • Brender McNair J. Theoretical Basis of Health IT Evaluation. pp. 39-52.
    • Scott PJ. Mixed Methods: A Paradigm for Holistic Evaluation of Health IT. pp. 102-113.
    • Hyppönen H, Ronchi E, Adler-Milstein J. Health Care Performance Indicators for Health Information Systems. pp. 181-194.
    • Nykänen P, Kaipio J. Quality of Health IT Evaluations. pp. 291-303.
    • Ammenwerth E, de Keizer N. Publishing Health IT Evaluation Studies. pp. 304-311.
    • Georgiou A. Finding, Appraising and Interpreting the Evidence of Health IT. pp. 312-323.
    • Ouloch T, De Keizer N. Evaluation of Health IT in Low-Income Countries. pp. 324-35.
    • Rigby M, Ammenwerth E, Talmon J. Forward Outlook: The Need for Evidence and for Action in Health Informatics. pp. 355-366.