Janne Pitkänen
Department of Computer Science, Aalto University, Finland / Adusso Ltd., Helsinki, Finland
Marko Nieminen
Department of Computer Science, Aalto University, Finland
Matti Pitkäranta
Adusso Ltd., Helsinki, Finland
Johanna Kaipio
Department of Computer Science, Aalto University, Finland
Mari Tyllinen
Department of Computer Science, Aalto University, Finland / Oy Apotti Ab, Helsinki, Finland
Antti K. Haapala
AnttiPatterns, Oulu, Finland
Download article
Published in: Proceedings from The 14th Scandinavian Conference on Health Informatics 2016, Gothenburg, Sweden, April 6-7 2016
Linköping Electronic Conference Proceedings 122:6, p. 37-41
Published: 2016-03-31
ISBN: 978-91-7685-776-2
ISSN: 1650-3686 (print), 1650-3740 (online)
”In healthcare IT system procurement we always need to choose the cheapest one.” Do we? In this paper we present a method and a procedure for effective extraction of usability test results for public procurement. Successful procurement necessitates the alternative products to be compared considering their realistic utility. We can significantly contribute to this comparison by measuring usability in a practical way. Our UXtract method enables the extraction of detailed, traceable and commensurate findings for objective evidence. The method extracts structured data straight from the test. Our case in large scale healthcare settings shows that this method is efficient for scoring usability in procurement. We elaborate the results and discuss about the impact and challenges of comparison testing when using it for decision making of multimillion investments on information technology.
healthcare information system; usability testing; summative evaluation; comparison; public procurement
1] Mats A. Bergman, Sofia Lundberg, Tender evaluation and supplier selection methods in public procurement, Journal of Purchasing and Supply Management, Volume 19, Issue 2, June 2013, Pages 73-83.
[2] HIMSS EHR Usability Task Force. Selecting an HER for Your Practice: Evaluating Usability. Apr. 2011; http://s3.amazonaws.com/rdcms-himss/files/produc-tion/public/HIMSSorg/Con-tent/files/HIMSS%20Guide%20to%20Usability_Select-ing%20an%20EMR.pdf
[3] Jokela T, Laine J & Nieminen M(2013). Usability in RFP’s: The Current Practice and Outline for the Future. In Kurosu, M. (Ed.) Human-Computer Interaction. Applications and Services, Springer Berlin Heidelberg, 2013, 8005, 101-106
[4] Kushniruk A, Beuscart-Zéphir MC, Grzes A, Borycki E, Watbled L and Kannry J. Increasing the safety of healthcare information systems through improved procurement: toward a framework for selection of safe healthcare systems. Healthcare quarterly (Toronto, Ont.), 13 (2010), 53-58.
[5] Kushniruk A, Kaipio J, Nieminen M, Hyppönen H, Lääveri T, Nøhr C, Kanstrup AN, Christiansen MB, Kuo M-H, Bo-rycki EM (2014). Human Factors in the Large: Experiences from Denmark, Finland and Canada in Moving Towards Regional and National Evaluations of Health Information System Usability. Contribution of the IMIA Human Factors Working Group. Yearbook of medical informatics 01/2014; 9(1): 67-81.
[6] Lowry S. et al. Technical evaluation, testing and validation of the usability of electronic health records (NIST IR 7804). (Feb. 2012); http://www.nist.gov/healthcare/usability/up-load/EUP_WERB_Version_2_23_12-Final-2.pdf
[7] Nielsen J (1993) Usability Engineering. Academic Press, Boston, USA.
[8] O’Malley CE, Draper SW and Riley MS (1984). Construc-tive interaction: A method for studying human-computer-human interaction. In Shackel, B. (Ed.) Human-computer interaction – INTERACT’84. pp. 269-274.
[9] Redish JG, Bias RG, Bailey R, Molich R, Dumas J and Spool JM (2002). Usability in practice: formative usability evaluations - evolution and revolution. In CHI ’02 Extended Abstracts on Human Factors in Computing Systems (CHI EA ’02). ACM, New York, NY, USA, 885-890.
[10] Riihiaho S, Nieminen M, Westman S, Addams-Mor-ing R, Katainen J (2015): Procuring Usability: Experiences from Usability Testing in Tender Evaluation. In H. Oinas-Kukkonen et al. (Eds.): Nordic Contributions in IS Research. Proceedings of the 6th Scandinavian Conference on Information Systems, SCIS 2015, Oulu, Finland, August 9-12, 2015. Lecture Notes in Business Information Processing Volume 223, 2015, pp 108-120.
[11] Sauro J and Kindlund J 2005. A method to standardize usability metrics into a single score. In Proceedings of the SIGCHI Conference on Human Factors in Computing Sys-tems (CHI ’05). ACM, New York, NY, USA, 401-409.
