Optimising mHealth helpdesk responsiveness in South Africa: towards automated message triage.


In South Africa, a national-level helpdesk was established in August 2014 as a social accountability mechanism for improving governance, allowing recipients of public sector services to send complaints, compliments and questions directly to a team of National Department of Health (NDoH) staff members via text message. As demand increases, mechanisms to streamline and improve the helpdesk must be explored. This work aims to evaluate the need for and feasibility of automated message triage to improve helpdesk responsiveness to high-priority messages. Drawing from 65 768 messages submitted between October 2016 and July 2017, the quality of helpdesk message handling was evaluated via detailed inspection of (1) a random sample of 481 messages and (2) messages reporting mistreatment of women, as identified using expert-curated keywords. Automated triage was explored by training a naïve Bayes classifier to replicate message labels assigned by NDoH staff. Classifier performance was evaluated on 12 526 messages withheld from the training set. 90 of 481 (18.7%) NDoH responses were scored as suboptimal or incorrect, with median response time of 4.0 hours. 32 reports of facility-based mistreatment and 39 of partner and family violence were identified; NDoH response time and appropriateness for these messages were not superior to the random sample (P>0.05). The naïve Bayes classifier had average accuracy of 85.4%, with ≥98% specificity for infrequently appearing (<50%) labels. These results show that helpdesk handling of mistreatment of women could be improved. Keyword matching and naïve Bayes effectively identified uncommon messages of interest and could support automated triage to improve handling of high-priority messages.





Published Version (Please cite this version)


Publication Info

Engelhard, Matthew, Charles Copley, Jacqui Watson, Yogan Pillay, Peter Barron and Amnesty Elizabeth LeFevre (2018). Optimising mHealth helpdesk responsiveness in South Africa: towards automated message triage. BMJ global health, 3(Suppl 2). p. e000567. 10.1136/bmjgh-2017-000567 Retrieved from https://hdl.handle.net/10161/18141.

This is constructed from limited available data and may be imprecise. To cite this article, please review & use the official citation provided by the journal.



Matthew M. Engelhard

Assistant Professor of Biostatistics & Bioinformatics

Developing new machine learning methods for multi-modal longitudinal clinical data to support clinical decision-making.

Unless otherwise indicated, scholarly articles published by Duke faculty members are made available here with a CC-BY-NC (Creative Commons Attribution Non-Commercial) license, as enabled by the Duke Open Access Policy. If you wish to use the materials in ways not already permitted under CC-BY-NC, please consult the copyright owner. Other materials are made available here through the author’s grant of a non-exclusive license to make their work openly accessible.