A multi-level evidence approach to evaluate training, performance, and system-level impact of Hamdard Force: a community mental health intervention

Abstract Background There is limited evidence to demonstrate the impact of training community mental health workers beyond knowledge acquisition. This study evaluated the training, workforce performance, and system-level outcomes of the Hamdard Force, a community mental health intervention, implemented in two districts of Khyber Pakhtunkhwa, Pakistan, as part of a multi-tiered, digital MHPSS service over 8 months. Methods We followed a multi-level evidence approach, guided by the New World Kirkpatrick Model, using a mixed-methods design that combined quantitative and qualitative indicators. We assessed participants' reactions, learning, behavioral outcomes, system-level outcomes of the intervention, and conducted Focus Group Discussions to identify contextual factors that influenced implementation. Results At Tier 1, a total of 403 community workers were trained, who reported satisfaction with the training content and relevance, and demonstrated significant improvement in knowledge post-training. Less than half of the trained workforce remained active in the field and referred people with mental healthcare needs under supervision. A total of 855 people were triaged by clinical psychologists at Tier 2 of the service model, and 549 cases met the criteria of being valid. The results of detailed clinical assessment and management demonstrated encouraging outcomes regarding the effectiveness of the intervention, both at the individual case and at the system-level. The Focus Group Discussions revealed the process of supervision, organizational support and digital tools of learning were significant drivers of the intervention. Challenges with internet connectivity, limited digital literacy, and persistent mental health stigma were identified as implementation barriers. Conclusion Overall, the integrated findings suggest that the intervention helped to build the capacity of the workforce, facilitated their performance through supervision, provided services through a task-sharing approach and strengthened referral pathways. These findings underscore the importance of implementation-focused evaluations for scaling up community mental healthcare in low-and middle-income countries.