How using Behavioural Insights (BI) methods helps in assessing its impact

Initiatives for strengthening social cohesion and building youth resilience (especially against involvement in anti-social and/or criminal activities) encounter numerous implementation and impact measurement challenges. The first being effective implementation of the intervention with vulnerable groups in line with the ‘do no harm’ principle, while the second relates to robust measurement of the impact. Traditional impact assessment measures fail to penetrate the complex web of behaviours, relationships, and individual and collective attitudes that determine an individual’s vulnerability to involvement in criminal or anti-social activities.  

These challenges are especially pronounced in a prison’s setting. Development practitioners working on the reintegration and rehabilitation of prisoners must ensure that the interventions are holistic (encompassing social, psychological, and economic elements), inclusive (benefitting the target population irrespective of their gender, ethnicity, or income status etc.), and realistic (responsive to the situation on ground). Furthermore, traditional impact assessment tools based on direct questioning from project beneficiaries are ineffective in prison settings because of the presence of jail authorities and law enforcement personnel during impact assessment exercises. Their presence often leads to overwhelmingly positive feedback which is not reflective of the actual situation on ground. 

Behavioural science provides a robust theoretical and methodological framework for assessing the impact of interventions to promote community resilience and positive behaviour among vulnerable populations. Research instruments from behavioural science can help in identifying the ‘deep structure’ of the target population’s vulnerability and measuring the behavioural changes in the pre and post intervention stages by linking them with the project’s theory of change. This is especially useful for eliciting accurate responses from young people in conflict with the law who are more comfortable with discussing proxy indicators of positive behavioural change (improved relationship with family members, community gatekeepers, friends etc.) rather than referring directly to the actual factors (violent thoughts, anger etc.) reflecting criminal behaviour. This allows practitioners to empirically assess project effectiveness while creating a scientific foundation for altering programme design and implementation strategy.

UNDP Pakistan has been employing methods from behavioural science to measure the impact of its ongoing programming to socially and economically empower young people, in conflict with the law, under its Youth Empowerment Programme (YEP). The specific interventions under the programme focused on the provision of livelihood skills and psychological counselling support to 50 young prisoners between the ages of 18 and 29 years incarcerated in District Jail Malir, Karachi. The overall purpose of the intervention was to promote social and economic rehabilitation of young people, in conflict with the law, by providing them with the necessary skills and knowledge to become successfully reintegrated in society following their release from prison. In this regard, the project’s theory of change states that, ‘If young people in prisons are provided with opportunities for economic empowerment, personal development, and constructive social engagement, then their vulnerability to involvement in violence and conflict (including violent extremism) will be decreased.’

Based on the above theory of change, we used the ‘Oneness’ tool to measure the extent to which young prisoners’ connectedness with their families, relatives, friends, workplace colleagues, and community gatekeepers improved or deteriorated following the intervention. The results showed an overwhelmingly positive impact of our interventions in terms of enhancing the young prisoners’ positive social connectedness, indicating encouraging prospects for their reintegration and rehabilitation after release. In this regard, the BI tool accurately determined the project’s impact by indirectly measuring intervention impact, using proxy indicators pertaining to social connectedness; where a direct inquiry would not have yielded correct results. Furthermore, by probing the impact of the project in terms of positively strengthening young people’s relationship with their social networks, the tool was able to measure the extent to which young people involved in petty offences were vulnerable to carrying out more serious criminal acts including violent extremism.

Following the successful application of the ‘Oneness’ tool, UNDP Pakistan will employ additional BI tools based on counter-factual methodologies in its upcoming work with young prisoners and probationers in Khyber Pakhtunkhwa. This will help in creating a robustly validated knowledge base on the social and psychological factors contributing to young people’s involvement in anti-social activities.

Authors:

Dr. Umar Taj 

Dr. Umar Taj is a Senior Behavioural Science Advisor at UNDP Pakistan and Associate Professor in Behavioural Science at the Warwick Business School, UK. 

Shumaila Waheed 

Shumaila Waheed is working as a Project Manager with the Society for the Protection of the Rights of the Child (SPARC). She is a development professional with 15 years of experience in child rights and youth empowerment sectors.

Hamza Hasan

Hamza Hasan works with UNDP's Youth Empowerment Programme as a Senior Social Inclusion Officer. He is a Cultural Anthropologist with experience of promoting and facilitating meaningful participation of vulnerable groups in the development process through research, strategic programme development, and inclusive project implementation