James Drummond has spent the past six years as a Policy Analyst at the Organisation for Economic Co-operation and Development (OECD). James looks at how to use behavioural insights (BI) to help ensure that regulatory policy in government are designed and delivered based on evidence. The Western Cape Government was lucky enough to work with James and the OECD to host the Behavioural Insights Conference in Cape Town in September 2018. We spoke to James about how to effectively apply behavioural insights in government, the challenges faced by developing countries in doing so, and how best to overcome them.
Under constant pressure to cut costs whilst grappling with ambitious transformation agendas, government will undeniably reap the benefits of behavioural insights (BI). Not only can BI improve the efficiency of policy design and frontline service delivery, but it can also be turned inward on the public service itself to transform internal working culture and ultimately drive better governing.
Sounds great—but where to start? With no prior exposure to behavioural insights work, policymakers weaving these processes into government for the first time often turn to expert advice and guidance. However, despite the comfort such theoretical tools can provide at the outset, James delves into the challenges that the real world can all too often present, along with some first-hand advice on how to tackle them.
CONDITIONS FOR SUCCESS: TOOLS AND GUIDANCE
Are there certain conditions for success when applying BI in government? This is what James and his colleagues in the OECD Public Governance Directorate spend their time exploring through three broad streams of work: providing tools and standards on how to approach BI; supporting governments in carrying out experiments; and fostering a global community of BI practitioners and policy makers. In this line of work, he also often finds himself involved in hosting conferences that bring together a mixture of experts, policymakers, academics, the private sector, and think tanks.
James explains that, “one thing that you always hear is that regulatory policy should be evidence-based, which BI is excellent at providing”. However, people don’t always have access to the tools and guidance on how to go about applying BI in practice, especially the process they need to go through from start to finish. To create this evidence-base, James explains that he tries to instil the use of concrete methodology in government, as well as in the day-to-day delivery of regulation.
Introducing the ‘BASIC’ framework
The ‘BASIC’ toolkit is one of the master tools developed by the OECD in partnership with Dr Pelle Guldborg Hansen of Roskilde University and iNudgeyou to help governments start drawing on behavioural insights. The toolkit provides practitioners and policymakers with a step-by-step process for analysing a policy problem, building strategies, and developing behaviourally informed interventions. Unlike other traditional BI frameworks, the OECD wanted to develop a tool that caters for the full public policy cycle, ranging from the behavioural analysis of a policy problem, right up until the experimental stages. They wanted to focus on creating a tool to help any policy maker start to use BI with minimal start-up costs, contributing to mainstreaming the methodology in policy making across government.
[1] The ‘BASIC’ framework
Adapting to the unexpected
As we move beyond the theoretical - demonstrated by the BASIC toolkit - and onto the practical side of applying BI, James shared details of several projects in which the OECD’s frameworks were confronted with the unexpected complications of a real-world setting. In his first example, the team received a request to work on a telecommunications regulation project in Colombia [2] that involved applying a behavioural lens to data analysis for a public policy problem. The OECD convened an expert team of policy analysts at the OECD headquarters in Paris and behavioural science experts around the world working predominantly remotely practising “systems from afar” where they “looked at the data, analysed it, [and] provided a set of recommendations”. However, James recounted that no matter how studied the approach, understanding culture can pose a great challenge, especially when working remotely.
James highlights a second example from a horizontal project carried out with Canada, Ireland, Mexico, and Oman, which focused on how regulatory policymakers can foster a strong culture of safety in the energy sector. The OECD team, which included policy analysts, three behavioural researchers and an academic panel of experts, created an online questionnaire that was sent via email to the regulators, who were then asked to send it to regulated entities. The OECD team discovered that differing administrative backgrounds and regulatory frameworks can introduce unique barriers at the testing stage of an intervention:
"a challenge with implementing the experiments was more the differences in each country in terms of what's allowed – both legally and practically - and what isn't".
This includes, for example, privacy legislation or centralised versus decentralised bureaucratic systems. This was a contributing factor to some sample sizes for countries being bigger than others.
The importance of cultural context
As the example above shows, the differences in cultural contexts have a significant impact on the implementation and results of a project. It's little wonder that when we asked James to pinpoint the lessons learnt, he emphasised the importance of understanding deeply the culture you’re working with, especially before implementing a BI experiment, to help develop a fuller understanding of the way it functions and working this knowledge into the experimental design.
This involves convening the key players on the ground, immersing them in the day-to-day operations and spending time engaging with local people. James also highlighted the large time investment that this strategy requires: "In OECD country-level work, we always spent significant time – often many months or more – scoping the project and working throughout project delivery with focal points in each one of the regulatory authorities in each of the countries… to make sure that the projects outputs and outcomes are actually in line with the reality of what they’re dealing with.”
James also recommended acquiring an expert with an existing background in the local environment or striking a working group of local stakeholders to act as advisors to the project team, to be able to go to them and say, “here’s what we’re hearing but we don’t understand... or can you give us some more background, is there something we’re missing?” They are also critical in analysis and report writing stages, where they can provide advice and context to help sharpen the final product. This falls in line with our research findings on the importance of partnerships: BI projects, especially in developing country contexts, depend on multiple stakeholders to pool together skill sets, perspectives, and resources to truly be successful.
Documenting your learnings
James identified a need for in depth documentation of projects and their unique learnings: "while we spent the time trying to understand the regulators and the cultural contexts from their perspectives, we didn't have the ability to include robust questions on culture and related intervening variables in the questionnaire sent to the study participants”. This created some issues when analysing the data. As the team processed the data from the experiment, they noticed that the countries varied considerably across all behavioural insights studied. For example, they tested both injunctive and descriptive social norms, which did not work for any country except for descriptive norms in Mexico. James noted that:
“we assume there were likely intervening variables related to each country’s context and culture, but since we could not include questions on these aspects in the experimental design we found it tough to find concrete explanations for the differences we noted”.
James noted that the experimental design did include descriptive statistics, but again how much they could include was limited by legal and practical issues. The team turned to Hofstede’s national cultural dimensions in the final report to help interpret the data and fill these holes. But James admits that this is suboptimal, as "not only does this create more work in the current project but you also lose some ability to share more in depth learnings from the data, especially going beyond a discussion of what worked and what did not to help policy makers understand why”. This said, James notes that there are always trade-offs:
“this was first of its kind research and was more diagnostic in nature, so we have to be realistic about what we can achieve. The report highlighted many ways in which further research could provide these deeper answers to these bigger questions, with initial analysis to guide researchers”.
A great example of the practical application of this tip can be found in OECD's publication 'Behavioural Insights and Public Policy: Lessons from Around the World', a global collection of over 100 case studies from around the world with key lessons for public institutions [3]. This documentation of projects shares valuable lessons with others attempting similar work and inspires those considering entering into the public policy BI space.
Recently, recognising the importance of sharing lessons learned, the OECD Observatory for Public Sector Innovation (OPSI) developed several online tools to support and enhance knowledge-sharing and collaboration on BI including an Interactive BI Units Map, which currently features institutions applying BI to public policy in 43 different countries, and an online Repository of BI projects, which currently includes more than 100 BI projects from 41 countries, ranging across various policy areas. By increasing the visibility and sharing of BI initiatives, practitioners and experts can consult on ongoing projects in real-time, offering advice and relevant evidence at the most critical phases of policy design, experimentation and evaluation. In addition to being a dedicated space to access data and information about the BI work led by governments, the tools are designed to facilitate engagement between policymakers, BI experts, practitioners and academics, and civil society, who may use the platform to identify and accelerate opportunities for coordinated international collaboration to address today’s policy challenges.
Locating BI projects from the OPSI BI Project Archive
Incorporating BI into the policy cycle
The field is starting to recognise the multitude of ways to incorporate BI in all stages of the policy cycle; whether it be external applications, such as improving the efficiency of frontline service delivery, or internally, to unify staff and stakeholders in their efforts, change attitudes and culture within organisations, and help to improve bureaucratic efficiency.
There is certainly room for innovation in public service delivery in the Western Cape, where high crime and murder rates are exacerbating the demand for healthcare services. A study carried out in 2018 by the Office of Health Standards Compliance showed that out of 65 primary healthcare facilities inspected in the Western Cape, none met the standard for an acceptable level of care, and the majority were scoring well below it [5].
James adds: “I think there’s a lot of opportunity there for public service delivery... it seems very difficult in low resource contexts for the government to get to the citizens and implement things like health services or community safety… so they seem to be very ripe grounds especially in systems that are cash-strapped.” BI interventions have great potential to improve areas of the healthcare service such as diagnosis, staff attitudes, administration, or appointment systems.
James also noted the OECD has been recently focused on developing work on BI applied to improving the efficiency of government. He recently co-authored a paper with a behavioural scientist, Daniel Shephard, and OECD colleague Daniel Trnka on “Behavioural insights and regulatory governance: Opportunities and challenges” [4]. James explains that “BI has been very successful at demonstrating how regulatory policies can be “humanised,” to coin a phrase from Cass Sunstein, but comparatively little research has shown us how to humanise the regulatory policy making process”. The paper goes through a number of behavioural barriers and biases present in the institutions, processes and tools of policy making, and offers possible practical solutions to test.
The benefit of BI in day-to-day government
Integrating BI in the day-to-day in government is something we’re thinking about a lot in the Western Cape. This has led to the implementation of several initiatives internally, including the Walk4Health competition, and the Climate Change pilot with the latter being aimed at reducing electricity consumption at one of the Provincial Government’s largest buildings. Electricity usage was tracked using smart energy meters, and a package of behavioural nudges was used to influence behaviour change, which led to a 13.5% reduction in energy consumption over four months. To add to this, such improvements in attitudes internally also have a positive impact on other areas of the policy cycle such as external service delivery—saving money on electricity, for example, frees up more budget to be allocated to health services.
A promising future
There is no doubt the work James and his colleagues at the OECD are doing to support governments applying behavioural insights will be a huge contributing factor to the field’s growth in the Global South. South Africa, and many other nations of the Global South, are active members of the OECD BI Expert’s Network where we can exchange ideas and foster learning with several hundred units from around the world. We hope that alongside the invaluable tools and guidance the OECD provides to budding and experienced BI practitioners in the public policy space, that this article adds another weapon to the arsenal through James’ personal experiences and advice.
Reflecting on what seems to be a recurrent theme in other interviews carried out for this series, and key takeaway from James is the importance for practitioners to consider context, and in order to sustain the benefits of these unique lessons, to properly document them. Finally, there is a lot of potential for incorporating BI in all aspects of the policy cycle, including turning it inward on the public service itself. We cannot deny that changing the narrative within highly bureaucratic structures like the government is no mean feat, but such an approach has the potential to be the “next frontier” for governments globally.
References
[1] OECD (2019), BASIC Brochure. Tools and Ethics for Applied Behavioural Insights: The BASIC Toolkit, OECD Publishing, Paris. Available here: https://www.OECD.org/gov/regulatory-policy/BASIC-Toolkit-web.pdf
[2] OECD (2016), Protecting Consumers through Behavioural Insights: Regulating the Communications Market in Colombia, OECD Publishing, Paris. Available here: https://doi.org/10.1787/9789264255463-en.
[3] OECD (2017), Behavioural Insights and Public Policy: Lessons from Around the World, OECD Publishing, Paris. Available here: https://doi.org/10.1787/9789264270480-en.
[4] Drummond, J., D. Shephard and D. Trnka (2021), "Behavioural insight and regulatory governance: Opportunities and challenges", OECD Regulatory Policy Working Papers, No. 16, OECD Publishing, Paris. Available here: https://doi.org/10.1787/ee46b4af-en.
[5] State of Provincial Healthcare System Spotlight on Western Cape May 2018. Available here: https://tac.org.za/files/tac-western-cape-state-of-health-report-may-2018.pdf