Getting a nudge: Moving from niche innovations to systems change

The bread and butter of many evaluators is to look at innovative pilots or prototypes, explore how they were developed and implemented, and in different ways assess their impact and overall value. Yet all too often, no matter how exciting the innovation, how successful it is in having a local impact, and how much value it creates, systems can be remarkably unresponsive to scaling pilots more widely. The pace of uptake is often slow or non-existent, with the consequences that benefits are spread unevenly, and the complex problems the innovations were looking to address remain embedded.

Working with colleagues from Kinnect Group, we had the privilege of following four innovative road safety projects working under the banner of the Signature Programme, led by ACC and the New Zealand Transport Agency. Each project was intended to trial innovative ways of working, and if successful, influence road safety policy and systems nationally.

With a three-year frame to work within, we had the luxury of being able to follow the projects to explore the outcomes they were achieving, and to develop an understanding of how the projects were able to nudge business as usual activity in road safety.

As we followed the trajectory of each project, socio-technical systems theory became a key reference point for us. This frame of thinking explores how niche innovations progress to become embedded as business as usual within systems and organisations.

Developed by Frank Geels, a ‘socio-technical transition’ is one where social and technological forces come together to create major shifts in transport, energy and other systems (see the reference to Geels 2011 below). Geels posits that socio-technical systems have three key elements:

  • Systems that include supply side (innovation) and demand-side (user environment)

  • Actors that are involved in maintaining and changing the system; they carry, reproduce and challenge the rules in their activities

  • Rules and institutions that guide actor’s perceptions and activities; they provide constraining and enabling contexts for actors and systems

Interaction of elements in a socio-technical regime

Interaction of elements in a socio-technical regime

Put briefly, socio-technical systems theory proposes that the combination of rules that are regulative (explicit and formal), normative (values, expectations, rights and responsibilities) or cognitive (the frames through which meaning or sense is made), together act within social technical regimes that are interlinked and maintain the stability of a system. These regimes combine socio-cultural, policy, science, technological and market forces that together can be very effective in resisting the pressures of change.

The diagram below shows the process through which socio-technical transitions are theorised to occur, or which may be prevented from occurring through the interrelationship of rules that govern systems. There are three levels of interplay:

  • Niches: the locus for radical innovation; these are often protected spaces where innovation is permitted to operate and even flourish.

  • Socio-technical regimes: the ‘deep structure’ of established rules and systems that stabilise current practice, and are shaped by technological, scientific, policy, socio-cultural and market forces.

  • Socio-technical landscape: the wider context that influences niche and regime dynamics, including trends, values, ideologies and macro-economic patterns (Geels 2011).

Socio-technical transitions from niche innovations to systemic change

Socio-technical transitions from niche innovations to systemic change

For a niche innovation to become sufficiently embedded to ‘nudge’ changes in systems, a combination of factors is needed that can work at macro (national/international); meso (city); and micro (community) levels (see Marletto 2014). This is a non-linear process involving a complex interplay of different factors. New innovations must break through to the socio-technical regime as opportunities arise, creating adjustments, stabilising and eventually establishing themselves as the new regime, in turn influencing or shifting the socio-technical landscape.

A socio-technical approach proposes that in many cases, the technical solutions to complex problems already exist; what matters is having the organisational structures and system processes that enable their proper use. This approach signals a shift in focus towards relationships rather than actors, on actions rather than functions. But such shifts are hugely challenging because there are many ways in which the actors within a dominant system work to preserve their position – as anyone trying to nudge the dominant position of the car in transport systems has found (see for example Cohen, 2012).

In the discussion that follows, we’ll look at two examples from the Signature Project and the extent to which they were able to bring about systemic change.

The first example was Behind the Wheel, an innovative driver licensing project, aimed at improving the driver licensing journey at a local level and improve safety outcomes for young drivers. Although itself a highly planned, well-developed and system-focused local intervention, it was also one of a large number of community-based drivers licensing projects in New Zealand, limiting its potential individual impact.

More fundamentally, at a national level, the drivers’ licensing system is highly constrained, with key aspects such as driver instruction skills, costs and payment systems, and test content and processes tightly set within a range of rules and regulations that prevent rapid change. The project’s greatest immediate influence was in the changes to the licensing environment locally, and its direct feed into the national licensing resources.

The figure below details the genesis of the project, and its influence on licensing systems from a socio-technical systems perspective:

  • At a landscape level, the road toll, the perceived needs of employers and demands for driver licensing programme together drive demand for community initiatives.

  • At the niche innovation level, the project was able to implement comprehensive changes to driver licensing across the pathway of pre-learner to fully licensed.

  • At the system/regime level, the project directly influenced resource development, but other elements that influence licensing uptake (such as school curriculum, instruction skills requirements, costs and payment systems, and test processes) have remained unchanged through regulatory and rule constraints. The initiative does however provide learning to inform wider practice, and potentially system changes.

It is worth noting that while the national licensing system has been resistant to change, should any of the current national level activity mentioned above result in regulatory changes, these are likely to become well-embedded and have long-lasting impacts.

Influences of community driver licensing project on national licensing systems

Influences of community driver licensing project on national licensing systems

The second initiative was a road safety programme targeting visitors to New Zealand. The Visiting Drivers project drew on the efforts of many organisations to support the safety and experience of visitors to New Zealand across the spectrum of travel to New Zealand, including planning and booking, in-flight information, on arrival information, and journey support.

A detailed evaluation report sets out the achievements of the project, and the figure below provides some context from a socio-technical systems perspective.

  • At a landscape level, the road toll, industry concerns of negative visitor experiences, and the inherent challenges of driving in New Zealand as a visitor created a demand for a central government-led intervention in this space.

  • At the niche innovation level of implementation, the Visiting Drivers partnership built an integrated system of support for visitors, spanning the continuum from planning and booking travel, in flight, on arrival and support through the journey. This was achieved through a coalition of players from central government agencies, industry and local government.

  • At the regime level, Visiting Drivers was able to exert rapid and widespread influence among partners, including industry practice in the target regions, road infrastructure, information sharing between operators and agencies, and coordinated messaging across the partnership. Council investment was not taken up to the degree intended. There were also national level changes made, including an industry code of practice, police practice, driver information resources, and to some extent, NZTA investment criteria.

Importantly, for Visiting Drivers (and in contrast to Behind the Wheel), legislative change was not needed, and enabled rapid mobilisation of activity across partners. But this also had the effect that when resources were reprioritised, some aspects of project activity ended quickly.

Influence of Visiting Drivers project on systems

Influence of Visiting Drivers project on systems

In evaluating these projects, we concluded that local niche innovations are as important for the systemic barriers and enablers that they reveal, as they are for the impacts that they have on a local community.
If we simply treat innovations as local level pilots, we lose sight of the system issues that can constrain or enable successful outcomes, and the broader system levers and changes that we need to scale local innovation. Institutions, power dynamics, incentives and politics – factors that contribute to the complexity of systems – all matter for bringing about change.

Policymakers should therefore explore where a system needs a nudge as much as whether or not a prototype should be continued or expanded.

We also found that that deviating from existing practice, even when the niche is sanctioned by the system owners, takes a lot of effort. And scaling the effort to wider practice and impact takes more than just doing more of the intervention; scaling requires a wider mandate to change existing rules and normative practice.

We’re very grateful for ACC and NZTA for funding and supporting the evaluation of the Signature Programme, and to the project leadership and teams who contributed across all aspects of the evaluation.

Further reading
Cabaj M. 2018. Evaluating efforts to scale social innovation. Here to There Consulting. Waterloo, Ontario.
Cohen MJ. 2012. The future of automobile society: a socio-technical transitions perspective. Technology analysis & strategic management, 24(4), 377-390.
Geels FW. 2011. The multi-level perspective on sustainability transitions: Responses to seven criticisms. Environmental Innovation and Societal Transitions, 1: 24-40
Marletto G. 2014. Car and the city: Socio-technical transition pathways to 2030. Technological Forecasting & Social Change, 87: 164–178
Opit S, Witten K. 2018. Unlocking Transport Innovation: A Sociotechnical Perspective of the Logics of Transport Planning Decision-Making within the Trial of a New Type of Pedestrian Crossing. SHORE & Whariki Research Centre, Massey University.

… and watch this space, we hope to have a report we’ve written on innovating road safety added to this list in the next few weeks.








Five years of Dovetail

In June 2014, I set off on a solo journey with Dovetail. It didn’t really stay solo for long – from its inception I’d coined the line of “a collaborative hub of changemakers”, and that’s how it quickly evolved. I’m genuinely honoured to have worked with so many people as collaborators in so many different projects, from rapid literature reviews to long-term research and evaluation projects.

It’s been quite a ride, with a huge amount of satisfaction, and some stress at times, but I’ve been happier working this way than in any other role I’ve had.

As this year’s infographic shows, it’s certainly been a productive five years. There have been lots of highlights, a few of which are

But the biggest highlight has been the privilege of working with people who are passionate about what they do and determined to change the world for the better, whether as clients or collaborators.

Dovetail 5-year report.JPG

For a clearer and printable version, you can also download a pdf here.

In the interests of scientific investigation (and a berry crumble)

Back in June I bought the new company vehicle – an eZee Forza e-bike with a gutsy 34Ah battery, which I was assured by Electric Bike Team in Auckland that it would take me far and wide.

Since I got the bike I’ve used it a lot, and significantly reduced my around-town car travel, especially during the week. It’s made travel from Grey Lynn to my office in the Viaduct completely door to door, there inside 15 minutes and literally no sweat. The latter has meant that I’m also able to visit many clients without feeling the need to take a shower.

e-bike view6.jpg

The Forza is not a thing of beauty, it’s heavy and solid and designed to be a good workhorse; a friend calls it the Landcruiser of e-bikes, built to last not to dazzle. And the battery is a beast – it weighs six kilograms on its own.

The battery has lasted well, I can go a week at a time without charging it, but I’d never really put it to the test, until today.

Last weekend I planted a couple of trees out at the family section at Karekare, and after a warm and dry week they were due a watering. And with a still, warm morning to wake up to, it was the ideal time to head out across the Waitakere Ranges and test the range and strength of the power pack under my seat.

I figured Westie hoons wouldn’t be up before 8 and on the road before 9, so with a 7am start I thought I’d get a couple of hours grace before I was at significant safety risk. So just before 7, wearing my most fluorescent of bike tops, I set off.

To start with, I was careful with the power settings, I wanted to be able to get there and back without having to pedal unassisted. But once I got to New Lynn and beginning the climb up to Titirangi, it was time to hit maximum power setting.

There are generally two types of e-bike motors, cadence and torque. Cadence will simply chug along at whatever power level it’s set to, regardless of how much effort you put in on the pedals. The lightest of touches will send the bike off at its power setting. Torque motors, which my Forza has, gives back proportional to the energy that you put in, as well as the power level that it’s set at. It’s a more natural way of riding, and requires you to put in some effort. It is also easier to go slow with, which when you’re on shared paths with pedestrians, is really helpful.

So if the cadence is like having cheerful minions pushing you along while you enjoy the scenery, the torque is more like a permanent tailwind to boost your effort.

With the torque tailwind I ascended to Titirangi easily, and still locking around 25kmh up the hill. From there it was on to Scenic Drive where the views are spectacular. The battery was doing fine, not even 10% used by that point.

Looking out towards Huia from Scenic Drive in Titirangi

Looking out towards Huia from Scenic Drive in Titirangi

The thing about riding in the Waitakere Ranges is that you don’t get to the top and go down the other side. You go up to the top of something, then down, then back up, then down, and so on until suddenly you’re going constantly down to sea level.

Anytime there was up involved, I was at full power, and the battery laughed in the face of everything it was presented with. And the disc brakes, god how I love disc brakes, were a joy down the narrow and steep decline of Lone Kauri Rd to Karekare.

Descent to Karekare

Descent to Karekare

I arrived at Karekare after about an hour and 45 minutes. I stayed there long enough to water the trees and brew a quick cup of tea, then hit the road again for the steepest incline. Karekare Rd is a tough one, a really steep and narrow road that only just lets two cars get past each other. The bike (and I) had to work hard here, never going above more than about 10kmh, but that’s about 5kmh more than I would manage on a road bike (not that I have ever tried). And there were no cars. None.

Piha

Piha

Piha cafe.jpg

From the top of Karekare Rd is the long and glorious descent to Piha, the farthest point of the journey at about the 48km mark. There I had a coffee at the Piha Café, and rewarded myself with their apple and berry crumble. It’s a crumble that deserves some effort to get to (apologies, I forgot to take a photo of that piece of evidence!).

The return journey covered the inevitable steep incline and rolling hilltops along Piha Rd, and then descended to Henderson, before joining the northwestern cycleway near Lincoln Rd, where I was back again at sea level. It was a fairly uneventful trip, the Westies were well behaved except one going just too close and way too fast near the Scenic Dv intersection with Piha Rd.

Lion Rock at Piha

Lion Rock at Piha

By this point the battery was half full, and at max power I rattled along the cycleway to Daily Bread café in Pt Chev, for another coffee and their morning cardamom bun. The odometer was saying 85km and the battery still had plenty of juice. So I returned to the cycleway and rode through to the Lightpath in the city, then down to the Viaduct, and the return home.

The Whau river from the Northwestern cycleway

The Whau river from the Northwestern cycleway

e-bike ride.png

All up, it was a 101km ride, with a total ascent of 20km and reaching a highest elevation of 402m. In four hours of actual riding time (i.e. without the tea, coffee and sweet treats), the ride averaged 25kmh, much faster than I would have managed on a road bike.

e-bike map.png

The battery still had at least a 25% charge left, despite not just the distance, but also the inclines it was having to pour its stored energy into.

And me? Well, it felt like a workout, certainly not a 100km road bike workout with 20km of uphill, but a good 60km ride on a gentler landscape.

I feel science was well-served today. As was the berry crumble.



And no, I don’t have shares in Electric Bike Team. But thanks team, I love the bike.

For those who saw a postscript in the last 24 hours regarding Electric Bike Hub withdrawing their eZee range, I have since discovered I was confusing Electric Bike Hub in Nelson with Electric Bike Team in Auckland. The Electric Bike Team have contacted me to say they still love the eZee range and have no intention of dropping the range. Apologies for confusion on my part.

Virtual health – evaluating in a shifting landscape

Earlier this year, Rachael Butler and I completed a series of rapid reviews for Waikato District Health Board, looking at current thinking in virtual health – where advances in technology enable remote forms of healthcare delivery. Some anticipate that it will comprise a substantial proportion, if not the majority, of patient interactions in the future.

What is virtual health?

Discussion about telehealth and telemedicine approaches are often used interchangeably with virtual health. In the work we developed with our partners, virtual health was proposed as an evolutionary step beyond telehealth and digital health approaches, which is patient-centred and engages people as partners with clinicians in maintaining and improving their health.

Reported benefits of virtual health include improved access to healthcare and health and wellness education, increased patient engagement and satisfaction, a reduction in direct and indirect costs, and improved health outcomes. Virtual health programmes have also been linked to positive outcomes for health providers, including improved operational efficiencies, a reduction in costs, alleviation of staff/provider shortages, and access to a pool of clinical resources.

There’s always a 'but'

We found that virtual health initiatives have the potential to deliver patient-centric care, create more collaborative ways of working and make health systems more efficient. But when you’re talking about potential, there’s always a but. And with virtual health, clinical uptake is often a key challenge, along with the readiness of organisations to implement technology, and the suitability of technology for the purposes intended. Furthermore, users of virtual health systems – both clinicians and consumer/patients – should be involved in developing new initiatives, be adequately trained, and feel confident that the technology is safe, offers clear benefits, and is easy to use.

A copy of the synthesised reviews can be accessed here, where the above discussion is dealt with in substantially more detail.

Evaluating virtual health

When we presented on these findings at the 2018 ANZEA Conference, I offered some reflections about evaluating these types of approaches, drawing on the writing of Trisha Greenhalgh, one of the leading thinkers in this space. Trisha Greenhalgh’s 2018 book, How to implement evidence-based healthcare, is an excellent starting point.

plans.png

The landscape of virtual health technologies is one that is constantly being iterated and refined. Maintaining a stable, unchanging intervention is really challenging in the context of apps that are being updated on our phones on a near-daily basis. So evaluations that are based on a linear assumption of stability and predictability may well be flawed from the outset. Evaluating technological interventions need to be seen in the light of something that is iterative, recursive and a long-term process – a developmental process.

Source: http://thedoghousediaries.com/5468

The six worlds

Greenhalgh writes of six ‘worlds’ with significantly different stakeholder perspectives and expectations of technology interventions; each with their own evaluation needs:

  • Political, where technology is a vehicle for delivering policy, improving efficiency and providing measurable benefits to patients
  • Clinical, where technology is a tool to support professional practice and improve quality of care
  • Personal (patient/carer), where technology supports the individual’s personal health and wellbeing
  • Technical, where the focus is on design and as a software development project
  • Commercial, as a way of delivering return on investment
  • Legal/regulatory, in terms of information governance.

Getting comfortable with the complex

Greenhalgh and colleagues are particularly critical of systematic reviews of randomised control trials that overly simplify both the technological interventions and the outcomes they expect, and which often fall back on needing more research to gain a greater effect size. Instead, she argues for greater use of mixed methods and ethnographic research that embrace both context and complexity, to better understand the pathways and processes through which technological interventions fail or succeed, and use real-world experience to make sense of multiple and imperfect data sources.

In short, virtual health approaches are here to stay, and they will continually be refined and adapted to better meet the needs of people – many of which will fail to do so. Understanding technological value and impact needs to embrace the complexity of the different worlds of technology, and work similarly developmentally and iteratively as their real-life experience unfolds.

 

The review that this post is based on was commissioned by Waikato District Health Board over late 2017 and early 2018. The Smarthealth and Healthtap initiatives were outside the scope of our review.

 

Dovetail in 2017

Another year goes by and another opportunity for reflection. Thanks again to all who have been part of Dovetail's success - as clients, collaborators, and conversations.

For Dovetail, it was another productive and satisfying year. One of the biggest highlights was being part of the Te Ara Mua Future Streets team which scooped up the Innovation Hub and Supreme Awards at the Bike to the Future Awards.

I was also very pleased to be part of publishing a few journal articles, my first for a while, on evaluation, bikelash and urban form.

Clients kept things busy as always, which is great because the variety of work is constantly engaging, and it gives an opportunity to help people working at the coalface. Some new research and evaluation directions emerged through this, including social procurement, financial capability, social enterprise support, and just at the tail end of the year, a new foray into ocean conservation.

The election blogs provided time for me to geek out to my heart's content, and to keep things on an even keel, I set myself an audacious goal to get to 52 gigs in 52 weeks.

And for me personally, 2017 had some pretty tough moments. I'm incredibly grateful to the friends, family, colleagues and clients who were there to get us through. I can't thank you enough.

Dovetail Annual Report 2017.jpg

For a clearer and printable version, you can also download a pdf here.