Month: November 2023

TL;DR: Better policy design and evaluation won’t save us 🙂

The APS Reform agenda provides a rare window of opportunity to address structural and systemic issues in the APS, so why not explore how might we transform the way policy is designed, delivered and managed end to end?

Why should we reform how we do policy? Simple. Because the gap between policy design and delivery has become the biggest barrier to delivering good public services and policy outcomes, and is a challenge most public servants experience daily, directly or indirectly. This gap wasn’t always the case, with policy design and delivery separated as part of the New Public Management reforms in the 90s. When you also consider the accelerating rate of change, increasing cadence of emergencies, and the massive speed and scale of new technologies, you could argue that end-to-end policy reform is our most urgent problem to solve.

Policy teams globally have been exploring new design methods like human-centred design, test-driven iteration (agile), and multi-disciplinary teams that get policy end users in the room (eg, NSW Policy Lab). There has also been an increased focus on improving policy evaluation across the world (eg, the Australian Centre for Evaluation). In both cases, I’m delighted to see innovative approaches being normalised across the policy profession, but it has become obvious that improving design and/or evaluation is still far from sufficient to drive better (or more humane) policy outcomes in an ever changing world. It is not only the current systemic inability to detect and respond to unintended consequences that emerge, but the lack of policy agility that perpetuates issues even long after they might be identified. 

Below I outline four current challenges for policy management and a couple of potential solutions, as something of a discussion starter 🙂

Current policy problems

Problem 1) The separation of (and mutual incomprehension between) policy design, delivery and the public

The lack of multi-disciplinary policy design, combined with a set-and-forget approach to policy, combined with delivery teams being left to interpret policy instructions without support, combined with a gap and interpretation inconsistency between policy modeling systems and policy delivery systems, all combined with a lack of feedback loops into improving policy over time, has led to a series of black holes throughout the process. Tweaking the process as it currently stands will not fix the black holes. We need a more holistic model for policy design, delivery and management.

CC-BY: Pia Andrews, 2023

There is also a significant gap with the public. From the start, there is usually a lack of diversity in expertise and experience in shaping a policy, and once an intervention is decided and rolled out, the people affected by policies have limited means to give feedback. Engaging the public early and often, and then providing clear feedback loops would help policies be better designed and improved over time.

Problem 2) The lack of real time monitoring of intended AND unintended impacts

The laudable efforts to improve policy evaluation are great, but formal evaluations usually have two limitations that could be better addressed with other mechanisms. Firstly, formal evaluations often tend to be positivist, in that they look for “has this initiative delivered what it said it would”, and aren’t often driven or set up to explore and understand unintended impacts, such as human or environmental patterns that emerged as a result of a new policy interacting in a complex domain.

Secondly, formal evaluations are usually a point in time assessment, rather than real time monitoring of policy impacts. Evaluation teams are not connected to the day to day delivery of policy interventions, creating a timeliness challenge in mitigating issues that are identified. Evolving and improving policy evaluation methods will create greater understanding, but perhaps too little, too late for those affected in between. Real time monitoring of intended and unintended impacts would nicely complement formal evaluation methods, while also providing a timely trigger if anything trended in the wrong direction.

Problem 3) A systemic inability to iterate policy in response to impact, feedback or change

Policies are often designed by a policy team, and then handed over to implementation, so that policy team can move on to the next policy priority, creating a systemic inability to iterate policies as the real impacts are felt in delivery. It doesn’t matter how collaborative or inclusive you are in designing a policy, there will always be perpetual change in the environment, and unintended impacts to mitigate. We need to take the lessons from the creation of “Continuous Integration and Continuous Delivery” (CI/CD) pipelines in service delivery, to create a “CI/CD Policy” approach which would manage policy design and delivery as part of the one continuum, drawing upon continuous feedback loops, monitoring and measurement of policy and human impacts to inform and iterate policies and the respective interventions. This would not only help policies to maximise the realisation of policy intent in a rapidly changing world, but would also provide the means to proactively identify and manage policy impacts (positive and negative) as they emerge.

Problem 4) Inconsistency in policy literacy and practice across the sector

Last, but not least, is the inconsistent definition, context and practice of “policy” across the sector, creating confusion and real issues of authority, decision making and accountability. Unfortunately today, many of the “policy guides” currently available limit themselves to Government Policy development, which has led to the common but dangerous assumption that Government Policies are the highest authority, and that the peak of good public service is to simply advise the Government.

To my mind, there are three highest level and fundamental categories of “policy”:

1. Foundational Policies: the constitution, legislation and regulations which provide the context, framing and highest legal authorities and accountabilities of a department;
2. Government Policies: the directions of the Government of the day via the respective Ministers, which is subject to foundational policy limitations; and
3. Operational Policies: which covers all the operational, whole-of-government, department-defined rules and delegated policies, which are subject to both the government and foundational policy directions, but are the authorititative domain of Secretaries.

The diagram below provides a useful reference on the hierarchy of authority of different policy types, as well as a guide to decision making involved in each. This should help public servants realise that different actors are needed for change to different policy types, and that even Ministerial directions are constrained by the Foundational Policies above. It also should provide public servants more understanding as to what decision making is actually within their delegated authority, such as operational policies. 

CC-BY: Pia Andrews, 2023

Potential solutions

Solution #1: Adaptive policy management

So what might adaptive policy management look like? Well, let’s start with what the characteristics for delivering great policy and human outcomes might look like, and then we can reverse engineer an ideal policy operating model we could work towards.

From	To
Narrowly informed, largely driven by generalist policy professionals, with occasional expertise or end user input.	Multidisciplinary and diverse expertise and experience informing the whole process, including early testing of several interventions with representatives of those affected.
Static policies are defined, the policy team moves on, policy change is slow and difficult, often principles-based and subject to varied interpretation in delivery.	Dynamic policies, with policy expertise present in policy interventions end to end (leg, services, reg, programs, grants, etc) with continuous, evidence-based policy iteration.
Reactive to issues, as they are identified. Constantly looking backwards, mitigating symptoms, without time to look forwards or address causes.	Responsive to change as it happens, monitoring for impact (intended and unintended) and constantly adaptive to change in a forward looking way.
Assumptions driven, policy interventions are based on past or current assumptions, without testing, exploring or co-designing a range of approaches.	Test driven, a diverse range of potential policy interventions are explored, with a range of stakeholders, with feasible options tested prior to finalising policy options or ratifications.
Culturally exclusive, policies are developed without culturally diverse experience or expertise.	Culturally inclusive, policies are developed in a culturally inclusive way, embracing diverse knowledge systems and methods.
Split policy infrastructure, where policy design and modeling happen in one place, but policy delivery happens in a different place, leading to inconsistencies in implementation assumptions, and the inability for policy owners to monitor the reality of policy implementation. Modeling is often limited in scope and domain, so policy conflicts are only identified in delivery, too late to inform design.	Shared policy infrastructure, common and shared digital policy models are used for both modeling/design and delivery, such that there is no gap between the two. Policy owners can have higher confidence in the likely impacts of change, whilst also keeping a finger on the pulse of actual policy impacts. Policy intent and impact are monitored alongside performance and CX measures, and feedback loops loop back to policy.
Policy realisation is slow, as the whole lifecycle requires policy options, legislation/regulation, operational policy development, with several opportunities for misinterpretation. Policy intent can take years to even start to be realised.	Policy realisation is fast, policies are developed in a faster way with reference implementations resulting from rapid and test driven drafting of human and machine readable policy. This results in better rules & dramatically speeds up implementation.
Community engagement, engaging the public in research or testing ideas is currently ad hoc and inconsistent.	Community empowerment, could refer to both the ability for communities to generate new policy ideas with government, but also that public sectors attempt to devolve more decision making on policy or investment to communities.

Perhaps policy making could be more of a team sport:

CC-BY: Pia Andrews, 2023

Below is a high level potential approach to the policy lifecycle, where policies are designed and delivered collaboratively, with shared policy infrastructure, and real impacts monitored,  escalated and fed into policy improvements over time, with formal evaluations able to be triggered when things go terribly wrong, not years later. Policy makers could, for instance, establish a theory of change between the vision / outcomes and the actions being taken, to ensure the indicators and measures are connected to and represented in delivery from the start. If all policies required a purpose statement, it would help implementers to ensure the delivery was aligned to the purpose and intent of the policies.

CC-BY: Pia Andrews, 2023

In this model, there is only two phases in the policy lifecycle:

• Policy purpose and authority – collaboratively developing the overarching policy purpose/intent, definition of success, and exploring options with a wide range of stakeholders, experts and those affected by the policy, including options testing, with clear definition of the measurable change(s) that should result, and the problem or opportunity the policy is trying to address. This all leads to a decision point, which varies according to the policy type above.
• Policy interventions design & delivery – this includes the end to end co-design and management of all related policy interventions, including the program(s), services, grants, rules/legislation/regulation, or operational policy development. Policy interventions are continuously monitored individually and at a portfolio level for intended and unintended impacts, constantly improved and iterated based on feedback loops, and improvements are fed where relevant back into iterating overarching policies based on evidence and expertise.

Any form of policy could follow this model. Whether Constitutional reform, legislation/regulation reform, advice/options to Government, whole of government policies or operational policies, the intended outcome can be better realised through being a little more test-driven, participatory, multidisciplinary, iterative and through managing the whole policy lifecycle as an end to end approach with real time and continuous improvements to interventions (like services, regulations, etc), while also continuously monitoring for policy impact that can feed into policy improvements.

Proposals for reforming how policy is done are often – understandably – met with concerns at “slowing things down”. But if you look at the full journey of policy today, policy intent realisation is already quite slow. If we had a more end to end and test driven approach, we’d get better policies designed that are easier and faster to implement, which would dramatically shorten the time to realise policy intent, even if it means a little more time up front. 

Solution #2: A focus and expansion of policy professionalism in the APS

We need to not only teach what all types of “good” public policy looks like, but create a culture of continuous learning and improvement for policy professionals. Perhaps we could start by complementing the excellent digital, data, HR and strategy professions coordinated by the APSC, with a “Policy Profession”? 🙂 

But we also need to teach public service craft to all public servants, including what a healthy, politically neutral and evidence-based approach to public administration looks like, and why we aren’t achieving it as a norm across the sector. For instance, we need to have clear and consistent guidance on how to engage with Ministerial offices appropriately, so that everyone can maintain the integrity, dignity and trustworthiness expected of our public institutions. We also need clear guidance on how to promote an open APS that engages appropriately and regularly with the community, something which will hopefully be addressed in the APS Reform Agenda proposed Charter of Partnerships and Engagement.

All public servants should be confident to maintain real and long term stewardship of public good, above and beyond day to day pressures or policy objectives, and also be knowledgeable of their foundational policy accountabilities, which are found in the constitution and relevant legislation and regulations. For instance, I have been surprised and somewhat horrified to hear people talk about how AI is a problem in government because it isn’t regulated, seemingly unaware that all government systems, regardless of the technology, are subject to Administrative Law, the Privacy Act, PGPA and many other foundational policies (leg/reg). We have many checks and balances we can use to ensure good governance, we just need to be aware of and apply them more consistently across the whole sector. For example, here is a paper where I documented the “special context of government” and then applied that special context to the use of AI in government. It resulted in a holistic approach that is complementary to the concept and practice of responsible government. When everyone has a shared and common understanding of the special context and responsibilities of the public service, we have a good chance to get shared and high integrity approaches to everything we design, deliver and administer in the public sector.

Solution #3: Shared and end to end “Policy Infrastructure”

Given how long this post has become, I’ll share more on this concept in a subsequent post, but here’s a teaser 😉 Basically, whilst difference teams have different tools, including distinct and separate interpretations of policy, then we’ll continue to see an interpretation gap, and a lack of end to end policy visibility, which impedes end to end policy management.

CC-BY: Pia Andrews, 2023

What do you think? 

What are the challenges you see, and what do you think needs to be done to improve policy management end to end? How might the APS Reform agenda help drive change, and how can we all do our part to improve things? How could we better deliver policy outcomes, and better public and community outcomes? How can we close the gap between policy and delivery? Would love to hear your thoughts and examples!

I want to acknowledge all the people who are contributing to this work, especially the folk involved in the Intended and Unintended Impact of Social Policy research project which is worth keeping an eye on!

Last week, I had the delightful opportunity to host some discussion tables at the 9th Annual FSTGov Government Summit in Canberra. It was an event designed just for public servants, to explore challenges and opportunities for reform and how to better serve the public. I hosted four groups in discussions about “how to build agile and adaptive public institutions”, which included 40 public servants from around 30 departments and agencies. The challenges, insights and highlights are captured below, for broader sharing and learning 🙂

What does agile and adaptive mean?

Challenge: The groups reflected that agile and adaptive still sound a bit buzzwordy, so we explored and documented roughly what they could and should mean in the context of public institutions. 

Insights: Several participants talked about the use of Agile in their orgs (usually in the IT departments) as a development methodology, which helped others to understand that context. When we discussed how to build agile and adaptive public institutions more broadly, we identified a few useful characteristics, which made it more broadly practical:

• Evidence-based and purpose-led at every step – too many people think agile just means fast and iterative, but you can’t iterative towards a destination that is undefined, and if you aren’t using evidence, testing and experimentation to validate and invalidate along the way, then you end up building a lot of unnecessary or even counterproductive things.
• Actively monitored and measured – we discussed how you can’t adapt to change if you don’t have the means and mechanisms to detect change in the first place. Active monitoring is already usually done for system performance (uptime, downtime, latency, etc) and for CX (user satisfaction, etc), but if we don’t also actively monitor for measurable policy impacts and for unintended quality of life impacts, then how can you adapt the policy interventions (which includes services) to ensure policy intent is being met in a humane way? In other words, how can you do no harm if you are unable to detect it? Active monitoring is critical to being adaptive, and to prioritising decision making about investment and efforts (including in the backlog).
• Operationally enabled for continuous change – it’s not enough to just detect change. You need to be able to respond in a timely manner. This is the heart of being both agile and adaptive. If you have a hard and unmovable plan for what you are doing, then you systemically remove the ability to naturally change or adapt according to the results of user testing, to new evidence  or to new extrinsic pressures. This means change only tends to happen under the pressures of urgency, which leads to a lot of short term and techno-centric prioritisation, rather than policy or user-centric prioritisation. Modern organisations needs to be operationally enabled for continuous and evidence based change. This includes delegating actual decision making as far down as possible, so that the people closest to impact and expertise are able to respond quickly to change, with oversight but also trust from their managers, a serious culture change for many teams. 
• Active feedback loops – Monitoring gives you quantitative data, but feedback gives you qualitative data, which can be key to identifying when things are not going well where the monitored measures might be missing something. Open and continuous mechanisms for feedback from end users AND from staff are key to keeping a finger on the pulse, to be adaptive to early indicators of problems before they snowball.
• Staff capacity: necessary to experiment, innovate and think – most public servants are working 100% on the most urgent thing, with no time to stop, think, plan or try something new. Under such conditions, it is little surprising that public institutions are generally slow to detect and respond to change and challenging for individuals to innovate. All public servants, at all levels, could choose to free up 5% or 10% capacity to experiment, innovate and even just to think and plan. For those horrified at the idea who are looking at the exponentially growing backlogs of work, I would suggest that throwing more resources at doing the same thing (a linear response) will only continue to fail at addressing the backlog, because we need exponential solutions to exponential problems. A little time to innovation, to re-engineer, to address causal issues and to work smarter, would create the conditions for more agility and adaptation at a grassroots level. The best way to scale is to support all public servants at all levels to improve their impactfulness, which can’t be done without a little capacity.
• Operational transparency – when you have easy to access visibility of your work program (what is currently being worked on, what’s done, what’s on the roadmap, etc) as well as in showing the outputs of your work (eg, sprint/code/policy reviews, showcases, blogs, research papers, etc), the you achieve two things that helps with institutional agility and adaptability. Firstly, you create an environment where anyone can offer peer review, expertise, experience and serendipitous networks of similarly motivated collaborators, providing the ability to deliver the best possible outcomes. This leads to the building of confidence in what you are doing, which speeds up delivery and adoption because we all work essentially at the speed of trust.
• Financial agility – all tables spoke about the challenges of “waterfall” budgeting and having to define every cent and deliverable years in advance of starting the work (through business cases, NPPs and the like). But even the Finance people in the discussion talked about wanting to shift to outcomes-based budgeting, and encouraging smaller investments in delivering an MVP rather than high risk big-bang launches of new systems at the end of the program plan. There is certainly opportunities to do small things within the cadence of budget planning that can help bring financial agility. For instance, choosing the use outcomes or epics as milestones in delivery roadmaps (rather than functionality or platform based milestones) which ensures you deliver something that works with flexibility on how to get there. We discussed sprints-based procurement, which I first saw at Dept Finance (APS), but none of the APS knew about it, so I dobbed in the incredible Sharyn Clarkson, from whom I learned about it 🙂

“We already have adopted agile in IT, what’s next?”

Challenge: We discussed how agile adoption in IT has helped, but not solved the big challenges facing service and policy delivery in government. When IT/dev teams adopt agile methods, they are usually still in the position of receiving “business requirements” from other parts of the department who are themselves disengaged from the process of designing or delivering the service/system. We identified the fact that many departments have maintained the issues of functionally segmentation structures, with multiple “product owners” emerging (eg, a business PO and a tech PO), which defeats the purpose and undermines the benefits of product management as a methodology. We also discussed how product management, where it has been adopted, still usually relates to managing platforms rather than services, so decision making, prioritisation and risk is analysed at a platform level, not at the service level, leading to cannablistic resourcing behaviours across “product teams” that are actually part of the same service. 

Insights: Ensuring each “product” being managed is at the service level to provide a more realistic and impactful way to priotise, manage risk, maximse intended policy impact and to actively manage the end user experience. Funding a diverse product team, with the design, dev, ops, business and policy expertise all represented (even if only part time, such as 3 hours a week for a policy person) dramatically helps to ensure the benefits, cadences and agility of a proper, agile, test driven and continuously improved product management approach. Explicitly adopting an MVP deployment strategy is also necessary for product management to work, otherwise the team is still driven to build and deploy everything all at once, which never works.

How do we balance risk and agility?

Challenge: Some of the group discussions reflected on what real risk is and isn’t, and we determined that the public sector reputation of being risk averse, has created a mythology that taking no action avoids risk, when the reality is that a culture of taking no action actually creates risk in a world that is continuously and unexpectedly changing around us.

Insights: Analysis of the risk of non-action should always be included in risk assessments, as well as risk to the public and those affected by the proposal. The SES reforms underway could include KPIs for executives to ensure that personal risk is well aligned to the portfolio and policy objectives, as well as aligned to the impact on the public, so that we avoid a situation where personal risk aversion can create risk for the public institutions and/or communities we serve. Risk needs to be assessed in the context of stewardship, looking at long term and short term implications, to ensure a balanced approach, that should also then be prioritised based on measurable policy and public impact.

These are just a few of the insights and observations from our discussion, what are your thoughts? How can you contribute to creating a more agile and adaptive organisation where you work? 🙂