On a recent trip to New Zealand I spent three action packed days working with Keitha Booth and Alison Stringer looking at open data. These two have an incredible amount of knowledge and experience to share, and it was an absolute pleasure to work with them, albeit briefly. They arranged meetings with about 3000* individuals from across different parts of the NZ government to talk about everything from open data, ICT policy, the role of government in a digital era, iterative policy, public engagement and the components that make up a feasible strategy for all of the above.

It’s important to note, I did this trip in a personal capacity only, and was sure to be clear I was not representing the Australian government in any official sense. I saw it as a bit of a public servant cultural exchange, which I think is probably a good idea even between agencies let alone governments 😉

I got to hear about some of the key NZ Government data projects, including data.govt.nz, data.linz.govt.nz, the statistical data service, some additional geospatial and linked data work, some NZ government planning and efforts around innovation and finding more efficient ways to do tech, and much more. I also found myself in various conversations with extremely clever people about science and government communications, public engagement, rockets, circus and more.

It was awesome, inspiring, informative and exhausting. But this blog post aims to capture the key ideas from the visit. I’d love your feedback on the ideas/frameworks below, and I’ll extrapolate on some of these ideas in followup posts.

I’m also looking forward to working more collaboratively with my colleagues in New Zealand, as well as from across all three spheres of government in Australia. I’d like to set up a way for government people in the open data and open government space across Australia/New Zealand to freely share information and technologies (in code), identify opportunities to collaborate, share their policies and planning for feedback and ideas, and generally work together for more awesome outcomes all round. Any suggestions for how best to do this? 🙂 GovDex? A new thing? Will continue public discussions on the Gov 2.0 mailing list, but I think it’ll be also useful to connect govvies privately whilst encouraging individuals and agencies to promote their work publicly.

This blog post is a collaboration with the wonderful Alison Stringer, in a personal capacity only. Enjoy!

* 3000 may be a wee stretch 🙂

Table of Contents

Open Data

• Strategic/Policy Building Blocks
• Technical Building Blocks
• References

Digital and Open Government

• Some imperatives for changing how we do government
• Policy/strategic components

Open data

Strategic/policy building blocks

Below are some basic building blocks we have found to be needed for an open data strategy to be sustainable and effective in gaining value for both the government and the broader community including industry, academia and civil society. It is based on the experiences in NZ, Aus and discussions with open data colleagues around the world. Would love your feedback, and I’ll expand this out to a broader post in the coming weeks.

• Policy– open as the default, specifically encouraging and supporting a proactive and automated disclosure of government information in an appropriate, secure and sustainable way. Ideally, each policy should be managed as an iterative and live document that responds to changing trends, opportunities and challenges:
  • Copyright and licensing – providing clear guidance that government information can be legally used. Using simple, permissive and known/trusted licences is important to avoid confusion.
  • Procurement – procurement policy creates a useful and efficient lever to establish proactive “business as usual” disclosure of information assets, by requiring new systems to support such functionality and publishing in open data formats from the start. This also means the security and privacy of data can be built into the system.
  • Proactive publishing – a policy of proactive disclosure helps avoid the inefficiencies of retrospective data publishing. It is important to also review existing assets and require an implementation plan from all parts of government on how they will open up their information assets, and then measure, monitor and report on the progress.
• Legislation – ensuring any legislative blockers to publishing data are sorted, for instance, in some jurisdictions civil servants are personally liable if someone takes umbrage to the publication of something. Indeed there may be some issues here that are perceptions as opposed to reality. A review of any relevant legislation and plan to fix any blockers to publishing information assets is recommended.
• Leadership/permission – this is vital, especially in early days whilst open data is still being integrated as business as usual. It should be as senior as possible.
• Resourcing – it is very hard to find new money in governments in the current fiscal environment. However, we do have people. Resourcing the technical aspects of an open data project would only need a couple of people and a little infrastructure that can both host and point to data and data services. The UK open data platform runs on less than £460K per year, including the costs of three staff). But there needs to be a policy of distributed publishing. In the UK there are ~760 registered publishers of data throughout government. It would be useful to have at least one data publisher (probably to work part of their job only and alongside the current senior agency data champion role) who spends a day or two a week just seeking out and publishing data for their department, and identifying opportunities to automate data publishing with the data.govt.nz team.
• Value realisation – including:
  • Improved policy development across government through better and early access to data and tools to use data
  • Knowledge transfer across government, especially given so many senior public servants are retiring in the coming years
  • Improved communication of complex issues to the public, better public engagement and exploration of data – especially with data visualisation tools
  • Monitoring, reporting, measuring clear outcomes (productivity savings, commercialisation, new business or products/projects, innovation in government, improved efficiency in Freedom of Information responses, efficiencies in not replicating data or reports, effectiveness and metrics around projects, programs and portfolios)
  • Application of data in developing citizen centric services and information
  • Supporting and facilitating commercialisation opportunities
• Agency collaboration – the importance of agency collaboration can not be overstated. Especially on sharing/using/reusing data, on sharing knowledge and skills, on public engagement and communications. Also on working together where projects or policy areas might be mutually beneficial and on public engagement such that there is a consistent and effective dialogue with citizens. This shouldn’t be a bottlenecked approach, but rather a distributed network of individuals at different levels and in different functions.
• Technology – need to have the right bits in place, or the best policy/vision won’t go anywhere 🙂 See below for an extrapolation on the technical building blocks.
• Public engagement – a public communications and engagement strategy is vital to build and support a community of interest and innovation around government data.

Technical building blocks

Below are some potential technical building blocks for supporting a whole of government(s) approach to information management, proactive publishing and collaboration. Let me know what you think I’m missing 🙂

Please note, I am not in any way suggesting this should be a functional scope for a single tool. On the contrary, I would suggest for each functional requirement the best of breed tool be found and that there be a modular approach such that you can replace components as they are upgraded or as better alternatives arise. There is no reason why a clever frontend tool couldn’t talk to a number of backend services.

• Copyright and licensing management – if an appropriately permissive copyright licence is applied to data/content at the point of creation, and stored in the metadata, it saves on the cost of administration down the track. The Australian Government default license has been determined as Creative Commons BY, so agencies and departments should use that, regardless of whether the data/content is ever publishing publicly. The New Zealand government recommends CC-BY as the default for data and information published for re-use.
• An effective data publishing platform(s) (see Craig Thomler’s useful postabout different generations of open data platforms) that supports the publishing, indexing and federation of data sources/services including:
  • Geospatial data – one of the pivotal data sets required for achieving citizen centric services, and in bringing the various other datasets together for analysis and policy development.
  • Real time data – eg, buses, weather, sensor networks
  • Statistical data – eg census and surveys, where raw access to data is only possible through an API that gives a minimum number of results so as to make individual identification difficult
  • Tabular data – such as spreadsheets or databases of records in structured format
• Identity management – for publishers at the very least.
• Linked data and metadata system(s) – particularly where such data can be automatically inferred or drawn from other systems.
• Change control – the ability to push or take updates to datasets, or multiple files in a dataset, including iterative updates from public or private sources in a verifiable way.
• Automation tools for publishing and updating datasets including where possible, from their source, proactive system-to-system publishing.
• Data analysis and visualisation tools – both to make it easier to communicate data, but also to help people (in government and the public) analyse and interact with any number of published datasets more effectively. This is far more efficient for government than each department trying to source their own data visualisation and analysis tools.
• Reporting tools – that clearly demonstrate status, progress, trends and value of open data and open government on an ongoing basis. Ideally this would also feed into a governance process to iteratively improve the relevant policies on an ongoing basis.

Some open data references

• There are a lot of good OECD reports on open data that include metrics around value realisation
• I linked to some key Australian government policies and principles in this space in my recent NetHui speech
• The Office of the Australian Information Commissioner does incredible work in this space
• Several NZers had a lot of good things to say about the Australian geospatial project SISStack
• It is worth looking at the technology, human resourcing and policies that support http://data.gov.uk/ and especially, the UK Government Digital Service office. A skunkworks for innovating, improving service delivery and saving money across the UK Government.
• Opening Government Data and Information – Some really useful NZ Govt content

Digital and Open Government

Although I was primarily in New Zealand to discuss open data, I ended up entering into a number of discussions about the broader aspects of digital and open government, which is entirely appropriate and a natural evolution. I was reminded of the three pillars of open government that we often discuss in Australia which roughly translate to:

• Transparency
• Participation
• Citizen centricity

There is a good speech by my old boss, Minister Kate Lundy, which explains these in some detail.

I got into a couple of discussions which went into the concept of public engagement at length. I highly recommend those people check out the Public Sphere consultation methodology that I developed with Minister Kate Lundy which is purposefully modular so that you can adapt it to any community and how they best communicate, digitally or otherwise. It also is focused on getting evidence based, peer reviewed, contextually analysed and useful actual outcomes. It got an international award from the World eDemocracy Forum, which was great to see. Particularly check out how we applied computer forensics tools to help figure out if a consultation is being gamed by any individual or group.

When I consider digital government, I find myself standing back in the first instance to consider the general role of government in a digital society. I think this is an important starting point as our understanding is broadly out of date. New Zealand has definitions in the State Sector Act 1988, but they aren’t necessarily very relevant to 2013, let alone an open and transparent digital government.

Some imperatives for changing how we do government

Below are some of the interesting imperatives I have identified as key drivers for changing how we do government:

• Changing public expectations – public expectations have fundamentally changed, not just with technology and everyone being connected to each other via ubiquitous mobile computing, but our basic assumptions and instincts are changing, such as the innate assumption of routing around damage, where damage might be technical or social. I’ve gone into my observations in some depth in a blog post called Online Culture – Part 1: Unicorns and Doom (2011).
• Tipping point of digital engagement with government – in 2009 Australia had more citizens engaging with government  online than through any other means. This digital tipping point creates a strong business case to move to digitally delivered services, as a digital approach enables more citizens to self serve online and frees up expensive human resources for our more vulnerable, complex or disengaged members of the community.
• Fiscal constraints over a number of years have largely led to IT Departments having done more for less for years, with limited investment in doing things differently, and effectively a legacy technology millstone. New investment is needed but no one has money for it, and IT Departments have in many cases, resorted to being focused on maintenance rather than project work (an upgrade of a system that maintains the status quo is still maintenance in my books). Systems have reached a difficult point where the fat has been trimmed and trimmed, but the demands have grown. In order to scale government services to growing needs in a way that enables more citizens to self service, new approaches are necessary, and the capability to aggregate services and information (through open APIs and open data) as well as user-centric design underpins this capability.
• Disconnect between business and IT – there has been for some time a growing problem of business units disengaging with IT. As cheap cloud services have started to appear, many parts of government (esp Comms and HR) have more recently started to just avoid IT altogether and do their own thing. On one hand this enables some more innovative approaches, but it also leads directly to a problem in whole of government consistency, reliability, standards and generally a distribution of services which is the exact opposite of a citizen centric approach. It’s important that we figure out how to get IT re-engaged in the business, policy and strategic development of government such that these approaches are more informed and implementable, and such that governments use, develop, fund and prioritise technology in alignment with a broader vision.
• Highly connected and mobile community and workforce – the opportunities (and risks) are immense, and it is important that governments take an informed and sustainable approach to this space. For instance, in developing public facing mobile services, a mobile optimised web services approach is more inclusive, cost efficient and sustainable than native applications development, but by making secure system APIs and open data available, the government can also facilitate public and private competition and innovation in services delivery.
• New opportunities for high speed Internet are obviously a big deal in Australia and New Zealand at the moment with the new infrastructure being rolled out (FTTP in both countries), and setting up to better support and engaging with citizens digitally now, before mainstream adoption, is rather important and urgent.
• Impact of politics and media on policy – the public service is generally to have an evidence-based approach to policy, and where this approach is developed in a transparent and iterative way, in collaboration with the broader society, it means government can engage directly with citizens rather than through the prism of politics or the media, each which have their own motivations and imperatives.
• Prioritisation of ICT spending – it is difficult to ensure the government investment and prioritisation of ICT projects aligns with the strategic goals of the organisation and government, especially where the goals are not clearly articulated.
• Communications and value realisation – with anyone able to publish pretty much anything, it is incumbent on governments to be a part of the public narrative as custodians of a lot of information and research. By doing this in a transparent and apolitical way, the public service can be a value and trusted source.
• The expensive overhead of replication of effort across governments – consolidating where possible is vital to improve efficiencies, but also to put in place the mechanisms to support whole of government approaches.
• Skills – a high technical literacy directly supports the capacity to innovate across government and across the society in every sector. As such this should be prioritised in our education systems, way above and well beyond “office productivity” tools.

Policy/strategic components

• Strategic approach to information policy – many people looking at information policy tend to look deeply at one or a small number of areas, but it is only in looking at all of the information created by government, and how we can share, link, re-use, and analyse that we will gain the significant policy, service delivery and social/economic benefits and opportunities. When one considers geospatial, tabular, real time and statistical (census and survey) data, and then the application of metadata and linked data, it gets rather complicated. But we need to be able to interface effectively with these different data types.
• Facilitating public and private innovation – taking a “government as a platform” approach, including open data and open APIs, such that industry and civil society can innovate on top of government systems and information assets, creating new value and services to the community.
• Sector and R&D investment – it is vital that government ensured that the investment in digital industries, internal innovation and indeed R&D more broadly, aligns with the strategic vision. This means understanding how to measure and monitor digital innovation more effectively and not through the lens of traditional approaches that may not be relevant, such as the number of patents and other IP metrics. The New Zealand and Australian business and research community need to make the most of their governments’ leadership in Open Government. The Open Government Partnership network might provide a way to build upon and export this expertise.
• Exports – by creating local capacity in the arena of improved and citizen-centric services delivery, Australia and New Zealand set themselves up nicely for exporting services and products to Asia Pacific, particularly given the rapid uptake of countries in the region to join the Open Government Partnership which requires signatories to develop plans around topics such as open data, citizen centricity and parliamentary transparency, all of which we are quite skilled in.
• Distributed skunkworks for government – developing the communities/spaces/tools across government to encourage and leverage the skills and enthusiasm of clever geeks both internally (internal hackdays, communities of practice) and externally (eg – GovHack). No one can afford new resources, but allocating a small amount of time from the existing workforce who are motivated to do great things is a cost efficient and empowering way to create a distributed skunkworks. And as people speak to each other about common problems and common solutions we should see less duplication of these solutions and improved efficiency across agencies.
• Iterative policy – rethinking how policy is developed, implemented, measured and governed to take a more iterative and agile approach that a) leverages the skills and expertise of the broader community for more evidence based and peer reviewed policy outcomes and b) is capable of responding effectively and in a timely manner to new challenges and opportunities as they arise. It would also be useful to build better internal intelligence systems for an improved understanding of the status of projects, and improved strategic planning for success.
• An Information Commissioner for New Zealand – an option for a policy lead on information management to work closely with departments to have a consolidated, consistent, effective and overall strategic approach to the management, sharing and benefits realisation of government information. This would also build the profile of Open Government in New Zealand and hopefully be the permanent solution to current resourcing challenges. The Office of the Australian Information Commissioner, and similar roles at State level, include the function of Information Commissioner, Privacy Commissioner and Free of Information Commissioner, and these combined give a holistic approach to government information policy that ideally balances open information and privacy. In New Zealand it could be a role that builds on recent information policies, such as NZGOAL which is designed, amongst other things, to replace bespoke content licences. Bespoke licences create an unnecessary liability issue for departments.
• Citizen centricity – the increasing importance of consolidating government service and information delivery, putting citizens (and business) at the centre of the design. This is achieved through open mechanisms (eg, APIs) to interface with government systems and information such that they can be managed in a distributed and secure way, but aggregated in a thematic way.
• Shared infrastructure and services – the shared services being taken up by some parts of the New Zealand Government is very encouraging to see, particularly when such an approach has been very successful in the ACT and SA state governments in Australia, and with several shared infrastructure and services projects at a national level in Australia including the AGIMO network and online services, and the NECTAR examples (free cloud stack tools for researchers). Shared services create the capacity for a consistent and consolidated approach, as well as enable the foundations of citizen centric design in a practical sense.

Some additional reading and thoughts

Digital literacy and ICT skills – should be embedded into curriculum and encouraged across the board. I did a paper on this as a contribution to the National Australian Curriculum consultation in 2010 with Senator Kate Lundy which identified three areas of ICT competency: 1) Productivity skills, 2) Online engagement skills, & 3) Automation skills as key skills for all citizens. It’s also worth looking at the NSW Digital Citizenship courseware. It’s worth noting that public libraries are a low cost and effective way to deliver digital services, information and skills to the broader community and minimise the issue of the digital divide.

Media data – often when talking about open data, media is completely forgotten. Video, audio, arts, etc. The GLAM (galleries, libraries, archives and museums) are all over this and should be part of the conversation about how to manage this kind of content across whole of government.

Just a few additional links for those interested, somewhat related to some of the things I discussed this last week.

• Creating Open Government (for a Digital Society) (2012)
• Speech from Nethui on Open Government and Gov 2.0 (2012)
• My talk on Data Liberation at Ignite Sydney (2012)
• Vivek Kundra and some lessons learnt about tech in gov (2012)
• Internet, government and society: reframing #ozlog & #openinternet (2012)
• Gov 2.0: Where to begin – Part 1 of 3 (2009)