What does ‘system update required’ say about Labour’s IT?

As part of the ‘drains up’ undertaken after the 2019 General Election, a coalition calling itself Labour Together undertook a review of what went wrong and as part of that review commissioned an organisation called the “common knowledge co-op” to look at Labour’s IT and its management. They produced a report called “System update required”. (original | mirror ) What did it say? I think this is important, but like so many learning opportunities that challenge power and the bad behaviour of the powerful it seems to me to be dramatically under-valued.

When I first read it, I was outraged. I hoped to summarise it in a sensationalist fashion to see if I could interest someone who might pick it and make things better. What I have written is not that exciting and I suspect little will change because the Party doesn’t have the knowledge and experience and today is led by people who care more about their control and position within the Party than they do in winning an election and becoming a government. I mean they’d be happy to be in Government but it’s more important to them that they control the Party.

In summary, the report says, portfolio management was unacceptably poor and not accountable to the highest levels of management although they too didn’t have clue. There weren’t enough IT staff and the more numerous IT management layer wasn’t good enough. The report makes no mention of ‘requirements management’, nor of any benefits analysis tools to allow an understanding the effectiveness of the software applications provided. Labour’s voter ID/GOTV software is no longer the best. Local adoption of the IT tools is low, partly because of poor commitment to training, partly due to a high turnover of local activists and partly because the Labour machine didn’t care.

In more detail,

1. Portfolio management of the applications was unacceptably poor i.e. there was no strategic ownership of the portfolio or its elements and the owning divisions and their heads did what they wanted including damaging the capability of other parts of the organisation. Much of this is down to dreadful talent management and probably factionalism. Factionalism will have impacted both staff selection, IT priority management and funding programmes. The report makes no comment on the effectiveness of  the “requirements management” process nor how the role of “senior user” in the project teams was undertaken.
2. There weren’t enough software engineering staff. The management were unable to scale the organisation appropriately as the 2019 election began; they had failed to do so for the Euro-elections. The contractor culture and poor management experience has led to poor documentation. The lack of documentation also inhibits user adoption and training and staff take-on. No effort was made to leverage Labour’s talented membership in systems development.
3. Labour no longer^[1] has leadership in the doorstep/campaigning apps and these, it would seem, are no longer considered competitive advantage; many progressive parties use either open source or third party programs to do this work. (I argue elsewhere and frequently, including below, that one should only build software where the organisation’s functionality is unique and gives competitive advantage).
4. Local i.e. CLP digital adoption is weakest where it’s needed. Data quality i.e. the contact rate is correlated with party membership and strength. Famously, when David Miliband stood down from South Shields the contact rate was alleged to be under 5%; the Party has been much more careful about releasing the information since then. Many of the IT tools are not used by local parties. Some/many regional staff were not appointed for their IT (or even campaigning) skills.

That’s the end of the summary, you can skip more of my précis by going to the Gaps & Lessons section of this post.

Tools and portfolio management

Labour’s tools are designed as point solutions to single problems and the ‘ownership’ of these systems is/was allocated across at least three divisional heads. There is no single IT strategy and the Heads of Department often refused to co-operate with each other. An attempt was made to the extent that a consultant produced a report to bring these competing managements into alignment, but this failed due to senior management fear of the power of fiefdoms. This envisaged a strong central IT function. This was compounded by an inappropriate appointment to a Director of IT^[2] who might have been expected to make this work but didn’t. While applications were owned divisionally, staff were not. There are/were too many managers with no clear chain of command on a decision.

The managerial contention led to frequent circumstances where constituencies and regional staff were denied access to critical tools for campaigning or permissions delayed and deadlines missed in some cases by holidays taken by key staff members. The report documents unprofessional interpersonal rivalries as the motivation for some of these delays.

At best the Labour Party’s compliance with the GDPR is often an excuse/reason for deny people access to the tools. This has led to an ineffective design and use of the tools.

IT staffing cadre and sourcing

There have never been enough development staff, in the summer of 2019, between the Euro-elections and the run-up to the general election, there was one software engineer. i.e. the Labour party had one developer responsible for 7 tools accountable to three divisions. The report recommends that a cadre of 7 would be more appropriate. The staff shortage and the use of contractors has led to inadequate documentation, which inhibits maintenance; both bug fixes and new functionality are too slow to arrive. The lack of documentation also inhibits user adoption and training.

The take-on of temporary staff for the 2019 election was too slow and inhibited by the previous hiring freeze. Tg=he Director of IT also left the organisation during the elction campaign; this can’t have helped scale the organisation to what was needed. This financial policy i.e. the hiring freeze and the delayed staff take-on almost certainly affected the result.

The 2019 campaign failed to take account of and use the vast pool of talented volunteers, unlike in 2017. This is exacerbated or due to the Party’s paranoia for control and secrecy. The code is, at least, held in git repos but volunteers are not given access. There may be good reasons why not, theft of code, or misuse and theft of data. The report recommends and I agree that more should be done to energise and utilise the skills and experience of Labour’s membership. They identify that code developed to leverage Labour’s unique advantage of motivating a large number of volunteers cannot be stolen.

Adoption

Digital adoption is weakest where it’s needed. The answer to this is training and an open source/train the trainer culture, and documentation that works. (I say, that an all members bugs database is needed, an effective trouble ticketing system with time to fix service levels i.e. effective incident and problem management and help desk functions. The long term inadequacy of Organise, a tool being replaced by an off the shelf package is unacceptable. (I write about the data protection risks of a broken membership system elsewhere on this blog.))

Gaps and Lessons

The party needs an IT Governance policy/strategy and needs someone to drive it and ensure it’s useful and meets the Party’s needs. It needs sponsorship from the NEC. I would adopt COBIT as it avoids inventing one, people that understand it are easy to find and I have seen it work exceptionally effectively. One of its central tools that will help Labour and any other organisation is the mantra, Plan, Do, Check, Adjust, where plans are written down. All projects and operational processes need to be approved and checked against the planned benefits. This doesn’t happen in the Labour Party.

Making the point that they had a divided management structure and more managers than coders is cute, accurate but easy. What is the ideal ratio between developers and their managers, operations staff (and their managers) and policy people including Governance, Compliance and Regulation and procurement/finance people? This will vary according to a number of factors, maybe those I explored in Software Programme Management on LinkedIn. i.e. it will depend on the importance of the system, its planned volatility, the maturity of the code and base technology but I am certain that having zero GCR people can’t work. (see also this article on Linkedin by me).

Reading the report makes me ask if the tools i.e applications Labour has, are the tools we need. What no-one questions is whether the tools are the right tools? I had not heard of Turnout, and while I had heard of Impact, no-one in our CLP is interested in using it. We do what we’ve always done. No-one is asking how effective is doorstep work? Let’s remember that Leave won that campaign without one!

Each project needs a business case that states the expected business benefit and each project needs to be evaluated to ensure that it still delivers as expected. (If possible they should be tested if only on paper, against the second best alternative.)

The development/acquisition strategies must take into account the skills market available, not just in the short term, but also longer term. The use of highly adopted technology will allow an ease of recruitment of staff and volunteers. The most important application of this principle/guidline maybe in applying it to a single UI borrowing or implementing common implementations or metaphors. It maybe that android is the most common user interface in use amongst Labour’s members and supporters; whatever is the case, Labour should not be investing in UI design. Despite this, I believe that stopping the use of Nationbuilder^[3] may have been a mistake, it had funnel management capability and potentially allow the encouragement of supporters or even voters to become more engaged; it was designed to act as a volunteer management engagement package. It does not meet my, “buy what people use” slogan.

Labour, well everyone really should only build for competitive advantage, no-one should adopt and change^[4]. For instance we no longer^[5] have leadership in the doorstep/campaigning apps and these are no longer considered competitive advantage; many progressive parties use open source or third party programs to do this work.

Labour needs to develop as if open source so that user authored applications can be easily incorporated.

Recommendations

I now try to briefly explain their recommendations although their words are admirably brief. They recommend slimming down the number of managers^[6] and having a single plan^[7] which is shared and understood and committed to.

1. They recommend hiring more staff, most importantly software engineers who can make the code better. They have some weak words on having a strategy and say nothing on requirements management.
2. They state that a Digital Roadmap is required which is generally available.
3. There should be a collaboration capability to allow collaboration by the broader membership, for bug fixes and even user authored apps.
4. They recommend that Labour de-commit from Experian and build an alternative system based on local knowledge for voter id and social media targeting.
5. Ensure everyone that needs it is trained that documentation is available and trouble ticketing works and is transparent.
6. That appropriate access permissions are granted; this would involve a roles and responsibilities register.
7. A common casework tool for all our public officials might be sensible so the Party can track issues although there are data protection issues here. They propose that contact creator becomes real time to avoid over contacting supporters.
8. They recommend broader GDPR training; in my view a complete rewrite, the policy is designed for control and neither compliance nor operational effectiveness are considered.
9. An availability tool should be made available so all potential users know what to expect.

---

[1] There’s a couple of citations in this para on P16.

[2] See page 10/11 of the report.

[3] This product charges on the basis of the database size, I wonder how much this was the motivation for leaving it rather than a skills shortage,  reliability and a weakness in meeting its business case.

[4] https://www.linkedin.com/pulse/software-programme-management-david-levy/

[5] There’s a couple of citations in this para on P16.

[6] The redundancy programme did not achieve this goal.

[7]This was clearly not the case in 2017.