Sounds like a massive, risky UK.gov scheme, but let's not keep too many tabs on it, OK?

The UK's spending watchdog has said it isn't possible to tell whether the biggest and most risky government projects are doing what they're supposed to because of poor records and incomplete reporting.

In 2011, Whitehall established a Government Major Project Portfolio (GMPP) to improve the way it monitored large, costly and important programmes, and to increase transparency on the performance of its most costly work.

As of September 2017, 29 of the 133 projects were classed as ICT, but there is also a tech element to a number of the 41 transformation projects.

These projects are budgeted to cost £423bn and are forecast to deliver £657bn in benefits, but the National Audit Office (NAO) noted in a report published today that "there is no corresponding figure for what benefits have been realised".

The projects in this portfolio change regularly as new programmes enter and others leave; this can be because they are completed, or because they are merged into another project or have been cancelled (which the NAO pointed out isn't always a bad thing – some projects just need putting down).

A total of 302 projects left between April 2011 and September 2017, but the NAO said the Infrastructure and Projects Authority (IPA) – which oversees the GMPP – lacked complete information on why these projects had left and what they had delivered by that point.

The watchdog also found that 44 of these projects also left at a point when the IPA said their delivery was either unachievable or in doubt – although since an update to the rules in November 2016, only one project has left with an amber-red rating, and none with a red rating have left.

The NAO also noted that some 176 projects had left without an exit review, although this again became less likely in more recent years.

However, there also wasn't information about what stage the projects were in when they left in 220 cases, which the NAO said makes it "difficult to determine whether projects left at the right time and for the right reasons".

It added that "poor records and incomplete reporting also reduce transparency, increasing the risk and perception that projects are removed inappropriately".

Of the 82 projects for which there was information on the stage of departure, less than half (39) left at the benefits realisation or operational phase, as would be expected.

ICT fared slightly better than the norm here, with 15 leaving in the operational phase and one in the benefit realisation stage. Three each left in initiation and planning or delivery phases, and three more left for other reasons. (Far be it from El Reg to point out that the 2016 crack down on red or amber-red departures would preclude a number from leaving.)

In comparison, 15 government transformation and service delivery projects left the portfolio during the delivery phase.

The NAO carried out a deeper dive into 48 of the projects, to see whether they had met their targets, but found that in almost half of cases (22), it was unable to determine whether they had achieved their intended outcomes.

In some cases this was because it was simply to early to tell, but in others it wasn't clear what the intended benefits were – seven projects didn't have a business case, for instance.

Another issue was that some projects are not measured against their original objectives, but on refreshed targets, which is "potentially providing an inaccurate picture of their success".

It cited the Mobile Infrastructure Project as an example. This initially committed to build 575 mobile telephone masts to expand coverage to 60,000 premises.

This was later branded unachievable and a revised target of 40 masts was set; when the project left the portfolio it was assessed against this and deemed a success as it had built 75 masts, which reached 7,199 premises.

The NAO also noted that a number of projects left the portfolio not because they had delivered on a benefit, but because a contract had been let. At this point, delivery is handed over to a third party – and the NAO questioned what happened to accountability if a project leaves in this way.

It also called for a requirement that all projects have a business case that is kept up to date to monitor changes in a project's scope, and that statements are published whenever a project exits the portfolio.

"The Infrastructure and Projects Authority is clearly contributing much-needed project management and evaluation techniques to the mammoth programme of major projects run by government," said NAO head Amyas Morse.

"We believe it could drive greater improvement if it adopted a clearer method of measuring the benefits of these projects, and tougher discipline over the terms on which projects are included, or more to the point, excluded from its oversight." ®