Reliable system was so reliable, no one noticed its licence had expired... until it was too late

On Call It's the end of the working week - for most of us. Time to kick back, brew a morning cuppa and delve into this week's On Call, our weekly readers' column of tech traumas.

This time, we meet "Ted", who tells us about a time he worked for an outsourcer in a government department that got into a pickle over licensing for a very old piece of kit.

"One of its core processes was a long serving scan and optical character recognition [OCR] program using a CDE interface running on propriety Unix servers fed by Windows work stations and other hardware," he said.

The hardware, firmware and OS were all unpatched and out of date – "to avoid the risks of changes breaking something", Ted revealed.

For years, the software had been scheduled for decommissioning – it was "always only six months away".

Regardless, everything ran as it should. "Like most departmentally written software it did all that was required, mostly light on old resources and ran reliably," he said. "Too reliably."

However, the day came when the data centre critical central server had to be briefly powered down for infrastructure work.

"Shutdown was fine," Ted said. "On power-up, the server came up smoothly."

The level 3 init scripts output flitted past and our man caught a glimpse of one line about licensing go whizzing by… "Fortunately it had a full file path."

But then the application user calls started – the OCR was non-functional.

Ted told us that while the main software was maintained by contractors who were still around, the OCR engine had gone through a couple of firms and no longer had a continental presence, let alone a local one.

"The support team confirmed the application itself was up but the core OCR part was down," Ted said. A manual software restart confirmed the worst: the licence had expired years ago.

The problem was, since the Unix servers hadn't been re-booted in years, no one had noticed because the licence was only checked on start-up.

But, to the client's credit, they had hung on to their licence entitlement  documents, which, Ted said, showed they were supposed to have a "permanent non-expiring licence".

And so the team called the parent company in the US, to be told that the Unix version of the OCR software was years out of support.

"The relevant compile server still existed but also had not been powered on for years," he said.

However, the customer was apparently quite persuasive, and was able to convince the OCR supplier to restart the server and generate a proper permanent key.

"Much to everyone's relief, the key server booted and a new key emailed. Once installed all worked as required," Ted said.

The root cause was that the previous expired key was only for five years – "the software maker was assured that the software would be decommissioned soon".

The moral of the story? "Never assume 'soon' means less than lifetime of this Universe when it comes to software," said Ted, sagely.

Have you ever had a licensing drama? Tell On Call and we might use your tale for a future column. ®