Wondering where your JavaScript libs went? Spam-detection snafu exiled npm packages

On the defensive after a malware kerfuffle last year, code registry npm shot first before asking questions over the weekend – and is now apologizing for the errant execution.

The keeper of hundreds of thousands of Node.js packages and other JavaScript libraries wrongly removed the account "floatdrop" belonging to developer Vsevolod Strukchinsky. This prevented anyone from finding the 102 software packages he had uploaded to the registry.

Making the code disappear set off alarm bells among developers: the code's vanishing act blocked programmers from deploying or updating their apps and their dependencies that relied on Strukchinsky's software, and sparked fears that people had been duped into installing malicious libraries at some point, malicious libraries that had now been deleted. In actual fact, the libraries were legit and had been wrongly flagged as spam.

npm on Sunday attributed the error to a breakdown in its system for catching malicious code. The software company said it relies on automated systems that perform static analysis to flag dubious content and authors, and identified packages and accounts are then reviewed by npm personnel who have to decide whether or not to pull the plug.

In a blog post over the weekend, npm said it caught the error within five minutes and set about reversing the block, but its repairs became complicated when members of the npm community, believing the removal to be malware-related, took it upon themselves to publish replacements for the removed packages.

How one developer just broke Node, Babel and thousands of projects in 11 lines of JavaScript

READ MORE

Today, npm followed up with a more detailed explanation of the mistake, a spam filtering false-positive: a spammer published a package that included the README file from floatdrop's legitimate timed-out package.

"Because of the matching READMEs, our spam system flagged floatdrop as associated with the spammer," npm's chief technology officer C J Silverio explained. "In the course of reviewing and acting on spam reports, an npm staffer acted on this flag without further investigating the user and removed the user and all of their packages from the registry."

The reason spammers bother to do this, Silverio said, is that npm is widely used, and popular packages rank highly in search engines. So spammers often copy code and text from npm packages to get their spam messages into search results.

Team npm said it is taking various steps to avoid similar snafus, including delaying the republication of a deleted package for 24 hours, new off-hours response guidelines, improved incident communication policies, and better tooling for employees to vet low-confidence spam flagging and to restore mistaken deletions.

The package registry is also working with its spam detection partner Smyte to ensure its detection process doesn't punish those who have their work hijacked by spammers.

"Our systems and processes balance the need to eliminate spam with the need to reduce false positives," Silverio said. "However, we failed to address the need to recover swiftly and cleanly from human error." ®