Software Bug Behind Biggest Telephony Outage in US History

[Faybert]

A software bug in a telecom provider's phone number blacklisting system caused the largest telephony outage in US history, according to a report released by the US Federal Communications Commission (FCC) at the start of the month.

The telco is Level 3, now part of CenturyLink, and the outage took place on October 4, 2016.

Software sees empty field as wildcard selector
According to the FCC's investigation, the outage began after a Level 3 employee entered phone numbers suspected of malicious activity in the company's network management software.

The employee wanted to block incoming phone calls from these numbers and had entered each number in fields provided by the software's GUI.
.....
.....

Level 3 telephony network goes down for 84 minutes
The event had massive repercussions, affecting the entire US. For 84 minutes between 10:06 and 11:30 AM Eastern Daylight Time (EDT), Level 3's network blocked all calls, a massive number of 111 million calls, 109 million of which where VoIP-based.

Approximately 29.4 million VoIP users and around 2.3 million wireless users were affected.

FCC said the event had "nationwide impact" and called it "the largest [outage] reported in the Federal Communications Commission’s Network Outage Reporting System (NORS)" history.
....
....