A recent outage affecting 911 services for nearly 90 minutes across multiple Canadian provinces was attributed to a previously unnoticed software bug in Bell’s system, according to a report released on Friday.
The report indicated that a “latent software issue” caused the disruption in Manitoba, Ontario, Quebec, New Brunswick, Nova Scotia, and Prince Edward Island, starting at 3:13 a.m. ET on September 17.
Philippe Gauvin, Bell’s assistant general counsel, clarified in the report to the Canadian Radio-television and Telecommunications Commission (CRTC) that the incident was not the result of a cyber attack or any planned or unplanned changes.
The software flaw impacted components of Bell’s next-generation 911 system, referred to as NG9-1-1. This system is intended to enable the public to provide dispatchers with more comprehensive details about emergencies, enhancing the understanding of incidents for police, fire services, and paramedics prior to their arrival.
Bell reported that it pinpointed the root cause and restored services by 4:40 a.m., after a duration of 87 minutes. The company also implemented “a permanent solution to the issue” later that same day.
On the day of the incident, Bell confirmed that the disruption impacted service in six provinces.
According to its report to the CRTC, the “vast majority” of calls made during the outage were effectively managed by backup systems; however, “legacy” voice calls routed through older systems were not received if they were directed to a call centre already utilizing the next-generation technology.
The report noted that none of the missed calls resulted in a “critical public safety incident.”
In a statement to CBC News, Bell refrained from disclosing the number of missed calls.
“Bell takes its responsibilities as a 9-1-1 provider seriously and recognizes the seriousness of any interruption in that service. We have implemented measures to ensure that the issue encountered on September 17 does not happen again,” the email stated, highlighting that this outage was the first of its kind since the system’s inception in early 2022.
“We are dedicated to the reliability, resilience, and ongoing enhancement of emergency calling services to ensure Canadians can seek help when needed.”
Initial alarms failed to activate, report reveals
The report from Friday also disclosed that two alarms during the initial phase of the outage, one at 3:13 and another at 3:19 a.m., did not prompt a “war room” crisis response as intended. Engineers and other specialists were not alerted to the problem until a separate alarm system detected an unusual decline in incoming calls 24 minutes later.
The response team assembled at 3:48 a.m. and resolved the issue by 4:40 a.m., according to the report.
Bell stated that the company is continuing to investigate the reasons behind the malfunction of the first two alarms and has since implemented “independent monitoring mechanisms” as an additional safety measure. The firm is also conducting a comprehensive review of the system, debriefing call centre personnel, and sharing insights with other providers.
Bell Canada’s logo is seen in Montreal in 2016. (Paul Chiasson/The Canadian Press)
In the publicly released report, Bell redacted specific details regarding the flaw that led to the outage and other aspects of its broader system, citing concerns that disclosing such information could enable hackers to uncover and exploit “vulnerabilities in the 911 infrastructure.”
“Revealing such information could pose direct risks to Bell Canada and the 9-1-1 network that serves the public, thereby impacting public interest,” Gauvin explained.
Across Canada, the 911 service is linked to regional legacy telephone providers.
Bell delivers this service in Manitoba, Ontario, and Quebec extending to the Atlantic provinces. Telus serves British Columbia and Alberta, while Saskatchewan is supported by SaskTel.
-
84 people couldn’t reach 911 during separate August outage that affected most of Manitoba, Bell says
Holly Barkwell, Canadian region director for the National Emergency Number Association (NENA), remarked last month that Canada’s 911 system is “extremely resilient,” but acknowledged that no network is flawless.
“Technological issues are always present,” Barkwell told CBC News at that time.
The CRTC has mandated that all legacy 911 systems in Canada must be decommissioned by March 31, a deadline that the agency extended by two years in early 2025.