Police Dispatch Phone Outage

December 16, 2024 12:23 p.m.
Outage category: 
Telephone
Location: 
All Dispatch phones (Nortel system) are not able to make outgoing calls, only receive them All users still on the PBX Nortel system (faxes, modems, elevators, some classrooms)
Status: 
Closed
Resolved alert: 
12/08/2024 2:00 pm

Dispatch showed a failed call when trying to out dial.

Initial symptoms: 

Police could not out dial. Received a failed message.

Duration: 
12/08/2024 9:41 am - 12/08/2024 2:00 pm
Impact to Mason: 

Police dispatch could not make outgoing phone calls. (Used backup POTS line)

Affected Services: 
Telephone Services
ROOT CAUSE ANALYSIS
Cause: 

Between GMU and th3 vendor sits a box that regulates what traffic is allowed to enter GMU or exit GMU.
This box is calls SBC. You have two; one for inbound traffic and one for outbound traffic. If one stops working the other takes on all traffic.
We rebooted only the inbound SBC. While it was rebooting the traffic started working, i.e.: the Outbound server took on all traffic. We could not confirm that both servers are working as we can’t trace any traffic on either server.

We worked with a third tier engineer. Since dispatch is now able to receive and make calls on the Nortel phones. We will work to get a third tier engineer Monday to confirm bother servers are working correctly.

Resolution: 

The SBC was rebooted.

Prevention: 

The issue was hardware failure thus only prevention is to have spare equipment or redundancy as is currently in place.

STATISTICS
Service Team: 
Telecom, Police, ITS, Onsite Carousel vendor