Network Connectivity Issue – All Campuses

September 30, 2019 10:32 a.m.
Outage category: 
Applications, VoIP phone service, VPN, Wired Network, Wireless Network
Location: 
All locations
Status: 
Open
Resolved alert: 
09/25/2019 4:50 pm

   All network services would have had an intermittent connectivity issues.

Initial symptoms: 

Customers were timing out trying to reach Mason applications and sites.

Duration: 
09/25/2019 3:00 pm - 09/25/2019 4:50 pm
Impact to Mason: 

Customers were timing out trying to reach Mason applications and sites.

Affected Services: 
Network Access
ROOT CAUSE ANALYSIS
Cause: 

The cause of the service outage was due to a Multicast router when restarted, this caused the Data Center Router to lose its links, building its routes.

Resolution: 

Multicast Router was shut down.

Prevention: 

Mason Engineers and Cisco TAC will be setting up a maintenance time day to figure out the root cause.  We do not want to do testing during production work days. Engineers will be monitoring and working with Cisco support, on why Multicast router affect the Data Center Router to crash, and to find out root cause?

STATISTICS
Service Team: 
Network Security & Engineering