Firewall Failure

August 23, 2019 12:05 p.m.
Outage category: 
Telephone, VoIP phone service, VPN, Webex, Wired Network, Wireless Network
Location: 
All Campuses
Status: 
Closed
Resolved alert: 
08/22/2019 2:45 pm

All network services had an short duration disruption of service.

Initial symptoms: 

Network Connectivity dropped.

Duration: 
08/22/2019 2:30 pm - 08/22/2019 2:45 pm
Impact to Mason: 

No internet based services were available for for roughly 3-5 minutes.

Affected Services: 
ROOT CAUSE ANALYSIS
Cause: 

The service outage was due to a crash on Firewall 1. Firewall 2, the back redundant node, took over thus just causing a quick blink.

Resolution: 

The redundant firewall took over traffic preventing a long term outage.

Prevention: 

We had Firewall redundancy in place. Once Firewall 2 took over all internet services came back online. All Firewalls are back up and redundancy is back in place.

Engineers will be monitoring and  working with Palo Alto support, on why this crash occurred to find out root cause?

STATISTICS
Service Team: 
NSENG