Unable to Access to Some Mason Services

September 14, 2022 9:07 a.m.
Outage category: 
Applications
Location: 
All Campuses
Status: 
Closed
Resolved alert: 
09/13/2022 3:00 pm

Customers using people finder or timesheets reporting very slow connection or login not work properly.  Other application were reported as being slow also.

Initial symptoms: 

DBA’s and others reporting that the applications are running slow or not working.

Duration: 
09/13/2022 1:40 pm - 09/13/2022 3:00 pm
Impact to Mason: 

Could not login to their timesheets, or use internal Mason applications.

Affected Services: 
Administrative Applications
ROOT CAUSE ANALYSIS
Cause: 

High CPU and traffic going through the Firewall being very slow.

Resolution: 

The Primary Firewall was shut down, and all traffic was moved to the DR secondary Firewall, witch became the Primary. Once this was done, all intermittent slowness and applications came back online with normal speeds.

Prevention: 

Engineers have shutdown Primary in Fairfax and rebooted it, also have a case with Palo Alto support to see what could they find was the issue causing the slowness.

STATISTICS
Service Team: 
NSENG, NSOPS, CCSO and CCSE