Virtual Machine Outage – Data Center
April 04, 2022 12:12 p.m.Outage category:
Hardware
Location:
Data Center - Nodus Server
Status:
Closed
Resolved alert:
04/03/2022 10:09 am
Service outage, with multiple hosts unreachable.
Initial symptoms:
SiteScope had systems alerted as down.
Duration:
04/03/2022 1:44 am - 04/03/2022 10:09 am
Impact to Mason:
Services on affected systems were down.
Affected Services:
Patriot Web
Other Affected Services:
Multiple servers were affected, a list of affected services is not currently available.
ROOT CAUSE ANALYSIS
Cause:
We are currently investigating the root issue.
Resolution:
Hardware restart of the NODUS3 system. Since then everything has been running well, and various VMs have been reassigned to other physical hosts.
Prevention:
We are going to reach out to Dell and VMware to discuss this, as VMware did not have any alerts for this host outage and Dell is the hardware vendor.
STATISTICS
Service Team:
Network Security Engineering, Network Operations, Cloud Compute Storage Operations, Cloud Compute Storage Engineering