Virtual Machine Outage
July 18, 2022 11:57 a.m.Outage category:
Applications, Website
Location:
All Campuses
Status:
Closed
Resolved alert:
04/03/2022 10:09 am
The following systems, some related to Banner, were down:
ADFSAZ02
admittop5
arubanetedit
build-1
clearpass
coligod
Corky1
Corky2
CORONA
DATA-01-A-FM01
DCOperation OLD
DEVARON
ENDEAVOR
FNDTADM3P
hestia
icsa4t
Lugnut
NALHUTTA
navigod
ptrain37
Ptrain9
shamu1
shamu2
shibdev1
STINGRAY
Trondheim
vCLS-be96ebbd-fe4f-4641-be61-346ac42e08c0
vernaculusp2
vskyline01
ZURG
ADFSAZ02
admittop5
arubanetedit
build-1
clearpass
coligod
Corky1
Corky2
CORONA
DATA-01-A-FM01
DCOperation OLD
DEVARON
ENDEAVOR
FNDTADM3P
hestia
icsa4t
Lugnut
NALHUTTA
navigod
ptrain37
Ptrain9
shamu1
shamu2
shibdev1
STINGRAY
Trondheim
vCLS-be96ebbd-fe4f-4641-be61-346ac42e08c0
vernaculusp2
vskyline01
ZURG
Initial symptoms:
Alerts noticed on SiteScope on multiple occasions, which were then shown to be a result of the Nodus 3 crashing.
Duration:
04/03/2022 1:44 am - 04/03/2022 10:09 am
Impact to Mason:
The following systems, some related to Banner, were down:
ADFSAZ02
admittop5
arubanetedit
build-1
clearpass
coligod
Corky1
Corky2
CORONA
DATA-01-A-FM01
DCOperation OLD
DEVARON
ENDEAVOR
FNDTADM3P
hestia
icsa4t
Lugnut
NALHUTTA
navigod
ptrain37
Ptrain9
shamu1
shamu2
shibdev1
STINGRAY
Trondheim
vCLS-be96ebbd-fe4f-4641-be61-346ac42e08c0
vernaculusp2
vskyline01
ZURG
Affected Services:
Banner Admin, Administrative Applications
ROOT CAUSE ANALYSIS
Cause:
Pending investigation. NODUS 3 had a similar event in April and it is being escalated to VMware.
Resolution:
Reboot NODUS 3 and migrated hosts and verified they were operational. More substantive troubleshooting is ongoing.
Prevention:
Working with vendor to determine issue.
STATISTICS
Service Team:
CCSO, Advanced Technologies, Data Center Operations