After Action Report: TDX Webhooks

Outage category: 
TeamDynamix
Location: 
All Campuses
Status: 
Closed
Resolved alert: 
04/08/2024 7:18 am

TDX users would not see the normal set up automations function in workflows or iPaaS flows called by webhooks.

Initial symptoms: 
Tickets were not updating properly for some automations.
Duration: 
04/05/2024 3:01 pm - 04/08/2024 7:18 am
Impact to Mason: 
Ticketing was impacted but still functioning.
Affected Services: 
Administrative Applications
Other Affected Services: 
The affected application was TeamDynamix
ROOT CAUSE ANALYSIS
Cause: 
From vendor: issue was caused by a concurrency lock not releasing appropriately for your instance.
Resolution: 
From Vendor: We were able to manually release the lock to restore service.
Prevention: 
Vendor is working on issue.
STATISTICS
Service Team: 
ITSM Tool Core Team, Vendor