Outage category: Web Conferencing
Location: Webex Conferencing Services
Status: Closed
Resolved Alert:
Initial Symptoms
Webex notified us of issues with service. ITS Support Center confirmed they were experiencing the issue within their own office staff.
Root Cause Analysis
Cause
Engineering identified that the settings cache for a subset of organizations had become corrupt. A software defect was exposed when a toggle feature update unexpectedly corrupted the settings cache for multiple organizations. While the cache was incorrect, the database values were unaffected. The Webex desktop app queried the incomplete configuration in the cache, which led to file upload and screenshare failures.
Resolution
Engineering manually repaired organizations that reported the issue, and a code update was deployed during the maintenance window on January 22nd to automatically refresh the settings cache and resolve the issue.
Prevention
1) Engineering manually repaired organizations that reported the issue, and a code update was deployed during the maintenance window on January 22nd which refreshed the settings cache for all affected organizations. 2) Engineering deployed code updates to address the defect and extra code hardening to protect the settings cache which will prevent recurrence.