Webex Outage
April 02, 2020 12:36 p.m.Webex Meetings was unavailable or inconsistent when joining. The site gmu.webex.com was down. On-going meetings were unaffected.
Users were not able to join meetings, the website gmu.webex.com was unavailable.
Users attempting to join meetings were unable to do so.
Cisco Engineering identified a service latency condition that occurred when high volumes of users connected to the Webex sites for web page joins, meeting launches, and Webex desktop/client updates. The root cause was a higher than normal I/O wait on the data for the Webex web servers when querying for the meeting client update availability. Due to the combination of traffic initiators during a high traffic period, the web server pools became unable to handle the additional requests, which led to small percentages of meeting join failures.
Cisco Engineering made a configuration change to add additional traffic shaping to the random client upgrade paths during the March 26th maintenance window.
Additional web service storage was deployed to production over the 3/27 weekend maintenance window.
Cisco Engineering made a configuration change to add additional traffic shaping to the random client upgrade paths during the March 26th maintenance window.
Additional web service storage was deployed to production over the 3/27 weekend maintenance window.
A client code change is also being deployed to add further load balancing and randomization intelligence to the update process.