Videoconferencing experiencing issues

Outage category: 
Webex
Location: 
Online - videoconferencing endpoints.
Status: 
Closed
Resolved alert: 
07/01/2019 3:54 pm

Videoconferencing endpoints were experiencing latency which caused intermittent problems with video and audio performance.

Initial symptoms: 

Audio and video quality was impacted causing baps in both during meetings and during recordings.

Duration: 
07/01/2019 12:00 pm - 07/01/2019 3:54 pm
Impact to Mason: 

All videoconferencing services were intermittent.

Affected Services: 
Webex Meetings
ROOT CAUSE ANALYSIS
Cause: 

The cause of the service outage was determined to be filesystem related error. It appears we were experiencing a tremendous amount of attempts in the same way that was blocking and this flooded a single CPU (instead of the load being balanced between them via round robin or some other method). Multicast traffic was causing this activity.

Resolution: 

The source of the multicast traffic was found and turned off, resulting in a clearing of the issue.

Prevention: 

Network & Security Engineering is working with the vendor to determine why this caused and error and how to prevent such errors in the future.

STATISTICS
Service Team: 
Network & Security Engineering (NSENG)