Outage category: Blackboard
Location: online users of Blackboard
Status: Closed
Resolved Alert:
Initial Symptoms
A small number of users are reporting issues accessing Blackboard. Blackboard is aware of the issue with their hosted service and confirms the issue is resolved for most users.
Root Cause Analysis
Cause
On April 20th, Anthology’s DNS servers experienced a targeted and excessive amount of DNS requests which was intended to deny servicing DNS requests. DNS servers became overwhelmed and unable to service requests. On April 25th, a change was executed to enhance our DNS services to prevent DDoS events like the one that took place on 4/20. This change was not successful which propagated incorrect DNS information across the internet.
Resolution
The change was rolled back but DNS is a pull- based request and took time to propagate the correction across the internet.
Prevention
Anthology fixed and re-executed the upgrade to DNS services to prevent issues like the one experienced between 4/20 – 4/25.