Resolved
Network Instability at Data Center - Increased Latency
Started 31 Jan at 06:38am UTC, resolved 31 Jan at 08:18am UTC.
Resolved
Resolved - our primary ISP, experienced an outage beginning at 12:35 AM CST. We failed over to our backup ISP (Spectrum), but the network load exceeded it's capacity, resulting in increased latency and intermittent failures rather than a complete outage.
Primary ISP service was restored at 1:27 AM CST, and all services are now operating normally.
Next Steps:
- We will increase our bandwidth with our backup ISP to handle full load during future failovers
- Begin process to test another primary ISP option as an additional redundant connection to our data centers
Created
We are experiencing network instability at one of our data centers due to an upstream provider issue. We have failed over to our backup ISP and are actively monitoring the situation. We will provide updates as more information becomes available.