Azure Issues causing some customers call quality issues

Incident Report for NTES IT Support

Postmortem

Since this event, NTES have had a case open with Microsoft, and previously with INEX. This week we received resolution.

The issue was tracked to an INEX2 peer sending micro-bursts of large traffic causing the issue. The resolution was that INEX blocked advertisements to the peer sending the traffic, forcing them to re-route the traffic to the LINX.

Posted Aug 11, 2023 - 14:43 IST

Resolved

This incident has been resolved.

Posted Jun 15, 2023 - 15:35 IST

Update

There is no more issue within any area we have influence over. Any customers with continuing issues should contact their ISP and ensure they are linked to "LAN2" with INEX.

Posted Jun 15, 2023 - 15:35 IST

Monitoring

Carriers have routed over INEX LAN2 and this has mitigated the issue, though the issue still exists within the Azure network

Posted Jun 15, 2023 - 13:01 IST

Update

Carriers are currently moving to second routing over INEX to MSFT, which appears to be mitigating this issue.

Posted Jun 15, 2023 - 11:09 IST

Update

For outbound we have routed a lot of traffic over an alternative route to and from Microsoft, which is mitigating the problem Microsoft are having.

Posted Jun 15, 2023 - 10:43 IST

Identified

Customers hosted in the Azure platform are experiencing breakup issues with calls. Engineers are tracing the issue at this time.

Posted Jun 15, 2023 - 10:14 IST

This incident affected: Voice / SIP Services.