VoIP Outage Resolved – so what happened ?

At 10:12am this morning, we started receiving reports that customers were experiencing problems making and receiving calls.

We immediately got in touch with our suppliers who had already identified the problem and were working to resolve it by adjusting internet network routes. We issued a series of tweets to try and keep everyone informed of the situation.

The situation broke down as the following (information as supplied by our supplier) :

11:01 All traffic in to our network is still affected as is the traffic in to other providers. All possible pressure is being put in to getting this resolved.

11:18 We have identified a major UK ISP (Global Crossing) as the cause of the problems. We have routed traffic away from them. It will take up to 15 minutes for this change to take effect as ISP routers recalculate routes

11.33 We are seeing traffic and call volumes coming back to normal levels due to the changes made. Some customers whose ISPs continue to route via the Global Crossing may continue to be affected. We have informed Global Crossing and they have acknowledged that they have an issue.

11:46 Customers may be interested in looking at the independent website https://downdetector.co.uk/ where it is demonstrated than many companies were affected by the same outage. The charts available show an increase in reports for many of the companies listed during the same period. These charts measure the reports from customers that they cannot reach the companies website

We apologise for any inconvenience that you may have suffered whilst this problem was identified and resolved, as you can see it affected a lot of the big internet websites as well.