M1 provides additional details, says contingency plan already in place

M1_LogoM1 today provided additional information on the outage that occurs from 15th to 17th January 2013 (last week). Contradicting to previous speculation, M1 mentioned in this Press Statement that during the few days of outage, the contingency plan was already activated. If not, this outage might last 12 to 16 weeks.

Nevertheless, M1 mentions that they will continue to “enhance the network resilience” that is targeted to complete in mid-2013. I think many will be surprised that even though the outage lasted 3 days, it is already considered a fast “rescue”.

In my opinion, the biggest problem might not be the outage. It should be the event that leads to the outage. For example, in the explanation, it was mentioned that sparks occurred when vendor tried to connect the power cables. I believe this should not even happen in the first place. It might be human error (thus causing the sparks and smoke that trigger the gas suppression system).

If this is a genuine equipment failure causing sparks, maybe the 3 days to restore the equipment is considered fast. In this case, it happens during a maintenance or upgrade, I guess someone might need to answer the lapse.

I am not here to judge but M1 or any telcos should know very well that many people nowadays depend on a lot on mobile phones. On the bright side, I guess everyone learned a good lesson and they should be now reviewing or checking their procedures to prevent similar occurrences.

Here is the Press Statement

Press Statement

M1 provides further details on service recovery process

SINGAPORE, 21 January 2013 – M1 takes a very serious view of the service incident which affected customers from 15 to 17 January 2013, and would like to provide more details on the consequent recovery activities.

See also  Get Free GrabHitch Rides Tomorrow in Action Cars (video)

On 15 January, upgrading works were being carried out by a vendor on the backhaul transmission network at one of our network centres. At around 2.30am, the vendor attempted to connect its power cables from the transmission equipment rack to the distribution rack. This caused sparks at the distribution rack, followed by smoke emitting from the transmission equipment rack and the distribution rack. The smoke triggered the FM 200 gas suppression system and gas was discharged. Shortly after, one of the 88 water sprinklers was set off, causing damage to one of our mobile network switches and resulting in the service disruption.

M1’s top priority was to restore service in the affected areas as quickly as possible and contingency plans were activated. To restore service, contingency plans for the core network was to increase network capacity and signaling links between all the other mobile network switches.

Plan A for the radio network was activated, with the radio network controller and its associated media gateway being re-linked to the alternative mobile network switch. While the links were being re-configured, Plan B was concurrently activated. This involved complex step-by-step restoration work including the reconfiguration of 416 base stations to the alternative mobile network switch, re-establishing communication links, and traffic rebalancing. This was followed by comprehensive testing, including drive tests in the coverage area of each individual base station, to ensure full call continuity for both voice and data services. Plan B resulted in a total of 199 sites being connected by 3am on 17 January 2013. Plan A took over and completed the remaining 217 sites.

See also  Something interesting is happening on 22nd July with Xiaomi

Partial 3G service to our customers in the affected areas was restored on the evening of 15 January 2013. Restoration works for all affected sites were completed at about 3pm on 17 January 2013 and full service was restored at 6pm on the same day.

We would like to reassure our customers that our contingency plans were in place and were activated in this incident. As a result, we were able to restore service in the shortest possible time under the circumstances. In the absence of these contingency plans, the services would have taken about 12-16 weeks to be fully restored. Plans to further enhance and improve our network resiliency incorporating the latest technology commenced in the second half of 2011, with target completion date of mid-2013.

[ad#boxlist]