Facing the challenges of modern risks and threats
From its founding in 1994 as the first all-electronic stock exchange in India, NSE has always relied heavily on technology. NSE Infotech Services leads the exchange’s ongoing digital transformation while keeping its IT systems up and running to avoid business disruption.
According to GM Shenoy, Chief Technology Officer (CTO) at NSE, business disruption can take a couple of forms: “It could be a cyber attack. It could be disruptions in the IT systems themselves. So, a comprehensive resilience, which means that our business continues to operate in the face of cyber or other incidents, is critical for NSE.”
For that reason, NSE conducts mock disaster recovery drills monthly, and it runs full switchover drills from its primary site in Mumbai to a backup site in Chennai every six months. Its regulatory-mandated recovery time objective (RTO) is four hours. However, that means that during the drill, the exchange could be offline for those four hours. “About a year ago, we started thinking that, while a three- to four-hour RTO was okay as far as the regulations went, we needed to look for a faster way to complete the switchover of the trading system alone to the disaster site,” says Shenoy.
The existing recovery drills relied heavily on highly skilled technicians to manually switch NSE’s heterogenous mix of physical and virtual systems over to the backup site in Chennai. The process was slow and cumbersome and, according to Shenoy, there were sometimes missed process steps and an overall lack of efficiency.
“Our first customers are the business units within NSE, and these business units experienced delays and outages,” states Shenoy. “We needed to find the right tool to help us automate the disaster recovery process, so we could continue to serve the internal business units as well as the external broker and trading community.”
Automating disaster recovery operations
NSE worked with Kyndryl Business Resiliency Services to implement an automated Kyndryl resiliency orchestration solution for disaster recovery for critical applications.
The automated solution is designed to reduce switchover time and provide simplified, predictable disaster recovery to improve business services. The Kyndryl team created custom workflows using the solution’s recovery automation library. Critically, the cloud-based solution is ideal for use within a hybrid or multicloud environment like that of NSE.
The Kyndryl team has taken a phased approach to solution implementation. In the first phase, it tackled NSE’s risk management system (RISK) and connect-to-NSE (C2N) applications. Later phases included its Index, Information Feed, Currency Feed and Surveillance (IICS), Clearing and Settlement (CNS), enterprise, third-party, and Readiness for Trading applications. NSE has now upgraded from implementation services to managed services support until 2022 after expanding the scope of the solution.
The resiliency orchestration solution automates disaster recovery exercises to help limit human error and effort, while ensuring that recovery activities are performed within NSE-set service level agreements (SLAs). NSE can easily conduct its monthly and six-month drills, including the data center switchover to its Chennai backup center, without a major impact on internal operations or external services.
Making disaster recovery "business as usual"
By implementing the resiliency orchestration solution, NSE has realized an 80% reduction in switchover and switchback times. “When we first used the tool from [Kyndryl], the time reduced to about an hour or so,” says Shenoy. “Now that we’ve successfully implemented the entire trading system disaster recovery activity, we’ve seen a leap forward to only 40 minutes.”
And the time savings has been consistent over time. “Not only did we implement it and try it out but we now use it for every mock drill which we’ve done monthly, so that our readiness is in place,” Shenoy says. NSE has seen a 100% success rate in its monthly drills. In addition, the solution has worked well during NSE’s major switchover drills. Shenoy explains: “Every six months when we conduct a planned switchover to our Chennai site we ensure that the entire switchover is done with the [Kyndryl] orchestration tool. That means we run the market from Chennai using the tool.”
Using the tool also reduces the human effort required for switchover thanks to its automated, intelligent workflows, which speed the process and reduce human intervention and error. This allows IT experts to focus their skills on the most pressing recovery needs during downtime. Shenoy states: “My own technology people are comfortable that the tool will run and will bring the best results in the shortest possible time.” Automation also means that disaster recovery drills can be launched at the push of a button—by the right person, of course. Previously, full teams of disaster recovery experts needed to be in position before NSE could launch its drills.
Shenoy notes that his internal and external customers are pleased. “The implementation has benefitted their organization, especially the business part,” he says, of the internal business units. “They’re happy not only because outages and delays have been taken care of, but also because the overall process has become very efficient.” He adds: “For the broker community and the investors, this is a totally transparent activity. For them a total switchover is just another normal day at the market. They are able to deliver a normal day with this process and they are satisfied that this is business as usual.”
Shenoy has great confidence in the tool. “If you ask me how comfortable we are if a disaster strikes, we would be ready for it,” he says. “We test it monthly in a mock environment. And we run switchover drills every six months. We have successfully tested it out.”
October 2021
