CEP system to rapidly process vast amounts of data, without affecting the performance of the network
24 June 2011, Stoke based eGaming operator bet365 has today announced that open-source, Complex Event Processing (CEP) platform, Esper, has been selected to provide the backbone of the company’s latest network monitoring initiative. The goal is to create a Panopticon of the bet365 network that can track the behaviour of every data packet and provide visibility of each system and their functions in real-time.
The system will be used to:
Monitor the delivery of banners out to bet365’s network of third party websites
Look for suspicious behaviour in in-bound network packets
Monitor events that occur across the network to ensure that each system is operating as it should
bet365’s In-Play betting system creates a huge amount of traffic. Packet throughput is currently clocked at around 1,000,000 high velocity packets per second. The company requires a monitoring solution that can keep up with its traffic profile and spot any network issues milliseconds after they have occurred. The size and complexity of the bet365 network, coupled with the massive amount of data moving in and out via its In-Play product, means that the retrospective view provided by many off-the-shelve monitoring systems fall short of their needs.
“Tolerance for glitches is very low amongst our users. They expect nothing less than a perfect service,” said Martin Davies, CTO, bet365. “We require a system that can spot issues as they happen and alert the appropriate support team in a fraction of a second after they have occurred. For us real-time means real-time.”
Instead of storing information in a static database and running queries on it, the CEP system will feed event streams from the network through a continuous query stream. It will then aggregate these event streams in memory with very low latency. This allows bet365 to construct performance analysis systems that run on commodity hardware. The result will be a completely passive monitoring system that puts no additional load on the network but sees everything that is occurring and can spot anomalous behaviour.
“Our system is like lots of pieces of string that are all tangled together. Each piece offers its own stream of events. The CEP system can take each individual stream and aggregate them together to provide a complete and straight forward view of the whole network. Rather than dealing with symptoms of issues we can go straight to the root cause. Essentially, through the use of CEP we can identify individual needles in a massive haystack,” said Davies.
Esper was chosen for a number of reasons. There was a good technology fit as the majority of bet365’s software is also written in .Net and Java. The open-source nature of the system meant that bet365 could inspect the quality of the coding and make sure it was fit for purpose. Its relative maturity offered further reassurance, as did the depth of the support available.
The design and build of the new system is being undertaken in-house.
