HOWZAT! Grad students turn their love of cricket into a winning project

Aravind Ravi doesn’t really know how many hours he logged while developing technology to find and save the highlights in video footage from cricket matches.

Combining his love of the world’s second most popular sport with an academic interest in image processing, the ambitious project he tackled with two partners in a master’s class at the University of Waterloo was more fun than work.

“We didn’t count,” says Ravi, a keen follower of the ICC Cricket World Cup that wraps up Sunday with the championship match. “It was a passion, so we just spent time until it started to work.”

The system he built with fellow engineering graduate students Harshwin Venugopal and Sruthy Paul uses deep-learning artificial intelligence (AI) to process video from matches frame by frame and recognize when an umpire is shown making an important call.

The program then automatically saves the previous 10 seconds of action in the match, a buffer sufficient to capture the events that led up to the umpire’s gesture.

It’s believed to be the first time that cricket umpires — who make simple, deliberate signals with their hands and arms after significant plays on the field — have been used with AI technology as the key to picking out highlights.

“We wanted to simplify the process and program software that is relatively easy to build,” says Venugopal. “It was more efficient.”

Still, the partners even surprised themselves when they came up with a working system by tuning pre-trained AI with just 400 Google and YouTube images to specifically recognize umpire gestures.

Results they published in a paper with their professor, Hamid Tizhoosh, and presented at a conference in India showed the system – which is compact enough to operate on a laptop computer — is capable of detecting and saving more than 90 per cent of the main events in cricket videos.

Included are plays when batsmen hit sixes — the rough equivalent of home runs in baseball — or get out by various means, plus run-scoring events involving violations by bowlers.

“The umpire is shown almost every time after these events,” says Ravi, a bowler who specialized in spin bowling during high school in his native India. “We just used that.”

The system was developed as a tool for television broadcasters to save time and effort while putting together highlight packages from matches.

And its creators believe the AI software could be modified to find and save highlights in any other sport that involves officials who signal important events with gestures.

“We were satisfied with the results with our proof-of-concept, but I think we can make a lot of improvements,” says Ravi, who is studying systems design engineering. “We had only a small data set. It could do much better if we had access to more images for training.”

Their research paper, A Dataset and Preliminary Results of Umpire Pose Detection Using SVM Classification of Deep Features, was published in the proceedings of the IEEE Symposium Series on Computational Intelligence.

For more information about engineering research at the University of Waterloo, visit here.