Sports teams are always looking for tweaks they can make to help athletes perform better and a new artificial intelligence system being developed at the University of Waterloo is focusing on baseball pitchers.

The project, called PitcherNet, started in 2022 and is a collaboration between the university and the Baltimore Orioles. It looks to use artificial intelligence to analyze how a pitcher throws a ball.

Many Major League Baseball stadiums are equipped with high-resolution camera systems from a company called Hawk-Eye Innovations, which allows the home teams to capture and analyze the movements of their players. The system has up to 12 high-speed cameras placed around the field.

But when teams are playing away games, or if teams want to analyze the movements of players in the minor league system, they don't have access to the same technology.

"They came to us and the project was basically to find a way in which we can mimic the Hawk-Eye tracking system with just one camera, or maybe you can just use the broadcast feed," Jerrin Bright, a PhD student who is part of the research project, told CBC K-W's The Morning Edition host Craig Norris.

"They wanted to go into quantitative metrics, like trying to find the release point, the extension. But the problem is when you go into a minor league or an amateur league, the scouts wouldn't be having access to this Hawk-Eye system, which means that they will have to go with qualitative analysis. And this is where our system comes into play."

Bright says the AI system developed at the university uses a video, from a broadcast feed or someone's cellphone, and figures out the pitcher's skeleton by using 3D human modelling. It then pinpoints 18 joints on the body.

"Once we get these 18 joint positions, we do quantitative analysis using machine learning models. And with that, we'll be able to extract different pitch metrics," Bright said.

To train the AI algorithm, the researchers create 3D avatars of pitchers to track their movements and view them from different vantage points. The system goes over pitch type — whether it was a fastball, slider, curveball or other type of pitch — the release point, velocity, release extension and handedness, which means they can isolate the pitcher's throwing hand to analyze how the ball is held.

"These pitch metrics could ultimately be used to do a lot of performance indications, look into the longevity of the players and look into ways in which they can optimize their pitching action," Bright said.

Scouts could use cellphone video

CBC News reached out to the Baltimore Orioles for comment on PitcherNet but did not receive a response. The team is partially funding the research.

Sig Mejdal, the team's assistant general manager, told the Baltimore Sun newspaper that the system is "still a work in progress, but there have been cases already where it's been useful to use."

Mejdal told the newspaper biomechanics is "relatively new" in the baseball world, but "there has to be many things we could uncover that right now remain out of reach."

John Zelek, a professor of systems design engineering and co-director of the Vision and Image Processing Lab at the University of Waterloo, says what they've designed is a simple system compared to Hawk-Eye, but it's effective.

Zelek said the system could be used in a variety of different ways, including by scouts who are looking at potential players to sign. They could take videos while sitting in the stands watching a ball game and the video could be sent through the PitcherNet system to analyze an up-and-coming player's performance.

On-site work this summer

Bright admits he's more of a cricket fan and "a numbers guy" but he "got into baseball after joining this project and it has been very interesting."

He said Waterloo researchers meet with team representatives about once a month to talk about progress and how the system can be improved so the team gets the metrics they need.

Bright says there are other applications for the software, and they're looking into how it could help hockey teams, too.

He'll be going to Baltimore this summer to work on the system in the Orioles' stadium and further develop it.

"I think the potential that AI systems like PitcherNet can have on a real game, that is very fascinating and I look forward to what I can contribute in this aspect," Bright said.