Computer vision, AI and using a phone app to quantify athlete movement

Unless people have been living under a rock for the last five years, they will know the validation work that has gone into the My Jump app. But if people have missed it, can you give us a brief overview?

I remember the first time I submitted a validation paper about “My Jump” (then renamed to My Jump 2, and finally to My Jump Lab). It was desk rejected in a top sports science journal. The editor didn’t even send it to peer review because he argued that “it’s not relevant that a smartphone can measure jumps.” Luckily, I went to another journal without dinosaurs on their editorial board, and it got accepted.

We systematically compared several jumps from several participants, measured simultaneously with My Jump and a Kistler force platform. The correlation was great (r > 0.98) and the precision was also high: about 2 cm bias compared with the force plate.

Also, since the app was based in manual slow motion video analysis, we had two independent observers analyze the same videos, and there were no differences between them. This paper now has more than 500 citations in ResearchGate. Not bad for not being relevant!

In the years since, different research groups have conducted validation studies of the app with different methodologies, including different populations (kids, elderly, athletes, disabled patients, etc.), different versions of the app (Android and iOS; My Jump 1, 2, Lab), different recording angles (crouching vs. standing in front of the athlete), and more. The result is always the same: high validity and reliability compared with the gold standard.

Right now, to the best of my knowledge, there are about 15 papers in peer reviewed journals.

Which features that you’ve added to My Jump Lab over the last two years have been most impactful to coaches?

Of course, we have the obvious features: jump, sprint, and real time, velocity based training just with the camera.

One feature that I think has helped coaches in a different but powerful way after the pandemic, and particularly since 2022, is its ability to import external videos.  My Jump Lab allows you to import videos that athletes send over WhatsApp, email, or anything else, making it possible to evaluate and monitor athletes online without coaches administering the test in person.

It’s funny, because I created this feature for my tests, so I didn’t have to do hundreds of jumps in my room while I was developing the app. It’s great to see that it has empowered coaches to give monitoring services to their online clients.

One question people may have before choosing the My Jump Lab is how they transfer the data to an athlete monitoring system or database to combine with other technologies. How easy is this with My Jump Lab?

The app does not have an API to integrate with other systems, because my philosophy has always been its simplicity.

When you use My Jump Lab, you don’t need to login or create an account. It works completely offline, and all the data is stored on device. There is an exception to this. On iOS, thanks to iCloud capabilities in Apple devices, data from all apps are shared between devices with the same Apple ID. But there is no way for a third party to access that data – not even me. This is great in terms of privacy, as well as simplicity.

For users who want to export the data to their AMS, there is a very simple option. In the settings, you can export the entire database to a CSV file. Then, you can work with that data as you would with any other Excel file. Some organizations have created their own custom macro to read the CSV file from My Jump Lab and then automatically import the data into their system.

AI is everywhere now. How are you using AI to help coaches do their jobs more effectively and efficiently?

It’s the big term, isn’t it? But AI has been around us for a long time, and we all have used it way before ChatGPT and the generative models became popular.

For example, when we take photos with our phones, we often see a yellow square around our faces. That’s an AI technique called computer vision, so everything related with image detection / tracking is basically AI.

I used AI in My Jump Lab in 2022 when I released an update to track barbell velocity in resistance training using computer vision. I trained a machine learning model to recognize plates by feeding a computer program about 1000 images of weight plates of different colors, perspectives, athletes, and exercises. I had to manually give the program the XY coordinates of the pixels that comprised each plate. Then I ran the program so it could “learn” what a weight plate was, and, therefore, detect one when it sees it. This was my first approach to AI and computer vision, and it was pretty successful, as Chris Bishop and I published a paper in the Journal of Strength and Conditioning Research validating it vs. GymAware.

A year later, and thanks to Apple’s updated SDK, iOS included a model to detect human bodies. That way, without the need to train a model myself, I was able to update My Jump Lab with the biggest update since its launch, trying to address the biggest drawback of the app: the need to manually detect the take off and landing of the jump.

With this update, the app can detect the human body in real time and measure jump performance automatically. An independent research group validated this feature vs. force plates, which was in JSCR in June 2024.

Finally, I’ve just incorporated a new feature in the app to track joints in real time to measure ROM of the selected body parts. For example, you can see in real time when the athlete reaches 110º in a back squat.

Overall, computer vision is the most used AI feature in video apps like mine and I think the potential is huge, although it still has many limitations.

Assessing sprint mechanics through the use of computer vision and AI is very popular now. How are you helping coaches with this at a price point everyone can afford?

As I mentioned above, computer vision has its limitations, and because of them I haven’t included AI features to sprint mechanics in my app yet.

The whole point of my app is to provide simple, affordable and real time metrics. That’s possible with jumping, VBT or mobility because the motion is simple, controlled, and slow compared to a sprint. I have tested the body detection computer vision model with sprints many times, and it’s still not at a point where it works the way I want.

Of course, there are computer vision solutions out there and I’m sure they are helpful for some people. But they entail recording a video, sending it to a server, waiting until it is analyzed by a third party, paying for each test, and then receiving the report. Again, I’m not criticizing this workflow – I’m sure it works for many people. But that is not my vision.

I will release an update to measure sprint mechanics with AI if and when I can do it on device, in real time, and without additional cost for the user.

What is next for the My Jump Lab suite of apps?

To be honest, I don’t know! I don’t really have a plan. I just like to watch the technology industry (overall, not just sports tech), learn from people who do it right, and learn even more from those who do it wrong.

The good thing of being an “app guy” is that I don’t need to think or work on hardware. I just wait until new things are released, and then I think about whether they could be helpful for my users. If so, I work hard to include it in my app ASAP.

Sometimes, though, the hype is not worth it.

Having had the opportunity to visit Apple at their London and California headquarters, I’m really looking forward to further developments of their Apple Vision Pro technology. I have tried My Jump Lab for hours with it, and it’s still not ready for my purpose due to its size, weight, battery life, software limitations, price… But I imagine a future where I would look through a cool pair of augmented reality glasses at an athlete and see how much power they develop in real time. Besides being extremely cool and the closest we will be to Dragon Ball power radar, it’s also very likely to happen in the future.

Finally, and most importantly to me, I take user feedback and research very seriously. Many of the features of the app are direct requests from users, both from the field and academia. For example, the latest update of the app included the Sprint Mechanics Assessment Score by Chris Bramah within a week after it was published, and it was thanks to a common friend, J-B Morin, who connected us.

If it’s simple, applicable, and science backed, I’m always interested!