RepRight

Framing the Problem

What is the problem you are trying to solve? Who does it affect?

Over 68% of Americans are obese. The reason is partially because many believe that fitness requires going to a gym physically to get fit. Some people can't afford paying up to $50/month for a gym membership or personal trainer to see if their form is correct. Some people also live too far from a gym to go consistently. However, working out at home makes it way more accessible for people to become fit, since it's free and people can workout wherever. The problem we're trying to solve is making fitness more accessible to people all over the world. Many people who try to start working out often don't have good form, which leads to injuries over time. Those injuries discourage people from continuing their fitness journey, and possibly never get back into fitness afterwards. People can watch a YouTube video online on how to do proper form, but they don't know whether or not their form is similar to the Youtube video. They can also ask the people around them on how to get into fitness, but most people are too busy to help critique someone's form every single time they want to workout.

Idea Explanation

What is your idea? How does it fix the problem?

Our idea was to create an AI Fitness coach that critiques people's forms on exercises like pushups and squats. Whether your elbows are extended too far outwards on pushups, or your knees are bending too far in/out, the AI recognizes faults in your form and provides real time feedback on how to fix it. Our program, RepRight, works any time of day and is able to provide feedback. RepRight prevents injuries before they happen when people try to exercise for the first time. More people would be able to work out for an extended period of time without getting injured. They would be able to workout safely for free, anywhere, at any time without depending on other people to critique their form.

Implementation

How do all the pieces fit together? Does your frontend make requests to your backend? Where does your database fit in?

Our program starts with training the model using Random Forest Classifiers to classify what exercise is being done. We recorded example videos of good and bad pushups, good and bad squats, each labeled. Then, we converted the videos into a .csv file of the vectors using built-in MediaPipe functions. We trained the model using that .csv on scikit learn. Then, we used OpenCV to get the vectors of the body in conjunction with MediaPipe. We found the ratio of the length of elbow to shoulder to the length of 0.4 * the distance from one shoulder to the next. We then used FastAPI to connect the app to a website. The database is created using AlchemySQL which holds prior training sessions so users can compare their progress from session to session, providing a platform to track progressive overload. Feedback is ran from the backend to the frontend so the user can see it as they're doing pushups. We also have a tutorial tab showing how to do pushups and squats for those exercising for the first time.

Challenges

What did you struggle with? How did you overcome it?

While developing the program, we struggled the most with was refining the algorithm that detects good reps and bad reps, especially for pushups. Finding the perfect ratio to be counted for a bad rep was difficult. We made multiple adjustments since the detection accuracy depended on several factors such as the user's distance from the camera, body positioning, lighting conditions and camera angle. We had repeatedly fine-tune thresholds and test with variety of users of different body sizes to make the system more reliable. We also wanted to implement a text to speech module and a soundboard that played whenever a rep was completed, differing on whether or not the rep was bad/good. Ultimately, due to time constraints, we were able to get the soundboard and text-to-speech to work on the app created using MediaPipe and OpenCV, but we weren't able to implement it on FastAPI.

Accomplishments

What did you learn? What did you accomplish?

We were to find working ratios of body parts for squats and pushups so that the algorithm would correctly detect a pushup as good or not. We learned how to collect manual data for a machine learning algorithm, process that data, and use it to train a small model using libraries like scikit-learn. We were able to create an OpenCV project that has real value and accessibility to everyone who's looking to get into the gym for the first time. We were also able to get experience working with text to speech and soundboard modules in an app. We were also able to get exposure to FastAPI to connect the OpenCV to the website.

Next Steps

What are the next steps for your project? How can you improve it?

Currently the project only has 2 exercise push-up and squat. We plan on increasing it to include more exercises such as lunges, planks, bicep curls and shoulder presses. In addition, we aim to improve the accuracy and adaptability of the detection by providing the machine learning model with a bigger data set. Additionally, we want to create a better algorithm for detecting the quality of the rep. We also want to optimize the code so that there's less redundancies. We would also implement the text to speech and soundboard into the FastAPI so that the users can tell if they're doing an exercise correct without having to get up and look at the program while they're in the middle of a set. Finally, we hope to develop a mobile interface to make the program more accessible and user-friendly.