GameLens

Framing the Problem

What is the problem you are trying to solve? Who does it affect?

The problem we are trying to solve is the inefficiency and lack of automation in the process surrounding the creation and sharing of sports highlights across social media platforms. Content creators, media managers, and sports organizations currently gather player statistics, write captions, and upload posts, a monotonous and inefficient process. Who this issue affects are social media managers, sports journalists, and marketing teams who need to quickly post updates and consistently during games to keep their audiences up-to-date and engaged.

Idea Explanation

What is your idea? How does it fix the problem?

Our idea was to design the startup of an application implementing pictures taken at live sports games and transforms them into automated, data-driven posts using AI to facilitate how sports highlights are created and publicized. By integrating image recognition through the Gemini API, real-time statistics accessed through the NBA API, and automated video/caption generation through Veo 3, the project transforms a single player photo into meaningful, shareable content. This fixes the problem of how tedious and time-consuming it is for creators and organizations to manually gather stats and relate them to specific highlights which must be captured by live play-by-play analysis during games. It makes the entire process faster, more efficient, and accessible to anyone that would like to create professional-style sports posts on their own similar to those from popular sports publishers on social media like ESPN, Bleacher Report, or House of Highlights.

Implementation

How do all the pieces fit together? Does your frontend make requests to your backend? Where does your database fit in?

This app, GameLens, functions like a digital assistant that runs on two parts: a website and a server. When the user uploads as picture of a player in the NBA the website instantly sends that picture to the server where the main program (Flask) inputs the image into Gemini AI, which determines the players full name. Then the server looks up the player's most recent stats using the public sports database NBA API. With all this information gathered, the server prompts Gemini to create a fun social media caption. Finally, the server sends the stats and caption back to the website, where the inputted image, the player's stats and the social media caption relevant to the image are outputted.

Challenges

What did you struggle with? How did you overcome it?

Backend: We struggled with implementing a way to correctly identify the player in the image. We overcame it by using the Gemini API and prompting Gemini to "Only tell me the full name of the player who is making the play in this image" to make sure Gemini response to the prompt in the same way the player is stored in the NBA API.

Frontend: On the front end part of our program we struggled with learning to use flask and figuring out how to properly connect the information received from the frontend to the backend so that it could be processed. In the end, we were able to figure everything out and we feel that we have learned a lot of new concepts.

Accomplishments

What did you learn? What did you accomplish?

Throughout the development of this project, we learned how to integrate multiple APIs into one cohesive workflow--specifically NBA_API, Google Gemini API, and the Instagram Graph API (in works for aftermath of project still). We gained direct experience using AI for image recognition, natural language generation, and video prompt creation, all while fetching and analyzing real-time player statistics from the NBA. We were able to build a system that takes a player's image, identifies them using Gemini's API and vision model through several factors like jersey number, height, physical appearance, and team name. It would then gather their all-time and current season/game stats through NBA_API and generate a personalized AI caption and video generated by creating a Veo 3 video prompt for Gemini to create. What we accomplished was how to merge AI and sports data into a practical tool that simplifies digital content creation.

Next Steps

What are the next steps for your project? How can you improve it?

The next steps for our project involves addressing the repetitive and time-consuming process in manually posting, scheduling, and synchronizing content across platforms like Instagram and Facebook. For sports highlights specifically, these platforms do not have automated posting directly from user-built tools without the use of a verified API app. To enhance our creation, we plan to include a feature that allows for the user to automatically publish an Instagram post with player image of the player, generated Veo 3 video, and auto-generated description into a carousel-- a series of content items of different types. This would be achieved by inserting an Instagram user access token and using the Instagram Graph API to seamlessly upload and manage multimedia content in a single automated workflow. This would allow verified users to connect their accounts, upload media, and automatically publish content without leaving the application. The security and authentication of the app can be improved by strengthening token management and securing token storage to ensure long-term, safe automation. We further plan to extend compatibility with other sports and social media API's like TikTok or X.