Squash AI Coach / Referee takes a short rally clip — YouTube link or file upload — and returns a WSF-style call (let, stroke, no let, and friends) with reasoning, confidence, and annotated key frames. A separate coach pipeline accepts longer clips (up to about two minutes) and returns a structured coaching report instead of a single decision.
Two pipelines
The judge path is built for clips under ten seconds: ingest with yt-dlp or ffmpeg, extract frames and a fixed set of key frames, run computer-vision analysis, then reason with either a local Gemma model via Ollama or Google Gemini. The last step draws the call onto the frames so you can see which moments drove the decision.
The coach path is longer and admin-gated. It downsamples the clip, streams a “thinking” pass through Gemini, and produces a report aimed at practice rather than a single referee call. Status updates flow worker → Redis pub/sub → API → SSE → browser, so the UI advances as each stage finishes rather than polling a blob of JSON at the end.
Stack
Five containers under Docker Compose: SvelteKit frontend (Svelte 5 runes, Tailwind), FastAPI + SQLAlchemy 2 + fastapi-users, a Celery worker sharing the API image, Postgres 16, and Redis as both broker and SSE fan-out. Auth is email/password with optional Google OAuth; quotas sit on the user role so free use cannot melt a GPU box overnight.
# Judge reasoning: Ollama (default) or Gemini Files API
JUDGE_LLM_PROVIDER=ollama # or gemini
OLLAMA_MODEL=gemma4:26b-qat
# Coach always uses Gemini regardless of the judge providerWhat was hard
A squash ball is often a handful of pixels. YOLO’s smallest anchors do not care about your feelings; ball tracking through a back-wall boast was the failure mode that kept eating demo videos. The useful fix was less “bigger model” and more “better key-frame selection plus structured rally data the LLM can actually argue from.”
If the vision layer lies, the referee layer writes beautiful wrong essays. Fix the ball first.
Beta today: judge and coach both run end-to-end in Docker; CV still has room before I would trust it on a league night without a human watching the annotated frames.