Pre-roll · scroll-scrubbed inference, 5 fps virtual playback

UFC 308 · Topuria vs. Holloway · Round 2 — the model reading every five frames as it goes.

scroll to advance

model

tsn · resnet-50 · k400 → mma

sequence

knockdown · 200f @ 12fps

○ neutral

knockdown

exchange

P(strike)0.220

frame 000 / 199window 00 / 39

↓ continue to the report

STR1KE·tsn · resnet-50 · k400 → mma·n=38 · acc 0.83pipeline results demo try

Technical report · 2026

Strike detection in MMA broadcast video with a 5-frame temporal classifier on top of SAM2 silhouettes.

A three-stage pipeline — segment, annotate, classify — that calls strike or neutral on every 5-frame window. Trained on 38 hand-labeled windows and a Kinetics-400 prior. This page presents the system, the design decisions, and the failure modes that survive them.

Author: Thomas Ou
Code: github.com/Smokeybear10/STR1KE
Source: UFC 308 · Topuria vs. Holloway
Read time: ≈ 7 min
Status: technical preview

Abstract

We fine-tune a Temporal Segment Network (ResNet-50, Kinetics-400 pretrain) for binary strike / neutral classification on 5-frame windows of MMA footage. SAM2 segmentation removes the broadcast (cage, crowd, overlays) before classification, reducing the domain-shift burden on a dataset of only 38 hand-labeled windows. The pipeline reaches 0.83 validation accuracy after under one minute of training on a free-tier Colab GPU. A from-scratch 3D CNN baseline never converges — we report both, and discuss where the small-data regime forces transfer learning to do almost all the work.

§ 01 · Problem

Detecting a strike in broadcast MMA video is, on the surface, a 1-class action-recognition problem. Three things make it specifically hard.

Temporal scale. A jab takes 100–200 ms from initiation to contact. At 30 fps that is five frames. Most action-recognition models read seconds of context; here the entire signal is shorter than a sneeze. Either the model reads all five frames at once, or it misses the action.

Domain noise. Broadcast footage carries cage geometry, crowd parallax, scorebugs, corporate overlays, and a camera that pans on contact. A naïve classifier will happily learn to predict strike whenever the Monster Energy logo enters frame.

No data. There is no public, labeled dataset for MMA strike detection. Every sample in this work was created from scratch.

§ 02 · First attempt — 3D CNN from scratch

The architecturally cleaner option came first: a small 3D CNN that ingests RGB and the SAM2 mask jointly as a 4-channel volume. The hypothesis was that explicit mask information, fed end-to-end, would let the network learn fighter-aware spatiotemporal features.

It never converged. With 38 training samples, no pretrain is available for 4-channel 3D convolutions; the model is initialized from scratch and asked to learn human-motion priors from a kitchen-table dataset. Validation accuracy flatlined at the majority baseline.

The pivot: trade architectural elegance for a strong prior. ResNet-50 has already seen ~300 K Kinetics-400 video clips covering human action. Fine-tuning only adapts the final layer. The backbone does the heavy lifting; the head learns the strike/neutral boundary. That move is the whole pipeline below.

FIG 01 · 20-epoch training comparisonn=38 · Colab T4

Solid: TSN with K400 pretrain, head-only fine-tune (introduced in §03). Dashed: 3D CNN trained from scratch on RGB + mask. The 3D-CNN curve is reconstructed from logged values — flat at 37.5% is the literal result we observed. The takeaway is not that 3D convolutions are wrong for this task; it is that, in the small-data regime, transfer learning beats theoretically- better-suited architectures that lack a useful prior.

§ 03 · Pipeline

The system is staged so each step is independently inspectable. Mask quality can be audited without re-running training; classifier output can be re-aggregated without re-running segmentation.

Stage 01

Segment

SAM2 · Meta

Promptable segmentation, prompted once per fighter on frame 0 of a clip. SAM2's streaming memory propagates masks forward through fast occlusion and overlapping bodies — exactly the failure modes that defeat simpler trackers.

Input: RGB frames
Output: per-frame binary mask
Prompt: 1 click / fighter

Stage 02

Annotate

Label Studio · SAM2 backend

38 five-frame windows hand-verified over one weekend at a kitchen table. Balanced 19 / 19. Without ML-assisted propagation (premium-only), every mask had to be inspected manually — the bottleneck is the eye, not the GPU.

Windows: 38 (19 strike · 19 neutral)
Frames each: 5 @ 30 fps
Split: 30 / 6 / 2

Stage 03

Classify

TSN · ResNet-50 · K400 pretrain

Temporal Segment Network. Each of the 5 frames passes through a shared ResNet-50; features are averaged into a single 2048-d vector; a fresh FC head produces the binary softmax. Backbone frozen, head fine-tuned.

Pretrain: tsn_r50_1x1x3_100e_k400
Epochs: 20
Train time: < 1 min · 1 GPU

Original broadcast frame from Topuria vs. Holloway, round 2 — The classifier never sees the left side; it sees only the right. Removing the broadcast before classification sidesteps the bulk of domain-shift, which the dataset is too small to absorb directly. Background-removal as preprocessing — not as architecture — is the single most consequential design decision in the project.

SAM2 silhouette mask of the same frame, fighters isolated on black — The classifier never sees the left side; it sees only the right. Removing the broadcast before classification sidesteps the bulk of domain-shift, which the dataset is too small to absorb directly. Background-removal as preprocessing — not as architecture — is the single most consequential design decision in the project.

FIG 03 · TSN forward pass on one 5-frame windowstage 03

Sparse sampling: TSN was designed for untrimmed video, but its consensus rule — average per-frame features before the head — is exactly right for our 5-frame window. The model is forced to pick up patterns that hold across the whole strike, not single-frame artifacts.

§ 04 · Results

The plot below is the model's softmax output, window-by-window, on the Knockdown clip. Hover anywhere to read the underlying numbers.

val accuracy

0.83

5 / 6 windows

peak confidence

0.98

pressure clip · w13

3D CNN baseline

0.375

majority-class

train time

< 60 s

colab T4 · 20 epochs

FIG 04 · Per-window P(strike) on Knockdown clip · interactive40 windows · 200 frames · 16.7 s

hover to scrub

Each x-tick is one 5-frame window. Red dots mark above-threshold local maxima; the dashed line is the 0.5 decision boundary. The three peaks at w3, w20 and w35correspond to the left hook, the follow-up ground strike, and the referee's intervention motion respectively — the model is detecting all three and quieting in between.

§ 05 · What the model gets wrong

Three failure modes recur across the test clips. They are not bugs; they are the load-bearing limits of a 38-sample classifier with no temporal context between windows.

Pressure clip · w35

Clinch break fires as strike

P(strike) = 0.82

Sudden separation looks like a recoiling punch. The pretrain has no concept of clinch.

Exchange clip · w3

Slipped jab missed

P(strike) = 0.33

Defender slips outside; striker's arm extends fully but never makes a connecting motion. The model splits the difference.

Knockdown clip · w14–w15

Strike crosses window boundary

P(strike) = 0.48 / 0.82

Wind-up sits in w14, contact in w15. Non-overlapping windows fragment the action; only w15 fires.

§ 06 · What ten times the data would change

The 0.83 number is a ceiling against the 38-window dataset, not against the task. The path from here is unglamorous: more annotation. Even 200–500 windows would be transformative. With ML-assisted propagation the same weekend yields ~10× the data.

Beyond volume, three structural changes are queued. Multi-class labels (jab, cross, hook, kick, elbow, knee, takedown) make the output useful for fight analytics, not just highlight detection. Overlapping sliding windows recover strikes that currently fragment across the boundary at w14/w15. Multi-fight, multi-fightertraining breaks the implicit overfit to Topuria's stance and Holloway's volume.

§ 07 · Reproducibility

The annotation format is intentionally simple. Each line in train.txtis a folder, a frame count, and an integer class label — readable by MMAction2's RawframeDataset out of the box.

# annotations/train.txt — MMAction2 RawframeDataset format
# <folder> <n_frames> <class_label>   (0 = neutral, 1 = strike)

strike/0  5 1
strike/1  5 1
neutral/0 5 0
neutral/1 5 0
...

# inference (per clip)
from mmaction.apis import inference_recognizer, init_recognizer

model  = init_recognizer('configs/tsn_r50_strike.py', 'work_dirs/best.pth')
result = inference_recognizer(model, 'window_w20.mp4')
# result.pred_score → tensor([0.04, 0.96])  →  STRIKE @ 0.96

Setting	Value	Notes
backbone	ResNet-50	ImageNet → Kinetics-400 → frozen during fine-tune
checkpoint	tsn_r50_1x1x3_100e_kinetics400_rgb	OpenMMLab model zoo
input	5 × 224 × 224 RGB	center crop · ImageNet norm
head	FC 2048 → 2 · softmax	only learnable parameters
optimizer	SGD · lr 0.001 · momentum 0.9	linear warmup · cosine decay
epochs	20	val acc converged by epoch 9
data format	RawframeDataset	JPEG sequences, not video files

§ 08 · Live inference

The model, running on three held-out clips.

live inference12 fps · 5-frame windows

f000 / 199w00 / 39

model call

neutral

P(strike)

0.220

0.00.51.0

clip

Knockdown

Topuria — R2 finishing sequence

commentary

Three confident detections in 6.7s. Peaks at w3 (0.93), w20 (0.96), w35 (0.85) align with the visible left hook → follow-up → referee stoppage.

per-window calls40 windows · 200 frames

0.00s / 16.58s

§ 09 · Try it

Run the pipeline on a clip you choose.

Drag a clip · or click to browse

Run the model on your own footage.

MP4 · ≤ 3 seconds · ≤ 10 MB. Frames are processed in five-frame windows; the page reports per-window confidence in line with the evaluation above.

uploaded video discarded after inferencemodal · single-gpu worker