Contextual ambiguity: Some distinctions are semantic, not visual: Stumble vs. foul, Gesture vs. structured signal, Screen vs. incidental proximity, Defensive rotation vs. blown coverage. Only people fluent in the sport’s grammar can label these consistently enough for models to learn them.

 Occlusions and overlaps: Pick-and-roll traffic, box-out scrums, goal-mouth melees—identity swaps and ball occlusions are common. Without judgment-driven adjudication and identity continuity checks, tracking models drift and downstream metrics degrade.

 Distribution shift: New arenas, camera heights, lighting, alternates, weather, broadcast overlays—small aesthetic changes can produce big prediction shifts without annotation that captures context and helps models generalize.

 Temporal causality: Sport is sequential decision-making. The significance of an action is often defined by what preceded it and what it created next. If labels don’t carry temporal links (screen → help tag → corner kick-out), models flatten cause-effect into isolated events.

 Class imbalance & the long tail: Rare but decisive events (e.g., “ghost screen”, “double move”, “goalie screen”) are precisely what analytics teams care about. They need oversampled, high-fidelity labels, not just broad event coverage.