The competition was a Visual Question Answering (VQA) challenge focused on autonomous driving safety, using dashcam videos from the 2COOOL dataset. Participants had to develop models that accurately answer structured, numeric questions about incident-related details (e.g., weather, road conditions, primary/secondary entities, prevention measures, and points of impact). The goal was to evaluate open-world robustness in rare or ambiguous scenarios, with submissions judged on their ability to provide reliable, safety-critical insights for incident analysis.

We present a robust and modular solution to AUTOPILOT-VQA that combines deterministic large VLMs with structured inference, hierarchical reasoning, and rule-based consistency enforcement. Our results demonstrate that controlling output structure and ontology alignmentareas critical as model capacity for multi-output video understanding tasks. The proposed conditioned hierarchy reduces ambiguity in incident reasoning, while deterministic post-processing ensures valid and coherent predictions. Additionally, selective ensemble-based completion effectively mitigates uncertainty without degrading confident outputs. Overall, the system highlights the effectiveness of combining strong single-model baselines with structured inference design and post-hoc corrections, offering a practical and reproducible framework for structured video question answering.

Proposed Approach

We formulate AUTOPILOT-VQA as a closed-set, multi-output classification problem over 25 question IDs (Q1.a ... Q9.b), where each video is mapped to one integer per question. The system predicts semantic labels first and converts them to competition IDs only at the final stage to enforce strict ontology control and reduce invalid outputs.

Our system consists of four main components:

1. deterministic vision-language inference with structured JSON output;
2. grouped and hierarchical prediction to improve incident-level consistency;
3. ontology-aware parsing and deterministic semantic consistency repair;
4. post-hoc multi-model completion for unknown predictions.

Together, these components form a hybrid pipeline that combines a strong single-model baseline with structured incident reasoning, consistency enforcement, and conservative ensemble-based completion.

Our primary submission used a Qwen3.5-27B in non-reasoning mode and achieved 0.63771 on the public leaderboard. Inference is deterministic (greedy decoding, no sampling), which improves reproducibility and stability across long runs. The pipeline supports both native video-token models (e.g., Qwen/Cosmos families) and image-sequence models (e.g., Gemma4): video-native models consume temporal video input directly, while image-sequence models consume FPS-sampled frames from the same video in chronological order.

The core inference stack uses structured prompting with strict JSON-only output constraints and explicit ego-vehicle perspective (“dashcam mounted on the ego-car”). We use grouped prediction mode to reduce prompt complexity by splitting questions into semantically coherent blocks, then aggregate into a complete 25-field record. A robust parser then normalizes model outputs by canonicalizing keys, mapping synonyms, and repairing minor lexical variations to valid ontology labels. If any keys are missing or values remain invalid, a dedicated one-step repair pass is executed to reconstruct a valid JSON response while preserving the original answer intent.

To improve incident reasoning consistency, we implement a conditioned hierarchical pass (enabled in the reported run):
1. Predict anchors Q5.b→Q5.f→ Q5.j;
2. Predict Q5.a conditioned on anchors;
3. Predict primary-dependent fields Q5.c, Q5.e, Q6.a, Q7.a, Q7.b;
4. Predict secondary-dependent fields Q5.g, Q5.i, Q6.b, Q7.c, Q7.d.

Each stage receives prior predictions as context, which reduces role confusion between primary and secondary entities. After model inference, we apply deterministic semantic consistency rules before integer conversion. The most important rule bundle enforces no-collision coherence: if Q5.j=1, then Q6.a=3, Q6.b=4, Q7.a=0, Q7.b=0, Q7.c=0, Q7.d=0, Q5.a=3, Q5.b=3, Q5.c=1. Additional rules align Q5.a with entity/behavior evidence (e.g., VRU, animals/objects, vehicle-to-vehicle, barrier impact, loss-of-control). Applicability-aware mapping is enforced for vehicle-only fields, and unknown/not-applicable outcomes are conservatively mapped to -1.

Finally, because -1 prevalence was empirically harmful for leaderboard performance, we performed post-hoc multi-model completion. Additional submissions were generated from diverse backbones (Cosmos2-8B, Qwen3.5-4B, Qwen3.5-27B with reasoning, Qwen3.5-35B, Gemma4-26B), combined with a weighted ensemble and consistency filtering, and used only as fallback for unknown cells. The final submission preserved all non–1 predictions from the main Qwen3.5-27B run and replaced only unknown entries by ID-aligned imputation from the ensemble output. This post-processing step improved the public leaderboard score from 0.63771 to 0.64991.

Technical Excellence and Future Impact

SRUKR’s success in the AUTOPILOT-VQA competition demonstrates their expertise in advancing structured Visual Question Answering for autonomous driving safety. Their solution integrated Vision-Language Model (VLM) specialization, hierarchical reasoning, and rule-based consistency enforcement to mitigate uncertainty and ensure reliable incident analysis.

Dmytro Kozii said that: "These techniques not only perform exceptionally in competition settings but also hold significant potential for real-world applications, such as improving accident analysis, enhancing autonomous vehicle decision-making, and enabling AI-driven safety systems".

Olena Vynokurova added: "While the competition focused on accuracy metrics, our bigger goal was proving that structured VQA can be deployed beyond benchmarks. The hierarchical pipeline and selective ensemble approach show how we can balance computational efficiency with the stringent requirements of autonomous driving - turning dashcam data into actionable safety intelligence".