The Perception Test Challenge, held as part of the European Conference on Computer Vision (ECCV) 2026, is a multimodal benchmark designed to comprehensively evaluate the perception and reasoning skills of multimodal video models and to advance the state of the art in visual perception.

The challenge focuses on several key research problems in multimodal perception for AI systems, particularly:

The Perception Test Challenge attracts a wide range of approaches and methods from teams leading companies and universities worldwide. Key open problems include generalization across domains (e.g., synthetic to real-world data), interpretability of AI decisions in perception tasks, and efficiency in real-time applications (e.g., autonomous systems).

This year, the fourth Perception Test Challenge received 1,560 submissions from 120 teams across 4 tracks.

Samsung R&D Institute Ukraine placed 3rd in the Grounded Video QA track.

In the grounded VideoQA task, a model receives a video-question pair and predicts bounding-box trajectories for the objects that must be localized to answer the question. The task therefore requires both semantic video understanding and accurate object tracking.

Our Visual Intelligence Team developed a four-stage solution for grounded video question answering that separates target class identification, object detection, identity matching, and gap filling. Semantic understanding and identity association are handled by a VLM, while dedicated detection and tracking models provide frame-level boxes. To limit cost, the VLM is applied only to a small set of key frames rather than every frame, reducing the number of required inferences.

The proposed pipeline achieved 0.6522 HOTA on the GVQA test set, which is 30% higher than the 2025 challenge’s winning results and only 0.3% below the second place and 1.7% below the first place of this year.

Its main advantage is efficiency combined with modularity: the VLM is used where semantic reasoning matters most, while dedicated detection and tracking models provide scalable frame-level localization across the benchmark. Although challenging questions involving subtle reasoning, similar objects, and long occlusions remain difficult, the staged design provides a practical and extensible foundation for future improvements.

Looking Ahead

The achievement of Samsung R&D Institute Ukraine’s Visual Intelligence Team in the Perception Test Challenge at ECCV 2026 highlights our commitment to advancing the state of the art in visual perception. This approach is expected to contribute to future real-world applications.

For more details, visit the https://perception-test-challenge.github.io/, https://eval.ai/web/challenges/challenge-page/2705/leaderboard/7004