Towards Primate-like Artificial Neural Networks for Visual Object Tracking

Horizon Europe Marie Skłodowska-Curie Action Postdoctoral Global Fellowship 2023

Description

The PRINNEVOT project embarks on a mission to bridge the gap between computer vision and the primate visual system in the context of Visual Object Tracking (VOT). VOT is the task of maintaining focus on a specific object amidst a dynamic visual environment. Our brains excel at it but replicating this ability in artificial vision systems remains a challenge. This project seeks to develop a novel class of VOT algorithms inspired by the primate visual system's prowess. Despite notable advancements in deep learning-based VOT over the past decade, these algorithms still fall short in emulating the robustness exhibited by primate vision. PRINNEVOT will address this gap through a multi-faceted approach. Firstly, PRINNEVOT will construct a comprehensive reference dataset, investigating both primate behavior and neural recordings. Secondly, among the existing artificial neural network (ANN)-based VOT methodologies, the project aims to identify those that align most closely with the primate brain's mechanisms. Lastly, PRINNEVOT will leverage the discovered inductive biases to develop a new ANN architecture for VOT that closely mirrors the primate's way of continuous object recognition and localization. By merging computer vision and computational neuroscience research, PRINNEVOT aspires to contribute to the development of more accurate and robust VOT algorithms. These algorithms, in alignment with the European Union's pursuit of safer and ethically grounded Artificial Intelligence, promise to enhance human-centric and trustworthy technologies. Furthermore, the project's outcomes will not only benefit AI and computer vision but also advance our understanding of the primate visual system, offering new empirical models of how the brain tracks objects in dynamic visual environments.

People

Matteo Dunnhofer

Kohitij Kar

Christian Micheloni

Institutions

University of Udine, Udine, Italy (host)

York University, Toronto, Canada (outgoing phase host)

Updates

[November 1st 2024] PRINNEVOT has started!

The outgoing phase of the PRINNEVOT project has started. For the next two years, I will work at the Centre for Vision Research at York Unviersity.

[February 8th 2025] Abstract accepted at VSS 2025.

The abstract titled "Dynamic Object Processing in Macaque IT Cortex: Temporal Dynamics and Model Limitations" (Dunnhofer et al., 2025) has been accepted for presentation at the Vision Sciences Society (VSS) Meeting 2025 to be held in St. Pete Beach, Florida, USA in May 2025.

Publications

Acknowledgements

The PRINNEVOT project has received funding from the European Union’s Horizon Europe research and innovation programme under the Marie Skłodowska-Curie grant agreement n. 101151834. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Research Executive Agency. Neither the European Union nor the granting authority can be held responsible for them.