-
Dynamics Based Neural Encoding with Inter-Intra Region Connectivity
Authors:
Mai Gamal,
Mohamed Rashad,
Eman Ehab,
Seif Eldawlatly,
Mennatullah Siam
Abstract:
Extensive literature has drawn comparisons between recordings of biological neurons in the brain and deep neural networks. This comparative analysis aims to advance and interpret deep neural networks and enhance our understanding of biological neural systems. However, previous works did not consider the time aspect and how the encoding of video and dynamics in deep networks relate to the biologica…
▽ More
Extensive literature has drawn comparisons between recordings of biological neurons in the brain and deep neural networks. This comparative analysis aims to advance and interpret deep neural networks and enhance our understanding of biological neural systems. However, previous works did not consider the time aspect and how the encoding of video and dynamics in deep networks relate to the biological neural systems within a large-scale comparison. Towards this end, we propose the first large-scale study focused on comparing video understanding models with respect to the visual cortex recordings using video stimuli. The study encompasses more than two million regression fits, examining image vs. video understanding, convolutional vs. transformer-based and fully vs. self-supervised models. Additionally, we propose a novel neural encoding scheme to better encode biological neural systems. We provide key insights on how video understanding models predict visual cortex responses; showing video understanding better than image understanding models, convolutional models are better in the early-mid visual cortical regions than transformer based ones except for multiscale transformers, and that two-stream models are better than single stream. Furthermore, we propose a novel neural encoding scheme that is built on top of the best performing video understanding models, while incorporating inter-intra region connectivity across the visual cortex. Our neural encoding leverages the encoded dynamics from video stimuli, through utilizing two-stream networks and multiscale transformers, while taking connectivity priors into consideration. Our results show that merging both intra and inter-region connectivity priors increases the encoding performance over each one of them standalone or no connectivity priors. It also shows the necessity for encoding dynamics to fully benefit from such connectivity priors.
△ Less
Submitted 8 December, 2024; v1 submitted 19 February, 2024;
originally announced February 2024.
-
The State of Computer Vision Research in Africa
Authors:
Abdul-Hakeem Omotayo,
Ashery Mbilinyi,
Lukman Ismaila,
Houcemeddine Turki,
Mahmoud Abdien,
Karim Gamal,
Idriss Tondji,
Yvan Pimi,
Naome A. Etori,
Marwa M. Matar,
Clifford Broni-Bediako,
Abigail Oppong,
Mai Gamal,
Eman Ehab,
Gbetondji Dovonon,
Zainab Akinjobi,
Daniel Ajisafe,
Oluwabukola G. Adegboro,
Mennatullah Siam
Abstract:
Despite significant efforts to democratize artificial intelligence (AI), computer vision which is a sub-field of AI, still lags in Africa. A significant factor to this, is the limited access to computing resources, datasets, and collaborations. As a result, Africa's contribution to top-tier publications in this field has only been 0.06% over the past decade. Towards improving the computer vision f…
▽ More
Despite significant efforts to democratize artificial intelligence (AI), computer vision which is a sub-field of AI, still lags in Africa. A significant factor to this, is the limited access to computing resources, datasets, and collaborations. As a result, Africa's contribution to top-tier publications in this field has only been 0.06% over the past decade. Towards improving the computer vision field and making it more accessible and inclusive, this study analyzes 63,000 Scopus-indexed computer vision publications from Africa. We utilize large language models to automatically parse their abstracts, to identify and categorize topics and datasets. This resulted in listing more than 100 African datasets. Our objective is to provide a comprehensive taxonomy of dataset categories to facilitate better understanding and utilization of these resources. We also analyze collaboration trends of researchers within and outside the continent. Additionally, we conduct a large-scale questionnaire among African computer vision researchers to identify the structural barriers they believe require urgent attention. In conclusion, our study offers a comprehensive overview of the current state of computer vision research in Africa, to empower marginalized communities to participate in the design and development of computer vision systems.
△ Less
Submitted 13 September, 2024; v1 submitted 21 January, 2024;
originally announced January 2024.
-
Towards a Better Understanding of the Computer Vision Research Community in Africa
Authors:
Abdul-Hakeem Omotayo,
Mai Gamal,
Eman Ehab,
Gbetondji Dovonon,
Zainab Akinjobi,
Ismaila Lukman,
Houcemeddine Turki,
Mahmod Abdien,
Idriss Tondji,
Abigail Oppong,
Yvan Pimi,
Karim Gamal,
Ro'ya-CV4Africa,
Mennatullah Siam
Abstract:
Computer vision is a broad field of study that encompasses different tasks (e.g., object detection). Although computer vision is relevant to the African communities in various applications, yet computer vision research is under-explored in the continent and constructs only 0.06% of top-tier publications in the last ten years. In this paper, our goal is to have a better understanding of the compute…
▽ More
Computer vision is a broad field of study that encompasses different tasks (e.g., object detection). Although computer vision is relevant to the African communities in various applications, yet computer vision research is under-explored in the continent and constructs only 0.06% of top-tier publications in the last ten years. In this paper, our goal is to have a better understanding of the computer vision research conducted in Africa and provide pointers on whether there is equity in research or not. We do this through an empirical analysis of the African computer vision publications that are Scopus indexed, where we collect around 63,000 publications over the period 2012-2022. We first study the opportunities available for African institutions to publish in top-tier computer vision venues. We show that African publishing trends in top-tier venues over the years do not exhibit consistent growth, unlike other continents such as North America or Asia. Moreover, we study all computer vision publications beyond top-tier venues in different African regions to find that mainly Northern and Southern Africa are publishing in computer vision with 68.5% and 15.9% of publications, resp. Nonetheless, we highlight that both Eastern and Western Africa are exhibiting a promising increase with the last two years closing the gap with Southern Africa. Additionally, we study the collaboration patterns in these publications to find that most of these exhibit international collaborations rather than African ones. We also show that most of these publications include an African author that is a key contributor as the first or last author. Finally, we present the most recurring keywords in computer vision publications per African region.
△ Less
Submitted 4 February, 2024; v1 submitted 11 May, 2023;
originally announced May 2023.
