Skip to main content

Showing 1–26 of 26 results for author: Lapedriza, A

Searching in archive cs. Search in all archives.
.
  1. arXiv:2501.16870  [pdf, other

    cs.CV

    Experimenting with Affective Computing Models in Video Interviews with Spanish-speaking Older Adults

    Authors: Josep Lopez Camunas, Cristina Bustos, Yanjun Zhu, Raquel Ros, Agata Lapedriza

    Abstract: Understanding emotional signals in older adults is crucial for designing virtual assistants that support their well-being. However, existing affective computing models often face significant limitations: (1) limited availability of datasets representing older adults, especially in non-English-speaking populations, and (2) poor generalization of models trained on younger or homogeneous demographics… ▽ More

    Submitted 28 January, 2025; originally announced January 2025.

    Journal ref: IEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2025)

  2. arXiv:2408.02143  [pdf, other

    cs.CL cs.AI

    Analyzing Cultural Representations of Emotions in LLMs through Mixed Emotion Survey

    Authors: Shiran Dudy, Ibrahim Said Ahmad, Ryoko Kitajima, Agata Lapedriza

    Abstract: Large Language Models (LLMs) have gained widespread global adoption, showcasing advanced linguistic capabilities across multiple of languages. There is a growing interest in academia to use these models to simulate and study human behaviors. However, it is crucial to acknowledge that an LLM's proficiency in a specific language might not fully encapsulate the norms and values associated with its cu… ▽ More

    Submitted 4 August, 2024; originally announced August 2024.

    Comments: Was accepted to ACII 2024

  3. arXiv:2310.13607  [pdf, ps, other

    cs.LG

    Analyzing the contribution of different passively collected data to predict Stress and Depression

    Authors: Irene Bonafonte, Cristina Bustos, Abraham Larrazolo, Gilberto Lorenzo Martinez Luna, Adolfo Guzman Arenas, Xavier Baro, Isaac Tourgeman, Mercedes Balcells, Agata Lapedriza

    Abstract: The possibility of recognizing diverse aspects of human behavior and environmental context from passively captured data motivates its use for mental health assessment. In this paper, we analyze the contribution of different passively collected sensor data types (WiFi, GPS, Social interaction, Phone Log, Physical Activity, Audio, and Academic features) to predict daily selfreport stress and PHQ-9 d… ▽ More

    Submitted 20 October, 2023; originally announced October 2023.

  4. arXiv:2310.12062  [pdf, other

    cs.CV

    On the use of Vision-Language models for Visual Sentiment Analysis: a study on CLIP

    Authors: Cristina Bustos, Carles Civit, Brian Du, Albert Sole-Ribalta, Agata Lapedriza

    Abstract: This work presents a study on how to exploit the CLIP embedding space to perform Visual Sentiment Analysis. We experiment with two architectures built on top of the CLIP embedding space, which we denote by CLIP-E. We train the CLIP-E models with WEBEmo, the largest publicly available and manually labeled benchmark for Visual Sentiment Analysis, and perform two sets of experiments. First, we test o… ▽ More

    Submitted 18 October, 2023; originally announced October 2023.

  5. arXiv:2207.02774  [pdf, other

    cs.CV cs.GR

    Local Relighting of Real Scenes

    Authors: Audrey Cui, Ali Jahanian, Agata Lapedriza, Antonio Torralba, Shahin Mahdizadehaghdam, Rohit Kumar, David Bau

    Abstract: We introduce the task of local relighting, which changes a photograph of a scene by switching on and off the light sources that are visible within the image. This new task differs from the traditional image relighting problem, as it introduces the challenge of detecting light sources and inferring the pattern of light that emanates from them. We propose an approach for local relighting that trains… ▽ More

    Submitted 6 July, 2022; originally announced July 2022.

    Comments: 15 pages, 15 figures

  6. arXiv:2202.01781  [pdf, other

    cs.CV cs.AI

    Predicting the impact of urban change in pedestrian and road safety

    Authors: Cristina Bustos, Daniel Rhoads, Agata Lapedriza, Javier Borge-Holthoefer, Albert Solé-Ribalta

    Abstract: Increased interaction between and among pedestrians and vehicles in the crowded urban environments of today gives rise to a negative side-effect: a growth in traffic accidents, with pedestrians being the most vulnerable elements. Recent work has shown that Convolutional Neural Networks are able to accurately predict accident rates exploiting Street View imagery along urban roads. The promising res… ▽ More

    Submitted 3 February, 2022; originally announced February 2022.

  7. arXiv:2201.04236  [pdf, other

    cs.CV

    Incidents1M: a large-scale dataset of images with natural disasters, damage, and incidents

    Authors: Ethan Weber, Dim P. Papadopoulos, Agata Lapedriza, Ferda Ofli, Muhammad Imran, Antonio Torralba

    Abstract: Natural disasters, such as floods, tornadoes, or wildfires, are increasingly pervasive as the Earth undergoes global warming. It is difficult to predict when and where an incident will occur, so timely emergency response is critical to saving the lives of those endangered by destructive events. Fortunately, technology can play a role in these situations. Social media posts can be used as a low-lat… ▽ More

    Submitted 11 January, 2022; originally announced January 2022.

  8. arXiv:2110.11672  [pdf, other

    cs.CV

    Explainable, automated urban interventions to improve pedestrian and vehicle safety

    Authors: Cristina Bustos, Daniel Rhoads, Albert Sole-Ribalta, David Masip, Alex Arenas, Agata Lapedriza, Javier Borge-Holthoefer

    Abstract: At the moment, urban mobility research and governmental initiatives are mostly focused on motor-related issues, e.g. the problems of congestion and pollution. And yet, we can not disregard the most vulnerable elements in the urban landscape: pedestrians, exposed to higher risks than other road users. Indeed, safe, accessible, and sustainable transport systems in cities are a core target of the UN'… ▽ More

    Submitted 8 November, 2021; v1 submitted 22 October, 2021; originally announced October 2021.

  9. arXiv:2109.13225  [pdf, other

    cs.CV

    Predicting Driver Self-Reported Stress by Analyzing the Road Scene

    Authors: Cristina Bustos, Neska Elhaouij, Albert Sole-Ribalta, Javier Borge-Holthoefer, Agata Lapedriza, Rosalind Picard

    Abstract: Several studies have shown the relevance of biosignals in driver stress recognition. In this work, we examine something important that has been less frequently explored: We develop methods to test if the visual driving scene can be used to estimate a drivers' subjective stress levels. For this purpose, we use the AffectiveROAD video recordings and their corresponding stress labels, a continuous hu… ▽ More

    Submitted 27 September, 2021; originally announced September 2021.

  10. Interpreting Face Inference Models using Hierarchical Network Dissection

    Authors: Divyang Teotia, Agata Lapedriza, Sarah Ostadabbas

    Abstract: This paper presents Hierarchical Network Dissection, a general pipeline to interpret the internal representation of face-centric inference models. Using a probabilistic formulation, our pipeline pairs units of the model with concepts in our "Face Dictionary", a collection of facial concepts with corresponding sample images. Our pipeline is inspired by Network Dissection, a popular interpretability… ▽ More

    Submitted 28 March, 2022; v1 submitted 23 August, 2021; originally announced August 2021.

    Journal ref: International Journal of Computer Vision (2022)

  11. arXiv:2107.14529  [pdf, other

    cs.CV

    Recognizing Emotions evoked by Movies using Multitask Learning

    Authors: Hassan Hayat, Carles Ventura, Agata Lapedriza

    Abstract: Understanding the emotional impact of movies has become important for affective movie analysis, ranking, and indexing. Methods for recognizing evoked emotions are usually trained on human annotated data. Concretely, viewers watch video clips and have to manually annotate the emotions they experienced while watching the videos. Then, the common practice is to aggregate the different annotations, by… ▽ More

    Submitted 30 July, 2021; originally announced July 2021.

    Comments: Accepted to the International Conference on Affective Computing and Intelligent Interaction (ACII) 2021

  12. arXiv:2011.14906  [pdf, other

    cs.CV cs.DB

    Person Perception Biases Exposed: Revisiting the First Impressions Dataset

    Authors: Julio C. S. Jacques Junior, Agata Lapedriza, Cristina Palmero, Xavier Baró, Sergio Escalera

    Abstract: This work revisits the ChaLearn First Impressions database, annotated for personality perception using pairwise comparisons via crowdsourcing. We analyse for the first time the original pairwise annotations, and reveal existing person perception biases associated to perceived attributes like gender, ethnicity, age and face attractiveness. We show how person perception bias can influence data label… ▽ More

    Submitted 30 November, 2020; originally announced November 2020.

    Comments: accepted on 11th International Workshop on Human Behavior Understanding (HBU), organized as part of WACV 2021

  13. arXiv:2011.08809  [pdf, other

    cs.CV

    Facial Expressions as a Vulnerability in Face Recognition

    Authors: Alejandro Peña, Ignacio Serna, Aythami Morales, Julian Fierrez, Agata Lapedriza

    Abstract: This work explores facial expression bias as a security vulnerability of face recognition systems. Despite the great performance achieved by state-of-the-art face recognition systems, the algorithms are still sensitive to a large range of covariates. We present a comprehensive analysis of how facial expression bias impacts the performance of face recognition technologies. Our study analyzes: i) fa… ▽ More

    Submitted 18 June, 2021; v1 submitted 17 November, 2020; originally announced November 2020.

    Comments: Proc. of IEEE Int. Conf. on Image Processing (ICIP)

  14. arXiv:2010.05848  [pdf, other

    cs.CL cs.LG

    Human-centric Dialog Training via Offline Reinforcement Learning

    Authors: Natasha Jaques, Judy Hanwen Shen, Asma Ghandeharioun, Craig Ferguson, Agata Lapedriza, Noah Jones, Shixiang Shane Gu, Rosalind Picard

    Abstract: How can we train a dialog model to produce better conversations by learning from human feedback, without the risk of humans teaching it harmful chat behaviors? We start by hosting models online, and gather human feedback from real-time, open-ended conversations, which we then use to train and improve the models using offline reinforcement learning (RL). We identify implicit conversational cues inc… ▽ More

    Submitted 12 October, 2020; originally announced October 2020.

    Comments: To appear in EMNLP 2020 (long paper)

  15. arXiv:2009.08704  [pdf, other

    cs.CV

    Learning Emotional-Blinded Face Representations

    Authors: Alejandro Peña, Julian Fierrez, Agata Lapedriza, Aythami Morales

    Abstract: We propose two face representations that are blind to facial expressions associated to emotional responses. This work is in part motivated by new international regulations for personal data protection, which enforce data controllers to protect any kind of sensitive information involved in automatic processes. The advances in Affective Computing have contributed to improve human-machine interfaces… ▽ More

    Submitted 18 September, 2020; originally announced September 2020.

    Comments: IAPR Intl. Conf. on Pattern Recognition, 2020

  16. arXiv:2009.05041  [pdf, other

    cs.CV cs.LG cs.NE

    Understanding the Role of Individual Units in a Deep Neural Network

    Authors: David Bau, Jun-Yan Zhu, Hendrik Strobelt, Agata Lapedriza, Bolei Zhou, Antonio Torralba

    Abstract: Deep neural networks excel at finding hierarchical representations that solve complex tasks over large data sets. How can we humans understand these learned representations? In this work, we present network dissection, an analytic framework to systematically identify the semantics of individual hidden units within image classification and image generation networks. First, we analyze a convolutiona… ▽ More

    Submitted 12 September, 2020; v1 submitted 10 September, 2020; originally announced September 2020.

    Comments: Proceedings of the National Academy of Sciences 2020. Code at https://github.com/davidbau/dissect/ and website at https://dissect.csail.mit.edu/

    MSC Class: 68T07 ACM Class: I.4; I.2

  17. arXiv:2009.03829  [pdf

    cs.CY cs.RO

    A Robotic Positive Psychology Coach to Improve College Students' Wellbeing

    Authors: Sooyeon Jeong, Sharifa Alghowinem, Laura Aymerich-Franch, Kika Arias, Agata Lapedriza, Rosalind Picard, Hae Won Park, Cynthia Breazeal

    Abstract: A significant number of college students suffer from mental health issues that impact their physical, social, and occupational outcomes. Various scalable technologies have been proposed in order to mitigate the negative impact of mental health disorders. However, the evaluation for these technologies, if done at all, often reports mixed results on improving users' mental health. We need to better… ▽ More

    Submitted 8 September, 2020; originally announced September 2020.

    Comments: 8 pages, 5 figures, RO-MAN 2020, Best paper award

  18. arXiv:2008.09188  [pdf, other

    cs.CV

    Detecting natural disasters, damage, and incidents in the wild

    Authors: Ethan Weber, Nuria Marzo, Dim P. Papadopoulos, Aritro Biswas, Agata Lapedriza, Ferda Ofli, Muhammad Imran, Antonio Torralba

    Abstract: Responding to natural disasters, such as earthquakes, floods, and wildfires, is a laborious task performed by on-the-ground emergency responders and analysts. Social media has emerged as a low-latency data source to quickly understand disaster situations. While most studies on social media are limited to text, images offer more information for understanding disaster and incident scenes. However, n… ▽ More

    Submitted 20 August, 2020; originally announced August 2020.

    Comments: ECCV 2020

  19. Context Based Emotion Recognition using EMOTIC Dataset

    Authors: Ronak Kosti, Jose M. Alvarez, Adria Recasens, Agata Lapedriza

    Abstract: In our everyday lives and social interactions we often try to perceive the emotional states of people. There has been a lot of research in providing machines with a similar capacity of recognizing emotions. From a computer vision perspective, most of the previous efforts have been focusing in analyzing the facial expressions and, in some cases, also the body pose. Some of these methods work remark… ▽ More

    Submitted 30 March, 2020; originally announced March 2020.

    ACM Class: I.4.8; I.5.4

  20. arXiv:1907.00456  [pdf, other

    cs.LG cs.AI stat.ML

    Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialog

    Authors: Natasha Jaques, Asma Ghandeharioun, Judy Hanwen Shen, Craig Ferguson, Agata Lapedriza, Noah Jones, Shixiang Gu, Rosalind Picard

    Abstract: Most deep reinforcement learning (RL) systems are not able to learn effectively from off-policy data, especially if they cannot explore online in the environment. These are critical shortcomings for applying RL to real-world problems where collecting data is expensive, and models must be tested offline before being deployed to interact with the environment -- e.g. systems that learn from human int… ▽ More

    Submitted 8 July, 2019; v1 submitted 30 June, 2019; originally announced July 2019.

  21. arXiv:1906.09308  [pdf, other

    cs.CL cs.AI cs.LG stat.ML

    Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog Systems

    Authors: Asma Ghandeharioun, Judy Hanwen Shen, Natasha Jaques, Craig Ferguson, Noah Jones, Agata Lapedriza, Rosalind Picard

    Abstract: Building an open-domain conversational agent is a challenging problem. Current evaluation methods, mostly post-hoc judgments of static conversation, do not capture conversation quality in a realistic interactive context. In this paper, we investigate interactive human evaluation and provide evidence for its necessity; we then introduce a novel, model-agnostic, and dataset-agnostic method to approx… ▽ More

    Submitted 3 November, 2019; v1 submitted 21 June, 2019; originally announced June 2019.

    Comments: 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada

  22. arXiv:1610.02055  [pdf, other

    cs.CV cs.AI

    Places: An Image Database for Deep Scene Understanding

    Authors: Bolei Zhou, Aditya Khosla, Agata Lapedriza, Antonio Torralba, Aude Oliva

    Abstract: The rise of multi-million-item dataset initiatives has enabled data-hungry machine learning algorithms to reach near-human semantic classification at tasks such as object and scene recognition. Here we describe the Places Database, a repository of 10 million scene photographs, labeled with scene semantic categories and attributes, comprising a quasi-exhaustive list of the types of environments enc… ▽ More

    Submitted 6 October, 2016; originally announced October 2016.

  23. arXiv:1512.04150  [pdf, other

    cs.CV

    Learning Deep Features for Discriminative Localization

    Authors: Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, Antonio Torralba

    Abstract: In this work, we revisit the global average pooling layer proposed in [13], and shed light on how it explicitly enables the convolutional neural network to have remarkable localization ability despite being trained on image-level labels. While this technique was previously proposed as a means for regularizing training, we find that it actually builds a generic localizable deep representation that… ▽ More

    Submitted 13 December, 2015; originally announced December 2015.

  24. arXiv:1504.07488  [pdf, ps, other

    cs.CV

    Speeding Up Neural Networks for Large Scale Classification using WTA Hashing

    Authors: Amir H. Bakhtiary, Agata Lapedriza, David Masip

    Abstract: In this paper we propose to use the Winner Takes All hashing technique to speed up forward propagation and backward propagation in fully connected layers in convolutional neural networks. The proposed technique reduces significantly the computational complexity, which in turn, allows us to train layers with a large number of kernels with out the associated time penalty. As a consequence we are a… ▽ More

    Submitted 28 April, 2015; originally announced April 2015.

    Comments: 9 pages, 9 figures, 1 table

  25. arXiv:1412.6856  [pdf, other

    cs.CV cs.NE

    Object Detectors Emerge in Deep Scene CNNs

    Authors: Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, Antonio Torralba

    Abstract: With the success of new computational architectures for visual processing, such as convolutional neural networks (CNN) and access to image databases with millions of labeled examples (e.g., ImageNet, Places), the state of the art in computer vision is advancing rapidly. One important factor for continued progress is to understand the representations that are learned by the inner layers of these de… ▽ More

    Submitted 15 April, 2015; v1 submitted 21 December, 2014; originally announced December 2014.

    Comments: 12 pages, ICLR 2015 conference paper

  26. arXiv:1311.6510  [pdf, other

    cs.CV cs.LG stat.ML

    Are all training examples equally valuable?

    Authors: Agata Lapedriza, Hamed Pirsiavash, Zoya Bylinskii, Antonio Torralba

    Abstract: When learning a new concept, not all training examples may prove equally useful for training: some may have higher or lower training value than others. The goal of this paper is to bring to the attention of the vision community the following considerations: (1) some examples are better than others for training detectors or classifiers, and (2) in the presence of better examples, some examples may… ▽ More

    Submitted 25 November, 2013; originally announced November 2013.