NeurIPS 2023 Competition: Privacy Preserving Federated Learning Document VQA
Authors:
Marlon Tobaben,
Mohamed Ali Souibgui,
Rubèn Tito,
Khanh Nguyen,
Raouf Kerkouche,
Kangsoo Jung,
Joonas Jälkö,
Lei Kang,
Andrey Barsky,
Vincent Poulain d'Andecy,
Aurélie Joseph,
Aashiq Muhamed,
Kevin Kuo,
Virginia Smith,
Yusuke Yamasaki,
Takumi Fukami,
Kenta Niwa,
Iifan Tyou,
Hiro Ishii,
Rio Yokota,
Ragul N,
Rintu Kutum,
Josep Llados,
Ernest Valveny,
Antti Honkela
, et al. (2 additional authors not shown)
Abstract:
The Privacy Preserving Federated Learning Document VQA (PFL-DocVQA) competition challenged the community to develop provably private and communication-efficient solutions in a federated setting for a real-life use case: invoice processing. The competition introduced a dataset of real invoice documents, along with associated questions and answers requiring information extraction and reasoning over…
▽ More
The Privacy Preserving Federated Learning Document VQA (PFL-DocVQA) competition challenged the community to develop provably private and communication-efficient solutions in a federated setting for a real-life use case: invoice processing. The competition introduced a dataset of real invoice documents, along with associated questions and answers requiring information extraction and reasoning over the document images. Thereby, it brings together researchers and expertise from the document analysis, privacy, and federated learning communities. Participants fine-tuned a pre-trained, state-of-the-art Document Visual Question Answering model provided by the organizers for this new domain, mimicking a typical federated invoice processing setup. The base model is a multi-modal generative language model, and sensitive information could be exposed through either the visual or textual input modality. Participants proposed elegant solutions to reduce communication costs while maintaining a minimum utility threshold in track 1 and to protect all information from each document provider using differential privacy in track 2. The competition served as a new testbed for developing and testing private federated learning methods, simultaneously raising awareness about privacy within the document image analysis and recognition community. Ultimately, the competition analysis provides best practices and recommendations for successfully running privacy-focused federated learning challenges in the future.
△ Less
Submitted 3 June, 2025; v1 submitted 6 November, 2024;
originally announced November 2024.
A Planning Framework for Persistent, Multi-UAV Coverage with Global Deconfliction
Authors:
Tushar Kusnur,
Shohin Mukherjee,
Dhruv Mauria Saxena,
Tomoya Fukami,
Takayuki Koyama,
Oren Salzman,
Maxim Likhachev
Abstract:
Planning for multi-robot coverage seeks to determine collision-free paths for a fleet of robots, enabling them to collectively observe points of interest in an environment. Persistent coverage is a variant of traditional coverage where coverage-levels in the environment decay over time. Thus, robots have to continuously revisit parts of the environment to maintain a desired coverage-level. Facilit…
▽ More
Planning for multi-robot coverage seeks to determine collision-free paths for a fleet of robots, enabling them to collectively observe points of interest in an environment. Persistent coverage is a variant of traditional coverage where coverage-levels in the environment decay over time. Thus, robots have to continuously revisit parts of the environment to maintain a desired coverage-level. Facilitating this in the real world demands we tackle numerous subproblems. While there exist standard solutions to these subproblems, there is no complete framework that addresses all of their individual challenges as a whole in a practical setting. We adapt and combine these solutions to present a planning framework for persistent coverage with multiple unmanned aerial vehicles (UAVs). Specifically, we run a continuous loop of goal assignment and globally deconflicting, kinodynamic path planning for multiple UAVs. We evaluate our framework in simulation as well as the real world. In particular, we demonstrate that (i) our framework exhibits graceful coverage given sufficient resources, we maintain persistent coverage; if resources are insufficient (e.g., having too few UAVs for a given size of the enviornment), coverage-levels decay slowly and (ii) planning with global deconfliction in our framework incurs a negligibly higher price compared to other weaker, more local collision-checking schemes. (Video: https://youtu.be/aqDs6Wymp5Q)
△ Less
Submitted 16 October, 2019; v1 submitted 24 August, 2019;
originally announced August 2019.
