-
A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America
Authors:
Laura Alonso Alemany,
Luciana Benotti,
Hernán Maina,
Lucía González,
Mariela Rajngewerc,
Lautaro Martínez,
Jorge Sánchez,
Mauro Schilman,
Guido Ivetta,
Alexia Halvorsen,
Amanda Mata Rojo,
Matías Bordone,
Beatriz Busaniche
Abstract:
Automated decision-making systems, especially those based on natural language processing, are pervasive in our lives. They are not only behind the internet search engines we use daily, but also take more critical roles: selecting candidates for a job, determining suspects of a crime, diagnosing autism and more. Such automated systems make errors, which may be harmful in many ways, be it because of…
▽ More
Automated decision-making systems, especially those based on natural language processing, are pervasive in our lives. They are not only behind the internet search engines we use daily, but also take more critical roles: selecting candidates for a job, determining suspects of a crime, diagnosing autism and more. Such automated systems make errors, which may be harmful in many ways, be it because of the severity of the consequences (as in health issues) or because of the sheer number of people they affect. When errors made by an automated system affect a population more than others, we call the system \textit{biased}.
Most modern natural language technologies are based on artifacts obtained from enormous volumes of text using machine learning, namely language models and word embeddings. Since they are created by applying subsymbolic machine learning, mostly artificial neural networks, they are opaque and practically uninterpretable by direct inspection, thus making it very difficult to audit them.
In this paper, we present a methodology that spells out how social scientists, domain experts, and machine learning experts can collaboratively explore biases and harmful stereotypes in word embeddings and large language models. Our methodology is based on the following principles:
* focus on the linguistic manifestations of discrimination on word embeddings and language models, not on the mathematical properties of the models * reduce the technical barrier for discrimination experts%, be it social scientists, domain experts or other * characterize through a qualitative exploratory process in addition to a metric-based approach * address mitigation as part of the training process, not as an afterthought
△ Less
Submitted 28 March, 2023; v1 submitted 13 July, 2022;
originally announced July 2022.
-
Long-term Control for Dialogue Generation: Methods and Evaluation
Authors:
Ramya Ramakrishnan,
Hashan Buddhika Narangodage,
Mauro Schilman,
Kilian Q. Weinberger,
Ryan McDonald
Abstract:
Current approaches for controlling dialogue response generation are primarily focused on high-level attributes like style, sentiment, or topic. In this work, we focus on constrained long-term dialogue generation, which involves more fine-grained control and requires a given set of control words to appear in generated responses. This setting requires a model to not only consider the generation of t…
▽ More
Current approaches for controlling dialogue response generation are primarily focused on high-level attributes like style, sentiment, or topic. In this work, we focus on constrained long-term dialogue generation, which involves more fine-grained control and requires a given set of control words to appear in generated responses. This setting requires a model to not only consider the generation of these control words in the immediate context, but also produce utterances that will encourage the generation of the words at some time in the (possibly distant) future. We define the problem of constrained long-term control for dialogue generation, identify gaps in current methods for evaluation, and propose new metrics that better measure long-term control. We also propose a retrieval-augmented method that improves performance of long-term controlled generation via logit modification techniques. We show through experiments on three task-oriented dialogue datasets that our metrics better assess dialogue control relative to current alternatives and that our method outperforms state-of-the-art constrained generation baselines.
△ Less
Submitted 15 May, 2022;
originally announced May 2022.
-
Multimessenger Search for Sources of Gravitational Waves and High-Energy Neutrinos: Results for Initial LIGO-Virgo and IceCube
Authors:
The IceCube Collaboration,
The LIGO Scientific Collaboration,
The Virgo Collaboration,
M. G. Aartsen,
M. Ackermann,
J. Adams,
J. A. Aguilar,
M. Ahlers,
M. Ahrens,
D. Altmann,
T. Anderson,
C. Arguelles,
T. C. Arlen,
J. Auffenberg,
X. Bai,
S. W. Barwick,
V. Baum,
J. J. Beatty,
J. Becker Tjus,
K. -H. Becker,
S. BenZvi,
P. Berghaus,
D. Berley,
E. Bernardini,
A. Bernhard
, et al. (1166 additional authors not shown)
Abstract:
We report the results of a multimessenger search for coincident signals from the LIGO and Virgo gravitational-wave observatories and the partially completed IceCube high-energy neutrino detector, including periods of joint operation between 2007-2010. These include parts of the 2005-2007 run and the 2009-2010 run for LIGO-Virgo, and IceCube's observation periods with 22, 59 and 79 strings. We find…
▽ More
We report the results of a multimessenger search for coincident signals from the LIGO and Virgo gravitational-wave observatories and the partially completed IceCube high-energy neutrino detector, including periods of joint operation between 2007-2010. These include parts of the 2005-2007 run and the 2009-2010 run for LIGO-Virgo, and IceCube's observation periods with 22, 59 and 79 strings. We find no significant coincident events, and use the search results to derive upper limits on the rate of joint sources for a range of source emission parameters. For the optimistic assumption of gravitational-wave emission energy of $10^{-2}$\,M$_\odot$c$^2$ at $\sim 150$\,Hz with $\sim 60$\,ms duration, and high-energy neutrino emission of $10^{51}$\,erg comparable to the isotropic gamma-ray energy of gamma-ray bursts, we limit the source rate below $1.6 \times 10^{-2}$\,Mpc$^{-3}$yr$^{-1}$. We also examine how combining information from gravitational waves and neutrinos will aid discovery in the advanced gravitational-wave detector era.
△ Less
Submitted 9 October, 2014; v1 submitted 3 July, 2014;
originally announced July 2014.
-
Methods and results of a search for gravitational waves associated with gamma-ray bursts using the GEO600, LIGO, and Virgo detectors
Authors:
The LIGO Scientific Collaboration,
the Virgo Collaboration,
J. Aasi,
B. P. Abbott,
R. Abbott,
T. Abbott,
M. R. Abernathy,
F. Acernese,
K. Ackley,
C. Adams,
T. Adams,
P. Addesso,
R. X. Adhikari,
C. Affeldt,
M. Agathos,
N. Aggarwal,
O. D. Aguiar,
P. Ajith,
A. Alemic,
B. Allen,
A. Allocca,
D. Amariutei,
M. Andersen,
R. A. Anderson,
S. B. Anderson
, et al. (868 additional authors not shown)
Abstract:
In this paper we report on a search for short-duration gravitational wave bursts in the frequency range 64 Hz-1792 Hz associated with gamma-ray bursts (GRBs), using data from GEO600 and one of the LIGO or Virgo detectors. We introduce the method of a linear search grid to analyse GRB events with large sky localisation uncertainties such as the localisations provided by the Fermi Gamma-ray Burst Mo…
▽ More
In this paper we report on a search for short-duration gravitational wave bursts in the frequency range 64 Hz-1792 Hz associated with gamma-ray bursts (GRBs), using data from GEO600 and one of the LIGO or Virgo detectors. We introduce the method of a linear search grid to analyse GRB events with large sky localisation uncertainties such as the localisations provided by the Fermi Gamma-ray Burst Monitor (GBM). Coherent searches for gravitational waves (GWs) can be computationally intensive when the GRB sky position is not well-localised, due to the corrections required for the difference in arrival time between detectors. Using a linear search grid we are able to reduce the computational cost of the analysis by a factor of O(10) for GBM events. Furthermore, we demonstrate that our analysis pipeline can improve upon the sky localisation of GRBs detected by the GBM, if a high-frequency GW signal is observed in coincidence. We use the linear search grid method in a search for GWs associated with 129 GRBs observed satellite-based gamma-ray experiments between 2006 and 2011. The GRBs in our sample had not been previously analysed for GW counterparts. A fraction of our GRB events are analysed using data from GEO600 while the detector was using squeezed-light states to improve its sensitivity; this is the first search for GWs using data from a squeezed-light interferometric observatory. We find no evidence for GW signals, either with any individual GRB in this sample or with the population as a whole. For each GRB we place lower bounds on the distance to the progenitor, assuming a fixed GW emission energy of $10^{-2} M_{\odot}c^{2}$, with a median exclusion distance of 0.8 Mpc for emission at 500 Hz and 0.3 Mpc at 1 kHz. The reduced computational cost associated with a linear search grid will enable rapid searches for GWs associated with Fermi GBM events in the Advanced detector era.
△ Less
Submitted 1 July, 2014; v1 submitted 5 May, 2014;
originally announced May 2014.
-
Search for gravitational waves associated with gamma-ray bursts detected by the InterPlanetary Network
Authors:
The LIGO Scientific Collaboration,
the Virgo Collaboration,
J. Aasi,
B. P. Abbott,
R. Abbott,
T. Abbott,
M. R. Abernathy,
F. Acernese,
K. Ackley,
C. Adams,
T. Adams,
P. Addesso,
R. X. Adhikari,
C. Affeldt,
M. Agathos,
N. Aggarwal,
O. D. Aguiar,
P. Ajith,
A. Alemic,
B. Allen,
A. Allocca,
D. Amariutei,
M. Andersen,
R. A. Anderson,
S. B. Anderson
, et al. (879 additional authors not shown)
Abstract:
We present the results of a search for gravitational waves associated with 223 gamma-ray bursts (GRBs) detected by the InterPlanetary Network (IPN) in 2005-2010 during LIGO's fifth and sixth science runs and Virgo's first, second and third science runs. The IPN satellites provide accurate times of the bursts and sky localizations that vary significantly from degree scale to hundreds of square degr…
▽ More
We present the results of a search for gravitational waves associated with 223 gamma-ray bursts (GRBs) detected by the InterPlanetary Network (IPN) in 2005-2010 during LIGO's fifth and sixth science runs and Virgo's first, second and third science runs. The IPN satellites provide accurate times of the bursts and sky localizations that vary significantly from degree scale to hundreds of square degrees. We search for both a well-modeled binary coalescence signal, the favored progenitor model for short GRBs, and for generic, unmodeled gravitational wave bursts. Both searches use the event time and sky localization to improve the gravitational-wave search sensitivity as compared to corresponding all-time, all-sky searches. We find no evidence of a gravitational-wave signal associated with any of the IPN GRBs in the sample, nor do we find evidence for a population of weak gravitational-wave signals associated with the GRBs. For all IPN-detected GRBs, for which a sufficient duration of quality gravitational-wave data is available, we place lower bounds on the distance to the source in accordance with an optimistic assumption of gravitational-wave emission energy of $10^{-2}M_{\odot}c^2$ at 150 Hz, and find a median of 13 Mpc. For the 27 short-hard GRBs we place 90% confidence exclusion distances to two source models: a binary neutron star coalescence, with a median distance of 12Mpc, or the coalescence of a neutron star and black hole, with a median distance of 22 Mpc. Finally, we combine this search with previously published results to provide a population statement for GRB searches in first-generation LIGO and Virgo gravitational-wave detectors, and a resulting examination of prospects for the advanced gravitational-wave detectors.
△ Less
Submitted 17 April, 2014; v1 submitted 26 March, 2014;
originally announced March 2014.
