Proximal Curriculum for Reinforcement Learning Agents
Authors:
Georgios Tzannetos,
Bárbara Gomes Ribeiro,
Parameswaran Kamalaruban,
Adish Singla
Abstract:
We consider the problem of curriculum design for reinforcement learning (RL) agents in contextual multi-task settings. Existing techniques on automatic curriculum design typically require domain-specific hyperparameter tuning or have limited theoretical underpinnings. To tackle these limitations, we design our curriculum strategy, ProCuRL, inspired by the pedagogical concept of Zone of Proximal De…
▽ More
We consider the problem of curriculum design for reinforcement learning (RL) agents in contextual multi-task settings. Existing techniques on automatic curriculum design typically require domain-specific hyperparameter tuning or have limited theoretical underpinnings. To tackle these limitations, we design our curriculum strategy, ProCuRL, inspired by the pedagogical concept of Zone of Proximal Development (ZPD). ProCuRL captures the intuition that learning progress is maximized when picking tasks that are neither too hard nor too easy for the learner. We mathematically derive ProCuRL by analyzing two simple learning settings. We also present a practical variant of ProCuRL that can be directly integrated with deep RL frameworks with minimal hyperparameter tuning. Experimental results on a variety of domains demonstrate the effectiveness of our curriculum strategy over state-of-the-art baselines in accelerating the training process of deep RL agents.
△ Less
Submitted 25 April, 2023;
originally announced April 2023.
Analyzing the "Sleeping Giants" Activism Model in Brazil
Authors:
Bárbara Gomes Ribeiro,
Manoel Horta Ribeiro,
Virgílio Almeida,
Wagner Meira Jr
Abstract:
In 2020, amidst the COVID pandemic and a polarized political climate, the Sleeping Giants online activist movement gained traction in Brazil. Its rationale was simple: to curb the spread of misinformation by harming the advertising revenue of sources that produce this type of content. Like its international counterparts, Sleeping Giants Brasil (SGB) campaigned against media outlets using Twitter t…
▽ More
In 2020, amidst the COVID pandemic and a polarized political climate, the Sleeping Giants online activist movement gained traction in Brazil. Its rationale was simple: to curb the spread of misinformation by harming the advertising revenue of sources that produce this type of content. Like its international counterparts, Sleeping Giants Brasil (SGB) campaigned against media outlets using Twitter to ask companies to remove ads from the targeted outlets. This work presents a thorough quantitative characterization of this activism model, analyzing the three campaigns carried out by SGB between May and September 2020. To do so, we use digital traces from both Twitter and Google Trends, toxicity and sentiment classifiers trained for the Portuguese language, and an annotated corpus of SGB's tweets. Our key findings were threefold. First, we found that SGB's requests to companies were largely successful (with 83.85\% of all 192 targeted companies responding positively) and that user pressure was correlated to the speed of companies' responses. Second, there were no significant changes in the online attention and the user engagement going towards the targeted media outlets in the six months that followed SGB's campaign (as measured by Google Trends and Twitter engagement). Third, we observed that user interactions with companies changed only transiently, even if the companies did not respond to SGB's request. Overall, our results paint a nuanced portrait of internet activism. On the one hand, they suggest that SGB was successful in getting companies to boycott specific media outlets, which may have harmed their advertisement revenue stream. On the other hand, they also suggest that the activist movement did not impact the online attention these media outlets received nor the online image of companies that did not respond positively to their requests.
△ Less
Submitted 25 February, 2022; v1 submitted 16 May, 2021;
originally announced May 2021.
