-
Multi-harmonic and special shape/pattern/template approximations of discrete signals with generally irregular arguments
Authors:
Ivan L. Andronov,
Hanna M. Akopian,
Vitalii V. Breus,
Lidiia L. Chinarova,
Larysa S. Kudashkina,
Nina V. Savchuk,
Serhii I. Iovchev,
Vladyslava I. Marsakova,
Serhii V. Kolesnikov,
Maksym Yu. Pyatnytskyy
Abstract:
The invited review of own algorithms and software (MAVKA and MCV) for the data analysis of astronomical signals - irregularly spaced, multi-periodic multi-harmonic, periodogram analysis and approximations with taking into account a polynomial trend simultaneously, instead of commonle used detrending and prewhitening. The references to original papers are listed.
The invited review of own algorithms and software (MAVKA and MCV) for the data analysis of astronomical signals - irregularly spaced, multi-periodic multi-harmonic, periodogram analysis and approximations with taking into account a polynomial trend simultaneously, instead of commonle used detrending and prewhitening. The references to original papers are listed.
△ Less
Submitted 15 April, 2025;
originally announced April 2025.
-
White Manin product and Hadamard product
Authors:
P. S. Kolesnikov,
B. K. Sartayev
Abstract:
In this paper, we consider three types of operads: alternative, assosymmetric, and bicommutative. We prove that the Hadamard product of these operads with the Novikov operad coincides with their white Manin product. As an application, we identify a variety of algebras in which all algebras are special.
In this paper, we consider three types of operads: alternative, assosymmetric, and bicommutative. We prove that the Hadamard product of these operads with the Novikov operad coincides with their white Manin product. As an application, we identify a variety of algebras in which all algebras are special.
△ Less
Submitted 27 March, 2025;
originally announced March 2025.
-
Magnetization reversal of finite-length Co and Fe atomic chains on Pt(332) surface: numerical calculations and a new theoretical approach
Authors:
S. V. Kolesnikov,
E. S. Glazova,
A. M. Saletsky
Abstract:
Different mechanisms of magnetization reversal in finite-length Co and Fe chains on the Pt(332) surface have been investigated, taking into account the Dzyaloshinskii-Moriya interaction. It has been found that the magnetization reversal in short atomic chains occurs through the simultaneous reversal of all magnetic moments. In contrast, the magnetization reversal in long atomic chains is facilitat…
▽ More
Different mechanisms of magnetization reversal in finite-length Co and Fe chains on the Pt(332) surface have been investigated, taking into account the Dzyaloshinskii-Moriya interaction. It has been found that the magnetization reversal in short atomic chains occurs through the simultaneous reversal of all magnetic moments. In contrast, the magnetization reversal in long atomic chains is facilitated by the formation of domain walls, which exhibit distinct structures for Co and Fe atomic chains. Using the geodesic nudged elastic band method, we have determined the energy barriers for magnetization reversal in chains consisting of 5 to 100 atoms. Additionally, the frequency prefactors have been calculated within the framework of the harmonic approximation of transition state theory. Notably, the dependencies of these prefactors on chain length and external magnetic field are significant and non-monotonic. We propose a theoretical approach that qualitatively describes the numerical dependencies for both the energy barriers and the frequency prefactors. The magnetization curves derived from our theoretical estimates show qualitative agreement with the results of numerical calculations. This analytical approach enables the estimation of the coercive force of atomic chains across a wide range of lengths, temperatures, sweeping rates, and model parameters. The proposed theoretical framework is applicable not only to the Co and Fe chains on the Pt(332) surface but also to a broad class of one-dimensional magnetic systems.
△ Less
Submitted 9 January, 2025;
originally announced January 2025.
-
On the Dong Property for a binary quadratic operad
Authors:
P. S. Kolesnikov,
B. K. Sartayev
Abstract:
The classical Dong Lemma for distributions over a Lie algebra lies in the foundation of vertex algebras theory. In this paper, we find necessary and sufficient condition for a variety of nonassociative algebras with binary operations to satisfy the analogue of the Dong Lemma. In particular, it turns out that Novikov and Novikov--Poisson algebras satisfy the Dong Lemma. The criterion is stated in t…
▽ More
The classical Dong Lemma for distributions over a Lie algebra lies in the foundation of vertex algebras theory. In this paper, we find necessary and sufficient condition for a variety of nonassociative algebras with binary operations to satisfy the analogue of the Dong Lemma. In particular, it turns out that Novikov and Novikov--Poisson algebras satisfy the Dong Lemma. The criterion is stated in the language of operads, so we determine for which binary quadratic operads the Dong Lemma holds true. As an application, we show the black Manin product of Dong operads is also a Dong operad.
△ Less
Submitted 27 December, 2024;
originally announced December 2024.
-
Revisiting BPR: A Replicability Study of a Common Recommender System Baseline
Authors:
Aleksandr Milogradskii,
Oleg Lashinin,
Alexander P,
Marina Ananyeva,
Sergey Kolesnikov
Abstract:
Bayesian Personalized Ranking (BPR), a collaborative filtering approach based on matrix factorization, frequently serves as a benchmark for recommender systems research. However, numerous studies often overlook the nuances of BPR implementation, claiming that it performs worse than newly proposed methods across various tasks. In this paper, we thoroughly examine the features of the BPR model, indi…
▽ More
Bayesian Personalized Ranking (BPR), a collaborative filtering approach based on matrix factorization, frequently serves as a benchmark for recommender systems research. However, numerous studies often overlook the nuances of BPR implementation, claiming that it performs worse than newly proposed methods across various tasks. In this paper, we thoroughly examine the features of the BPR model, indicating their impact on its performance, and investigate open-source BPR implementations. Our analysis reveals inconsistencies between these implementations and the original BPR paper, leading to a significant decrease in performance of up to 50% for specific implementations. Furthermore, through extensive experiments on real-world datasets under modern evaluation settings, we demonstrate that with proper tuning of its hyperparameters, the BPR model can achieve performance levels close to state-of-the-art methods on the top-n recommendation tasks and even outperform them on specific datasets. Specifically, on the Million Song Dataset, the BPR model with hyperparameters tuning statistically significantly outperforms Mult-VAE by 10% in NDCG@100 with binary relevance function.
△ Less
Submitted 18 October, 2024; v1 submitted 21 September, 2024;
originally announced September 2024.
-
Differential envelopes of Novikov conformal algebras
Authors:
P. S. Kolesnikov,
A. A. Nesterenko
Abstract:
A Novikov conformal algebra is a conformal algebra such that its coefficient algebra is right-symmetric and left commutative (i.e., it is an ``ordinary'' Novikov algebra). We prove that every Novikov conformal algebra with a uniformly bounded locality function on a set of generators can be embedded into a commutative conformal algebra with a derivation. In particular, every finitely generated Novi…
▽ More
A Novikov conformal algebra is a conformal algebra such that its coefficient algebra is right-symmetric and left commutative (i.e., it is an ``ordinary'' Novikov algebra). We prove that every Novikov conformal algebra with a uniformly bounded locality function on a set of generators can be embedded into a commutative conformal algebra with a derivation. In particular, every finitely generated Novikov conformal algebra has a commutative conformal differential envelope. For infinitely generated algebras this statement is not true in general.
△ Less
Submitted 16 September, 2024;
originally announced September 2024.
-
On the locality of formal distributions over pre-Lie and Novikov algebras
Authors:
L. A. Bokut,
P. S. Kolesnikov
Abstract:
The Dong Lemma in the theory of vertex algebras states that the locality property of formal distributions over a Lie algebra is preserved under the action of a vertex operator. A~similar statement is known for associative algebras. We study local formal distributions over pre-Lie (right-symmetric), pre-associative (dendriform), and Novikov algebras to show that the analogue of the Dong Lemma holds…
▽ More
The Dong Lemma in the theory of vertex algebras states that the locality property of formal distributions over a Lie algebra is preserved under the action of a vertex operator. A~similar statement is known for associative algebras. We study local formal distributions over pre-Lie (right-symmetric), pre-associative (dendriform), and Novikov algebras to show that the analogue of the Dong Lemma holds for Novikov algebras but does not hold for pre-Lie and pre-associative ones.
△ Less
Submitted 26 April, 2024;
originally announced April 2024.
-
The Table of the Structure Constants for the Complex Simple Lie Algebra of Type E_6 and Chevalley Commutator Formulas in the Chevalley Group of Type E_6 over a Field
Authors:
Anna I. Polovinkina,
Sergey G. Kolesnikov
Abstract:
This article is the third in the series. It is devoted the calculation of the structure constants for the complex simple Lie algebra of type E_6 and Chevalley commutator formulas.
This article is the third in the series. It is devoted the calculation of the structure constants for the complex simple Lie algebra of type E_6 and Chevalley commutator formulas.
△ Less
Submitted 15 March, 2024;
originally announced March 2024.
-
The Table of the Structure Constants for the Complex Simple Lie Algebra of Type G_2 and Chevalley Commutator Formulas in the Chevalley Group of Type G_2 over a Field
Authors:
Sergey G. Kolesnikov
Abstract:
This article is the second in the series and is devoted to the type G_2. The work consists of two parts. In the first part we calculate the structure constants of the complex simple Lie algebra of type G_2. All structure constants are represented as functions of the structure constants corresponding to extraspecial pairs. The results obtained are used to calculate the commutator Chevalley formulas…
▽ More
This article is the second in the series and is devoted to the type G_2. The work consists of two parts. In the first part we calculate the structure constants of the complex simple Lie algebra of type G_2. All structure constants are represented as functions of the structure constants corresponding to extraspecial pairs. The results obtained are used to calculate the commutator Chevalley formulas for [x_r(u),x_s(y)], when the sum r+s is a root.
Further, in the second part there is a table of structure constants and Chevalley commutator formulas in the special case, when all structure constants corresponding to extraspecial pairs are positive.
△ Less
Submitted 23 December, 2023;
originally announced December 2023.
-
In-Context Reinforcement Learning for Variable Action Spaces
Authors:
Viacheslav Sinii,
Alexander Nikulin,
Vladislav Kurenkov,
Ilya Zisman,
Sergey Kolesnikov
Abstract:
Recently, it has been shown that transformers pre-trained on diverse datasets with multi-episode contexts can generalize to new reinforcement learning tasks in-context. A key limitation of previously proposed models is their reliance on a predefined action space size and structure. The introduction of a new action space often requires data re-collection and model re-training, which can be costly f…
▽ More
Recently, it has been shown that transformers pre-trained on diverse datasets with multi-episode contexts can generalize to new reinforcement learning tasks in-context. A key limitation of previously proposed models is their reliance on a predefined action space size and structure. The introduction of a new action space often requires data re-collection and model re-training, which can be costly for some applications. In our work, we show that it is possible to mitigate this issue by proposing the Headless-AD model that, despite being trained only once, is capable of generalizing to discrete action spaces of variable size, semantic content and order. By experimenting with Bernoulli and contextual bandits, as well as a gridworld environment, we show that Headless-AD exhibits significant capability to generalize to action spaces it has never encountered, even outperforming specialized models trained for a specific set of actions on several environment configurations. Implementation is available at: https://github.com/corl-team/headless-ad.
△ Less
Submitted 1 July, 2024; v1 submitted 20 December, 2023;
originally announced December 2023.
-
Emergence of In-Context Reinforcement Learning from Noise Distillation
Authors:
Ilya Zisman,
Vladislav Kurenkov,
Alexander Nikulin,
Viacheslav Sinii,
Sergey Kolesnikov
Abstract:
Recently, extensive studies in Reinforcement Learning have been carried out on the ability of transformers to adapt in-context to various environments and tasks. Current in-context RL methods are limited by their strict requirements for data, which needs to be generated by RL agents or labeled with actions from an optimal policy. In order to address this prevalent problem, we propose AD…
▽ More
Recently, extensive studies in Reinforcement Learning have been carried out on the ability of transformers to adapt in-context to various environments and tasks. Current in-context RL methods are limited by their strict requirements for data, which needs to be generated by RL agents or labeled with actions from an optimal policy. In order to address this prevalent problem, we propose AD$^\varepsilon$, a new data acquisition approach that enables in-context Reinforcement Learning from noise-induced curriculum. We show that it is viable to construct a synthetic noise injection curriculum which helps to obtain learning histories. Moreover, we experimentally demonstrate that it is possible to alleviate the need for generation using optimal policies, with in-context RL still able to outperform the best suboptimal policy in a learning dataset by a 2x margin.
△ Less
Submitted 12 June, 2024; v1 submitted 19 December, 2023;
originally announced December 2023.
-
XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAX
Authors:
Alexander Nikulin,
Vladislav Kurenkov,
Ilya Zisman,
Artem Agarkov,
Viacheslav Sinii,
Sergey Kolesnikov
Abstract:
Inspired by the diversity and depth of XLand and the simplicity and minimalism of MiniGrid, we present XLand-MiniGrid, a suite of tools and grid-world environments for meta-reinforcement learning research. Written in JAX, XLand-MiniGrid is designed to be highly scalable and can potentially run on GPU or TPU accelerators, democratizing large-scale experimentation with limited resources. Along with…
▽ More
Inspired by the diversity and depth of XLand and the simplicity and minimalism of MiniGrid, we present XLand-MiniGrid, a suite of tools and grid-world environments for meta-reinforcement learning research. Written in JAX, XLand-MiniGrid is designed to be highly scalable and can potentially run on GPU or TPU accelerators, democratizing large-scale experimentation with limited resources. Along with the environments, XLand-MiniGrid provides pre-sampled benchmarks with millions of unique tasks of varying difficulty and easy-to-use baselines that allow users to quickly start training adaptive agents. In addition, we have conducted a preliminary analysis of scaling and generalization, showing that our baselines are capable of reaching millions of steps per second during training and validating that the proposed benchmarks are challenging. XLand-MiniGrid is open-source and available at https://github.com/dunnolab/xland-minigrid.
△ Less
Submitted 19 November, 2024; v1 submitted 19 December, 2023;
originally announced December 2023.
-
Identity Curvature Laplace Approximation for Improved Out-of-Distribution Detection
Authors:
Maksim Zhdanov,
Stanislav Dereka,
Sergey Kolesnikov
Abstract:
Uncertainty estimation is crucial in safety-critical applications, where robust out-of-distribution (OOD) detection is essential. Traditional Bayesian methods, though effective, are often hindered by high computational demands. As an alternative, Laplace approximation offers a more practical and efficient approach to uncertainty estimation. In this paper, we introduce the Identity Curvature Laplac…
▽ More
Uncertainty estimation is crucial in safety-critical applications, where robust out-of-distribution (OOD) detection is essential. Traditional Bayesian methods, though effective, are often hindered by high computational demands. As an alternative, Laplace approximation offers a more practical and efficient approach to uncertainty estimation. In this paper, we introduce the Identity Curvature Laplace Approximation (ICLA), a novel method that challenges the conventional posterior covariance formulation by using identity curvature and optimizing prior precision. This innovative design significantly enhances OOD detection performance on well-known datasets such as CIFAR-10, CIFAR-100, and ImageNet, while maintaining calibration scores. We attribute this improvement to the alignment issues between typical feature embeddings and curvature as measured by the Fisher information matrix. Our findings are further supported by demonstrating that incorporating Fisher penalty or sharpness-aware minimization techniques can greatly enhance the uncertainty estimation capabilities of standard Laplace approximation.
△ Less
Submitted 5 November, 2024; v1 submitted 16 December, 2023;
originally announced December 2023.
-
The Table of the Structure Constants for the Complex Simple Lie Algebra of Type F_4 and its Application to the Calculation of Commutators in the Chevalley Group of Type F_4 over Fields and Rings
Authors:
Sergey G. Kolesnikov,
Anna I. Polovinkina
Abstract:
This work is the first in a series of papers devoted to constructing tables of structure constants for the complex simple Lie algebras and to finding an explicit form of Chevalley commutator formulas.
The work consists of three parts. In the first part, expressions are found for the structure constants of the complex simple Lie algebra of type F_4 in the form of functions of structure constants…
▽ More
This work is the first in a series of papers devoted to constructing tables of structure constants for the complex simple Lie algebras and to finding an explicit form of Chevalley commutator formulas.
The work consists of three parts. In the first part, expressions are found for the structure constants of the complex simple Lie algebra of type F_4 in the form of functions of structure constants corresponding to extraspecial pairs of roots. As a consequence, all Chevalley commutator formulas [x_r(u),x_s(y)] are calculated when the sum r+s is a root.
Further, in the second part, tables of structure constants and Chevalley commutator formulas are given in the special case when all constants corresponding to extraspecial pairs are equal to one.
Finally, in the third part, directed and weighted graphs associated with root systems are constructed. It is shown that the elements of the exponent of the adjacency matrices of directed graphs are the numbers P_{rs}, where P_{rs} is the number of representations of the root r in the form of a sum of the root s and fundamental roots such that any initial segment of the sum is a root. It is also shown that the elements of an exponent of the weight matrix of a weighted graph are the values of sums arising when calculating complex commutators in Chevalley groups.
△ Less
Submitted 6 December, 2023;
originally announced December 2023.
-
Wild-Tab: A Benchmark For Out-Of-Distribution Generalization In Tabular Regression
Authors:
Sergey Kolesnikov
Abstract:
Out-of-Distribution (OOD) generalization, a cornerstone for building robust machine learning models capable of handling data diverging from the training set's distribution, is an ongoing challenge in deep learning. While significant progress has been observed in computer vision and natural language processing, its exploration in tabular data, ubiquitous in many industrial applications, remains nas…
▽ More
Out-of-Distribution (OOD) generalization, a cornerstone for building robust machine learning models capable of handling data diverging from the training set's distribution, is an ongoing challenge in deep learning. While significant progress has been observed in computer vision and natural language processing, its exploration in tabular data, ubiquitous in many industrial applications, remains nascent. To bridge this gap, we present Wild-Tab, a large-scale benchmark tailored for OOD generalization in tabular regression tasks. The benchmark incorporates 3 industrial datasets sourced from fields like weather prediction and power consumption estimation, providing a challenging testbed for evaluating OOD performance under real-world conditions. Our extensive experiments, evaluating 10 distinct OOD generalization methods on Wild-Tab, reveal nuanced insights. We observe that many of these methods often struggle to maintain high-performance levels on unseen data, with OOD performance showing a marked drop compared to in-distribution performance. At the same time, Empirical Risk Minimization (ERM), despite its simplicity, delivers robust performance across all evaluations, rivaling the results of state-of-the-art methods. Looking forward, we hope that the release of Wild-Tab will facilitate further research on OOD generalization and aid in the deployment of machine learning models in various real-world contexts where handling distribution shifts is a crucial requirement.
△ Less
Submitted 4 December, 2023;
originally announced December 2023.
-
Groebner--Shirshov bases method for vertex algebras
Authors:
R. A. Kozlov,
P. S. Kolesnikov
Abstract:
In this note we show how to apply the Gröbner--Shirshov bases (GSB) method for modules over an associative algebra to the study of vertex algebras defined by generators and relations. We compute GSBs for a series of vertex algebras and study the problem of embedding of a left-symmetric algebra into a vertex one preserving the normally ordered product.
In this note we show how to apply the Gröbner--Shirshov bases (GSB) method for modules over an associative algebra to the study of vertex algebras defined by generators and relations. We compute GSBs for a series of vertex algebras and study the problem of embedding of a left-symmetric algebra into a vertex one preserving the normally ordered product.
△ Less
Submitted 4 December, 2023;
originally announced December 2023.
-
Time-Aware Item Weighting for the Next Basket Recommendations
Authors:
Aleksey Romanov,
Oleg Lashinin,
Marina Ananyeva,
Sergey Kolesnikov
Abstract:
In this paper we study the next basket recommendation problem. Recent methods use different approaches to achieve better performance. However, many of them do not use information about the time of prediction and time intervals between baskets. To fill this gap, we propose a novel method, Time-Aware Item-based Weighting (TAIW), which takes timestamps and intervals into account. We provide experimen…
▽ More
In this paper we study the next basket recommendation problem. Recent methods use different approaches to achieve better performance. However, many of them do not use information about the time of prediction and time intervals between baskets. To fill this gap, we propose a novel method, Time-Aware Item-based Weighting (TAIW), which takes timestamps and intervals into account. We provide experiments on three real-world datasets, and TAIW outperforms well-tuned state-of-the-art baselines for next-basket recommendations. In addition, we show the results of an ablation study and a case study of a few items.
△ Less
Submitted 30 July, 2023;
originally announced July 2023.
-
RecBaselines2023: a new dataset for choosing baselines for recommender models
Authors:
Veronika Ivanova,
Oleg Lashinin,
Marina Ananyeva,
Sergey Kolesnikov
Abstract:
The number of proposed recommender algorithms continues to grow. The authors propose new approaches and compare them with existing models, called baselines. Due to the large number of recommender models, it is difficult to estimate which algorithms to choose in the article. To solve this problem, we have collected and published a dataset containing information about the recommender models used in…
▽ More
The number of proposed recommender algorithms continues to grow. The authors propose new approaches and compare them with existing models, called baselines. Due to the large number of recommender models, it is difficult to estimate which algorithms to choose in the article. To solve this problem, we have collected and published a dataset containing information about the recommender models used in 903 papers, both as baselines and as proposed approaches. This dataset can be seen as a typical dataset with interactions between papers and previously proposed models. In addition, we provide a descriptive analysis of the dataset and highlight possible challenges to be investigated with the data. Furthermore, we have conducted extensive experiments using a well-established methodology to build a good recommender algorithm under the dataset. Our experiments show that the selection of the best baselines for proposing new recommender approaches can be considered and successfully solved by existing state-of-the-art collaborative filtering models. Finally, we discuss limitations and future work.
△ Less
Submitted 25 June, 2023;
originally announced June 2023.
-
Katakomba: Tools and Benchmarks for Data-Driven NetHack
Authors:
Vladislav Kurenkov,
Alexander Nikulin,
Denis Tarasov,
Sergey Kolesnikov
Abstract:
NetHack is known as the frontier of reinforcement learning research where learning-based methods still need to catch up to rule-based solutions. One of the promising directions for a breakthrough is using pre-collected datasets similar to recent developments in robotics, recommender systems, and more under the umbrella of offline reinforcement learning (ORL). Recently, a large-scale NetHack datase…
▽ More
NetHack is known as the frontier of reinforcement learning research where learning-based methods still need to catch up to rule-based solutions. One of the promising directions for a breakthrough is using pre-collected datasets similar to recent developments in robotics, recommender systems, and more under the umbrella of offline reinforcement learning (ORL). Recently, a large-scale NetHack dataset was released; while it was a necessary step forward, it has yet to gain wide adoption in the ORL community. In this work, we argue that there are three major obstacles for adoption: resource-wise, implementation-wise, and benchmark-wise. To address them, we develop an open-source library that provides workflow fundamentals familiar to the ORL community: pre-defined D4RL-style tasks, uncluttered baseline implementations, and reliable evaluation tools with accompanying configs and logs synced to the cloud.
△ Less
Submitted 26 October, 2023; v1 submitted 14 June, 2023;
originally announced June 2023.
-
Diversifying Deep Ensembles: A Saliency Map Approach for Enhanced OOD Detection, Calibration, and Accuracy
Authors:
Stanislav Dereka,
Ivan Karpukhin,
Maksim Zhdanov,
Sergey Kolesnikov
Abstract:
Deep ensembles are capable of achieving state-of-the-art results in classification and out-of-distribution (OOD) detection. However, their effectiveness is limited due to the homogeneity of learned patterns within ensembles. To overcome this issue, our study introduces Saliency Diversified Deep Ensemble (SDDE), a novel approach that promotes diversity among ensemble members by leveraging saliency…
▽ More
Deep ensembles are capable of achieving state-of-the-art results in classification and out-of-distribution (OOD) detection. However, their effectiveness is limited due to the homogeneity of learned patterns within ensembles. To overcome this issue, our study introduces Saliency Diversified Deep Ensemble (SDDE), a novel approach that promotes diversity among ensemble members by leveraging saliency maps. Through incorporating saliency map diversification, our method outperforms conventional ensemble techniques and improves calibration in multiple classification and OOD detection tasks. In particular, the proposed method achieves state-of-the-art OOD detection quality, calibration, and accuracy on multiple benchmarks, including CIFAR10/100 and large-scale ImageNet datasets.
△ Less
Submitted 5 November, 2024; v1 submitted 19 May, 2023;
originally announced May 2023.
-
Revisiting the Minimalist Approach to Offline Reinforcement Learning
Authors:
Denis Tarasov,
Vladislav Kurenkov,
Alexander Nikulin,
Sergey Kolesnikov
Abstract:
Recent years have witnessed significant advancements in offline reinforcement learning (RL), resulting in the development of numerous algorithms with varying degrees of complexity. While these algorithms have led to noteworthy improvements, many incorporate seemingly minor design choices that impact their effectiveness beyond core algorithmic advances. However, the effect of these design choices o…
▽ More
Recent years have witnessed significant advancements in offline reinforcement learning (RL), resulting in the development of numerous algorithms with varying degrees of complexity. While these algorithms have led to noteworthy improvements, many incorporate seemingly minor design choices that impact their effectiveness beyond core algorithmic advances. However, the effect of these design choices on established baselines remains understudied. In this work, we aim to bridge this gap by conducting a retrospective analysis of recent works in offline RL and propose ReBRAC, a minimalistic algorithm that integrates such design elements built on top of the TD3+BC method. We evaluate ReBRAC on 51 datasets with both proprioceptive and visual state spaces using D4RL and V-D4RL benchmarks, demonstrating its state-of-the-art performance among ensemble-free methods in both offline and offline-to-online settings. To further illustrate the efficacy of these design choices, we perform a large-scale ablation study and hyperparameter sensitivity analysis on the scale of thousands of experiments.
△ Less
Submitted 24 October, 2023; v1 submitted 16 May, 2023;
originally announced May 2023.
-
Anti-Exploration by Random Network Distillation
Authors:
Alexander Nikulin,
Vladislav Kurenkov,
Denis Tarasov,
Sergey Kolesnikov
Abstract:
Despite the success of Random Network Distillation (RND) in various domains, it was shown as not discriminative enough to be used as an uncertainty estimator for penalizing out-of-distribution actions in offline reinforcement learning. In this paper, we revisit these results and show that, with a naive choice of conditioning for the RND prior, it becomes infeasible for the actor to effectively min…
▽ More
Despite the success of Random Network Distillation (RND) in various domains, it was shown as not discriminative enough to be used as an uncertainty estimator for penalizing out-of-distribution actions in offline reinforcement learning. In this paper, we revisit these results and show that, with a naive choice of conditioning for the RND prior, it becomes infeasible for the actor to effectively minimize the anti-exploration bonus and discriminativity is not an issue. We show that this limitation can be avoided with conditioning based on Feature-wise Linear Modulation (FiLM), resulting in a simple and efficient ensemble-free algorithm based on Soft Actor-Critic. We evaluate it on the D4RL benchmark, showing that it is capable of achieving performance comparable to ensemble-based methods and outperforming ensemble-free approaches by a wide margin.
△ Less
Submitted 17 May, 2023; v1 submitted 31 January, 2023;
originally announced January 2023.
-
Let Offline RL Flow: Training Conservative Agents in the Latent Space of Normalizing Flows
Authors:
Dmitriy Akimov,
Vladislav Kurenkov,
Alexander Nikulin,
Denis Tarasov,
Sergey Kolesnikov
Abstract:
Offline reinforcement learning aims to train a policy on a pre-recorded and fixed dataset without any additional environment interactions. There are two major challenges in this setting: (1) extrapolation error caused by approximating the value of state-action pairs not well-covered by the training data and (2) distributional shift between behavior and inference policies. One way to tackle these p…
▽ More
Offline reinforcement learning aims to train a policy on a pre-recorded and fixed dataset without any additional environment interactions. There are two major challenges in this setting: (1) extrapolation error caused by approximating the value of state-action pairs not well-covered by the training data and (2) distributional shift between behavior and inference policies. One way to tackle these problems is to induce conservatism - i.e., keeping the learned policies closer to the behavioral ones. To achieve this, we build upon recent works on learning policies in latent action spaces and use a special form of Normalizing Flows for constructing a generative model, which we use as a conservative action encoder. This Normalizing Flows action encoder is pre-trained in a supervised manner on the offline dataset, and then an additional policy model - controller in the latent space - is trained via reinforcement learning. This approach avoids querying actions outside of the training dataset and therefore does not require additional regularization for out-of-dataset actions. We evaluate our method on various locomotion and navigation tasks, demonstrating that our approach outperforms recently proposed algorithms with generative action models on a large portion of datasets.
△ Less
Submitted 30 January, 2023; v1 submitted 20 November, 2022;
originally announced November 2022.
-
Q-Ensemble for Offline RL: Don't Scale the Ensemble, Scale the Batch Size
Authors:
Alexander Nikulin,
Vladislav Kurenkov,
Denis Tarasov,
Dmitry Akimov,
Sergey Kolesnikov
Abstract:
Training large neural networks is known to be time-consuming, with the learning duration taking days or even weeks. To address this problem, large-batch optimization was introduced. This approach demonstrated that scaling mini-batch sizes with appropriate learning rate adjustments can speed up the training process by orders of magnitude. While long training time was not typically a major issue for…
▽ More
Training large neural networks is known to be time-consuming, with the learning duration taking days or even weeks. To address this problem, large-batch optimization was introduced. This approach demonstrated that scaling mini-batch sizes with appropriate learning rate adjustments can speed up the training process by orders of magnitude. While long training time was not typically a major issue for model-free deep offline RL algorithms, recently introduced Q-ensemble methods achieving state-of-the-art performance made this issue more relevant, notably extending the training duration. In this work, we demonstrate how this class of methods can benefit from large-batch optimization, which is commonly overlooked by the deep offline RL community. We show that scaling the mini-batch size and naively adjusting the learning rate allows for (1) a reduced size of the Q-ensemble, (2) stronger penalization of out-of-distribution actions, and (3) improved convergence time, effectively shortening training duration by 3-4x times on average.
△ Less
Submitted 30 January, 2023; v1 submitted 20 November, 2022;
originally announced November 2022.
-
CORL: Research-oriented Deep Offline Reinforcement Learning Library
Authors:
Denis Tarasov,
Alexander Nikulin,
Dmitry Akimov,
Vladislav Kurenkov,
Sergey Kolesnikov
Abstract:
CORL is an open-source library that provides thoroughly benchmarked single-file implementations of both deep offline and offline-to-online reinforcement learning algorithms. It emphasizes a simple developing experience with a straightforward codebase and a modern analysis tracking tool. In CORL, we isolate methods implementation into separate single files, making performance-relevant details easie…
▽ More
CORL is an open-source library that provides thoroughly benchmarked single-file implementations of both deep offline and offline-to-online reinforcement learning algorithms. It emphasizes a simple developing experience with a straightforward codebase and a modern analysis tracking tool. In CORL, we isolate methods implementation into separate single files, making performance-relevant details easier to recognize. Additionally, an experiment tracking feature is available to help log metrics, hyperparameters, dependencies, and more to the cloud. Finally, we have ensured the reliability of the implementations by benchmarking commonly employed D4RL datasets providing a transparent source of results that can be reused for robust evaluation tools such as performance profiles, probability of improvement, or expected online performance.
△ Less
Submitted 26 October, 2023; v1 submitted 13 October, 2022;
originally announced October 2022.
-
Deep Image Retrieval is not Robust to Label Noise
Authors:
Stanislav Dereka,
Ivan Karpukhin,
Sergey Kolesnikov
Abstract:
Large-scale datasets are essential for the success of deep learning in image retrieval. However, manual assessment errors and semi-supervised annotation techniques can lead to label noise even in popular datasets. As previous works primarily studied annotation quality in image classification tasks, it is still unclear how label noise affects deep learning approaches to image retrieval. In this wor…
▽ More
Large-scale datasets are essential for the success of deep learning in image retrieval. However, manual assessment errors and semi-supervised annotation techniques can lead to label noise even in popular datasets. As previous works primarily studied annotation quality in image classification tasks, it is still unclear how label noise affects deep learning approaches to image retrieval. In this work, we show that image retrieval methods are less robust to label noise than image classification ones. Furthermore, we, for the first time, investigate different types of label noise specific to image retrieval tasks and study their effect on model performance.
△ Less
Submitted 23 May, 2022;
originally announced May 2022.
-
EXACT: How to Train Your Accuracy
Authors:
Ivan Karpukhin,
Stanislav Dereka,
Sergey Kolesnikov
Abstract:
Classification tasks are usually evaluated in terms of accuracy. However, accuracy is discontinuous and cannot be directly optimized using gradient ascent. Popular methods minimize cross-entropy, hinge loss, or other surrogate losses, which can lead to suboptimal results. In this paper, we propose a new optimization framework by introducing stochasticity to a model's output and optimizing expected…
▽ More
Classification tasks are usually evaluated in terms of accuracy. However, accuracy is discontinuous and cannot be directly optimized using gradient ascent. Popular methods minimize cross-entropy, hinge loss, or other surrogate losses, which can lead to suboptimal results. In this paper, we propose a new optimization framework by introducing stochasticity to a model's output and optimizing expected accuracy, i.e. accuracy of the stochastic model. Extensive experiments on linear models and deep image classification show that the proposed optimization method is a powerful alternative to widely used classification losses.
△ Less
Submitted 24 July, 2024; v1 submitted 19 May, 2022;
originally announced May 2022.
-
CVTT: Cross-Validation Through Time
Authors:
Mikhail Andronov,
Sergey Kolesnikov
Abstract:
The evaluation of recommender systems from a practical perspective is a topic of ongoing discourse within the research community. While many current evaluation methods reduce performance to a single value metric as an easy way to compare models, it relies on the assumption that the methods' performance remains constant over time. In this study, we examine this assumption and propose the Cross-Vali…
▽ More
The evaluation of recommender systems from a practical perspective is a topic of ongoing discourse within the research community. While many current evaluation methods reduce performance to a single value metric as an easy way to compare models, it relies on the assumption that the methods' performance remains constant over time. In this study, we examine this assumption and propose the Cross-Validation Thought Time (CVTT) technique as a more comprehensive evaluation method, focusing on model performance over time. By utilizing the proposed technique, we conduct an in-depth analysis of the performance of popular RecSys algorithms. Our findings indicate that (1) the performance of the recommenders varies over time for all reviewed datasets, (2) using simple evaluation approaches can lead to a substantial decrease in performance in real-world evaluation scenarios, and (3) excessive data usage can lead to suboptimal results.
△ Less
Submitted 10 February, 2023; v1 submitted 11 May, 2022;
originally announced May 2022.
-
Probabilistic Embeddings Revisited
Authors:
Ivan Karpukhin,
Stanislav Dereka,
Sergey Kolesnikov
Abstract:
In recent years, deep metric learning and its probabilistic extensions claimed state-of-the-art results in the face verification task. Despite improvements in face verification, probabilistic methods received little attention in the research community and practical applications. In this paper, we, for the first time, perform an in-depth analysis of known probabilistic methods in verification and r…
▽ More
In recent years, deep metric learning and its probabilistic extensions claimed state-of-the-art results in the face verification task. Despite improvements in face verification, probabilistic methods received little attention in the research community and practical applications. In this paper, we, for the first time, perform an in-depth analysis of known probabilistic methods in verification and retrieval tasks. We study different design choices and propose a simple extension, achieving new state-of-the-art results among probabilistic methods. Finally, we study confidence prediction and show that it correlates with data quality, but contains little information about prediction error probability. We thus provide a new confidence evaluation benchmark and establish a baseline for future confidence prediction research. PyTorch implementation is publicly released.
△ Less
Submitted 10 November, 2022; v1 submitted 14 February, 2022;
originally announced February 2022.
-
High time resolution broad-band polarimetry: technique, calibration and standards
Authors:
V. Breus,
S. V. Kolesnikov,
I. L. Andronov
Abstract:
Regular large-scale polarimetric observations in Crimean astrophysical observatory began in the early 1960s. In 2002 - 2017 the single-channel aperture photometer-polarimeter with a quarter-wave plate at the 2.6-m Shajn mirror telescope (SMT) was used. We accumulated a large homogeneous data set of polarimetric observations of different types of objects that are to be published separately. Correct…
▽ More
Regular large-scale polarimetric observations in Crimean astrophysical observatory began in the early 1960s. In 2002 - 2017 the single-channel aperture photometer-polarimeter with a quarter-wave plate at the 2.6-m Shajn mirror telescope (SMT) was used. We accumulated a large homogeneous data set of polarimetric observations of different types of objects that are to be published separately. Correct polarimetric data processing requires high polarization standards and zero-polarization stars. We aim to improve the data reduction and calibration process to obtain further results with highest possible accuracy. High time resolution broad-band (WR, R, V, B, U) polarization observations are made of 98 known standard stars (527 time series with total duration about 184 hours). We determined values of linear and circular polarization for 98 nearby Northern bright stars. This catalogue is not compilative, but obtained using the same instrument and technique during large time interval. It will be used for our future research and it may be used by other authors. We implemented the least squares approach for determination of the Stokes parameters. It allowed us to obtain results with the accuracy better then we obtained using previously used methods. We report suspicious or variable stars that are not suitable as standards for high precision polarimetry.
△ Less
Submitted 22 December, 2021;
originally announced December 2021.
-
Next Period Recommendation Reality Check
Authors:
Sergey Kolesnikov,
Oleg Lashinin,
Michail Pechatov,
Alexander Kosov
Abstract:
Over the past decade, tremendous progress has been made in Recommender Systems (RecSys) for well-known tasks such as next-item and next-basket prediction. On the other hand, the recently proposed next-period recommendation (NPR) task is not covered as much. Current works about NPR are mostly based around distinct problem formulations, methods, and proprietary datasets, making solutions difficult t…
▽ More
Over the past decade, tremendous progress has been made in Recommender Systems (RecSys) for well-known tasks such as next-item and next-basket prediction. On the other hand, the recently proposed next-period recommendation (NPR) task is not covered as much. Current works about NPR are mostly based around distinct problem formulations, methods, and proprietary datasets, making solutions difficult to reproduce. In this article, we aim to fill the gap in RecSys methods evaluation on the NPR task using publicly available datasets and (1) introduce the TTRS, a large-scale financial transactions dataset suitable for RecSys methods evaluation; (2) benchmark popular RecSys approaches on several datasets for the NPR task. When performing our analysis, we found a strong repetitive consumption pattern in several real-world datasets. With this setup, our results suggest that the repetitive nature of data is still hard to generalize for the evaluated RecSys methods, and novel item prediction performance is still questionable.
△ Less
Submitted 20 December, 2022; v1 submitted 11 October, 2021;
originally announced October 2021.
-
Showing Your Offline Reinforcement Learning Work: Online Evaluation Budget Matters
Authors:
Vladislav Kurenkov,
Sergey Kolesnikov
Abstract:
In this work, we argue for the importance of an online evaluation budget for a reliable comparison of deep offline RL algorithms. First, we delineate that the online evaluation budget is problem-dependent, where some problems allow for less but others for more. And second, we demonstrate that the preference between algorithms is budget-dependent across a diverse range of decision-making domains su…
▽ More
In this work, we argue for the importance of an online evaluation budget for a reliable comparison of deep offline RL algorithms. First, we delineate that the online evaluation budget is problem-dependent, where some problems allow for less but others for more. And second, we demonstrate that the preference between algorithms is budget-dependent across a diverse range of decision-making domains such as Robotics, Finance, and Energy Management. Following the points above, we suggest reporting the performance of deep offline RL algorithms under varying online evaluation budgets. To facilitate this, we propose to use a reporting tool from the NLP field, Expected Validation Performance. This technique makes it possible to reliably estimate expected maximum performance under different budgets while not requiring any additional computation beyond hyperparameter search. By employing this tool, we also show that Behavioral Cloning is often more favorable to offline RL algorithms when working within a limited budget.
△ Less
Submitted 5 June, 2022; v1 submitted 8 October, 2021;
originally announced October 2021.
-
LRWR: Large-Scale Benchmark for Lip Reading in Russian language
Authors:
Evgeniy Egorov,
Vasily Kostyumov,
Mikhail Konyk,
Sergey Kolesnikov
Abstract:
Lipreading, also known as visual speech recognition, aims to identify the speech content from videos by analyzing the visual deformations of lips and nearby areas. One of the significant obstacles for research in this field is the lack of proper datasets for a wide variety of languages: so far, these methods have been focused only on English or Chinese. In this paper, we introduce a naturally dist…
▽ More
Lipreading, also known as visual speech recognition, aims to identify the speech content from videos by analyzing the visual deformations of lips and nearby areas. One of the significant obstacles for research in this field is the lack of proper datasets for a wide variety of languages: so far, these methods have been focused only on English or Chinese. In this paper, we introduce a naturally distributed large-scale benchmark for lipreading in Russian language, named LRWR, which contains 235 classes and 135 speakers. We provide a detailed description of the dataset collection pipeline and dataset statistics. We also present a comprehensive comparison of the current popular lipreading methods on LRWR and conduct a detailed analysis of their performance. The results demonstrate the differences between the benchmarked languages and provide several promising directions for lipreading models finetuning. Thanks to our findings, we also achieved new state-of-the-art results on the LRW benchmark.
△ Less
Submitted 14 September, 2021;
originally announced September 2021.
-
An improved kinetic Monte Carlo model for computational and analytical investigations of the magnetic properties of finite-size atomic chains
Authors:
Sergey Kolesnikov
Abstract:
Two improved kMC models for investigations of the magnetic properties of finite-size atomic chains are presented. These models take the possible noncollinearity of magnetic moments into account. The spontaneous remagnetization of ferromagnetic Co chains on Pt(997) surface and antiferromagnetic Fe chains on $\text{Cu}_2\text{N/Cu(001)}$ surface is investigated in the framework of our models. The re…
▽ More
Two improved kMC models for investigations of the magnetic properties of finite-size atomic chains are presented. These models take the possible noncollinearity of magnetic moments into account. The spontaneous remagnetization of ferromagnetic Co chains on Pt(997) surface and antiferromagnetic Fe chains on $\text{Cu}_2\text{N/Cu(001)}$ surface is investigated in the framework of our models. The results are compared with the results of the simple kMC model. It is also shown that a single domain-wall approximation can be successfully used to estimation of the reversal time of the magnetization. Therefore, the improved kMC models can be used for analytical calculations as well as for computer simulations.
△ Less
Submitted 25 August, 2021;
originally announced August 2021.
-
On the embedding of left-symmetric algebras into differential perm-algebras
Authors:
P. S. Kolesnikov,
B. K. Sartayev
Abstract:
Given an associative algebra satisfying the left commutativity identity $abc=bac$ (Perm-algebra) with a derivation $d$, the new operation $a\circ b = a d(b)$ is left-symmetric (pre-Lie). We derive necessary and sufficient conditions for a left-symmetric algebra to be embeddable into a differential Perm-algebra.
Given an associative algebra satisfying the left commutativity identity $abc=bac$ (Perm-algebra) with a derivation $d$, the new operation $a\circ b = a d(b)$ is left-symmetric (pre-Lie). We derive necessary and sufficient conditions for a left-symmetric algebra to be embeddable into a differential Perm-algebra.
△ Less
Submitted 1 June, 2021;
originally announced June 2021.
-
On the special identities of Gelfand--Dorfman algebras
Authors:
P. S. Kolesnikov,
B. K. Sartayev
Abstract:
In this paper, we prove that the class of all special Gelfand--Dorfman algebras (GD-algebras) is closed with respect to homomorphisms and thus forms a variety. We also prove that every 2-dimensional GD-algebra is special. For the latter, we give a technical method to find all special identities of GD-algebras and compute the degree 6 component of the Gröbner basis for the shuffle operad constructe…
▽ More
In this paper, we prove that the class of all special Gelfand--Dorfman algebras (GD-algebras) is closed with respect to homomorphisms and thus forms a variety. We also prove that every 2-dimensional GD-algebra is special. For the latter, we give a technical method to find all special identities of GD-algebras and compute the degree 6 component of the Gröbner basis for the shuffle operad constructed on the symmetric operad governing the class of GD-algebras.
△ Less
Submitted 24 October, 2023; v1 submitted 28 May, 2021;
originally announced May 2021.
-
On dimension theory of supermodules, super-rings and superschemes
Authors:
A. N. Zubkov,
P. S. Kolesnikov
Abstract:
We introduce the notion of Krull super-dimension of supermodules over certain super-commutative Noetherian super-rings. We investigate how this notion relates to the notion of odd regular sequence introduced by T.Schmitt and how it behaves with respect to the transition to the graded and bigraded supermodules and super-rings associated with the original ones. We also apply these results to the sup…
▽ More
We introduce the notion of Krull super-dimension of supermodules over certain super-commutative Noetherian super-rings. We investigate how this notion relates to the notion of odd regular sequence introduced by T.Schmitt and how it behaves with respect to the transition to the graded and bigraded supermodules and super-rings associated with the original ones. We also apply these results to the super-dimension theory of superschemes of finite type and their morphisms.
△ Less
Submitted 22 May, 2021;
originally announced May 2021.
-
Defining relations and Gröbner--Shirshov bases of Poisson algebras as of conformal modules
Authors:
P. S. Kolesnikov,
A. S. Panasenko
Abstract:
We study the relation between Poisson algebras and representations of Lie conformal algebras. We establish a setting for the calculation of a Gröbner--Shirshov basis in a module over an associative conformal algebra and apply this technique to Poisson algebras considered as conformal modules over appropriate associative envelopes of current Lie conformal algebras. As a result, we obtain a setting…
▽ More
We study the relation between Poisson algebras and representations of Lie conformal algebras. We establish a setting for the calculation of a Gröbner--Shirshov basis in a module over an associative conformal algebra and apply this technique to Poisson algebras considered as conformal modules over appropriate associative envelopes of current Lie conformal algebras. As a result, we obtain a setting for the calculation of a Gröbner--Shirshov basis in a Poisson algebra.
△ Less
Submitted 14 November, 2020;
originally announced November 2020.
-
Standard bases for the universal associative conformal envelopes of Kac--Moody conformal algebras
Authors:
P. S. Kolesnikov,
R. A. Kozlov
Abstract:
We study the universal enveloping associative conformal algebra for the central extension of a current Lie conformal algebra at the locality level $N=3$. A standard basis of defining relations for this algebra is explicitly calculated. As a corollary, we find a linear basis of the free commutative conformal algebra relative to the locality $N=3$ on the generators.
We study the universal enveloping associative conformal algebra for the central extension of a current Lie conformal algebra at the locality level $N=3$. A standard basis of defining relations for this algebra is explicitly calculated. As a corollary, we find a linear basis of the free commutative conformal algebra relative to the locality $N=3$ on the generators.
△ Less
Submitted 30 September, 2020; v1 submitted 25 September, 2020;
originally announced September 2020.
-
Sample Efficient Ensemble Learning with Catalyst.RL
Authors:
Sergey Kolesnikov,
Valentin Khrulkov
Abstract:
We present Catalyst.RL, an open-source PyTorch framework for reproducible and sample efficient reinforcement learning (RL) research. Main features of Catalyst.RL include large-scale asynchronous distributed training, efficient implementations of various RL algorithms and auxiliary tricks, such as n-step returns, value distributions, hyperbolic reinforcement learning, etc. To demonstrate the effect…
▽ More
We present Catalyst.RL, an open-source PyTorch framework for reproducible and sample efficient reinforcement learning (RL) research. Main features of Catalyst.RL include large-scale asynchronous distributed training, efficient implementations of various RL algorithms and auxiliary tricks, such as n-step returns, value distributions, hyperbolic reinforcement learning, etc. To demonstrate the effectiveness of Catalyst.RL, we applied it to a physics-based reinforcement learning challenge "NeurIPS 2019: Learn to Move -- Walk Around" with the objective to build a locomotion controller for a human musculoskeletal model. The environment is computationally expensive, has a high-dimensional continuous action space and is stochastic. Our team took the 2nd place, capitalizing on the ability of Catalyst.RL to train high-quality and sample-efficient RL agents in only a few hours of training time. The implementation along with experiments is open-sourced so results can be reproduced and novel ideas tried out.
△ Less
Submitted 7 April, 2020; v1 submitted 29 March, 2020;
originally announced March 2020.
-
Quadratic Lie conformal superalgebras related to Novikov superalgebras
Authors:
P. S. Kolesnikov,
R. A. Kozlov,
A. S. Panasenko
Abstract:
We study quadratic Lie conformal superalgebras associated with No\-vikov superalgebras. For every Novikov superalgebra $(V,\circ)$, we construct an enveloping differential Poisson superalgebra $U(V)$ with a derivation $d$ such that $u\circ v = ud(v)$ and $\{u,v\} = u\circ v - (-1)^{|u||v|} v\circ u$ for $u,v\in V$. The latter means that the commutator Gelfand--Dorfman superalgebra of $V$ is specia…
▽ More
We study quadratic Lie conformal superalgebras associated with No\-vikov superalgebras. For every Novikov superalgebra $(V,\circ)$, we construct an enveloping differential Poisson superalgebra $U(V)$ with a derivation $d$ such that $u\circ v = ud(v)$ and $\{u,v\} = u\circ v - (-1)^{|u||v|} v\circ u$ for $u,v\in V$. The latter means that the commutator Gelfand--Dorfman superalgebra of $V$ is special. Next, we prove that every quadratic Lie conformal superalgebra constructed on a finite-dimensional special Gel'fand--Dorfman superalgebra has a finite faithful conformal representation. This statement is a step toward a solution of the following open problem: whether a finite Lie conformal (super)algebra has a finite faithful conformal representation.
△ Less
Submitted 28 December, 2021; v1 submitted 9 December, 2019;
originally announced December 2019.
-
Magnetic properties of the finite-length biatomic chains in the framework of the single domain-wall approximation
Authors:
S. V. Kolesnikov,
I. N. Kolesnikova
Abstract:
A simple analytical method for study the magnetic properties of the finite-length biatomic chains in the framework of Heisenberg model with uniaxial magnetic anisotropy is proposed. The method allows to estimate the reversal time of the magnetization of ferromagnetic and antiferromagnetic biatomic chains. Three cases are considered: the spontaneous remagnetization, the remagnetization under the in…
▽ More
A simple analytical method for study the magnetic properties of the finite-length biatomic chains in the framework of Heisenberg model with uniaxial magnetic anisotropy is proposed. The method allows to estimate the reversal time of the magnetization of ferromagnetic and antiferromagnetic biatomic chains. Three cases are considered: the spontaneous remagnetization, the remagnetization under the interaction with a scanning tunneling microscope, and the remagnetization in the external magnetic field. The applicability limits of the method are discussed. Within its limits of applicability the method gives results which are in perfect agreement with the results of the kinetic Monte Carlo simulations. As the examples, two physical systems are considered: biatomic Fe chains on Cu2N/Cu(001) surface and biatomic Co chains on Pt(997) surface. The presented method is incomparably less time-consuming than the standard kMC simulations, especially in the cases of low temperatures or long chains.
△ Less
Submitted 30 July, 2019;
originally announced July 2019.
-
Gelfand--Dorfman algebras, derived identities, and the Manin product of operads
Authors:
P. S. Kolesnikov,
B. Sartayev,
A. Orazgaliev
Abstract:
Gelfand--Dorfman bialgebras (GD-algebras) are nonassociative systems with two bilinear operations satisfying a series of identities that express Hamiltonian property of an operator in the formal calculus of variations. The paper is devoted to the study of GD-algebras related with differential Poisson algebras. As a byproduct, we obtain a general description of identities that hold for operations…
▽ More
Gelfand--Dorfman bialgebras (GD-algebras) are nonassociative systems with two bilinear operations satisfying a series of identities that express Hamiltonian property of an operator in the formal calculus of variations. The paper is devoted to the study of GD-algebras related with differential Poisson algebras. As a byproduct, we obtain a general description of identities that hold for operations $a\succ b = d(a)b$ and $a\prec b = ad(b)$ on a (non-associative) differential algebra with a derivation~$d$.
△ Less
Submitted 6 March, 2019;
originally announced March 2019.
-
Catalyst.RL: A Distributed Framework for Reproducible RL Research
Authors:
Sergey Kolesnikov,
Oleksii Hrinchuk
Abstract:
Despite the recent progress in deep reinforcement learning field (RL), and, arguably because of it, a large body of work remains to be done in reproducing and carefully comparing different RL algorithms. We present catalyst.RL, an open source framework for RL research with a focus on reproducibility and flexibility. Main features of our library include large-scale asynchronous distributed training…
▽ More
Despite the recent progress in deep reinforcement learning field (RL), and, arguably because of it, a large body of work remains to be done in reproducing and carefully comparing different RL algorithms. We present catalyst.RL, an open source framework for RL research with a focus on reproducibility and flexibility. Main features of our library include large-scale asynchronous distributed training, easy-to-use configuration files with the complete list of hyperparameters for the particular experiments, efficient implementations of various RL algorithms and auxiliary tricks, such as frame stacking, n-step returns, value distributions, etc. To vindicate the usefulness of our framework, we evaluate it on a range of benchmarks in a continuous control, as well as on the task of developing a controller to enable a physiologically-based human model with a prosthetic leg to walk and run. The latter task was introduced at NeurIPS 2018 AI for Prosthetics Challenge, where our team took the 3rd place, capitalizing on the ability of catalyst.RL to train high-quality and sample-efficient RL agents.
△ Less
Submitted 28 February, 2019;
originally announced March 2019.
-
Universal enveloping Poisson conformal algebras
Authors:
P. S. Kolesnikov
Abstract:
Lie conformal algebras are useful tools for studying vertex operator algebras and their representations. In this paper, we establish close relations between Poisson conformal algebras and representations of Lie conformal algebras. We also calculate explicitly Poisson conformal brackets on the associated graded conformal algebras of universal associative conformal envelopes of Virasoro conformal al…
▽ More
Lie conformal algebras are useful tools for studying vertex operator algebras and their representations. In this paper, we establish close relations between Poisson conformal algebras and representations of Lie conformal algebras. We also calculate explicitly Poisson conformal brackets on the associated graded conformal algebras of universal associative conformal envelopes of Virasoro conformal algebra and Neveu--Schwartz conformal superalgebra.
△ Less
Submitted 8 February, 2019;
originally announced February 2019.
-
Artificial Intelligence for Prosthetics - challenge solutions
Authors:
Łukasz Kidziński,
Carmichael Ong,
Sharada Prasanna Mohanty,
Jennifer Hicks,
Sean F. Carroll,
Bo Zhou,
Hongsheng Zeng,
Fan Wang,
Rongzhong Lian,
Hao Tian,
Wojciech Jaśkowski,
Garrett Andersen,
Odd Rune Lykkebø,
Nihat Engin Toklu,
Pranav Shyam,
Rupesh Kumar Srivastava,
Sergey Kolesnikov,
Oleksii Hrinchuk,
Anton Pechenko,
Mattias Ljungström,
Zhen Wang,
Xu Hu,
Zehong Hu,
Minghui Qiu,
Jun Huang
, et al. (25 additional authors not shown)
Abstract:
In the NeurIPS 2018 Artificial Intelligence for Prosthetics challenge, participants were tasked with building a controller for a musculoskeletal model with a goal of matching a given time-varying velocity vector. Top participants were invited to describe their algorithms. In this work, we describe the challenge and present thirteen solutions that used deep reinforcement learning approaches. Many s…
▽ More
In the NeurIPS 2018 Artificial Intelligence for Prosthetics challenge, participants were tasked with building a controller for a musculoskeletal model with a goal of matching a given time-varying velocity vector. Top participants were invited to describe their algorithms. In this work, we describe the challenge and present thirteen solutions that used deep reinforcement learning approaches. Many solutions use similar relaxations and heuristics, such as reward shaping, frame skipping, discretization of the action space, symmetry, and policy blending. However, each team implemented different modifications of the known algorithms by, for example, dividing the task into subtasks, learning low-level control, or by incorporating expert knowledge and using imitation learning.
△ Less
Submitted 6 February, 2019;
originally announced February 2019.
-
Derived identities of differential algebras
Authors:
P. S. Kolesnikov
Abstract:
Suppose $A$ is a not necessarily associative algebra with a derivation $d$. Then $A$ may be considered as a system with two binary operations $\succ $ and $\prec $ defined by $x\succ y = d(x)y$, $x\prec y = xd(y)$, $x,y\in A$. Suppose $A$ satisfies some multi-linear polynomial identities. We show how to find the identities that hold for operations $\prec $ and $\succ $. It turns out that if $A$ be…
▽ More
Suppose $A$ is a not necessarily associative algebra with a derivation $d$. Then $A$ may be considered as a system with two binary operations $\succ $ and $\prec $ defined by $x\succ y = d(x)y$, $x\prec y = xd(y)$, $x,y\in A$. Suppose $A$ satisfies some multi-linear polynomial identities. We show how to find the identities that hold for operations $\prec $ and $\succ $. It turns out that if $A$ belongs to a variety governed by an operad Var then $\succ $ and $\prec $ satisfy the defining relations of the operad Var$\circ $Nov, where $\circ $ is the Manin white product of operads, Nov is the operad of Novikov algebras. Moreover, there are no other identities that hold for operations $\succ $, $\prec $ on an arbitrary differential Var-algebra.
△ Less
Submitted 30 December, 2018;
originally announced December 2018.
-
On the Hochschild cohomologies of associative conformal algebras with a finite faithful representation
Authors:
P. S. Kolesnikov,
R. A. Kozlov
Abstract:
Associative conformal algebras of conformal endomorphisms are of essential importance for the study of finite representations of conformal Lie algebras (Lie vertex algebras). We describe all semisimple algebras of conformal endomorphisms which have the trivial second Hochschild cohomology group with coefficients in every conformal bimodule. As a consequence, we state a complete solution of the rad…
▽ More
Associative conformal algebras of conformal endomorphisms are of essential importance for the study of finite representations of conformal Lie algebras (Lie vertex algebras). We describe all semisimple algebras of conformal endomorphisms which have the trivial second Hochschild cohomology group with coefficients in every conformal bimodule. As a consequence, we state a complete solution of the radical splitting problem in the class of associative conformal algebras with a finite faithful representation.
△ Less
Submitted 30 October, 2018;
originally announced October 2018.
-
Learning to Run challenge solutions: Adapting reinforcement learning methods for neuromusculoskeletal environments
Authors:
Łukasz Kidziński,
Sharada Prasanna Mohanty,
Carmichael Ong,
Zhewei Huang,
Shuchang Zhou,
Anton Pechenko,
Adam Stelmaszczyk,
Piotr Jarosik,
Mikhail Pavlov,
Sergey Kolesnikov,
Sergey Plis,
Zhibo Chen,
Zhizheng Zhang,
Jiale Chen,
Jun Shi,
Zhuobin Zheng,
Chun Yuan,
Zhihui Lin,
Henryk Michalewski,
Piotr Miłoś,
Błażej Osiński,
Andrew Melnik,
Malte Schilling,
Helge Ritter,
Sean Carroll
, et al. (4 additional authors not shown)
Abstract:
In the NIPS 2017 Learning to Run challenge, participants were tasked with building a controller for a musculoskeletal model to make it run as fast as possible through an obstacle course. Top participants were invited to describe their algorithms. In this work, we present eight solutions that used deep reinforcement learning approaches, based on algorithms such as Deep Deterministic Policy Gradient…
▽ More
In the NIPS 2017 Learning to Run challenge, participants were tasked with building a controller for a musculoskeletal model to make it run as fast as possible through an obstacle course. Top participants were invited to describe their algorithms. In this work, we present eight solutions that used deep reinforcement learning approaches, based on algorithms such as Deep Deterministic Policy Gradient, Proximal Policy Optimization, and Trust Region Policy Optimization. Many solutions use similar relaxations and heuristics, such as reward shaping, frame skipping, discretization of the action space, symmetry, and policy blending. However, each of the eight teams implemented different modifications of the known algorithms.
△ Less
Submitted 1 April, 2018;
originally announced April 2018.
-
Fitting of the TB-SMA interatomic potentials for Pt/Cu(111) surface alloy
Authors:
S. A. Dokukin,
S. V. Kolesnikov,
A. M. Saletsky,
A. L. Klavsyuk
Abstract:
In this paper we present new parameters of the TB-SMA interatomic potentials for the Pt/Cu(111) surface alloy. The parameters are fitted using both the experimental and {\it ab initio} data. The potentials reproduce not only the bulk properties of copper and platinum, but also the energy characteristics of the Pt/Cu(111) surface alloy. The potentials can be used for the simulations of the growth o…
▽ More
In this paper we present new parameters of the TB-SMA interatomic potentials for the Pt/Cu(111) surface alloy. The parameters are fitted using both the experimental and {\it ab initio} data. The potentials reproduce not only the bulk properties of copper and platinum, but also the energy characteristics of the Pt/Cu(111) surface alloy. The potentials can be used for the simulations of the growth of the Pt/Cu(111) surface alloy on the atomic scale.
△ Less
Submitted 27 February, 2018;
originally announced February 2018.
