-
AdUE: Improving uncertainty estimation head for LoRA adapters in LLMs
Authors:
Artem Zabolotnyi,
Roman Makarov,
Mile Mitrovic,
Polina Proskura,
Oleg Travkin,
Roman Alferov,
Alexey Zaytsev
Abstract:
Uncertainty estimation remains a critical challenge in adapting pre-trained language models to classification tasks, particularly under parameter-efficient fine-tuning approaches such as adapters. We introduce AdUE1, an efficient post-hoc uncertainty estimation (UE) method, to enhance softmax-based estimates. Our approach (1) uses a differentiable approximation of the maximum function and (2) appl…
▽ More
Uncertainty estimation remains a critical challenge in adapting pre-trained language models to classification tasks, particularly under parameter-efficient fine-tuning approaches such as adapters. We introduce AdUE1, an efficient post-hoc uncertainty estimation (UE) method, to enhance softmax-based estimates. Our approach (1) uses a differentiable approximation of the maximum function and (2) applies additional regularization through L2-SP, anchoring the fine-tuned head weights and regularizing the model. Evaluations on five NLP classification datasets across four language models (RoBERTa, ELECTRA, LLaMA-2, Qwen) demonstrate that our method consistently outperforms established baselines such as Mahalanobis distance and softmax response. Our approach is lightweight (no base-model changes) and produces better-calibrated confidence.
△ Less
Submitted 21 May, 2025;
originally announced May 2025.
-
Supervised Learning based Method for Condition Monitoring of Overhead Line Insulators using Leakage Current Measurement
Authors:
Mile Mitrovic,
Dmitry Titov,
Klim Volkhov,
Irina Lukicheva,
Andrey Kudryavzev,
Petr Vorobev,
Qi Li,
Vladimir Terzija
Abstract:
As a new practical and economical solution to the aging problem of overhead line (OHL) assets, the technical policies of most power grid companies in the world experienced a gradual transition from scheduled preventive maintenance to a risk-based approach in asset management. Even though the accumulation of contamination is predictable within a certain degree, there are currently no effective ways…
▽ More
As a new practical and economical solution to the aging problem of overhead line (OHL) assets, the technical policies of most power grid companies in the world experienced a gradual transition from scheduled preventive maintenance to a risk-based approach in asset management. Even though the accumulation of contamination is predictable within a certain degree, there are currently no effective ways to identify the risk of the insulator flashover in order to plan its replacement. This paper presents a novel machine learning (ML) based method for estimating the flashover probability of the cup-and-pin glass insulator string. The proposed method is based on the Extreme Gradient Boosting (XGBoost) supervised ML model, in which the leakage current (LC) features and applied voltage are used as the inputs. The established model can estimate the critical flashover voltage (U50%) for various designs of OHL insulators with different voltage levels. The proposed method is also able to accurately determine the condition of the insulator strings and instruct asset management engineers to take appropriate actions.
△ Less
Submitted 26 July, 2024;
originally announced July 2024.
-
Data-Driven Stochastic AC-OPF using Gaussian Processes
Authors:
Mile Mitrovic
Abstract:
The thesis focuses on developing a data-driven algorithm, based on machine learning, to solve the stochastic alternating current (AC) chance-constrained (CC) Optimal Power Flow (OPF) problem. Although the AC CC-OPF problem has been successful in academic circles, it is highly nonlinear and computationally demanding, which limits its practical impact. The proposed approach aims to address this limi…
▽ More
The thesis focuses on developing a data-driven algorithm, based on machine learning, to solve the stochastic alternating current (AC) chance-constrained (CC) Optimal Power Flow (OPF) problem. Although the AC CC-OPF problem has been successful in academic circles, it is highly nonlinear and computationally demanding, which limits its practical impact. The proposed approach aims to address this limitation and demonstrate its empirical efficiency through applications to multiple IEEE test cases. To solve the non-convex and computationally challenging CC AC-OPF problem, the proposed approach relies on a machine learning Gaussian process regression (GPR) model. The full Gaussian process (GP) approach is capable of learning a simple yet non-convex data-driven approximation to the AC power flow equations that can incorporate uncertain inputs. The proposed approach uses various approximations for GP-uncertainty propagation. The full GP CC-OPF approach exhibits highly competitive and promising results, outperforming the state-of-the-art sample-based chance constraint approaches. To further improve the robustness and complexity/accuracy trade-off of the full GP CC-OPF, a fast data-driven setup is proposed. This setup relies on the sparse and hybrid Gaussian processes (GP) framework to model the power flow equations with input uncertainty.
△ Less
Submitted 17 February, 2024;
originally announced February 2024.
-
Constructive semigroups with apartness -- a state of the art
Authors:
Melanija Mitrovic,
Mahouton Norbert Hounkonnou,
Paula Catarino
Abstract:
This chapter aims to provide a clear and understandable picture of constructive semigroups with apartness in Bishop's style of constructive mathematics, BISH. Our theory is partly inspired by the classical case, but it is distinguished from it in two significant aspects: we use intuitionistic logic rather than classical throughout; our work is based on the notion of apartness (between elements of…
▽ More
This chapter aims to provide a clear and understandable picture of constructive semigroups with apartness in Bishop's style of constructive mathematics, BISH. Our theory is partly inspired by the classical case, but it is distinguished from it in two significant aspects: we use intuitionistic logic rather than classical throughout; our work is based on the notion of apartness (between elements of the set, and, later, between elements and its subsets). Following Heyting, at least initially, classical semigroup theory is seen as a guide that helps us to develop the constructive theory of semigroups with apartness. To have a structure, we need a set, a relation, and rules establishing how we will put them together. Working within classical or intuitionistic logic, in order to analyze algebraic structures, it is necessary to start with study on sets and ordered sets, relational systems, etc. A comparative analysis between presented classical and constructive results is also a part of this chapter. All proofs can be found in the Appendix.
△ Less
Submitted 24 April, 2023; v1 submitted 4 April, 2023;
originally announced April 2023.
-
Supporting Future Electrical Utilities: Using Deep Learning Methods in EMS and DMS Algorithms
Authors:
Ognjen Kundacina,
Gorana Gojic,
Mile Mitrovic,
Dragisa Miskovic,
Dejan Vukobratovic
Abstract:
Electrical power systems are increasing in size, complexity, as well as dynamics due to the growing integration of renewable energy resources, which have sporadic power generation. This necessitates the development of near real-time power system algorithms, demanding lower computational complexity regarding the power system size. Considering the growing trend in the collection of historical measur…
▽ More
Electrical power systems are increasing in size, complexity, as well as dynamics due to the growing integration of renewable energy resources, which have sporadic power generation. This necessitates the development of near real-time power system algorithms, demanding lower computational complexity regarding the power system size. Considering the growing trend in the collection of historical measurement data and recent advances in the rapidly developing deep learning field, the main goal of this paper is to provide a review of recent deep learning-based power system monitoring and optimization algorithms. Electrical utilities can benefit from this review by re-implementing or enhancing the algorithms traditionally used in energy management systems (EMS) and distribution management systems (DMS).
△ Less
Submitted 1 March, 2023;
originally announced March 2023.
-
GP CC-OPF: Gaussian Process based optimization tool for Chance-Constrained Optimal Power Flow
Authors:
Mile Mitrovic,
Ognjen Kundacina,
Aleksandr Lukashevich,
Petr Vorobev,
Vladimir Terzija,
Yury Maximov,
Deepjyoti Deka
Abstract:
The Gaussian Process (GP) based Chance-Constrained Optimal Power Flow (CC-OPF) is an open-source Python code developed for solving economic dispatch (ED) problem in modern power grids. In recent years, integrating a significant amount of renewables into a power grid causes high fluctuations and thus brings a lot of uncertainty to power grid operations. This fact makes the conventional model-based…
▽ More
The Gaussian Process (GP) based Chance-Constrained Optimal Power Flow (CC-OPF) is an open-source Python code developed for solving economic dispatch (ED) problem in modern power grids. In recent years, integrating a significant amount of renewables into a power grid causes high fluctuations and thus brings a lot of uncertainty to power grid operations. This fact makes the conventional model-based CC-OPF problem non-convex and computationally complex to solve. The developed tool presents a novel data-driven approach based on the GP regression model for solving the CC-OPF problem with a trade-off between complexity and accuracy. The proposed approach and developed software can help system operators to effectively perform ED optimization in the presence of large uncertainties in the power grid.
△ Less
Submitted 16 February, 2023;
originally announced February 2023.
-
Power System Anomaly Detection and Classification Utilizing WLS-EKF State Estimation and Machine Learning
Authors:
Sajjad Asefi,
Mile Mitrovic,
Dragan Ćetenović,
Victor Levi,
Elena Gryazina,
Vladimir Terzija
Abstract:
Power system state estimation is being faced with different types of anomalies. These might include bad data caused by gross measurement errors or communication system failures. Sudden changes in load or generation can be considered as anomaly depending on the implemented state estimation method. Additionally, considering power grid as a cyber physical system, state estimation becomes vulnerable t…
▽ More
Power system state estimation is being faced with different types of anomalies. These might include bad data caused by gross measurement errors or communication system failures. Sudden changes in load or generation can be considered as anomaly depending on the implemented state estimation method. Additionally, considering power grid as a cyber physical system, state estimation becomes vulnerable to false data injection attacks. The existing methods for anomaly classification cannot accurately classify (discriminate between) the above mentioned three types of anomalies, especially when it comes to discrimination between sudden load changes and false data injection attacks. This paper presents a new algorithm for detecting anomaly presence, classifying the anomaly type and identifying the origin of the anomaly, i.e., measurements that contain gross errors in case of bad data, or buses associated with loads experiencing a sudden change, or state variables targeted by false data injection attack. The algorithm combines analytical and machine learning (ML) approaches. The first stage exploits an analytical approach to detect anomaly presence by combining $χ^2$-test and anomaly detection index. The second stage utilizes ML for classification of anomaly type and identification of its origin, with particular reference to discrimination between sudden load changes and false data injection attacks. The proposed ML based method is trained to be independent of the network configuration which eliminates retraining of the algorithm after network topology changes. The results obtained by implementing the proposed algorithm on IEEE 14 bus test system demonstrate the accuracy and effectiveness of the proposed algorithm.
△ Less
Submitted 1 October, 2022; v1 submitted 26 September, 2022;
originally announced September 2022.
-
Data-Driven Chance Constrained AC-OPF using Hybrid Sparse Gaussian Processes
Authors:
Mile Mitrovic,
Aleksandr Lukashevich,
Petr Vorobev,
Vladimir Terzija,
Yury Maximov,
Deepjyoti Deka
Abstract:
The alternating current (AC) chance-constrained optimal power flow (CC-OPF) problem addresses the economic efficiency of electricity generation and delivery under generation uncertainty. The latter is intrinsic to modern power grids because of the high amount of renewables. Despite its academic success, the AC CC-OPF problem is highly nonlinear and computationally demanding, which limits its pract…
▽ More
The alternating current (AC) chance-constrained optimal power flow (CC-OPF) problem addresses the economic efficiency of electricity generation and delivery under generation uncertainty. The latter is intrinsic to modern power grids because of the high amount of renewables. Despite its academic success, the AC CC-OPF problem is highly nonlinear and computationally demanding, which limits its practical impact. For improving the AC-OPF problem complexity/accuracy trade-off, the paper proposes a fast data-driven setup that uses the sparse and hybrid Gaussian processes (GP) framework to model the power flow equations with input uncertainty. We advocate the efficiency of the proposed approach by a numerical study over multiple IEEE test cases showing up to two times faster and more accurate solutions compared to the state-of-the-art methods.
△ Less
Submitted 30 August, 2022;
originally announced August 2022.
-
Data-Driven Stochastic AC-OPF using Gaussian Processes
Authors:
Mile Mitrovic,
Aleksandr Lukashevich,
Petr Vorobev,
Vladimir Terzija,
Semen Budenny,
Yury Maximov,
Deepjyoti Deka
Abstract:
In recent years, electricity generation has been responsible for more than a quarter of the greenhouse gas emissions in the US. Integrating a significant amount of renewables into a power grid is probably the most accessible way to reduce carbon emissions from power grids and slow down climate change. Unfortunately, the most accessible renewable power sources, such as wind and solar, are highly fl…
▽ More
In recent years, electricity generation has been responsible for more than a quarter of the greenhouse gas emissions in the US. Integrating a significant amount of renewables into a power grid is probably the most accessible way to reduce carbon emissions from power grids and slow down climate change. Unfortunately, the most accessible renewable power sources, such as wind and solar, are highly fluctuating and thus bring a lot of uncertainty to power grid operations and challenge existing optimization and control policies. The chance-constrained alternating current (AC) optimal power flow (OPF) framework finds the minimum cost generation dispatch maintaining the power grid operations within security limits with a prescribed probability. Unfortunately, the AC-OPF problem's chance-constrained extension is non-convex, computationally challenging, and requires knowledge of system parameters and additional assumptions on the behavior of renewable distribution. Known linear and convex approximations to the above problems, though tractable, are too conservative for operational practice and do not consider uncertainty in system parameters. This paper presents an alternative data-driven approach based on Gaussian process (GP) regression to close this gap. The GP approach learns a simple yet non-convex data-driven approximation to the AC power flow equations that can incorporate uncertainty inputs. The latter is then used to determine the solution of CC-OPF efficiently, by accounting for both input and parameter uncertainty. The practical efficiency of the proposed approach using different approximations for GP-uncertainty propagation is illustrated over numerous IEEE test cases.
△ Less
Submitted 21 July, 2022;
originally announced July 2022.
-
Einstein field equation, recursion operators, Noether and master symmetries in conformable Poisson manifolds
Authors:
Mahouton Norbert Hounkonnou,
Mahougnon Justin Landalidji,
Melanija Mitrovic
Abstract:
We show that a Minkowski phase space endowed with a bracket relatively to a conformable differential realizes a Poisson algebra, confering a bi-Hamiltonian structure to the resulting manifold. We infer that the related Hamiltonian vector field is an infinitesimal Noether symmetry, and compute the corresponding deformed recursion operator. Besides, using the Hamiltonian-Jacobi separability, we cons…
▽ More
We show that a Minkowski phase space endowed with a bracket relatively to a conformable differential realizes a Poisson algebra, confering a bi-Hamiltonian structure to the resulting manifold. We infer that the related Hamiltonian vector field is an infinitesimal Noether symmetry, and compute the corresponding deformed recursion operator. Besides, using the Hamiltonian-Jacobi separability, we construct recursion operators for Hamiltonian vector fields in conformable Poisson-Schwarzschild and Friedmann-Lemaître-Robertson-Walker (FLRW) manifolds, and derive related constants of motion, Christoffel symbols, components of Riemann and Ricci tensors, Ricci constant and components of Einstein tensor. We highlight the existence of a hierarchy of bi-Hamiltonian structures in both the manifolds, and compute a family of recursion operators and master symmetries generating the constants of motion.
△ Less
Submitted 18 March, 2022;
originally announced March 2022.
-
Hamiltonian Dynamics of a spaceship in Alcubierre and Gödel metrics: Recursion operators and underlying master symmetries
Authors:
Mahouton Norbert Hounkonnou,
Mahougnon Justin Landalidji,
Melanija Mitrovíc
Abstract:
We study the Hamiltonian dynamics of a spaceship in the background of Alcubierre and Gödel metrics. We derive the Hamiltonian vector fields governing the system evolution, construct and discuss related recursion operators generating the constants of motion. Besides, we characterize relevant master symmetries.
We study the Hamiltonian dynamics of a spaceship in the background of Alcubierre and Gödel metrics. We derive the Hamiltonian vector fields governing the system evolution, construct and discuss related recursion operators generating the constants of motion. Besides, we characterize relevant master symmetries.
△ Less
Submitted 2 January, 2022;
originally announced January 2022.
-
Noncommutative Kepler Dynamics: symmetry groups and bi-Hamiltonian structures
Authors:
Mahouton Norbert Hounkonnou,
Mahougnon Justin Landalidji,
Melanija Mitrovic
Abstract:
Integrals of motion are constructed from noncommutative (NC) Kepler dynamics, generating $SO(3),$ $SO(4),$ and $SO(1,3)$ dynamical symmetry groups. The Hamiltonian vector field is derived in action-angle coordinates, and the existence of a hierarchy of bi-Hamiltonian structures is highlighted. Then, a family of Nijenhuis recursion operators is computed and discussed.
Integrals of motion are constructed from noncommutative (NC) Kepler dynamics, generating $SO(3),$ $SO(4),$ and $SO(1,3)$ dynamical symmetry groups. The Hamiltonian vector field is derived in action-angle coordinates, and the existence of a hierarchy of bi-Hamiltonian structures is highlighted. Then, a family of Nijenhuis recursion operators is computed and discussed.
△ Less
Submitted 31 August, 2021;
originally announced September 2021.
-
Some results in constructive semigroup theory
Authors:
Erik Darpö,
Melanija Mitrović
Abstract:
We give a constructive treatment of some basic concepts and results in semigroup theory. Focusing on semigroups equipped with an apartness relation, we give analogues, from the point of view of apartness, of several classical constructions and results, including transitive closure and congruence closure, free semigroups, periodicity, Rees factors, and Green's relations.
We give a constructive treatment of some basic concepts and results in semigroup theory. Focusing on semigroups equipped with an apartness relation, we give analogues, from the point of view of apartness, of several classical constructions and results, including transitive closure and congruence closure, free semigroups, periodicity, Rees factors, and Green's relations.
△ Less
Submitted 4 September, 2021; v1 submitted 12 March, 2021;
originally announced March 2021.
-
Theory of constructive semigroups with apartness -- foundations, development and practice
Authors:
Melanija Mitrovic,
Mahouton Norbert Hounkonnou,
Marian Alexandru Baroni
Abstract:
This paper has several purposes. We present through a critical review the results from already published papers on the constructive semigroup theory, and contribute to its further development by giving solutions to open problems. We also draw attention to its possible applications in other (constructive) mathematics disciplines, in computer science, social sciences, economics, etc. Another importa…
▽ More
This paper has several purposes. We present through a critical review the results from already published papers on the constructive semigroup theory, and contribute to its further development by giving solutions to open problems. We also draw attention to its possible applications in other (constructive) mathematics disciplines, in computer science, social sciences, economics, etc. Another important goal of this paper is to provide a clear, understandable picture of constructive semigroups with apartness in Bishop's style both to (classical) algebraists and the ones who apply algebraic knowledge.
△ Less
Submitted 24 January, 2022; v1 submitted 6 August, 2020;
originally announced August 2020.
-
Generalized Witt and Witt n-algebras, Virasoro algebras and constraints, and KdV equations from R(p,q)-deformed quantum algebras
Authors:
Mahouton Norbert Hounkonnou,
Fridolin Melong,
Melanija Mitrovic
Abstract:
We perform generalizations of Witt and Virasoro algebras, and derive the corresponding Korteweg-de Vries equations from known R(p,q)-deformed quantum algebras previously introduced in J. Math. Phys. 51, 063518, (2010). Related relevant properties are investigated and discussed. Besides, we construct the R(p,q)-deformed Witt n- algebra, and determine the Virasoro constraints for a toy model, which…
▽ More
We perform generalizations of Witt and Virasoro algebras, and derive the corresponding Korteweg-de Vries equations from known R(p,q)-deformed quantum algebras previously introduced in J. Math. Phys. 51, 063518, (2010). Related relevant properties are investigated and discussed. Besides, we construct the R(p,q)-deformed Witt n- algebra, and determine the Virasoro constraints for a toy model, which play an important role in the study of matrix models. Finally, as matter of illustration, explicit results are provided for main particular deformed quantum algebras known in the literature.
△ Less
Submitted 8 August, 2020;
originally announced August 2020.
-
The radioscience LaRa instrument onboard ExoMars 2020 to investigate the rotation and interior of Mars
Authors:
Veronique Dehant,
Sebastien Le Maistre,
Rose-Marie Baland,
Nicolas Bergeot,
Ozgur Karatekin,
Marie-Julie Peters,
Attilio Rivoldini,
Luca Ruiz Lozano,
Orkun Temel,
Tim Van Hoolst,
Marie Yseboodt,
Michel Mitrovic,
Alexander Kosov,
Vaclav Valenta,
Lieven Thomassen,
Sumit Karki,
Khaldoun Al Khalifeh,
Christophe Craeye,
Leonid Gurvits,
Jean-Charles Marty,
Sami Asmar,
William Folkner,
the LaRa Team
Abstract:
LaRa (Lander Radioscience) is an experiment on the ExoMars 2020 mission that uses the Doppler shift on the radio link due to the motion of the ExoMars platform tied to the surface of Mars with respect to the Earth ground stations (e.g. the deep space network stations of NASA), in order to precisely measure the relative velocity of the lander on Mars with respect to the Earth. The LaRa measurements…
▽ More
LaRa (Lander Radioscience) is an experiment on the ExoMars 2020 mission that uses the Doppler shift on the radio link due to the motion of the ExoMars platform tied to the surface of Mars with respect to the Earth ground stations (e.g. the deep space network stations of NASA), in order to precisely measure the relative velocity of the lander on Mars with respect to the Earth. The LaRa measurements shall improve the understanding of the structure and processes in the deep interior of Mars by obtaining the rotation and orientation of Mars with a better precision compared to the previous missions. In this paper, we provide the analysis done until now for the best realization of these objectives. We explain the geophysical observation that will be reached with LaRa (Length-of-day variations, precession, nutation, and possibly polar motion). We develop the experiment set up, which includes the ground stations on Earth (so-called ground segment). We describe the instrument, i.e. the transponder and its three antennas. We further detail the link budget and the expected noise level that will be reached. Finally, we detail the expected results, which encompasses the explanation of how we shall determine Mars' orientation parameters, and the way we shall deduce Mars' interior structure and Mars' atmosphere from them. Lastly, we explain briefly how we will be able to determine the Surface platform position.
△ Less
Submitted 10 October, 2019; v1 submitted 9 October, 2019;
originally announced October 2019.
-
Submodular Streaming in All its Glory: Tight Approximation, Minimum Memory and Low Adaptive Complexity
Authors:
Ehsan Kazemi,
Marko Mitrovic,
Morteza Zadimoghaddam,
Silvio Lattanzi,
Amin Karbasi
Abstract:
Streaming algorithms are generally judged by the quality of their solution, memory footprint, and computational complexity. In this paper, we study the problem of maximizing a monotone submodular function in the streaming setting with a cardinality constraint $k$. We first propose Sieve-Streaming++, which requires just one pass over the data, keeps only $O(k)$ elements and achieves the tight…
▽ More
Streaming algorithms are generally judged by the quality of their solution, memory footprint, and computational complexity. In this paper, we study the problem of maximizing a monotone submodular function in the streaming setting with a cardinality constraint $k$. We first propose Sieve-Streaming++, which requires just one pass over the data, keeps only $O(k)$ elements and achieves the tight $(1/2)$-approximation guarantee. The best previously known streaming algorithms either achieve a suboptimal $(1/4)$-approximation with $Θ(k)$ memory or the optimal $(1/2)$-approximation with $O(k\log k)$ memory. Next, we show that by buffering a small fraction of the stream and applying a careful filtering procedure, one can heavily reduce the number of adaptive computational rounds, thus substantially lowering the computational complexity of Sieve-Streaming++. We then generalize our results to the more challenging multi-source streaming setting. We show how one can achieve the tight $(1/2)$-approximation guarantee with $O(k)$ shared memory while minimizing not only the required rounds of computations but also the total number of communicated bits. Finally, we demonstrate the efficiency of our algorithms on real-world data summarization tasks for multi-source streams of tweets and of YouTube videos.
△ Less
Submitted 13 May, 2019; v1 submitted 2 May, 2019;
originally announced May 2019.
-
Adaptive Sequence Submodularity
Authors:
Marko Mitrovic,
Ehsan Kazemi,
Moran Feldman,
Andreas Krause,
Amin Karbasi
Abstract:
In many machine learning applications, one needs to interactively select a sequence of items (e.g., recommending movies based on a user's feedback) or make sequential decisions in a certain order (e.g., guiding an agent through a series of states). Not only do sequences already pose a dauntingly large search space, but we must also take into account past observations, as well as the uncertainty of…
▽ More
In many machine learning applications, one needs to interactively select a sequence of items (e.g., recommending movies based on a user's feedback) or make sequential decisions in a certain order (e.g., guiding an agent through a series of states). Not only do sequences already pose a dauntingly large search space, but we must also take into account past observations, as well as the uncertainty of future outcomes. Without further structure, finding an optimal sequence is notoriously challenging, if not completely intractable. In this paper, we view the problem of adaptive and sequential decision making through the lens of submodularity and propose an adaptive greedy policy with strong theoretical guarantees. Additionally, to demonstrate the practical utility of our results, we run experiments on Amazon product recommendation and Wikipedia link prediction tasks.
△ Less
Submitted 20 June, 2019; v1 submitted 15 February, 2019;
originally announced February 2019.
-
Data Summarization at Scale: A Two-Stage Submodular Approach
Authors:
Marko Mitrovic,
Ehsan Kazemi,
Morteza Zadimoghaddam,
Amin Karbasi
Abstract:
The sheer scale of modern datasets has resulted in a dire need for summarization techniques that identify representative elements in a dataset. Fortunately, the vast majority of data summarization tasks satisfy an intuitive diminishing returns condition known as submodularity, which allows us to find nearly-optimal solutions in linear time. We focus on a two-stage submodular framework where the go…
▽ More
The sheer scale of modern datasets has resulted in a dire need for summarization techniques that identify representative elements in a dataset. Fortunately, the vast majority of data summarization tasks satisfy an intuitive diminishing returns condition known as submodularity, which allows us to find nearly-optimal solutions in linear time. We focus on a two-stage submodular framework where the goal is to use some given training functions to reduce the ground set so that optimizing new functions (drawn from the same distribution) over the reduced set provides almost as much value as optimizing them over the entire ground set. In this paper, we develop the first streaming and distributed solutions to this problem. In addition to providing strong theoretical guarantees, we demonstrate both the utility and efficiency of our algorithms on real-world tasks including image summarization and ride-share optimization.
△ Less
Submitted 7 June, 2018;
originally announced June 2018.
-
Submodularity on Hypergraphs: From Sets to Sequences
Authors:
Marko Mitrovic,
Moran Feldman,
Andreas Krause,
Amin Karbasi
Abstract:
In a nutshell, submodular functions encode an intuitive notion of diminishing returns. As a result, submodularity appears in many important machine learning tasks such as feature selection and data summarization. Although there has been a large volume of work devoted to the study of submodular functions in recent years, the vast majority of this work has been focused on algorithms that output sets…
▽ More
In a nutshell, submodular functions encode an intuitive notion of diminishing returns. As a result, submodularity appears in many important machine learning tasks such as feature selection and data summarization. Although there has been a large volume of work devoted to the study of submodular functions in recent years, the vast majority of this work has been focused on algorithms that output sets, not sequences. However, in many settings, the order in which we output items can be just as important as the items themselves.
To extend the notion of submodularity to sequences, we use a directed graph on the items where the edges encode the additional value of selecting items in a particular order. Existing theory is limited to the case where this underlying graph is a directed acyclic graph. In this paper, we introduce two new algorithms that provably give constant factor approximations for general graphs and hypergraphs having bounded in or out degrees. Furthermore, we show the utility of our new algorithms for real-world applications in movie recommendation, online link prediction, and the design of course sequences for MOOCs.
△ Less
Submitted 15 March, 2018; v1 submitted 25 February, 2018;
originally announced February 2018.
-
The Nobel Prize delay
Authors:
Francesco Becattini,
Arnab Chatterjee,
Santo Fortunato,
Marija Mitrović,
Raj Kumar Pan,
Pietro Della Briotta Parolo
Abstract:
The time lag between the publication of a Nobel discovery and the conferment of the prize has been rapidly increasing for all disciplines, especially for Physics. Does this mean that fundamental science is running out of groundbreaking discoveries?
The time lag between the publication of a Nobel discovery and the conferment of the prize has been rapidly increasing for all disciplines, especially for Physics. Does this mean that fundamental science is running out of groundbreaking discoveries?
△ Less
Submitted 28 May, 2014;
originally announced May 2014.
-
Inferring human mobility using communication patterns
Authors:
Vasyl Palchykov,
Marija Mitrović,
Hang-Hyun Jo,
Jari Saramäki,
Raj Kumar Pan
Abstract:
Understanding the patterns of mobility of individuals is crucial for a number of reasons, from city planning to disaster management. There are two common ways of quantifying the amount of travel between locations: by direct observations that often involve privacy issues, e.g., tracking mobile phone locations, or by estimations from models. Typically, such models build on accurate knowledge of the…
▽ More
Understanding the patterns of mobility of individuals is crucial for a number of reasons, from city planning to disaster management. There are two common ways of quantifying the amount of travel between locations: by direct observations that often involve privacy issues, e.g., tracking mobile phone locations, or by estimations from models. Typically, such models build on accurate knowledge of the population size at each location. However, when this information is not readily available, their applicability is rather limited. As mobile phones are ubiquitous, our aim is to investigate if mobility patterns can be inferred from aggregated mobile phone call data alone. Using data released by Orange for Ivory Coast, we show that human mobility is well predicted by a simple model based on the frequency of mobile phone calls between two locations and their geographical distance. We argue that the strength of the model comes from directly incorporating the social dimension of mobility. Furthermore, as only aggregated call data is required, the model helps to avoid potential privacy problems.
△ Less
Submitted 22 August, 2014; v1 submitted 30 April, 2014;
originally announced April 2014.
-
Universality in voting behavior: an empirical analysis
Authors:
Arnab Chatterjee,
Marija Mitrović,
Santo Fortunato
Abstract:
Election data represent a precious source of information to study human behavior at a large scale. In proportional elections with open lists, the number of votes received by a candidate, rescaled by the average performance of all competitors in the same party list, has the same distribution regardless of the country and the year of the election. Here we provide the first thorough assessment of thi…
▽ More
Election data represent a precious source of information to study human behavior at a large scale. In proportional elections with open lists, the number of votes received by a candidate, rescaled by the average performance of all competitors in the same party list, has the same distribution regardless of the country and the year of the election. Here we provide the first thorough assessment of this claim. We analyzed election datasets of 15 countries with proportional systems. We confirm that a class of nations with similar election rules fulfill the universality claim. Discrepancies from this trend in other countries with open-lists elections are always associated with peculiar differences in the election rules, which matter more than differences between countries and historical periods. Our analysis shows that the role of parties in the electoral performance of candidates is crucial: alternative scalings not taking into account party affiliations lead to poor results.
△ Less
Submitted 24 January, 2013; v1 submitted 10 December, 2012;
originally announced December 2012.
-
How the online social networks are used: Dialogs-based structure of MySpace
Authors:
Milovan Suvakov,
Marija Mitrovic,
Vladimir Gligorijevic,
Bosiljka Tadic
Abstract:
Quantitative study of collective dynamics in online social networks is a new challenge based on the abundance of empirical data. Conclusions, however, may depend on factors as user's psychology profiles and their reasons to use the online contacts. In this paper we have compiled and analyzed two datasets from \texttt{MySpace}. The data contain networked dialogs occurring within a specified time de…
▽ More
Quantitative study of collective dynamics in online social networks is a new challenge based on the abundance of empirical data. Conclusions, however, may depend on factors as user's psychology profiles and their reasons to use the online contacts. In this paper we have compiled and analyzed two datasets from \texttt{MySpace}. The data contain networked dialogs occurring within a specified time depth, high temporal resolution, and texts of messages, in which the emotion valence is assessed by using SentiStrength classifier. Performing a comprehensive analysis we obtain three groups of results: Dynamic topology of the dialogs-based networks have characteristic structure with Zipf's distribution of communities, low link reciprocity, and disassortative correlations. Overlaps supporting "weak-ties" hypothesis are found to follow the laws recently conjectured for online games. Long-range temporal correlations and persistent fluctuations occur in the time series of messages carrying positive (negative) emotion. Patterns of user communications have dominant positive emotion (attractiveness) and strong impact of circadian cycles and nteractivity times longer than one day. Taken together, these results give a new insight into functioning of the online social networks and unveil importance of the amount of information and emotion that is communicated along the social links. (All data used in this study are fully anonymized.)
△ Less
Submitted 28 June, 2012;
originally announced June 2012.
-
Statistical analysis of emotions and opinions at Digg website
Authors:
Piotr Pohorecki,
Julian Sienkiewicz,
Marija Mitrovic,
Georgios Paltoglou,
Janusz A. Holyst
Abstract:
We performed statistical analysis on data from the Digg.com website, which enables its users to express their opinion on news stories by taking part in forum-like discussions as well as directly evaluate previous posts and stories by assigning so called "diggs". Owing to fact that the content of each post has been annotated with its emotional value, apart from the strictly structural properties, t…
▽ More
We performed statistical analysis on data from the Digg.com website, which enables its users to express their opinion on news stories by taking part in forum-like discussions as well as directly evaluate previous posts and stories by assigning so called "diggs". Owing to fact that the content of each post has been annotated with its emotional value, apart from the strictly structural properties, the study also includes an analysis of the average emotional response of the posts commenting the main story. While analysing correlations at the story level, an interesting relationship between the number of diggs and the number of comments received by a story was found. The correlation between the two quantities is high for data where small threads dominate and consistently decreases for longer threads. However, while the correlation of the number of diggs and the average emotional response tends to grow for longer threads, correlations between numbers of comments and the average emotional response are almost zero. We also show that the initial set of comments given to a story has a substantial impact on the further "life" of the discussion: high negative average emotions in the first 10 comments lead to longer threads while the opposite situation results in shorter discussions. We also suggest presence of two different mechanisms governing the evolution of the discussion and, consequently, its length.
△ Less
Submitted 16 April, 2012; v1 submitted 26 January, 2012;
originally announced January 2012.
-
Patterns of Emotional Blogging and Emergence of Communities: Agent-Based Model on Bipartite Networks
Authors:
Marija Mitrović,
Bosiljka Tadić
Abstract:
Background: We study mechanisms underlying the collective emotional behavior of Bloggers by using the agent-based modeling and the parameters inferred from the related empirical data.
Methodology/Principal Findings: A bipartite network of emotional agents and posts evolves through the addition of agents and their actions on posts. The emotion state of an agent,quantified by the arousal and the v…
▽ More
Background: We study mechanisms underlying the collective emotional behavior of Bloggers by using the agent-based modeling and the parameters inferred from the related empirical data.
Methodology/Principal Findings: A bipartite network of emotional agents and posts evolves through the addition of agents and their actions on posts. The emotion state of an agent,quantified by the arousal and the valence, fluctuates in time due to events on the connected posts, and in the moments of agent's action it is transferred to a selected post. We claim that the indirect communication of the emotion in the model rules, combined with the action-delay time and the circadian rhythm extracted from the empirical data, can explain the genesis of emotional bursts by users on popular Blogs and similar Web portals. The model also identifies the parameters and how they influence the course of the dynamics.
Conclusions: The collective behavior is here recognized by the emergence of communities on the network and the fractal time-series of their emotional comments, powered by the negative emotion (critique). The evolving agents communities leave characteristic patterns of the activity in the phase space of the arousal--valence variables, where each segment represents a common emotion described in psychology.
△ Less
Submitted 23 October, 2011;
originally announced October 2011.
-
Quantitative Analysis of Bloggers Collective Behavior Powered by Emotions
Authors:
Marija Mitrović,
Georgios Paltoglou,
Bosiljka Tadić
Abstract:
Large-scale data resulting from users online interactions provide the ultimate source of information to study emergent social phenomena on the Web. From individual actions of users to observable collective behaviors, different mechanisms involving emotions expressed in the posted text play a role. Here we combine approaches of statistical physics with machine-learning methods of text analysis to s…
▽ More
Large-scale data resulting from users online interactions provide the ultimate source of information to study emergent social phenomena on the Web. From individual actions of users to observable collective behaviors, different mechanisms involving emotions expressed in the posted text play a role. Here we combine approaches of statistical physics with machine-learning methods of text analysis to study emergence of the emotional behavior among Web users. Mapping the high-resolution data from digg.com onto bipartite network of users and their comments onto posted stories, we identify user communities centered around certain popular posts and determine emotional contents of the related comments by the emotion-classifier developed for this type of texts. Applied over different time periods, this framework reveals strong correlations between the excess of negative emotions and the evolution of communities. We observe avalanches of emotional comments exhibiting significant self-organized critical behavior and temporal correlations. To explore robustness of these critical states, we design a network automaton model on realistic network connections and several control parameters, which can be inferred from the dataset. Dissemination of emotions by a small fraction of very active users appears to critically tune the collective states.
△ Less
Submitted 29 November, 2010;
originally announced November 2010.
-
Network theory approach for data evaluation in the dynamic force spectroscopy of biomolecular interactions
Authors:
Jelena Zivković,
Marija Mitrović,
Luuk Janssen,
Hans A. Heus,
Bosiljka Tadić,
Sylvia Speller
Abstract:
Investigations of molecular bonds between single molecules and molecular complexes by the dynamic force spectroscopy are subject to large fluctuations at nanoscale and possible other aspecific binding, which mask the experimental output. Big efforts are devoted to develop methods for effective selection of the relevant experimental data, before taking the quantitative analysis of bond parameters…
▽ More
Investigations of molecular bonds between single molecules and molecular complexes by the dynamic force spectroscopy are subject to large fluctuations at nanoscale and possible other aspecific binding, which mask the experimental output. Big efforts are devoted to develop methods for effective selection of the relevant experimental data, before taking the quantitative analysis of bond parameters. Here we present a methodology which is based on the application of graph theory. The force-distance curves corresponding to repeated pulling events are mapped onto their correlation network (mathematical graph). On these graphs the groups of similar curves appear as topological modules, which are identified using the spectral analysis of graphs. We demonstrate the approach by analyzing a large ensemble of the force-distance curves measured on: ssDNA-ssDNA, peptide-RNA (system from HIV1), and peptide-Au surface. Within our data sets the methodology systematically separates subgroups of curves which are related to different intermolecular interactions and to spatial arrangements in which the molecules are brought together and/or pulling speeds. This demonstrates the sensitivity of the method to the spatial degrees of freedom, suggesting potential applications in the case of large molecular complexes and situations with multiple binding sites.
△ Less
Submitted 12 November, 2009;
originally announced November 2009.
-
Bloggers Behavior and Emergent Communities in Blog Space
Authors:
Marija Mitrović,
Bosiljka Tadić
Abstract:
Interactions between users in cyberspace may lead to phenomena different from those observed in common social networks. Here we analyse large data sets about users and Blogs which they write and comment, mapped onto a bipartite graph. In such enlarged Blog space we trace user activity over time, which results in robust temporal patterns of user--Blog behavior and the emergence of communities. Wi…
▽ More
Interactions between users in cyberspace may lead to phenomena different from those observed in common social networks. Here we analyse large data sets about users and Blogs which they write and comment, mapped onto a bipartite graph. In such enlarged Blog space we trace user activity over time, which results in robust temporal patterns of user--Blog behavior and the emergence of communities. With the spectral methods applied to the projection on weighted user network we detect clusters of users related to their common interests and habits. Our results suggest that different mechanisms may play the role in the case of very popular Blogs. Our analysis makes a suitable basis for theoretical modeling of the evolution of cyber communities and for practical study of the data, in particular for an efficient search of interesting Blog clusters and further retrieval of their contents by text analysis.
△ Less
Submitted 15 October, 2009;
originally announced October 2009.
-
Jamming and Correlation Patterns in Traffic of Information on Sparse Modular Networks
Authors:
Bosiljka Tadić,
Marija Mitrović
Abstract:
We study high-density traffic of information packets on sparse modular networks with scale-free subgraphs. With different statistical measures we distinguish between the free flow and congested regime and point out the role of modules in the jamming transition. We further consider correlations between traffic signals collected at each node in the network. The correlation matrix between pairs of…
▽ More
We study high-density traffic of information packets on sparse modular networks with scale-free subgraphs. With different statistical measures we distinguish between the free flow and congested regime and point out the role of modules in the jamming transition. We further consider correlations between traffic signals collected at each node in the network. The correlation matrix between pairs of signals reflects the network modularity in the eigenvalue spectrum and the structure of eigenvectors. The internal structure of the modules has an important role in the diffusion dynamics, leading to enhanced correlations between the modular hubs, which can not be filtered out by standard methods. Implications for the analysis of real networks with unknown modular structure are discussed.
△ Less
Submitted 7 April, 2009;
originally announced April 2009.
-
Spectral and Dynamical Properties in Classes of Sparse Networks with Mesoscopic Inhomogeneities
Authors:
Marija Mitrović,
Bosiljka Tadić
Abstract:
We study structure, eigenvalue spectra and diffusion dynamics in a wide class of networks with subgraphs (modules) at mesoscopic scale. The networks are grown within the model with three parameters controlling the number of modules, their internal structure as scale-free and correlated subgraphs, and the topology of connecting network. Within the exhaustive spectral analysis for both the adjacen…
▽ More
We study structure, eigenvalue spectra and diffusion dynamics in a wide class of networks with subgraphs (modules) at mesoscopic scale. The networks are grown within the model with three parameters controlling the number of modules, their internal structure as scale-free and correlated subgraphs, and the topology of connecting network. Within the exhaustive spectral analysis for both the adjacency matrix and the normalized Laplacian matrix we identify the spectral properties which characterize the mesoscopic structure of sparse cyclic graphs and trees. The minimally connected nodes, clustering, and the average connectivity affect the central part of the spectrum. The number of distinct modules leads to an extra peak at the lower part of the Laplacian spectrum in cyclic graphs. Such a peak does not occur in the case of topologically distinct tree-subgraphs connected on a tree. Whereas the associated eigenvectors remain localized on the subgraphs both in trees and cyclic graphs. We also find a characteristic pattern of periodic localization along the chains on the tree for the eigenvector components associated with the largest eigenvalue equal 2 of the Laplacian. We corroborate the results with simulations of the random walk on several types of networks. Our results for the distribution of return-time of the walk to the origin (autocorrelator) agree well with recent analytical solution for trees, and it appear to be independent on their mesoscopic and global structure. For the cyclic graphs we find new results with twice larger stretching exponent of the tail of the distribution, which is virtually independent on the size of cycles. The modularity and clustering contribute to a power-law decay at short return times.
△ Less
Submitted 25 August, 2009; v1 submitted 29 September, 2008;
originally announced September 2008.
