-
Dataset of artefacts for machine learning applications in astronomy
Authors:
Sreevarsha Sreejith,
Maria V. Pruzhinskaya,
Alina A. Volnova,
Vadim V. Krushinsky,
Konstantin L. Malanchev,
Emille E. O. Ishida,
Anastasia D. Lavrukhina,
Timofey A. Semenikhin,
Emmanuel Gangler,
Matwey V. Kornilov,
Vladimir S. Korolev
Abstract:
Accurate photometry in astronomical surveys is challenged by image artefacts, which affect measurements and degrade data quality. Due to the large amount of available data, this task is increasingly handled using machine learning algorithms, which often require a labelled training set to learn data patterns. We present an expert-labelled dataset of 1127 artefacts with 1213 labels from 26 fields in…
▽ More
Accurate photometry in astronomical surveys is challenged by image artefacts, which affect measurements and degrade data quality. Due to the large amount of available data, this task is increasingly handled using machine learning algorithms, which often require a labelled training set to learn data patterns. We present an expert-labelled dataset of 1127 artefacts with 1213 labels from 26 fields in ZTF DR3, along with a complementary set of nominal objects. The artefact dataset was compiled using the active anomaly detection algorithm PineForest, developed by the SNAD team. These datasets can serve as valuable resources for real-bogus classification, catalogue cleaning, anomaly detection, and educational purposes. Both artefacts and nominal images are provided in FITS format in two sizes (28 x 28 and 63 x 63 pixels). The datasets are publicly available for further scientific applications.
△ Less
Submitted 10 April, 2025;
originally announced April 2025.
-
Exploring the Universe with SNAD: Anomaly Detection in Astronomy
Authors:
Alina A. Volnova,
Patrick D. Aleo,
Anastasia Lavrukhina,
Etienne Russeil,
Timofey Semenikhin,
Emmanuel Gangler,
Emille E. O. Ishida,
Matwey V. Kornilov,
Vladimir Korolev,
Konstantin Malanchev,
Maria V. Pruzhinskaya,
Sreevarsha Sreejith
Abstract:
SNAD is an international project with a primary focus on detecting astronomical anomalies within large-scale surveys, using active learning and other machine learning algorithms. The work carried out by SNAD not only contributes to the discovery and classification of various astronomical phenomena but also enhances our understanding and implementation of machine learning techniques within the fiel…
▽ More
SNAD is an international project with a primary focus on detecting astronomical anomalies within large-scale surveys, using active learning and other machine learning algorithms. The work carried out by SNAD not only contributes to the discovery and classification of various astronomical phenomena but also enhances our understanding and implementation of machine learning techniques within the field of astrophysics. This paper provides a review of the SNAD project and summarizes the advancements and achievements made by the team over several years.
△ Less
Submitted 24 October, 2024;
originally announced October 2024.
-
Coniferest: a complete active anomaly detection framework
Authors:
M. V. Kornilov,
V. S. Korolev,
K. L. Malanchev,
A. D. Lavrukhina,
E. Russeil,
T. A. Semenikhin,
E. Gangler,
E. E. O. Ishida,
M. V. Pruzhinskaya,
A. A. Volnova,
S. Sreejith
Abstract:
We present coniferest, an open source generic purpose active anomaly detection framework written in Python. The package design and implemented algorithms are described. Currently, static outlier detection analysis is supported via the Isolation forest algorithm. Moreover, Active Anomaly Discovery (AAD) and Pineforest algorithms are available to tackle active anomaly detection problems. The algorit…
▽ More
We present coniferest, an open source generic purpose active anomaly detection framework written in Python. The package design and implemented algorithms are described. Currently, static outlier detection analysis is supported via the Isolation forest algorithm. Moreover, Active Anomaly Discovery (AAD) and Pineforest algorithms are available to tackle active anomaly detection problems. The algorithms and package performance are evaluated on a series of synthetic datasets. We also describe a few success cases which resulted from applying the package to real astronomical data in active anomaly detection tasks within the SNAD project.
△ Less
Submitted 15 November, 2024; v1 submitted 22 October, 2024;
originally announced October 2024.
-
Real-bogus scores for active anomaly detection
Authors:
T. A. Semenikhin,
M. V. Kornilov,
M. V. Pruzhinskaya,
A. D. Lavrukhina,
E. Russeil,
E. Gangler,
E. E. O. Ishida,
V. S. Korolev,
K. L. Malanchev,
A. A. Volnova,
S. Sreejith
Abstract:
In the task of anomaly detection in modern time-domain photometric surveys, the primary goal is to identify astrophysically interesting, rare, and unusual objects among a large volume of data. Unfortunately, artifacts -- such as plane or satellite tracks, bad columns on CCDs, and ghosts -- often constitute significant contaminants in results from anomaly detection analysis. In such contexts, the A…
▽ More
In the task of anomaly detection in modern time-domain photometric surveys, the primary goal is to identify astrophysically interesting, rare, and unusual objects among a large volume of data. Unfortunately, artifacts -- such as plane or satellite tracks, bad columns on CCDs, and ghosts -- often constitute significant contaminants in results from anomaly detection analysis. In such contexts, the Active Anomaly Discovery (AAD) algorithm allows tailoring the output of anomaly detection pipelines according to what the expert judges to be scientifically interesting. We demonstrate how the introduction real-bogus scores, obtained from a machine learning classifier, improves the results from AAD. Using labeled data from the SNAD ZTF knowledge database, we train four real-bogus classifiers: XGBoost, CatBoost, Random Forest, and Extremely Randomized Trees. All the models perform real-bogus classification with similar effectiveness, achieving ROC-AUC scores ranging from 0.93 to 0.95. Consequently, we select the Random Forest model as the main model due to its simplicity and interpretability. The Random Forest classifier is applied to 67 million light curves from ZTF DR17. The output real-bogus score is used as an additional feature for two anomaly detection algorithms: static Isolation Forest and AAD. While results from Isolation Forest remained unchanged, the number of artifacts detected by the active approach decreases significantly with the inclusion of the real-bogus score, from 27 to 3 out of 100. We conclude that incorporating the real-bogus classifier result as an additional feature in the active anomaly detection pipeline significantly reduces the number of artifacts in the outputs, thereby increasing the incidence of astrophysically interesting objects presented to human experts.
△ Less
Submitted 20 December, 2024; v1 submitted 16 September, 2024;
originally announced September 2024.
-
SNAD catalogue of M-dwarf flares from the Zwicky Transient Facility
Authors:
A. S. Voloshina,
A. D. Lavrukhina,
M. V. Pruzhinskaya,
K. L. Malanchev,
E. E. O. Ishida,
V. V. Krushinsky,
P. D. Aleo,
E. Gangler,
M. V. Kornilov,
V. S. Korolev,
E. Russeil,
T. A. Semenikhin,
S. Sreejith,
A. A. Volnova
Abstract:
Most of the stars in the Universe are M spectral class dwarfs, which are known to be the source of bright and frequent stellar flares. In this paper, we propose new approaches to discover M-dwarf flares in ground-based photometric surveys. We employ two approaches: a modification of a traditional method of parametric fit search and a machine learning algorithm based on active anomaly detection. Th…
▽ More
Most of the stars in the Universe are M spectral class dwarfs, which are known to be the source of bright and frequent stellar flares. In this paper, we propose new approaches to discover M-dwarf flares in ground-based photometric surveys. We employ two approaches: a modification of a traditional method of parametric fit search and a machine learning algorithm based on active anomaly detection. The algorithms are applied to Zwicky Transient Facility (ZTF) data release 8, which includes the data from the ZTF high-cadence survey, allowing us to reveal flares lasting from minutes to hours. We analyze over 35 million ZTF light curves and visually scrutinize 1168 candidates suggested by the algorithms to filter out artifacts, occultations of a star by an asteroid, and other types of known variable objects. The result of this analysis is the largest catalogue of ZTF flaring stars to date, representing 134 flares with amplitudes ranging from -0.2 to -4.6 magnitudes, including repeated flares. Using Pan-STARRS DR2 colors, we assign a spectral subclass to each object in the sample. For 13 flares with well-sampled light curves and available geometric distances from Gaia DR3, we estimate the bolometric energy. This research shows that the proposed methods combined with the ZTF's cadence strategy are suitable for identifying M-dwarf flares and other fast transients, allowing for the extraction of significant astrophysical information from their light curves.
△ Less
Submitted 29 September, 2024; v1 submitted 11 April, 2024;
originally announced April 2024.
-
Rainbow: a colorful approach on multi-passband light curve estimation
Authors:
E. Russeil,
K. L. Malanchev,
P. D. Aleo,
E. E. O. Ishida,
M. V. Pruzhinskaya,
E. Gangler,
A. D. Lavrukhina,
A. A. Volnova,
A. Voloshina,
T. Semenikhin,
S. Sreejith,
M. V. Kornilov,
V. S. Korolev
Abstract:
We present Rainbow, a physically motivated framework which enables simultaneous multi-band light curve fitting. It allows the user to construct a 2-dimensional continuous surface across wavelength and time, even in situations where the number of observations in each filter is significantly limited. Assuming the electromagnetic radiation emission from the transient can be approximated by a black-bo…
▽ More
We present Rainbow, a physically motivated framework which enables simultaneous multi-band light curve fitting. It allows the user to construct a 2-dimensional continuous surface across wavelength and time, even in situations where the number of observations in each filter is significantly limited. Assuming the electromagnetic radiation emission from the transient can be approximated by a black-body, we combined an expected temperature evolution and a parametric function describing its bolometric light curve. These three ingredients allow the information available in one passband to guide the reconstruction in the others, thus enabling a proper use of multi-survey data. We demonstrate the effectiveness of our method by applying it to simulated data from the Photometric LSST Astronomical Time-series Classification Challenge (PLAsTiCC) as well as real data from the Young Supernova Experiment (YSE DR1). We evaluate the quality of the estimated light curves according to three different tests: goodness of fit, time of peak prediction and ability to transfer information to machine learning (ML) based classifiers. Results confirm that Rainbow leads to equivalent (SNII) or up to 75% better (SN Ibc) goodness of fit when compared to the Monochromatic approach. Similarly, accuracy when using Rainbow best-fit values as a parameter space in multi-class ML classification improves for all classes in our sample. An efficient implementation of Rainbow has been publicly released as part of the light curve package at https://github.com/light-curve/light-curve-python. Our approach enables straight forward light curve estimation for objects with observations in multiple filters and from multiple experiments. It is particularly well suited for situations where light curve sampling is sparse.
△ Less
Submitted 5 October, 2023; v1 submitted 4 October, 2023;
originally announced October 2023.
-
Automatic detection of plateau phases in light curves of variable stars
Authors:
Anastasia Lavrukhina,
Konstantin Malanchev,
Matwey V. Kornilov
Abstract:
Modern astronomical surveys produce millions of light curves of variable sources. These massive data sets challenge the community to create automatic light-curve processing methods for detection, classification, and characterisation of variable stars. In this paper, we present a novel method for extracting the variable components of a light curve based on Otsu's thresholding method. To validate th…
▽ More
Modern astronomical surveys produce millions of light curves of variable sources. These massive data sets challenge the community to create automatic light-curve processing methods for detection, classification, and characterisation of variable stars. In this paper, we present a novel method for extracting the variable components of a light curve based on Otsu's thresholding method. To validate the effectiveness of this method, we apply it to the light curves of detached eclipsing binaries and dwarf novae, sourced from OGLE catalogues.
△ Less
Submitted 18 September, 2023;
originally announced September 2023.
-
Reduction of supernova light curves by vector Gaussian processes
Authors:
Matwey V. Kornilov,
T. A. Semenikhin,
M. V. Pruzhinskaya
Abstract:
Bolometric light curves play an important role in understanding the underlying physics of various astrophysical phenomena, as they allow for a comprehensive modeling of the event and enable comparison between different objects. However, constructing these curves often requires the approximation and extrapolation from multicolor photometric observations. In this study, we introduce vector Gaussian…
▽ More
Bolometric light curves play an important role in understanding the underlying physics of various astrophysical phenomena, as they allow for a comprehensive modeling of the event and enable comparison between different objects. However, constructing these curves often requires the approximation and extrapolation from multicolor photometric observations. In this study, we introduce vector Gaussian processes as a new method for reduction of supernova light curves. This method enables us to approximate vector functions, even with inhomogeneous time-series data, while considering the correlation between light curves in different passbands. We applied this methodology to a sample of 29 superluminous supernovae (SLSNe) assembled using the Open Supernova Catalog. Their multicolor light curves were approximated using vector Gaussian processes. Subsequently, under the black-body assumption for the SLSN spectra at each moment of time, we reconstructed the bolometric light curves. The vector Gaussian processes developed in this work are accessible via the Python library gp-multistate-kernel on GitHub. Our approach provides an efficient tool for analyzing light curve data, opening new possibilities for astrophysical research.
△ Less
Submitted 28 August, 2023;
originally announced August 2023.
-
The SNAD Viewer: Everything You Want to Know about Your Favorite ZTF Object
Authors:
Konstantin Malanchev,
Matwey V. Kornilov,
Maria V. Pruzhinskaya,
Emille E. O. Ishida,
Patrick D. Aleo,
Vladimir S. Korolev,
Anastasia Lavrukhina,
Etienne Russeil,
Sreevarsha Sreejith,
Alina A. Volnova,
Anastasiya Voloshina,
Alberto Krone-Martins
Abstract:
We describe the SNAD Viewer, a web portal for astronomers which presents a centralized view of individual objects from the Zwicky Transient Facility's (ZTF) data releases, including data gathered from multiple publicly available astronomical archives and data sources. Initially built to enable efficient expert feedback in the context of adaptive machine learning applications, it has evolved into a…
▽ More
We describe the SNAD Viewer, a web portal for astronomers which presents a centralized view of individual objects from the Zwicky Transient Facility's (ZTF) data releases, including data gathered from multiple publicly available astronomical archives and data sources. Initially built to enable efficient expert feedback in the context of adaptive machine learning applications, it has evolved into a full-fledged community asset that centralizes public information and provides a multi-dimensional view of ZTF sources. For users, we provide detailed descriptions of the data sources and choices underlying the information displayed in the portal. For developers, we describe our architectural choices and their consequences such that our experience can help others engaged in similar endeavors or in adapting our publicly released code to their requirements. The infrastructure we describe here is scalable and flexible and can be personalized and used by other surveys and for other science goals. The Viewer has been instrumental in highlighting the crucial roles domain experts retain in the era of big data in astronomy. Given the arrival of the upcoming generation of large-scale surveys, we believe similar systems will be paramount in enabling an optimal exploitation of the scientific potential enclosed in current terabyte and future petabyte-scale data sets. The Viewer is publicly available online at https://ztf.snad.space
△ Less
Submitted 3 March, 2023; v1 submitted 14 November, 2022;
originally announced November 2022.
-
Supernova search with active learning in ZTF DR3
Authors:
Maria V. Pruzhinskaya,
Emille E. O. Ishida,
Alexandra K. Novinskaya,
Etienne Russeil,
Alina A. Volnova,
Konstantin L. Malanchev,
Matwey V. Kornilov,
Patrick D. Aleo,
Vladimir S. Korolev,
Vadim V. Krushinsky,
Sreevarsha Sreejith,
Emmanuel Gangler
Abstract:
We provide the first results from the complete SNAD adaptive learning pipeline in the context of a broad scope of data from large-scale astronomical surveys. The main goal of this work is to explore the potential of adaptive learning techniques in application to big data sets. Our SNAD team used Active Anomaly Discovery (AAD) as a tool to search for new supernova (SN) candidates in the photometric…
▽ More
We provide the first results from the complete SNAD adaptive learning pipeline in the context of a broad scope of data from large-scale astronomical surveys. The main goal of this work is to explore the potential of adaptive learning techniques in application to big data sets. Our SNAD team used Active Anomaly Discovery (AAD) as a tool to search for new supernova (SN) candidates in the photometric data from the first 9.4 months of the Zwicky Transient Facility (ZTF) survey, namely, between March 17 and December 31 2018 (58194 < MJD < 58483). We analysed 70 ZTF fields at a high galactic latitude and visually inspected 2100 outliers. This resulted in 104 SN-like objects being found, 57 of which were reported to the Transient Name Server for the first time and with 47 having previously been mentioned in other catalogues, either as SNe with known types or as SN candidates. We visually inspected the multi-colour light curves of the non-catalogued transients and performed fittings with different supernova models to assign it to a probable photometric class: Ia, Ib/c, IIP, IIL, or IIn. Moreover, we also identified unreported slow-evolving transients that are good superluminous SN candidates, along with a few other non-catalogued objects, such as red dwarf flares and active galactic nuclei. Beyond confirming the effectiveness of human-machine integration underlying the AAD strategy, our results shed light on potential leaks in currently available pipelines. These findings can help avoid similar losses in future large-scale astronomical surveys. Furthermore, the algorithm enables direct searches of any type of data and based on any definition of an anomaly set by the expert.
△ Less
Submitted 27 March, 2023; v1 submitted 18 August, 2022;
originally announced August 2022.
-
SNAD Transient Miner: Finding Missed Transient Events in ZTF DR4 using k-D trees
Authors:
P. D. Aleo,
K. L. Malanchev,
M. V. Pruzhinskaya,
E. E. O. Ishida,
E. Russeil,
M. V. Kornilov,
V. S. Korolev,
S. Sreejith,
A. A. Volnova,
G. S. Narayan
Abstract:
We report the automatic detection of 11 transients (7 possible supernovae and 4 active galactic nuclei candidates) within the Zwicky Transient Facility fourth data release (ZTF DR4), all of them observed in 2018 and absent from public catalogs. Among these, three were not part of the ZTF alert stream. Our transient mining strategy employs 41 physically motivated features extracted from both real l…
▽ More
We report the automatic detection of 11 transients (7 possible supernovae and 4 active galactic nuclei candidates) within the Zwicky Transient Facility fourth data release (ZTF DR4), all of them observed in 2018 and absent from public catalogs. Among these, three were not part of the ZTF alert stream. Our transient mining strategy employs 41 physically motivated features extracted from both real light curves and four simulated light curve models (SN Ia, SN II, TDE, SLSN-I). These features are input to a k-D tree algorithm, from which we calculate the 15 nearest neighbors. After pre-processing and selection cuts, our dataset contained approximately a million objects among which we visually inspected the 105 closest neighbors from seven of our brightest, most well-sampled simulations, comprising 89 unique ZTF DR4 sources. Our result illustrates the potential of coherently incorporating domain knowledge and automatic learning algorithms, which is one of the guiding principles directing the SNAD team. It also demonstrates that the ZTF DR is a suitable testing ground for data mining algorithms aiming to prepare for the next generation of astronomical data.
△ Less
Submitted 4 May, 2022; v1 submitted 22 November, 2021;
originally announced November 2021.
-
Useful relations for the analysis of stellar scintillation at the entrance pupil of a telescope
Authors:
Victor Kornilov,
Boris Safonov,
Matwey Kornilov
Abstract:
The development of new techniques for characterizing atmospheric optical turbulence (OT) has become an active topic of research again in recent years. In order to facilitate these studies, we reconsidered known theoretical results and obtained some new practically useful conclusions. We introduce a dimensionless Fresnel filter, which allows us to approximate a polychromatic weighting function (WF)…
▽ More
The development of new techniques for characterizing atmospheric optical turbulence (OT) has become an active topic of research again in recent years. In order to facilitate these studies, we reconsidered known theoretical results and obtained some new practically useful conclusions. We introduce a dimensionless Fresnel filter, which allows us to approximate a polychromatic weighting function (WF) by a monochromatic one with a typical precision of several percent. A so-called dimensionless WF can be easily scaled for a receiving aperture of any size. For the case of a circular aperture and monochromatic radiation, an analytical expression for the WF was found. The WFs for a square aperture and for a circular aperture match with relative difference less than 0.01 if the circular aperture diameter is 1.15 times larger than the square aperture side.
A linear digital filter can be applied to the scintillation signal from an image detector. As an example of digital filtering, we considered the power law filter $\propto f^{5/3}$ with the WF being constant in a wide range of altitudes. We discuss the main limitations of this approach for measuring OT integral: finite pixel size, aliasing, and finite image detector size.
△ Less
Submitted 16 August, 2021;
originally announced August 2021.
-
New SU UMa-type star ZTF18abdlzhd in the Zwicky Transient Facility data
Authors:
Sergei V. Antipin,
Alexandra M. Zubareva,
Aleksandr A. Belinski,
Marina A. Burlak,
Natalia P. Ikonnikova,
Konstantin L. Malanchev,
Matwey V. Kornilov,
Egor O. Mishin
Abstract:
We carried out a search for unknown dwarf novae in a public data release of the Zwicky Transient Facility survey and suspected that the object ZTF18abdlzhd is a SU UMa-type star. Performed multicolor CCD observations permit us to follow its fading from an outburst in August and an entire superoutburst in October 2020. The duration of the superoutburst is 13 days. We detected superhumps with period…
▽ More
We carried out a search for unknown dwarf novae in a public data release of the Zwicky Transient Facility survey and suspected that the object ZTF18abdlzhd is a SU UMa-type star. Performed multicolor CCD observations permit us to follow its fading from an outburst in August and an entire superoutburst in October 2020. The duration of the superoutburst is 13 days. We detected superhumps with period P = 0.06918(3) d that are characteristic of UGSU type stars.
△ Less
Submitted 4 March, 2021;
originally announced March 2021.
-
Anomaly detection in the Zwicky Transient Facility DR3
Authors:
K. L. Malanchev,
M. V. Pruzhinskaya,
V. S. Korolev,
P. D. Aleo,
M. V. Kornilov,
E. E. O. Ishida,
V. V. Krushinsky,
F. Mondon,
S. Sreejith,
A. A. Volnova,
A. A. Belinski,
A. V. Dodin,
A. M. Tatarnikov,
S. G. Zheltoukhov
Abstract:
We present results from applying the SNAD anomaly detection pipeline to the third public data release of the Zwicky Transient Facility (ZTF DR3). The pipeline is composed of 3 stages: feature extraction, search of outliers with machine learning algorithms and anomaly identification with followup by human experts. Our analysis concentrates in three ZTF fields, comprising more than 2.25 million obje…
▽ More
We present results from applying the SNAD anomaly detection pipeline to the third public data release of the Zwicky Transient Facility (ZTF DR3). The pipeline is composed of 3 stages: feature extraction, search of outliers with machine learning algorithms and anomaly identification with followup by human experts. Our analysis concentrates in three ZTF fields, comprising more than 2.25 million objects. A set of 4 automatic learning algorithms was used to identify 277 outliers, which were subsequently scrutinised by an expert. From these, 188 (68%) were found to be bogus light curves -- including effects from the image subtraction pipeline as well as overlapping between a star and a known asteroid, 66 (24%) were previously reported sources whereas 23 (8%) correspond to non-catalogued objects, with the two latter cases of potential scientific interest (e. g. 1 spectroscopically confirmed RS Canum Venaticorum star, 4 supernovae candidates, 1 red dwarf flare). Moreover, using results from the expert analysis, we were able to identify a simple bi-dimensional relation which can be used to aid filtering potentially bogus light curves in future studies. We provide a complete list of objects with potential scientific application so they can be further scrutinised by the community. These results confirm the importance of combining automatic machine learning algorithms with domain knowledge in the construction of recommendation systems for astronomy. Our code is publicly available at https://github.com/snad-space/zwad
△ Less
Submitted 2 February, 2021; v1 submitted 2 December, 2020;
originally announced December 2020.
-
Active Anomaly Detection for time-domain discoveries
Authors:
Emille E. O. Ishida,
Matwey V. Kornilov,
Konstantin L. Malanchev,
Maria V. Pruzhinskaya,
Alina A. Volnova,
Vladimir S. Korolev,
Florian Mondon,
Sreevarsha Sreejith,
Anastasia Malancheva,
Shubhomoy Das
Abstract:
We present the first evidence that adaptive learning techniques can boost the discovery of unusual objects within astronomical light curve data sets. Our method follows an active learning strategy where the learning algorithm chooses objects which can potentially improve the learner if additional information about them is provided. This new information is subsequently used to update the machine le…
▽ More
We present the first evidence that adaptive learning techniques can boost the discovery of unusual objects within astronomical light curve data sets. Our method follows an active learning strategy where the learning algorithm chooses objects which can potentially improve the learner if additional information about them is provided. This new information is subsequently used to update the machine learning model, allowing its accuracy to evolve with each new information. For the case of anomaly detection, the algorithm aims to maximize the number of scientifically interesting anomalies presented to the expert by slightly modifying the weights of a traditional Isolation Forest (IF) at each iteration. In order to demonstrate the potential of such techniques, we apply the Active Anomaly Discovery (AAD) algorithm to 2 data sets: simulated light curves from the PLAsTiCC challenge and real light curves from the Open Supernova Catalog. We compare the AAD results to those of a static IF. For both methods, we performed a detailed analysis for all objects with the ~2% highest anomaly scores. We show that, in the real data scenario, AAD was able to identify ~80\% more true anomalies than the IF. This result is the first evidence that AAD algorithms can play a central role in the search for new physics in the era of large scale sky surveys.
△ Less
Submitted 14 July, 2020; v1 submitted 29 September, 2019;
originally announced September 2019.
-
Maximum likelihood estimation for disk image parameters
Authors:
Matwey V. Kornilov
Abstract:
We present a novel technique for estimating disk parameters (the centre and the radius) from its 2D image. It is based on the maximal likelihood approach utilising both edge pixels coordinates and the image intensity gradients. We emphasise the following advantages of our likelihood model. It has closed-form formulae for parameter estimating, requiring less computational resources than iterative a…
▽ More
We present a novel technique for estimating disk parameters (the centre and the radius) from its 2D image. It is based on the maximal likelihood approach utilising both edge pixels coordinates and the image intensity gradients. We emphasise the following advantages of our likelihood model. It has closed-form formulae for parameter estimating, requiring less computational resources than iterative algorithms therefore. The likelihood model naturally distinguishes the outer and inner annulus edges. The proposed technique was evaluated on both synthetic and real data.
△ Less
Submitted 18 March, 2020; v1 submitted 24 July, 2019;
originally announced July 2019.
-
Anomaly Detection in the Open Supernova Catalog
Authors:
Maria V. Pruzhinskaya,
Konstantin L. Malanchev,
Matwey V. Kornilov,
Emille E. O. Ishida,
Florian Mondon,
Alina A. Volnova,
Vladimir S. Korolev
Abstract:
In the upcoming decade large astronomical surveys will discover millions of transients raising unprecedented data challenges in the process. Only the use of the machine learning algorithms can process such large data volumes. Most of the discovered transients will belong to the known classes of astronomical objects. However, it is expected that some transients will be rare or completely new events…
▽ More
In the upcoming decade large astronomical surveys will discover millions of transients raising unprecedented data challenges in the process. Only the use of the machine learning algorithms can process such large data volumes. Most of the discovered transients will belong to the known classes of astronomical objects. However, it is expected that some transients will be rare or completely new events of unknown physical nature. The task of finding them can be framed as an anomaly detection problem. In this work, we perform for the first time an automated anomaly detection analysis in the photometric data of the Open Supernova Catalog (OSC), which serves as a proof of concept for the applicability of these methods to future large scale surveys. The analysis consists of the following steps: 1) data selection from the OSC and approximation of the pre-processed data with Gaussian processes, 2) dimensionality reduction, 3) searching for outliers with the use of the isolation forest algorithm, 4) expert analysis of the identified outliers. The pipeline returned 81 candidate anomalies, 27 (33%) of which were confirmed to be from astrophysically peculiar objects. Found anomalies correspond to a selected sample of 1.4% of the initial automatically identified data sample of ~2000 objects. Among the identified outliers we recognised superluminous supernovae, non-classical Type Ia supernovae, unusual Type II supernovae, one active galactic nucleus and one binary microlensing event. We also found that 16 anomalies classified as supernovae in the literature are likely to be quasars or stars. Our proposed pipeline represents an effective strategy to guarantee we shall not overlook exciting new science hidden in the data we fought so hard to acquire. All code and products of this investigation are made publicly available.
△ Less
Submitted 22 August, 2019; v1 submitted 27 May, 2019;
originally announced May 2019.
-
Fips: an OpenGL based FITS viewer
Authors:
Matwey Kornilov,
Konstantin Malanchev
Abstract:
FITS (Flexible Image Transport System) is a common format for astronomical data storage. It was first standardised in the early 1980s. Even though astronomical data is now processed mostly using software, visual data inspection by a human is still important during equipment or software commissioning and while observing. We present Fips, a cross-platform FITS file viewer open source software. To th…
▽ More
FITS (Flexible Image Transport System) is a common format for astronomical data storage. It was first standardised in the early 1980s. Even though astronomical data is now processed mostly using software, visual data inspection by a human is still important during equipment or software commissioning and while observing. We present Fips, a cross-platform FITS file viewer open source software. To the best of our knowledge, it is for the first time that the image rendering algorithms are implemented mostly on GPU (graphics processing unit). We show that it is possible to implement a fully-capable FITS viewer using OpenGL interface. We also emphasise the advantages of using GPUs for efficient image handling.
△ Less
Submitted 29 January, 2019;
originally announced January 2019.
-
Astronomical observation tasks short-term scheduling using PDDS algorithm
Authors:
Matwey V. Kornilov
Abstract:
A concept of the ground-based optical astronomical observations efficiency is considered in this paper. We believe that a telescope efficiency can be increased by properly allocating observation tasks with respect to the current environment state and probability to obtain the data with required properties under the current conditions. An online observations scheduling is assumed to be essential pa…
▽ More
A concept of the ground-based optical astronomical observations efficiency is considered in this paper. We believe that a telescope efficiency can be increased by properly allocating observation tasks with respect to the current environment state and probability to obtain the data with required properties under the current conditions. An online observations scheduling is assumed to be essential part for raising the efficiency. The short-term online scheduling is treated as the discrete optimisation problems which are stated using several abstraction levels. The optimisation problems are solved using a parallel depth-bounded discrepancy search (PDDS) algorithm [13]. Some aspects of the algorithm performance are discussed. The presented algorithm is a core of open-source chelyabinsk C++ library which is supposed to be used at 2.5 m telescope of Sternberg Astronomical Institude of Lomonosov Moscow State University.
△ Less
Submitted 25 June, 2018;
originally announced June 2018.
-
Forecasting seeing and parameters of long-exposure images by means of ARIMA
Authors:
Matwey V. Kornilov
Abstract:
Atmospheric turbulence is the one of the major limiting factors for ground-based astronomical observations. In this paper, the problem of short-term forecasting seeing is discussed. The real data that were obtained by atmospheric optical turbulence (OT) measurements above Mount Shatdzhatmaz in 2007--2013 have been analysed. Linear auto-regressive integrated moving average (ARIMA) models are used f…
▽ More
Atmospheric turbulence is the one of the major limiting factors for ground-based astronomical observations. In this paper, the problem of short-term forecasting seeing is discussed. The real data that were obtained by atmospheric optical turbulence (OT) measurements above Mount Shatdzhatmaz in 2007--2013 have been analysed. Linear auto-regressive integrated moving average (ARIMA) models are used for the forecasting. A new procedure for forecasting the image characteristics of direct astronomical observations (central image intensity, full width at half maximum, radius encircling 80% of the energy) has been proposed. Probability density functions of the forecast of these quantities are 1.5--2 times thinner than the respective unconditional probability density functions. Overall, this study found that the described technique could adequately describe temporal stochastic variations of the OT power.
△ Less
Submitted 25 June, 2018;
originally announced June 2018.
-
Night-sky brightness and extinction at Mt. Shatdzhatmaz
Authors:
V. Kornilov,
M. Kornilov,
O. Voziakova,
N. Shatsky,
B. Safonov,
I. Gorbunov,
S. Potanin,
D. Cheryasov,
V. Senik
Abstract:
The photometric sky quality of Mt. Shatdzhatmaz, the site of Sternberg Astronomical Institute Caucasian Observatory 2.5 m telescope, is characterized here by the statistics of the night-time sky brightness and extinction. The data were obtained as a by-product of atmospheric optical turbulence measurements with the MASS (Multi-Aperture Scintillation Sensor) device conducted in 2007--2013. The fact…
▽ More
The photometric sky quality of Mt. Shatdzhatmaz, the site of Sternberg Astronomical Institute Caucasian Observatory 2.5 m telescope, is characterized here by the statistics of the night-time sky brightness and extinction. The data were obtained as a by-product of atmospheric optical turbulence measurements with the MASS (Multi-Aperture Scintillation Sensor) device conducted in 2007--2013. The factors biasing night-sky brightness measurements are considered and a technique to reduce their impact on the statistics is proposed.
The single-band photometric estimations provided by MASS are easy to transform to the standard photometric bands. The median moonless night-sky brightness is 22.1, 21.1, 20.3, and 19.0 mag per square arcsec for the $B$, $V$, $R$, and $I$ spectral bands, respectively. The median extinction coefficients for the same photometric bands are 0.28, 0.17, 0.13, and 0.09 mag. The best atmospheric transparency is observed in winter.
△ Less
Submitted 26 July, 2016;
originally announced July 2016.
-
Resolved photometry of the binary components of RW Aur
Authors:
S. Antipin,
A. Belinski,
A. Cherepashchuk,
D. Cherjasov,
A. Dodin,
I. Gorbunov,
S. Lamzin,
M. Kornilov,
V. Kornilov,
S. Potanin,
B. Safonov,
V. Senik,
N. Shatsky,
O. Voziakova
Abstract:
Resolved UBVRI photometry of RW Aur binary was performed on November 13/14, 2014 during the deep dimming of RW Aur with a newly installed 2.5 meter telescope of the Caucasus observatory of Lomonosov Moscow State University at the mount Shatzhatmaz. At that moment RW Aur A was $\simeq 3^m$ fainter than in November 1994 in all spectral bands. We explain the current RW Aur A dimming as a result of ec…
▽ More
Resolved UBVRI photometry of RW Aur binary was performed on November 13/14, 2014 during the deep dimming of RW Aur with a newly installed 2.5 meter telescope of the Caucasus observatory of Lomonosov Moscow State University at the mount Shatzhatmaz. At that moment RW Aur A was $\simeq 3^m$ fainter than in November 1994 in all spectral bands. We explain the current RW Aur A dimming as a result of eclipse of the star by dust particles with size $>1 μm.$ We found that RW Aur B is also a variable star: it was brighter than 20 years ago at $0.7^m$ in each of UBVRI band (gray brightening).
△ Less
Submitted 24 December, 2014;
originally announced December 2014.
-
Study on atmospheric optical turbulence above Mt. Shatdzhatmaz in 2007--2013
Authors:
Victor Kornilov,
Boris Safonov,
Matwey Kornilov,
Nicolay Shatsky,
Olga Voziakova,
Sergei Potanin,
Igor Gorbunov,
Victor Senik,
Dmitry Cheryasov
Abstract:
We present the results of the atmospheric optical turbulence (OT) measurements performed atop Mt. Shatdzhatmaz at the installation site of new 2.5-m telescope of Sternberg Astronomical Institute. Nearly 300 000 vertical OT profiles from the ground up to an altitude of 23 km were obtained in the period November 2007 - June 2013 with the combined multi-aperture scintillation sensor (MASS) and differ…
▽ More
We present the results of the atmospheric optical turbulence (OT) measurements performed atop Mt. Shatdzhatmaz at the installation site of new 2.5-m telescope of Sternberg Astronomical Institute. Nearly 300 000 vertical OT profiles from the ground up to an altitude of 23 km were obtained in the period November 2007 - June 2013 with the combined multi-aperture scintillation sensor (MASS) and differential image motion monitor (DIMM) instrument.
The medians of the main OT characteristics computed over the whole dataset are as follows: the integral seeing $β_0 = 0.96$ arcsec, the free-atmosphere seeing $β_{free} = 0.43$ arcsec, and the isoplanatic angle $θ_0 = 2.07$ arcsec. The median atmospheric time constant is $τ_0 = 6.57 \mbox{ ms}$. The revealed long-term variability of these parameters on scales of months and years implies the need to take it into account in astroclimatic campaign planning. For example, the annual variation in the monthly $θ_0$ estimate amounts up to 30% while the time constant $τ_0$ changes by a factor of 2.5.
Evaluation of the potential of Mt. Shatdzhatmaz in terms of high angular resolution observations indicates that in October--November, this site is as good as the best of studied summits in the world.
△ Less
Submitted 26 March, 2014;
originally announced March 2014.
-
Prompt, early, and afterglow optical observations of five gamma-ray bursts (GRBs 100901A, 100902A, 100905A, 100906A, and 101020A)
Authors:
E. S. Gorbovskoy,
G. V. Lipunova,
V. M. Lipunov,
V. G. Kornilov,
A. A. Belinski,
N. I. Shatskiy,
N. V. Tyurina,
D. A. Kuvshinov,
P. V. Balanutsa,
V. V. Chazov,
A. Kuznetsov,
D. S. Zimnukhov,
M. V. Kornilov,
A. V. Sankovich,
A. Krylov,
K. I. Ivanov,
O. Chvalaev,
V. A. Poleschuk,
E. N. Konstantinov,
O. A. Gress,
S. A. Yazev,
N. M. Budnev,
V. V. Krushinski,
I. S. Zalozhnich,
A. A. Popov
, et al. (13 additional authors not shown)
Abstract:
We present results of the prompt, early, and afterglow optical observations of five gamma-ray bursts, GRBs 100901A, 100902A, 100905A, 100906A, and 101020A, made with the Mobile Astronomical System of TElescope-Robots in Russia (MASTER-II net), the 1.5-m telescope of Sierra-Nevada Observatory, and the 2.56-m Nordic Optical Telescope. For two sources, GRB 100901A and GRB 100906A, we detected optical…
▽ More
We present results of the prompt, early, and afterglow optical observations of five gamma-ray bursts, GRBs 100901A, 100902A, 100905A, 100906A, and 101020A, made with the Mobile Astronomical System of TElescope-Robots in Russia (MASTER-II net), the 1.5-m telescope of Sierra-Nevada Observatory, and the 2.56-m Nordic Optical Telescope. For two sources, GRB 100901A and GRB 100906A, we detected optical counterparts and obtained light curves starting before cessation of gamma-ray emission, at 113 s and 48 s after the trigger, respectively. Observations of GRB 100906A were conducted with two polarizing filters. Observations of the other three bursts gave the upper limits on the optical flux; their properties are briefly discussed. More detailed analysis of GRB 100901A and GRB 100906A supplemented by Swift data provides the following results and indicates different origins of the prompt optical radiation in the two bursts. The light curves patterns and spectral distributions suggest a common production site of the prompt optical and high-energy emission in GRB 100901A. Results of spectral fits for GRB 100901A in the range from the optical to X-rays favor power-law energy distributions with similar values of the optical extinction in the host galaxy. GRB 100906A produced a smoothly peaking optical light curve suggesting that the prompt optical radiation in this GRB originated in a front shock. This is supported by a spectral analysis. We have found that the Amati and Ghirlanda relations are satisfied for GRB 100906A. An upper limit on the value of the optical extinction on the host of GRB 100906A is obtained.
△ Less
Submitted 15 November, 2011;
originally announced November 2011.
-
First results of site testing program at Mt. Shatdzhatmaz in 2007 - 2009
Authors:
V. Kornilov,
N. Shatsky,
O. Voziakova,
B. Safonov,
S. Potanin,
M. Kornilov
Abstract:
We present the first results of the site testing performed at Mt.~Shatdzhatmaz at Northern Caucasus, where the new Sternberg astronomical institute 2.5-m telescope will be installed. An automatic site monitor instrumentation and functionality are described together with the methods of measurement of the basic astroclimate and weather parameters. The clear night sky time derived on the basis of 200…
▽ More
We present the first results of the site testing performed at Mt.~Shatdzhatmaz at Northern Caucasus, where the new Sternberg astronomical institute 2.5-m telescope will be installed. An automatic site monitor instrumentation and functionality are described together with the methods of measurement of the basic astroclimate and weather parameters. The clear night sky time derived on the basis of 2006 -- 2009 data amounts to 1340 hours per year. Principle attention is given to the measurement of the optical turbulence altitude distribution which is the most important characteristic affecting optical telescopes performance. For the period from November 2007 to October 2009 more than 85\,000 turbulence profiles were collected using the combined MASS/DIMM instrument. The statistical properties of turbulent atmosphere above the summit are derived and the median values for seeing $β_0 = 0.93$~arcsec and free-atmosphere seeing $β_{free} = 0.51$~arcsec are determined. Together with the estimations of isoplanatic angle $θ_0 = 2.07$~arcsec and time constant $τ_0 = 2.58 \mbox{ ms}$, these are the first representative results obtained for Russian sites which are necessary for development of modern astronomical observation techniques like adaptive optics.
△ Less
Submitted 17 June, 2010;
originally announced June 2010.
-
The revision of the turbulence profiles restoration from MASS scintillation indices
Authors:
V. Kornilov,
M. Kornilov
Abstract:
The altitude distribution of optical turbulence is derived from the MASS instrument data by solving an inverse problem. In this paper, some modifications of the profile restoration are described. The principal change is the introduction of the Non Negative Least Squares algorithm which has good regularizing properties. An averaging of scintillation indices was replaced with averaging of obtained s…
▽ More
The altitude distribution of optical turbulence is derived from the MASS instrument data by solving an inverse problem. In this paper, some modifications of the profile restoration are described. The principal change is the introduction of the Non Negative Least Squares algorithm which has good regularizing properties. An averaging of scintillation indices was replaced with averaging of obtained solutions what leads to clearer physical results. It is shown that restoration with a number of turbulent layers as large as 14-15 can be successfully performed.
△ Less
Submitted 22 October, 2010; v1 submitted 25 May, 2010;
originally announced May 2010.
