-
A new discrimination measure for assessing predictive performance of non-linear survival models
Authors:
Alfensi Faruk,
Jan Palczewski,
Georgios Aivaliotis
Abstract:
Non-linear survival models are flexible models in which the proportional hazard assumption is not required. This poses difficulties in their evaluation. We introduce a new discrimination measure, time-dependent Uno's C-index, to assess the discrimination performance of non-linear survival models. This is an unbiased version of Antolini's time-dependent concordance. We prove convergence of both mea…
▽ More
Non-linear survival models are flexible models in which the proportional hazard assumption is not required. This poses difficulties in their evaluation. We introduce a new discrimination measure, time-dependent Uno's C-index, to assess the discrimination performance of non-linear survival models. This is an unbiased version of Antolini's time-dependent concordance. We prove convergence of both measures employing Nolan and Pollard's results on U-statistics. We explore the relationship between these measures and, in particular, the bias of Antolini's concordance in the presence of censoring using simulated data. We demonstrate the value of time-dependent Uno's C-index for the evaluation of models trained on censored real data and for model tuning.
△ Less
Submitted 7 April, 2025;
originally announced April 2025.
-
Exact Bayesian inference for Markov switching diffusions
Authors:
Timothée Stumpf-Fétizon,
Krzysztof Łatuszyński,
Jan Palczewski,
Gareth Roberts
Abstract:
We give the first exact Bayesian methodology for the problem of inference in discretely observed regime switching diffusions. We design an MCMC and an MCEM algorithm that target the exact posterior of diffusion parameters and the latent regime process. The algorithms are exact in the sense that they target the correct posterior distribution of the continuous model, so that the errors are due to Mo…
▽ More
We give the first exact Bayesian methodology for the problem of inference in discretely observed regime switching diffusions. We design an MCMC and an MCEM algorithm that target the exact posterior of diffusion parameters and the latent regime process. The algorithms are exact in the sense that they target the correct posterior distribution of the continuous model, so that the errors are due to Monte Carlo only. Switching diffusion models extend ordinary diffusions by allowing for jumps in instantaneous drift and volatility. The jumps are driven by a latent, continuous time Markov switching process. We illustrate the method on numerical examples, including an empirical analysis of the method's scalability in the length of the time series, and find that it is comparable in computational cost with discrete approximations while avoiding their shortcomings.
△ Less
Submitted 13 February, 2025;
originally announced February 2025.
-
A Bayesian Mixture Model Approach to Expected Possession Values in Rugby League
Authors:
Thomas Sawczuk,
Anna Palczewska,
Ben Jones,
Jan Palczewski
Abstract:
The aim of this study was to improve previous zonal approaches to expected possession value (EPV) models in low data availability sports by introducing a Bayesian Mixture Model approach to an EPV model in rugby league. 99,966 observations from the 2021 Super League season were used. A set of 33 centres (30 in the field of play, 3 in the try area) were located across the pitch. Each centre held the…
▽ More
The aim of this study was to improve previous zonal approaches to expected possession value (EPV) models in low data availability sports by introducing a Bayesian Mixture Model approach to an EPV model in rugby league. 99,966 observations from the 2021 Super League season were used. A set of 33 centres (30 in the field of play, 3 in the try area) were located across the pitch. Each centre held the probability of five possession outcomes occurring (converted/unconverted try, penalty, drop goal and no points). Weights for the model were provided for each location on the pitch using linear and bilinear interpolation techniques. Probabilities at each centre were estimated using a Bayesian approach and extrapolated to all locations on the pitch. An EPV measure was derived from the possession outcome probabilities and their points value. The model produced a smooth pitch surface, which was able to provide different possession outcome probabilities and EPVs for every location on the pitch. Differences between team attacking and defensive plots were visualised and an actual vs expected player rating system was developed. The model provides significantly more flexibility than previous approaches and could be adapted to other sports where data is similarly sparse.
△ Less
Submitted 21 December, 2022;
originally announced December 2022.
-
Use of Kernel Density Estimation to understand the spatial trends of attacking possessions in rugby league
Authors:
Thomas Sawczuk,
Anna Palczewska,
Ben Jones,
Jan Palczewski
Abstract:
Despite having the potential to provide significant insights into tactical preparations for future matches, very few studies have considered the spatial trends of team attacking possessions in rugby league. Those which have considered these trends have used grid based aggregation methods, which provide a discrete understanding of rugby league match play but may fail to provide a complete understan…
▽ More
Despite having the potential to provide significant insights into tactical preparations for future matches, very few studies have considered the spatial trends of team attacking possessions in rugby league. Those which have considered these trends have used grid based aggregation methods, which provide a discrete understanding of rugby league match play but may fail to provide a complete understanding of the spatial trends of attacking possessions due to the dynamic nature of the sport. In this study, we use Kernel Density Estimation (KDE) to provide a continuous understanding of the spatial trends of attacking possessions in rugby league on a team by team basis. We use the Wasserstein distance to understand the differences between teams (i.e. using all of each team's data) and within teams (i.e. using a single team's data against different opponents). Our results show that KDEs are able to provide interesting tactical insights at the between team level. Furthermore, at the within team level, the results are able to show patterns of spatial trends for attacking teams, which are present against some opponents but not others. The results could help sports practitioners to understand opposition teams' previous performances and prepare tactical strategies for matches against them.
△ Less
Submitted 16 June, 2022;
originally announced June 2022.
-
Automatic model training under restrictive time constraints
Authors:
Lukas Cironis,
Jan Palczewski,
Georgios Aivaliotis
Abstract:
We develop a hyperparameter optimisation algorithm, Automated Budget Constrained Training (AutoBCT), which balances the quality of a model with the computational cost required to tune it. The relationship between hyperparameters, model quality and computational cost must be learnt and this learning is incorporated directly into the optimisation problem. At each training epoch, the algorithm decide…
▽ More
We develop a hyperparameter optimisation algorithm, Automated Budget Constrained Training (AutoBCT), which balances the quality of a model with the computational cost required to tune it. The relationship between hyperparameters, model quality and computational cost must be learnt and this learning is incorporated directly into the optimisation problem. At each training epoch, the algorithm decides whether to terminate or continue training, and, in the latter case, what values of hyperparameters to use. This decision weighs optimally potential improvements in the quality with the additional training time and the uncertainty about the learnt quantities. The performance of our algorithm is verified on a number of machine learning problems encompassing random forests and neural networks. Our approach is rooted in the theory of Markov decision processes with partial information and we develop a numerical method to compute the value function and an optimal strategy.
△ Less
Submitted 21 April, 2021;
originally announced April 2021.
-
Bayesian calibration and number of jump components in electricity spot price models
Authors:
Jhonny Gonzalez,
John Moriarty,
Jan Palczewski
Abstract:
We find empirical evidence that mean-reverting jump processes are not statistically adequate to model electricity spot price spikes but independent, signed sums of such processes are statistically adequate. Further we demonstrate a change in the composition of these sums after a major economic event. This is achieved by developing a Markov Chain Monte Carlo (MCMC) procedure for Bayesian model cali…
▽ More
We find empirical evidence that mean-reverting jump processes are not statistically adequate to model electricity spot price spikes but independent, signed sums of such processes are statistically adequate. Further we demonstrate a change in the composition of these sums after a major economic event. This is achieved by developing a Markov Chain Monte Carlo (MCMC) procedure for Bayesian model calibration and a Bayesian assessment of model adequacy (posterior predictive checking). In particular we determine the number of signed mean-reverting jump components required in the APXUK and EEX markets, in time periods both before and after the recent global financial crises. Statistically, consistent structural changes occur across both markets, with a reduction of the intensity and size, or the disappearance, of positive price spikes in the later period. All code and data are provided to enable replication of results.
△ Less
Submitted 5 May, 2017; v1 submitted 12 January, 2016;
originally announced January 2016.
-
Asymptotics of Monte Carlo maximum likelihood estimators
Authors:
Blazej Miasojedow,
Wojciech Niemiro,
Jan Palczewski,
Wojciech Rejchel
Abstract:
We describe Monte Carlo approximation to the maximum likelihood estimator in models with intractable norming constants and explanatory variables. We consider both sources of randomness (due to the initial sample and to Monte Carlo simulations) and prove asymptotical normality of the estimator.
We describe Monte Carlo approximation to the maximum likelihood estimator in models with intractable norming constants and explanatory variables. We consider both sources of randomness (due to the initial sample and to Monte Carlo simulations) and prove asymptotical normality of the estimator.
△ Less
Submitted 19 December, 2014;
originally announced December 2014.
-
Adaptive Monte Carlo Maximum Likelihood
Authors:
Blazej Miasojedow,
Wojciech Niemiro,
Jan Palczewski,
Wojciech Rejchel
Abstract:
We consider Monte Carlo approximations to the maximum likelihood estimator in models with intractable norming constants. This paper deals with adaptive Monte Carlo algorithms, which adjust control parameters in the course of simulation. We examine asymptotics of adaptive importance sampling and a new algorithm, which uses resampling and MCMC. This algorithm is designed to reduce problems with dege…
▽ More
We consider Monte Carlo approximations to the maximum likelihood estimator in models with intractable norming constants. This paper deals with adaptive Monte Carlo algorithms, which adjust control parameters in the course of simulation. We examine asymptotics of adaptive importance sampling and a new algorithm, which uses resampling and MCMC. This algorithm is designed to reduce problems with degeneracy of importance weights. Our analysis is based on martingale limit theorems. We also describe how adaptive maximization algorithms of Newton-Raphson type can be combined with the resampling techniques. The paper includes results of a small scale simulation study in which we compare the performance of adaptive and non-adaptive Monte Carlo maximum likelihood algorithms.
△ Less
Submitted 19 December, 2014;
originally announced December 2014.
