Skip to main content

Showing 1–29 of 29 results for author: Kormilitzin, A

.
  1. arXiv:2404.16461  [pdf, other

    cs.CL

    Large Language Models Perform on Par with Experts Identifying Mental Health Factors in Adolescent Online Forums

    Authors: Isabelle Lorge, Dan W. Joyce, Andrey Kormilitzin

    Abstract: Mental health in children and adolescents has been steadily deteriorating over the past few years. The recent advent of Large Language Models (LLMs) offers much hope for cost and time efficient scaling of monitoring and intervention, yet despite specifically prevalent issues such as school bullying and eating disorders, previous studies on have not investigated performance in this domain or for op… ▽ More

    Submitted 26 April, 2024; v1 submitted 25 April, 2024; originally announced April 2024.

  2. arXiv:2403.19802  [pdf, other

    cs.CL cs.AI

    Developing Healthcare Language Model Embedding Spaces

    Authors: Niall Taylor, Dan Schofield, Andrey Kormilitzin, Dan W Joyce, Alejo Nevado-Holgado

    Abstract: Pre-trained Large Language Models (LLMs) often struggle on out-of-domain datasets like healthcare focused text. We explore specialized pre-training to adapt smaller LLMs to different healthcare datasets. Three methods are assessed: traditional masked language modeling, Deep Contrastive Learning for Unsupervised Textual Representations (DeCLUTR), and a novel pre-training objective utilizing metadat… ▽ More

    Submitted 28 March, 2024; originally announced March 2024.

  3. arXiv:2403.19790  [pdf, other

    cs.AI

    Bespoke Large Language Models for Digital Triage Assistance in Mental Health Care

    Authors: Niall Taylor, Andrey Kormilitzin, Isabelle Lorge, Alejo Nevado-Holgado, Dan W Joyce

    Abstract: Contemporary large language models (LLMs) may have utility for processing unstructured, narrative free-text clinical data contained in electronic health records (EHRs) -- a particularly important use-case for mental health where a majority of routinely-collected patient data lacks structured, machine-readable content. A significant problem for the the United Kingdom's National Health Service (NH… ▽ More

    Submitted 28 March, 2024; originally announced March 2024.

  4. arXiv:2402.10597  [pdf, other

    cs.CL cs.AI

    Efficiency at Scale: Investigating the Performance of Diminutive Language Models in Clinical Tasks

    Authors: Niall Taylor, Upamanyu Ghose, Omid Rohanian, Mohammadmahdi Nouriborji, Andrey Kormilitzin, David Clifton, Alejo Nevado-Holgado

    Abstract: The entry of large language models (LLMs) into research and commercial spaces has led to a trend of ever-larger models, with initial promises of generalisability, followed by a widespread desire to downsize and create specialised models without the need for complete fine-tuning, using Parameter Efficient Fine-tuning (PEFT) methods. We present an investigation into the suitability of different PEFT… ▽ More

    Submitted 16 February, 2024; originally announced February 2024.

  5. arXiv:2402.07645  [pdf, other

    cs.CL

    Detecting the Clinical Features of Difficult-to-Treat Depression using Synthetic Data from Large Language Models

    Authors: Isabelle Lorge, Dan W. Joyce, Niall Taylor, Alejo Nevado-Holgado, Andrea Cipriani, Andrey Kormilitzin

    Abstract: Difficult-to-treat depression (DTD) has been proposed as a broader and more clinically comprehensive perspective on a person's depressive disorder where despite treatment, they continue to experience significant burden. We sought to develop a Large Language Model (LLM)-based tool capable of interrogating routinely-collected, narrative (free-text) electronic health record (EHR) data to locate publi… ▽ More

    Submitted 12 February, 2024; originally announced February 2024.

  6. arXiv:2205.05535  [pdf, other

    cs.CL

    Clinical Prompt Learning with Frozen Language Models

    Authors: Niall Taylor, Yi Zhang, Dan Joyce, Alejo Nevado-Holgado, Andrey Kormilitzin

    Abstract: Prompt learning is a new paradigm in the Natural Language Processing (NLP) field which has shown impressive performance on a number of natural language tasks with common benchmarking text datasets in full, few-shot, and zero-shot train-evaluation setups. Recently, it has even been observed that large but frozen pre-trained language models (PLMs) with prompt learning outperform smaller but fine-tun… ▽ More

    Submitted 11 May, 2022; originally announced May 2022.

    Comments: 18 pages, 6 figures, 6 tables

    MSC Class: ACM-class: J.2

  7. arXiv:2111.07611  [pdf, other

    cs.CL cs.AI

    Rationale production to support clinical decision-making

    Authors: Niall Taylor, Lei Sha, Dan W Joyce, Thomas Lukasiewicz, Alejo Nevado-Holgado, Andrey Kormilitzin

    Abstract: The development of neural networks for clinical artificial intelligence (AI) is reliant on interpretability, transparency, and performance. The need to delve into the black-box neural network and derive interpretable explanations of model output is paramount. A task of high clinical importance is predicting the likelihood of a patient being readmitted to hospital in the near future to enable effic… ▽ More

    Submitted 15 November, 2021; originally announced November 2021.

    Comments: Machine Learning for Health (ML4H) - Extended Abstract

  8. arXiv:2010.12260  [pdf, other

    cs.LG cs.CV stat.ML

    Population Gradients improve performance across data-sets and architectures in object classification

    Authors: Yurika Sakai, Andrey Kormilitzin, Qiang Liu, Alejo Nevado-Holgado

    Abstract: The most successful methods such as ReLU transfer functions, batch normalization, Xavier initialization, dropout, learning rate decay, or dynamic optimizers, have become standards in the field due, particularly, to their ability to increase the performance of Neural Networks (NNs) significantly and in almost all situations. Here we present a new method to calculate the gradients while training NNs… ▽ More

    Submitted 23 October, 2020; originally announced October 2020.

  9. arXiv:2010.08433  [pdf, other

    cs.CL cs.IR

    An efficient representation of chronological events in medical texts

    Authors: Andrey Kormilitzin, Nemanja Vaci, Qiang Liu, Hao Ni, Goran Nenadic, Alejo Nevado-Holgado

    Abstract: In this work we addressed the problem of capturing sequential information contained in longitudinal electronic health records (EHRs). Clinical notes, which is a particular type of EHR data, are a rich source of information and practitioners often develop clever solutions how to maximise the sequential information contained in free-texts. We proposed a systematic methodology for learning from chron… ▽ More

    Submitted 24 October, 2020; v1 submitted 16 October, 2020; originally announced October 2020.

    Comments: 4 pages, 2 figures, 7 tables

  10. arXiv:2003.01271  [pdf, other

    cs.CL cs.IR cs.LG

    Med7: a transferable clinical natural language processing model for electronic health records

    Authors: Andrey Kormilitzin, Nemanja Vaci, Qiang Liu, Alejo Nevado-Holgado

    Abstract: The field of clinical natural language processing has been advanced significantly since the introduction of deep learning models. The self-supervised representation learning and the transfer learning paradigm became the methods of choice in many natural language processing application, in particular in the settings with the dearth of high quality manually annotated data. Electronic health record s… ▽ More

    Submitted 24 April, 2020; v1 submitted 2 March, 2020; originally announced March 2020.

    Comments: 16 pages, 1 figure, 15 tables

  11. arXiv:1908.11399  [pdf, other

    eess.IV cs.LG q-bio.QM stat.ML

    Deep Learning for Estimating Synaptic Health of Primary Neuronal Cell Culture

    Authors: Andrey Kormilitzin, Xinyu Yang, William H. Stone, Caroline Woffindale, Francesca Nicholls, Elena Ribe, Alejo Nevado-Holgado, Noel Buckley

    Abstract: Understanding the morphological changes of primary neuronal cells induced by chemical compounds is essential for drug discovery. Using the data from a single high-throughput imaging assay, a classification model for predicting the biological activity of candidate compounds was introduced. The image recognition model which is based on deep convolutional neural network (CNN) architecture with residu… ▽ More

    Submitted 29 August, 2019; originally announced August 2019.

    Comments: 11 pages, 5 figures

  12. arXiv:1901.01592  [pdf

    cs.CL cs.AI stat.ML

    Named Entity Recognition in Electronic Health Records Using Transfer Learning Bootstrapped Neural Networks

    Authors: Luka Gligic, Andrey Kormilitzin, Paul Goldberg, Alejo Nevado-Holgado

    Abstract: Neural networks (NNs) have become the state of the art in many machine learning applications, especially in image and sound processing [1]. The same, although to a lesser extent [2,3], could be said in natural language processing (NLP) tasks, such as named entity recognition. However, the success of NNs remains dependent on the availability of large labelled datasets, which is a significant hurdle… ▽ More

    Submitted 29 July, 2019; v1 submitted 6 January, 2019; originally announced January 2019.

    Comments: 11 pages, 4 figures, 8 tables

  13. arXiv:1811.05468  [pdf

    cs.CL cs.LG stat.ML

    Few-shot Learning for Named Entity Recognition in Medical Text

    Authors: Maximilian Hofer, Andrey Kormilitzin, Paul Goldberg, Alejo Nevado-Holgado

    Abstract: Deep neural network models have recently achieved state-of-the-art performance gains in a variety of natural language processing (NLP) tasks (Young, Hazarika, Poria, & Cambria, 2017). However, these gains rely on the availability of large amounts of annotated examples, without which state-of-the-art performance is rarely achievable. This is especially inconvenient for the many NLP fields where ann… ▽ More

    Submitted 13 November, 2018; originally announced November 2018.

    Comments: 10 pages, 4 figures, 4 tables

  14. arXiv:1708.01206  [pdf, other

    stat.ML

    Detecting early signs of depressive and manic episodes in patients with bipolar disorder using the signature-based model

    Authors: Andrey Kormilitzin, Kate E. A. Saunders, Paul J. Harrison, John R. Geddes, Terry Lyons

    Abstract: Recurrent major mood episodes and subsyndromal mood instability cause substantial disability in patients with bipolar disorder. Early identification of mood episodes enabling timely mood stabilisation is an important clinical goal. Recent technological advances allow the prospective reporting of mood in real time enabling more accurate, efficient data capture. The complex nature of these data stre… ▽ More

    Submitted 3 August, 2017; originally announced August 2017.

    Comments: 12 pages, 3 tables, 10 figures

  15. arXiv:1606.02074  [pdf, ps, other

    stat.AP stat.ML

    Application of the Signature Method to Pattern Recognition in the CEQUEL Clinical Trial

    Authors: A. B. Kormilitzin, K. E. A. Saunders, P. J. Harrison, J. R. Geddes, T. J. Lyons

    Abstract: The classification procedure of streaming data usually requires various ad hoc methods or particular heuristic models. We explore a novel non-parametric and systematic approach to analysis of heterogeneous sequential data. We demonstrate an application of this method to classification of the delays in responding to the prompts, from subjects with bipolar disorder collected during a clinical trial,… ▽ More

    Submitted 7 June, 2016; originally announced June 2016.

    Comments: 16 pages, 7 figures

  16. arXiv:1603.03788  [pdf, other

    stat.ML cs.LG stat.ME

    A Primer on the Signature Method in Machine Learning

    Authors: Ilya Chevyrev, Andrey Kormilitzin

    Abstract: We provide an introduction to the signature method, focusing on its theoretical properties and machine learning applications. Our presentation is divided into two parts. In the first part, we present the definition and fundamental properties of the signature of a path. The signature is a sequence of numbers associated with a path that captures many of its important analytic and geometric propertie… ▽ More

    Submitted 17 January, 2025; v1 submitted 11 March, 2016; originally announced March 2016.

    Comments: 61 pages, 26 figures, 3 tables. Expanded Part 1 and simplified the presentation in Part 2. To appear in Open Access in a forthcoming Springer volume "Signatures Methods in Finance: An Introduction with Computational Applications"

  17. Analytic structure of the $n = 7$ scattering amplitude in $\mathcal{N}=4$ SYM theory in multi-Regge kinematics: Conformal Regge cut contribution

    Authors: Jochen Bartels, Andrey Kormilitzin, Lev N. Lipatov

    Abstract: In this second part of our investigation of the analytic structure of the $2\to5$ scattering amplitude in the planar limit of $\mathcal{N}=4$ SYM in multi-Regge kinematics we compute, in all kinematic regions, the Regge cut contributions in leading order. The results are infrared finite and conformally invariant.

    Submitted 9 November, 2014; originally announced November 2014.

    Comments: 44 pages, 14 figures, 2 tables

    Journal ref: Phys. Rev. D 91, 045005 (2015)

  18. Analytic structure of the $n=7$ scattering amplitude in $\mathcal{N}=4$ SYM theory at multi-Regge kinematics: Conformal Regge pole contribution

    Authors: Jochen Bartels, Andrey Kormilitzin, Lev Lipatov

    Abstract: We investigate the analytic structure of the $2\to5$ scattering amplitude in the planar limit of $\mathcal{N}=4$ SYM in multi-Regge kinematics in all physical regions. We demonstrate the close connection between Regge pole and Regge cut contributions: in a selected class of kinematic regions (Mandelstam regions) the usual factorizing Regge pole formula develops unphysical singularities which have… ▽ More

    Submitted 19 May, 2014; v1 submitted 8 November, 2013; originally announced November 2013.

    Comments: 46 pages, references added, typos corrected, journal version

    Report number: DESY 13-209

    Journal ref: Phys. Rev. D 89 (2014), 065002

  19. BFKL approach and 2->5 MHV amplitude

    Authors: J. Bartels, A. Kormilitzin, L. N. Lipatov, A. Prygarin

    Abstract: We study MHV amplitude for the 2 -> 5 scattering in the multi-Regge kinematics. The Mandelstam cut correction to the BDS amplitude is calculated in the leading logarithmic approximation (LLA) and the corresponding remainder function is given to any loop order in a closed integral form. We show that the LLA remainder function at two loops for 2 -> 5 amplitude can be written as a sum of two 2 -> 4 r… ▽ More

    Submitted 19 March, 2012; v1 submitted 29 December, 2011; originally announced December 2011.

    Comments: 24 pages, 17 figures

  20. Geometric scaling behavior of the scattering amplitude for DIS with nuclei

    Authors: Andrey Kormilitzin, Eugene Levin, Sebastian Tapia

    Abstract: The main question, that we answer in this paper, is whether the initial condition can influence on the geometric scaling behavior of the amplitude for DIS at high energy. We re-write the non-linear Balitsky-Kovchegov equation in the form which is useful for treating the interaction with nuclei. Using the simplified BFKL kernel, we find the analytical solution to this equation with the initial cond… ▽ More

    Submitted 16 June, 2011; originally announced June 2011.

    Comments: 19pp, 11 figures in .eps files

    Report number: TAUP 2929/11

  21. arXiv:1011.1248  [pdf, ps, other

    hep-ph hep-ex nucl-ex nucl-th

    On the Nuclear Modification Factor at RHIC and LHC

    Authors: Andrey Kormilitzin, Eugene Levin, Amir H. Rezaeian

    Abstract: We show that pQCD factorization incorporated with pre-haronization energy-loss effect naturally leads to flatness of the nuclear modification factor R_{AA} for produced hadrons at high transverse momentum p_T. We consider two possible scenarios for the pre-hadronization: In scenario 1, the produced gluon propagates through dense QCD medium and loses energy. In scenario 2, all gluons first decay to… ▽ More

    Submitted 16 May, 2011; v1 submitted 4 November, 2010; originally announced November 2010.

    Comments: 14 pages, 10 figures; v2: results unchanged, more discussion and references added. The version to appear in Nucl. Phys. A

    Journal ref: Nucl.Phys.A860:84-101,2011

  22. Non-linear equation: energy conservation and impact parameter dependence

    Authors: Andrey Kormilitzin, Eugene Levin

    Abstract: In this paper we address two questions: how energy conservation affects the solution to the non-linear equation, and how impact parameter dependence influences the inclusive production. Answering the first question we solve the modified BK equation which takes into account energy conservation. In spite of the fact that we used the simplified kernel, we believe that the main result of the paper: th… ▽ More

    Submitted 8 September, 2010; originally announced September 2010.

    Comments: 24 pp. 8 figures in eps files

    Report number: TAUP 2920/10

    Journal ref: Nucl.Phys.A849:98-119,2011

  23. arXiv:1009.1329  [pdf, ps, other

    hep-ph hep-th nucl-th

    High density QCD and nucleus-nucleus scattering deeply in the saturation region

    Authors: Andrey Kormilitzin, Eugene Levin, Jeremy S. Miller

    Abstract: In this paper we solve the equations that describe nucleus nucleus scattering, in high density QCD,in the framework of the BFKL Pomeron calculus. We found that (i) the contribution of short distances to the opacity for nucleus-nucleus scattering dies at high energies, (ii) the opacity tends to unity at high energy, and (iii) the main contribution that survives comes from soft (long distance) proce… ▽ More

    Submitted 4 July, 2011; v1 submitted 7 September, 2010; originally announced September 2010.

    Comments: 25pp and 12 figures in eps format

    Report number: TAUP 2919/10

    Journal ref: Nucl.Phys.A859:87-113,2011

  24. QCD motivated approach to soft interactions at high energies: nucleus-nucleus and hadron-nucleus collisions

    Authors: E. Gotsman, A. Kormilitzin, E. Levin, U. Maor

    Abstract: In this paper we consider nucleus-nucleus and hadron-nucleus reactions in the kinematic region: $g A^{1/3} G_{3\pom} \exp\Lb ΔY\Rb \approx 1 G^2_{3\pom} \exp\Lb ΔY\Rb \approx 1 $, where $G_{3\pom}$ is the triple Pomeron coupling, $g$ is the vertex of Pomeron nucleon interaction, and 1 + $Δ_{\pom}$ denotes the Pomeron intercept. We find that in this kinematic region the traditional Glauber-Grib… ▽ More

    Submitted 23 December, 2009; originally announced December 2009.

    Comments: 18pp. 14 fugures

    Report number: TAUP -2907-09

    Journal ref: Nucl.Phys.A842:82-101,2010

  25. A QCD motivated model for soft processes

    Authors: A. Kormilitzin, E. Levin

    Abstract: In this talk we give a brief description of a QCD motivated model for both hard and soft interactions at high energies. In this model the long distance behaviour of the scattering amplitude is determined by the dipole scattering amplitude in the saturation domain. All phenomenological parameters for dipole-proton interaction were fitted from the deep inelastic scattering data and the soft proces… ▽ More

    Submitted 6 November, 2008; originally announced November 2008.

    Comments: 5 pages, figures, talk at "Diffraction'08"

    Report number: TAUP 2888-06

  26. arXiv:0809.3886  [pdf, ps, other

    hep-ph

    Soft processes at high energy without soft Pomeron: a QCD motivated model

    Authors: A. Kormilitzin, E. Levin

    Abstract: In this paper we develop a QCD motivated model for both hard and soft interactions at high energies. In this model the long distance behavior of the scattering amplitude is determined by the approximate solution to the non-linear evolution equation for parton system in the saturation domain. All phenomenological parameters for dipole-proton interaction were fitted from the deep inelastic scatter… ▽ More

    Submitted 1 March, 2009; v1 submitted 23 September, 2008; originally announced September 2008.

    Comments: 33pages, 16 figures

    Report number: TAUP 2884/08

  27. Multiparticle production in the mean field approximation of high density QCD

    Authors: Andrey Kormilitzin, Eugene Levin, Alex Prygarin

    Abstract: The generating functional is suggested for multiparticle generation processes. In mean field approximation of high density QCD two equations for new generating functional are derived: linear functional equation for an arbitrary initial condition and non-linear one for a specific initial condition. The non-linear equation has the form of Kovchegov-Levin equation for diffraction production and giv… ▽ More

    Submitted 22 July, 2008; originally announced July 2008.

    Comments: 11 pages, 5 figures

    Report number: TAUP-2879/08

    Journal ref: Nucl.Phys.A813:1-13,2008

  28. arXiv:0707.2202  [pdf, ps, other

    hep-ph

    Saturation model in the non-Glauber approach

    Authors: Andrey Kormilitzin

    Abstract: In this paper a new saturation model is presented. This model is based on the theoretical solution for the generating functional, and it is quite different and not more complicated than the Glauber-like approach used before. The model describes the structure function F_{2} of the proton, as well as the diffractive structure function F_{2}^{D}. We show the difference between our model, and the ei… ▽ More

    Submitted 31 July, 2007; v1 submitted 15 July, 2007; originally announced July 2007.

    Comments: 27 pages, 18 figures and one table, typos were corrected

  29. Survival probability for high mass diffraction

    Authors: E. Gotsman, A. Kormilitzin, E. Levin, U. Maor

    Abstract: Based on the calculation of survival probabilities, we discuss the problem of extracting the value of $G_{3P}$, the triple Pomeron 'bare' coupling constant, by comparing the large rapidity gap single high mass diffraction data in proton-proton scattering and $J/Ψ$ photo and DIS production. For p-p scattering the calculation in a three amplitude rescattering eikonal model, predicts the survival p… ▽ More

    Submitted 1 August, 2007; v1 submitted 5 February, 2007; originally announced February 2007.

    Comments: 17 pages, 8 pictures and one table

    Report number: TAUP -2846-07

    Journal ref: Eur.Phys.J.C52:295-304,2007