BERT for Long Documents: A Case Study of Automated ICD Coding
Authors:
Arash Afkanpour,
Shabir Adeel,
Hansenclever Bassani,
Arkady Epshteyn,
Hongbo Fan,
Isaac Jones,
Mahan Malihi,
Adrian Nauth,
Raj Sinha,
Sanjana Woonna,
Shiva Zamani,
Elli Kanal,
Mikhail Fomitchev,
Donny Cheung
Abstract:
Transformer models have achieved great success across many NLP problems. However, previous studies in automated ICD coding concluded that these models fail to outperform some of the earlier solutions such as CNN-based models. In this paper we challenge this conclusion. We present a simple and scalable method to process long text with the existing transformer models such as BERT. We show that this…
▽ More
Transformer models have achieved great success across many NLP problems. However, previous studies in automated ICD coding concluded that these models fail to outperform some of the earlier solutions such as CNN-based models. In this paper we challenge this conclusion. We present a simple and scalable method to process long text with the existing transformer models such as BERT. We show that this method significantly improves the previous results reported for transformer models in ICD coding, and is able to outperform one of the prominent CNN-based methods.
△ Less
Submitted 4 November, 2022;
originally announced November 2022.
Interpretable Sequence Learning for COVID-19 Forecasting
Authors:
Sercan O. Arik,
Chun-Liang Li,
Jinsung Yoon,
Rajarishi Sinha,
Arkady Epshteyn,
Long T. Le,
Vikas Menon,
Shashank Singh,
Leyou Zhang,
Nate Yoder,
Martin Nikoltchev,
Yash Sonthalia,
Hootan Nakhost,
Elli Kanal,
Tomas Pfister
Abstract:
We propose a novel approach that integrates machine learning into compartmental disease modeling to predict the progression of COVID-19. Our model is explainable by design as it explicitly shows how different compartments evolve and it uses interpretable encoders to incorporate covariates and improve performance. Explainability is valuable to ensure that the model's forecasts are credible to epide…
▽ More
We propose a novel approach that integrates machine learning into compartmental disease modeling to predict the progression of COVID-19. Our model is explainable by design as it explicitly shows how different compartments evolve and it uses interpretable encoders to incorporate covariates and improve performance. Explainability is valuable to ensure that the model's forecasts are credible to epidemiologists and to instill confidence in end-users such as policy makers and healthcare institutions. Our model can be applied at different geographic resolutions, and here we demonstrate it for states and counties in the United States. We show that our model provides more accurate forecasts, in metrics averaged across the entire US, than state-of-the-art alternatives, and that it provides qualitatively meaningful explanatory insights. Lastly, we analyze the performance of our model for different subgroups based on the subgroup distributions within the counties.
△ Less
Submitted 13 January, 2021; v1 submitted 3 August, 2020;
originally announced August 2020.
