Showing 1–37 of 37 results for author: Pepe, A

Efficient MedSAMs: Segment Anything in Medical Images on Laptop

Authors: Jun Ma, Feifei Li, Sumin Kim, Reza Asakereh, Bao-Hiep Le, Dang-Khoa Nguyen-Vu, Alexander Pfefferle, Muxin Wei, Ruochen Gao, Donghang Lyu, Songxiao Yang, Lennart Purucker, Zdravko Marinov, Marius Staring, Haisheng Lu, Thuy Thanh Dao, Xincheng Ye, Zhi Li, Gianluca Brugnara, Philipp Vollmuth, Martha Foltyn-Dumitru, Jaeyoung Cho, Mustafa Ahmed Mahmutoglu, Martin Bendszus, Irada Pflüger , et al. (57 additional authors not shown)

Abstract: Promptable segmentation foundation models have emerged as a transformative approach to addressing the diverse needs in medical images, but most existing models require expensive computing, posing a big barrier to their adoption in clinical practice. In this work, we organized the first international competition dedicated to promptable medical image segmentation, featuring a large-scale dataset spa… ▽ More Promptable segmentation foundation models have emerged as a transformative approach to addressing the diverse needs in medical images, but most existing models require expensive computing, posing a big barrier to their adoption in clinical practice. In this work, we organized the first international competition dedicated to promptable medical image segmentation, featuring a large-scale dataset spanning nine common imaging modalities from over 20 different institutions. The top teams developed lightweight segmentation foundation models and implemented an efficient inference pipeline that substantially reduced computational requirements while maintaining state-of-the-art segmentation accuracy. Moreover, the post-challenge phase advanced the algorithms through the design of performance booster and reproducibility tasks, resulting in improved algorithms and validated reproducibility of the winning solution. Furthermore, the best-performing algorithms have been incorporated into the open-source software with a user-friendly interface to facilitate clinical adoption. The data and code are publicly available to foster the further development of medical image segmentation foundation models and pave the way for impactful real-world applications. △ Less

Submitted 20 December, 2024; originally announced December 2024.

Comments: CVPR 2024 MedSAM on Laptop Competition Summary: https://www.codabench.org/competitions/1847/

STAResNet: a Network in Spacetime Algebra to solve Maxwell's PDEs

Authors: Alberto Pepe, Sven Buchholz, Joan Lasenby

Abstract: We introduce STAResNet, a ResNet architecture in Spacetime Algebra (STA) to solve Maxwell's partial differential equations (PDEs). Recently, networks in Geometric Algebra (GA) have been demonstrated to be an asset for truly geometric machine learning. In \cite{brandstetter2022clifford}, GA networks have been employed for the first time to solve partial differential equations (PDEs), demonstrating… ▽ More We introduce STAResNet, a ResNet architecture in Spacetime Algebra (STA) to solve Maxwell's partial differential equations (PDEs). Recently, networks in Geometric Algebra (GA) have been demonstrated to be an asset for truly geometric machine learning. In \cite{brandstetter2022clifford}, GA networks have been employed for the first time to solve partial differential equations (PDEs), demonstrating an increased accuracy over real-valued networks. In this work we solve Maxwell's PDEs both in GA and STA employing the same ResNet architecture and dataset, to discuss the impact that the choice of the right algebra has on the accuracy of GA networks. Our study on STAResNet shows how the correct geometric embedding in Clifford Networks gives a mean square error (MSE), between ground truth and estimated fields, up to 2.6 times lower than than obtained with a standard Clifford ResNet with 6 times fewer trainable parameters. STAREsNet demonstrates consistently lower MSE and higher correlation regardless of scenario. The scenarios tested are: sampling period of the dataset; presence of obstacles with either seen or unseen configurations; the number of channels in the ResNet architecture; the number of rollout steps; whether the field is in 2D or 3D space. This demonstrates how choosing the right algebra in Clifford networks is a crucial factor for more compact, accurate, descriptive and better generalising pipelines. △ Less

Submitted 24 August, 2024; originally announced August 2024.

Deep Medial Voxels: Learned Medial Axis Approximations for Anatomical Shape Modeling

Authors: Antonio Pepe, Richard Schussnig, Jianning Li, Christina Gsaxner, Dieter Schmalstieg, Jan Egger

Abstract: Shape reconstruction from imaging volumes is a recurring need in medical image analysis. Common workflows start with a segmentation step, followed by careful post-processing and,finally, ad hoc meshing algorithms. As this sequence can be timeconsuming, neural networks are trained to reconstruct shapes through template deformation. These networks deliver state-ofthe-art results without manual inter… ▽ More Shape reconstruction from imaging volumes is a recurring need in medical image analysis. Common workflows start with a segmentation step, followed by careful post-processing and,finally, ad hoc meshing algorithms. As this sequence can be timeconsuming, neural networks are trained to reconstruct shapes through template deformation. These networks deliver state-ofthe-art results without manual intervention, but, so far, they have primarily been evaluated on anatomical shapes with little topological variety between individuals. In contrast, other works favor learning implicit shape models, which have multiple benefits for meshing and visualization. Our work follows this direction by introducing deep medial voxels, a semi-implicit representation that faithfully approximates the topological skeleton from imaging volumes and eventually leads to shape reconstruction via convolution surfaces. Our reconstruction technique shows potential for both visualization and computer simulations. △ Less

Submitted 18 March, 2024; originally announced March 2024.

Comments: 10 pages

GroomGen: A High-Quality Generative Hair Model Using Hierarchical Latent Representations

Authors: Yuxiao Zhou, Menglei Chai, Alessandro Pepe, Markus Gross, Thabo Beeler

Abstract: Despite recent successes in hair acquisition that fits a high-dimensional hair model to a specific input subject, generative hair models, which establish general embedding spaces for encoding, editing, and sampling diverse hairstyles, are way less explored. In this paper, we present GroomGen, the first generative model designed for hair geometry composed of highly-detailed dense strands. Our appro… ▽ More Despite recent successes in hair acquisition that fits a high-dimensional hair model to a specific input subject, generative hair models, which establish general embedding spaces for encoding, editing, and sampling diverse hairstyles, are way less explored. In this paper, we present GroomGen, the first generative model designed for hair geometry composed of highly-detailed dense strands. Our approach is motivated by two key ideas. First, we construct hair latent spaces covering both individual strands and hairstyles. The latent spaces are compact, expressive, and well-constrained for high-quality and diverse sampling. Second, we adopt a hierarchical hair representation that parameterizes a complete hair model to three levels: single strands, sparse guide hairs, and complete dense hairs. This representation is critical to the compactness of latent spaces, the robustness of training, and the efficiency of inference. Based on this hierarchical latent representation, our proposed pipeline consists of a strand-VAE and a hairstyle-VAE that encode an individual strand and a set of guide hairs to their respective latent spaces, and a hybrid densification step that populates sparse guide hairs to a dense hair model. GroomGen not only enables novel hairstyle sampling and plausible hairstyle interpolation, but also supports interactive editing of complex hairstyles, or can serve as strong data-driven prior for hairstyle reconstruction from images. We demonstrate the superiority of our approach with qualitative examples of diverse sampled hairstyles and quantitative evaluation of generation quality regarding every single component and the entire pipeline. △ Less

Submitted 16 November, 2023; v1 submitted 3 November, 2023; originally announced November 2023.

Comments: SIGGRAPH Asia 2023

Journal ref: ACM Trans. Graph. 42, 6, Article 267 (December 2023)

Anatomy Completor: A Multi-class Completion Framework for 3D Anatomy Reconstruction

Authors: Jianning Li, Antonio Pepe, Gijs Luijten, Christina Schwarz-Gsaxner, Jens Kleesiek, Jan Egger

Abstract: In this paper, we introduce a completion framework to reconstruct the geometric shapes of various anatomies, including organs, vessels and muscles. Our work targets a scenario where one or multiple anatomies are missing in the imaging data due to surgical, pathological or traumatic factors, or simply because these anatomies are not covered by image acquisition. Automatic reconstruction of the miss… ▽ More In this paper, we introduce a completion framework to reconstruct the geometric shapes of various anatomies, including organs, vessels and muscles. Our work targets a scenario where one or multiple anatomies are missing in the imaging data due to surgical, pathological or traumatic factors, or simply because these anatomies are not covered by image acquisition. Automatic reconstruction of the missing anatomies benefits many applications, such as organ 3D bio-printing, whole-body segmentation, animation realism, paleoradiology and forensic imaging. We propose two paradigms based on a 3D denoising auto-encoder (DAE) to solve the anatomy reconstruction problem: (i) the DAE learns a many-to-one mapping between incomplete and complete instances; (ii) the DAE learns directly a one-to-one residual mapping between the incomplete instances and the target anatomies. We apply a loss aggregation scheme that enables the DAE to learn the many-to-one mapping more effectively and further enhances the learning of the residual mapping. On top of this, we extend the DAE to a multiclass completor by assigning a unique label to each anatomy involved. We evaluate our method using a CT dataset with whole-body segmentations. Results show that our method produces reasonable anatomy reconstructions given instances with different levels of incompleteness (i.e., one or multiple random anatomies are missing). Codes and pretrained models are publicly available at https://github.com/Jianningli/medshapenet-feedback/ tree/main/anatomy-completor △ Less

Submitted 10 September, 2023; originally announced September 2023.

Comments: 15 pages

MedShapeNet -- A Large-Scale Dataset of 3D Medical Shapes for Computer Vision

Authors: Jianning Li, Zongwei Zhou, Jiancheng Yang, Antonio Pepe, Christina Gsaxner, Gijs Luijten, Chongyu Qu, Tiezheng Zhang, Xiaoxi Chen, Wenxuan Li, Marek Wodzinski, Paul Friedrich, Kangxian Xie, Yuan Jin, Narmada Ambigapathy, Enrico Nasca, Naida Solak, Gian Marco Melito, Viet Duc Vu, Afaque R. Memon, Christopher Schlachta, Sandrine De Ribaupierre, Rajnikant Patel, Roy Eagleson, Xiaojun Chen , et al. (132 additional authors not shown)

Abstract: Prior to the deep learning era, shape was commonly used to describe the objects. Nowadays, state-of-the-art (SOTA) algorithms in medical imaging are predominantly diverging from computer vision, where voxel grids, meshes, point clouds, and implicit surface models are used. This is seen from numerous shape-related publications in premier vision conferences as well as the growing popularity of Shape… ▽ More Prior to the deep learning era, shape was commonly used to describe the objects. Nowadays, state-of-the-art (SOTA) algorithms in medical imaging are predominantly diverging from computer vision, where voxel grids, meshes, point clouds, and implicit surface models are used. This is seen from numerous shape-related publications in premier vision conferences as well as the growing popularity of ShapeNet (about 51,300 models) and Princeton ModelNet (127,915 models). For the medical domain, we present a large collection of anatomical shapes (e.g., bones, organs, vessels) and 3D models of surgical instrument, called MedShapeNet, created to facilitate the translation of data-driven vision algorithms to medical applications and to adapt SOTA vision algorithms to medical problems. As a unique feature, we directly model the majority of shapes on the imaging data of real patients. As of today, MedShapeNet includes 23 dataset with more than 100,000 shapes that are paired with annotations (ground truth). Our data is freely accessible via a web interface and a Python application programming interface (API) and can be used for discriminative, reconstructive, and variational benchmarks as well as various applications in virtual, augmented, or mixed reality, and 3D printing. Exemplary, we present use cases in the fields of classification of brain tumors, facial and skull reconstructions, multi-class anatomy completion, education, and 3D printing. In future, we will extend the data and improve the interfaces. The project pages are: https://medshapenet.ikim.nrw/ and https://github.com/Jianningli/medshapenet-feedback △ Less

Submitted 12 December, 2023; v1 submitted 30 August, 2023; originally announced August 2023.

Comments: 16 pages

MSC Class: 68T01

CGA-PoseNet: Camera Pose Regression via a 1D-Up Approach to Conformal Geometric Algebra

Authors: Alberto Pepe, Joan Lasenby

Abstract: We introduce CGA-PoseNet, which uses the 1D-Up approach to Conformal Geometric Algebra (CGA) to represent rotations and translations with a single mathematical object, the motor, for camera pose regression. We do so starting from PoseNet, which successfully predicts camera poses from small datasets of RGB frames. State-of-the-art methods, however, require expensive tuning to balance the orientatio… ▽ More We introduce CGA-PoseNet, which uses the 1D-Up approach to Conformal Geometric Algebra (CGA) to represent rotations and translations with a single mathematical object, the motor, for camera pose regression. We do so starting from PoseNet, which successfully predicts camera poses from small datasets of RGB frames. State-of-the-art methods, however, require expensive tuning to balance the orientational and translational components of the camera pose.This is usually done through complex, ad-hoc loss function to be minimized, and in some cases also requires 3D points as well as images. Our approach has the advantage of unifying the camera position and orientation through the motor. Consequently, the network searches for a single object which lives in a well-behaved 4D space with a Euclidean signature. This means that we can address the case of image-only datasets and work efficiently with a simple loss function, namely the mean squared error (MSE) between the predicted and ground truth motors. We show that it is possible to achieve high accuracy camera pose regression with a significantly simpler problem formulation. This 1D-Up approach to CGA can be employed to overcome the dichotomy between translational and orientational components in camera pose regression in a compact and elegant way. △ Less

Submitted 10 February, 2023; originally announced February 2023.

The HoloLens in Medicine: A systematic Review and Taxonomy

Authors: Christina Gsaxner, Jianning Li, Antonio Pepe, Yuan Jin, Jens Kleesiek, Dieter Schmalstieg, Jan Egger

Abstract: The HoloLens (Microsoft Corp., Redmond, WA), a head-worn, optically see-through augmented reality display, is the main player in the recent boost in medical augmented reality research. In medical settings, the HoloLens enables the physician to obtain immediate insight into patient information, directly overlaid with their view of the clinical scenario, the medical student to gain a better understa… ▽ More The HoloLens (Microsoft Corp., Redmond, WA), a head-worn, optically see-through augmented reality display, is the main player in the recent boost in medical augmented reality research. In medical settings, the HoloLens enables the physician to obtain immediate insight into patient information, directly overlaid with their view of the clinical scenario, the medical student to gain a better understanding of complex anatomies or procedures, and even the patient to execute therapeutic tasks with improved, immersive guidance. In this systematic review, we provide a comprehensive overview of the usage of the first-generation HoloLens within the medical domain, from its release in March 2016, until the year of 2021, were attention is shifting towards it's successor, the HoloLens 2. We identified 171 relevant publications through a systematic search of the PubMed and Scopus databases. We analyze these publications in regard to their intended use case, technical methodology for registration and tracking, data sources, visualization as well as validation and evaluation. We find that, although the feasibility of using the HoloLens in various medical scenarios has been shown, increased efforts in the areas of precision, reliability, usability, workflow and perception are necessary to establish AR in clinical practice. △ Less

Submitted 6 September, 2022; originally announced September 2022.

Comments: 35 pages, 11 figures

Back to the Roots: Reconstructing Large and Complex Cranial Defects using an Image-based Statistical Shape Model

Authors: Jianning Li, David G. Ellis, Antonio Pepe, Christina Gsaxner, Michele R. Aizenberg, Jens Kleesiek, Jan Egger

Abstract: Designing implants for large and complex cranial defects is a challenging task, even for professional designers. Current efforts on automating the design process focused mainly on convolutional neural networks (CNN), which have produced state-of-the-art results on reconstructing synthetic defects. However, existing CNN-based methods have been difficult to translate to clinical practice in craniopl… ▽ More Designing implants for large and complex cranial defects is a challenging task, even for professional designers. Current efforts on automating the design process focused mainly on convolutional neural networks (CNN), which have produced state-of-the-art results on reconstructing synthetic defects. However, existing CNN-based methods have been difficult to translate to clinical practice in cranioplasty, as their performance on complex and irregular cranial defects remains unsatisfactory. In this paper, a statistical shape model (SSM) built directly on the segmentation masks of the skulls is presented. We evaluate the SSM on several cranial implant design tasks, and the results show that, while the SSM performs suboptimally on synthetic defects compared to CNN-based approaches, it is capable of reconstructing large and complex defects with only minor manual corrections. The quality of the resulting implants is examined and assured by experienced neurosurgeons. In contrast, CNN-based approaches, even with massive data augmentation, fail or produce less-than-satisfactory implants for these cases. Codes are publicly available at https://github.com/Jianningli/ssm △ Less

Submitted 12 April, 2022; originally announced April 2022.

Comments: 9 pages

Stochastic Modeling of Inhomogeneities in the Aortic Wall and Uncertainty Quantification using a Bayesian Encoder-Decoder Surrogate

Authors: Sascha Ranftl, Malte Rolf-Pissarczyk, Gloria Wolkerstorfer, Antonio Pepe, Jan Egger, Wolfgang von der Linden, Gerhard A. Holzapfel

Abstract: Inhomogeneities in the aortic wall can lead to localized stress accumulations, possibly initiating dissection. In many cases, a dissection results from pathological changes such as fragmentation or loss of elastic fibers. But it has been shown that even the healthy aortic wall has an inherent heterogeneous microstructure. Some parts of the aorta are particularly susceptible to the development of i… ▽ More Inhomogeneities in the aortic wall can lead to localized stress accumulations, possibly initiating dissection. In many cases, a dissection results from pathological changes such as fragmentation or loss of elastic fibers. But it has been shown that even the healthy aortic wall has an inherent heterogeneous microstructure. Some parts of the aorta are particularly susceptible to the development of inhomogeneities due to pathological changes, however, the distribution in the aortic wall and the spatial extent, such as size, shape, and type, are difficult to predict. Motivated by this observation, we describe the heterogeneous distribution of elastic fiber degradation in the dissected aortic wall using a stochastic constitutive model. For this purpose, random field realizations, which model the stochastic distribution of degraded elastic fibers, are generated over a non-equidistant grid. The random field then serves as input for a uni-axial extension test of the pathological aortic wall, solved with the finite-element (FE) method. To include the microstructure of the dissected aortic wall, a constitutive model developed in a previous study is applied, which also includes an approach to model the degradation of inter-lamellar elastic fibers. Then to assess the uncertainty in the output stress distribution due to this stochastic constitutive model, a convolutional neural network, specifically a Bayesian encoder-decoder, was used as a surrogate model that maps the random input fields to the output stress distribution obtained from the FE analysis. The results show that the neural network is able to predict the stress distribution of the FE analysis while significantly reducing the computational time. In addition, it provides the probability for exceeding critical stresses within the aortic wall, which could allow for the prediction of delamination or fatal rupture. △ Less

Submitted 21 February, 2022; originally announced February 2022.

Comments: Preprint. 55 pages, 8 figures

Automated cross-sectional view selection in CT angiography of aortic dissections with uncertainty awareness and retrospective clinical annotations

Authors: Antonio Pepe, Jan Egger, Marina Codari, Martin J. Willemink, Christina Gsaxner, Jianning Li, Peter M. Roth, Gabriel Mistelbauer, Dieter Schmalstieg, Dominik Fleischmann

Abstract: Objective: Surveillance imaging of chronic aortic diseases, such as dissections, relies on obtaining and comparing cross-sectional diameter measurements at predefined aortic landmarks, over time. Due to a lack of robust tools, the orientation of the cross-sectional planes is defined manually by highly trained operators. We show how manual annotations routinely collected in a clinic can be efficien… ▽ More Objective: Surveillance imaging of chronic aortic diseases, such as dissections, relies on obtaining and comparing cross-sectional diameter measurements at predefined aortic landmarks, over time. Due to a lack of robust tools, the orientation of the cross-sectional planes is defined manually by highly trained operators. We show how manual annotations routinely collected in a clinic can be efficiently used to ease this task, despite the presence of a non-negligible interoperator variability in the measurements. Impact: Ill-posed but repetitive imaging tasks can be eased or automated by leveraging imperfect, retrospective clinical annotations. Methodology: In this work, we combine convolutional neural networks and uncertainty quantification methods to predict the orientation of such cross-sectional planes. We use clinical data randomly processed by 11 operators for training, and test on a smaller set processed by 3 independent operators to assess interoperator variability. Results: Our analysis shows that manual selection of cross-sectional planes is characterized by 95% limits of agreement (LOA) of $10.6^\circ$ and $21.4^\circ$ per angle. Our method showed to decrease static error by $3.57^\circ$ ($40.2$%) and $4.11^\circ$ ($32.8$%) against state of the art and LOA by $5.4^\circ$ ($49.0$%) and $16.0^\circ$ ($74.6$%) against manual processing. Conclusion: This suggests that pre-existing annotations can be an inexpensive resource in clinics to ease ill-posed and repetitive tasks like cross-section extraction for surveillance of aortic dissections. △ Less

Submitted 22 November, 2021; originally announced November 2021.

Comments: 28 pages

Learning to Rearrange Voxels in Binary Segmentation Masks for Smooth Manifold Triangulation

Authors: Jianning Li, Antonio Pepe, Christina Gsaxner, Yuan Jin, Jan Egger

Abstract: Medical images, especially volumetric images, are of high resolution and often exceed the capacity of standard desktop GPUs. As a result, most deep learning-based medical image analysis tasks require the input images to be downsampled, often substantially, before these can be fed to a neural network. However, downsampling can lead to a loss of image quality, which is undesirable especially in reco… ▽ More Medical images, especially volumetric images, are of high resolution and often exceed the capacity of standard desktop GPUs. As a result, most deep learning-based medical image analysis tasks require the input images to be downsampled, often substantially, before these can be fed to a neural network. However, downsampling can lead to a loss of image quality, which is undesirable especially in reconstruction tasks, where the fine geometric details need to be preserved. In this paper, we propose that high-resolution images can be reconstructed in a coarse-to-fine fashion, where a deep learning algorithm is only responsible for generating a coarse representation of the image, which consumes moderate GPU memory. For producing the high-resolution outcome, we propose two novel methods: learned voxel rearrangement of the coarse output and hierarchical image synthesis. Compared to the coarse output, the high-resolution counterpart allows for smooth surface triangulation, which can be 3D-printed in the highest possible quality. Experiments of this paper are carried out on the dataset of AutoImplant 2021 (https://autoimplant2021.grand-challenge.org/), a MICCAI challenge on cranial implant design. The dataset contains high-resolution skulls that can be viewed as 2D manifolds embedded in a 3D space. Codes associated with this study can be accessed at https://github.com/Jianningli/voxel_rearrangement. △ Less

Submitted 11 August, 2021; originally announced August 2021.

Comments: 18 pages, 11 figures, 1 table, 14 references

AI-based Aortic Vessel Tree Segmentation for Cardiovascular Diseases Treatment: Status Quo

Authors: Yuan Jin, Antonio Pepe, Jianning Li, Christina Gsaxner, Fen-hua Zhao, Kelsey L. Pomykala, Jens Kleesiek, Alejandro F. Frangi, Jan Egger

Abstract: The aortic vessel tree is composed of the aorta and its branching arteries, and plays a key role in supplying the whole body with blood. Aortic diseases, like aneurysms or dissections, can lead to an aortic rupture, whose treatment with open surgery is highly risky. Therefore, patients commonly undergo drug treatment under constant monitoring, which requires regular inspections of the vessels thro… ▽ More The aortic vessel tree is composed of the aorta and its branching arteries, and plays a key role in supplying the whole body with blood. Aortic diseases, like aneurysms or dissections, can lead to an aortic rupture, whose treatment with open surgery is highly risky. Therefore, patients commonly undergo drug treatment under constant monitoring, which requires regular inspections of the vessels through imaging. The standard imaging modality for diagnosis and monitoring is computed tomography (CT), which can provide a detailed picture of the aorta and its branching vessels if completed with a contrast agent, called CT angiography (CTA). Optimally, the whole aortic vessel tree geometry from consecutive CTAs is overlaid and compared. This allows not only detection of changes in the aorta, but also of its branches, caused by the primary pathology or newly developed. When performed manually, this reconstruction requires slice by slice contouring, which could easily take a whole day for a single aortic vessel tree, and is therefore not feasible in clinical practice. Automatic or semi-automatic vessel tree segmentation algorithms, however, can complete this task in a fraction of the manual execution time and run in parallel to the clinical routine of the clinicians. In this paper, we systematically review computing techniques for the automatic and semi-automatic segmentation of the aortic vessel tree. The review concludes with an in-depth discussion on how close these state-of-the-art approaches are to an application in clinical practice and how active this research field is, taking into account the number of publications, datasets and challenges. △ Less

Submitted 3 April, 2023; v1 submitted 6 August, 2021; originally announced August 2021.

Deep Learning -- A first Meta-Survey of selected Reviews across Scientific Disciplines, their Commonalities, Challenges and Research Impact

Authors: Jan Egger, Antonio Pepe, Christina Gsaxner, Yuan Jin, Jianning Li, Roman Kern

Abstract: Deep learning belongs to the field of artificial intelligence, where machines perform tasks that typically require some kind of human intelligence. Similar to the basic structure of a brain, a deep learning algorithm consists of an artificial neural network, which resembles the biological brain structure. Mimicking the learning process of humans with their senses, deep learning networks are fed wi… ▽ More Deep learning belongs to the field of artificial intelligence, where machines perform tasks that typically require some kind of human intelligence. Similar to the basic structure of a brain, a deep learning algorithm consists of an artificial neural network, which resembles the biological brain structure. Mimicking the learning process of humans with their senses, deep learning networks are fed with (sensory) data, like texts, images, videos or sounds. These networks outperform the state-of-the-art methods in different tasks and, because of this, the whole field saw an exponential growth during the last years. This growth resulted in way over 10,000 publications per year in the last years. For example, the search engine PubMed alone, which covers only a sub-set of all publications in the medical field, provides already over 11,000 results in Q3 2020 for the search term 'deep learning', and around 90% of these results are from the last three years. Consequently, a complete overview over the field of deep learning is already impossible to obtain and, in the near future, it will potentially become difficult to obtain an overview over a subfield. However, there are several review articles about deep learning, which are focused on specific scientific fields or applications, for example deep learning advances in computer vision or in specific tasks like object detection. With these surveys as a foundation, the aim of this contribution is to provide a first high-level, categorized meta-survey of selected reviews on deep learning across different scientific disciplines. The categories (computer vision, language processing, medical informatics and additional works) have been chosen according to the underlying data sources (image, language, medical, mixed). In addition, we review the common architectures, methods, pros, cons, evaluations, challenges and future directions for every sub-category. △ Less

Submitted 17 November, 2021; v1 submitted 16 November, 2020; originally announced November 2020.

Comments: 83 pages, 22 figures, 9 tables, 100 references

Journal ref: PeerJ Computer Science 7:e773, 2021

Medical Deep Learning -- A systematic Meta-Review

Authors: Jan Egger, Christina Gsaxner, Antonio Pepe, Kelsey L. Pomykala, Frederic Jonske, Manuel Kurz, Jianning Li, Jens Kleesiek

Abstract: Deep learning (DL) has remarkably impacted several different scientific disciplines over the last few years. E.g., in image processing and analysis, DL algorithms were able to outperform other cutting-edge methods. Additionally, DL has delivered state-of-the-art results in tasks like autonomous driving, outclassing previous attempts. There are even instances where DL outperformed humans, for examp… ▽ More Deep learning (DL) has remarkably impacted several different scientific disciplines over the last few years. E.g., in image processing and analysis, DL algorithms were able to outperform other cutting-edge methods. Additionally, DL has delivered state-of-the-art results in tasks like autonomous driving, outclassing previous attempts. There are even instances where DL outperformed humans, for example with object recognition and gaming. DL is also showing vast potential in the medical domain. With the collection of large quantities of patient records and data, and a trend towards personalized treatments, there is a great need for automated and reliable processing and analysis of health information. Patient data is not only collected in clinical centers, like hospitals and private practices, but also by mobile healthcare apps or online websites. The abundance of collected patient data and the recent growth in the DL field has resulted in a large increase in research efforts. In Q2/2020, the search engine PubMed returned already over 11,000 results for the search term 'deep learning', and around 90% of these publications are from the last three years. However, even though PubMed represents the largest search engine in the medical field, it does not cover all medical-related publications. Hence, a complete overview of the field of 'medical deep learning' is almost impossible to obtain and acquiring a full overview of medical sub-fields is becoming increasingly more difficult. Nevertheless, several review and survey articles about medical DL have been published within the last few years. They focus, in general, on specific medical scenarios, like the analysis of medical images containing specific pathologies. With these surveys as a foundation, the aim of this article is to provide the first high-level, systematic meta-review of medical DL surveys. △ Less

Submitted 18 May, 2022; v1 submitted 28 October, 2020; originally announced October 2020.

Comments: 22 pages, 7 figures, 7 tables, 159 references. Computer Methods and Programs in Biomedicine (CMPB), Elsevier, May 2022

A Baseline Approach for AutoImplant: the MICCAI 2020 Cranial Implant Design Challenge

Authors: Jianning Li, Antonio Pepe, Christina Gsaxner, Gord von Campe, Jan Egger

Abstract: In this study, we present a baseline approach for AutoImplant (https://autoimplant.grand-challenge.org/) - the cranial implant design challenge, which, as suggested by the organizers, can be formulated as a volumetric shape learning task. In this task, the defective skull, the complete skull and the cranial implant are represented as binary voxel grids. To accomplish this task, the implant can be… ▽ More In this study, we present a baseline approach for AutoImplant (https://autoimplant.grand-challenge.org/) - the cranial implant design challenge, which, as suggested by the organizers, can be formulated as a volumetric shape learning task. In this task, the defective skull, the complete skull and the cranial implant are represented as binary voxel grids. To accomplish this task, the implant can be either reconstructed directly from the defective skull or obtained by taking the difference between a defective skull and a complete skull. In the latter case, a complete skull has to be reconstructed given a defective skull, which defines a volumetric shape completion problem. Our baseline approach for this task is based on the former formulation, i.e., a deep neural network is trained to predict the implants directly from the defective skulls. The approach generates high-quality implants in two steps: First, an encoder-decoder network learns a coarse representation of the implant from down-sampled, defective skulls; The coarse implant is only used to generate the bounding box of the defected region in the original high-resolution skull. Second, another encoder-decoder network is trained to generate a fine implant from the bounded area. On the test set, the proposed approach achieves an average dice similarity score (DSC) of 0.8555 and Hausdorff distance (HD) of 5.1825 mm. The code is publicly available at https://github.com/Jianningli/autoimplant. △ Less

Submitted 24 June, 2020; v1 submitted 22 June, 2020; originally announced June 2020.

Comments: 12 pages

An Online Platform for Automatic Skull Defect Restoration and Cranial Implant Design

Authors: Jianning Li, Antonio Pepe, Christina Gsaxner, Jan Egger

Abstract: We introduce a fully automatic system for cranial implant design, a common task in cranioplasty operations. The system is currently integrated in Studierfenster (http://studierfenster.tugraz.at/), an online, cloud-based medical image processing platform for medical imaging applications. Enhanced by deep learning algorithms, the system automatically restores the missing part of a skull (i.e., skull… ▽ More We introduce a fully automatic system for cranial implant design, a common task in cranioplasty operations. The system is currently integrated in Studierfenster (http://studierfenster.tugraz.at/), an online, cloud-based medical image processing platform for medical imaging applications. Enhanced by deep learning algorithms, the system automatically restores the missing part of a skull (i.e., skull shape completion) and generates the desired implant by subtracting the defective skull from the completed skull. The generated implant can be downloaded in the STereoLithography (.stl) format directly via the browser interface of the system. The implant model can then be sent to a 3D printer for in loco implant manufacturing. Furthermore, thanks to the standard format, the user can thereafter load the model into another application for post-processing whenever necessary. Such an automatic cranial implant design system can be integrated into the clinical practice to improve the current routine for surgeries related to skull defect repair (e.g., cranioplasty). Our system, although currently intended for educational and research use only, can be seen as an application of additive manufacturing for fast, patient-specific implant design. △ Less

Submitted 1 June, 2020; originally announced June 2020.

Comments: 5 pages

Design and Development of a Web-based Tool for Inpainting of Dissected Aortae in Angiography Images

Authors: Alexander Prutsch, Antonio Pepe, Jan Egger

Abstract: Medical imaging is an important tool for the diagnosis and the evaluation of an aortic dissection (AD); a serious condition of the aorta, which could lead to a life-threatening aortic rupture. AD patients need life-long medical monitoring of the aortic enlargement and of the disease progression, subsequent to the diagnosis of the aortic dissection. Since there is a lack of 'healthy-dissected' imag… ▽ More Medical imaging is an important tool for the diagnosis and the evaluation of an aortic dissection (AD); a serious condition of the aorta, which could lead to a life-threatening aortic rupture. AD patients need life-long medical monitoring of the aortic enlargement and of the disease progression, subsequent to the diagnosis of the aortic dissection. Since there is a lack of 'healthy-dissected' image pairs from medical studies, the application of inpainting techniques offers an alternative source for generating them by doing a virtual regression from dissected aortae to healthy aortae; an indirect way to study the origin of the disease. The proposed inpainting tool combines a neural network, which was trained on the task of inpainting aortic dissections, with an easy-to-use user interface. To achieve this goal, the inpainting tool has been integrated within the 3D medical image viewer of StudierFenster (www.studierfenster.at). By designing the tool as a web application, we simplify the usage of the neural network and reduce the initial learning curve. △ Less

Submitted 6 May, 2020; originally announced May 2020.

Comments: 9 figures, 14 references

Journal ref: The 24th Central European Seminar on Computer Graphics (CESCG), pp. 1-8, May. 2020

The arXiv of the future will not look like the arXiv

Authors: Alberto Pepe, Matteo Cantiello, Josh Nicholson

Abstract: The arXiv is the most popular preprint repository in the world. Since its inception in 1991, the arXiv has allowed researchers to freely share publication-ready articles prior to formal peer review. The growth and the popularity of the arXiv emerged as a result of new technologies that made document creation and dissemination easy, and cultural practices where collaboration and data sharing were d… ▽ More The arXiv is the most popular preprint repository in the world. Since its inception in 1991, the arXiv has allowed researchers to freely share publication-ready articles prior to formal peer review. The growth and the popularity of the arXiv emerged as a result of new technologies that made document creation and dissemination easy, and cultural practices where collaboration and data sharing were dominant. The arXiv represents a unique place in the history of research communication and the Web itself, however it has arguably changed very little since its creation. Here we look at the strengths and weaknesses of arXiv in an effort to identify what possible improvements can be made based on new technologies not previously available. Based on this, we argue that a modern arXiv might in fact not look at all like the arXiv of today. △ Less

Submitted 20 September, 2017; originally announced September 2017.

Comments: The authors of this document welcome public comments and ideas from its readers, at the online version of this article (https://www.authorea.com/173764)

10 Simple Rules for the Care and Feeding of Scientific Data

Authors: Alyssa Goodman, Alberto Pepe, Alexander W. Blocker, Christine L. Borgman, Kyle Cranmer, Mercè Crosas, Rosanne Di Stefano, Yolanda Gil, Paul Groth, Margaret Hedstrom, David W. Hogg, Vinay Kashyap, Ashish Mahabal, Aneta Siemiginowska, Aleksandra Slavkovic

Abstract: This article offers a short guide to the steps scientists can take to ensure that their data and associated analyses continue to be of value and to be recognized. In just the past few years, hundreds of scholarly papers and reports have been written on questions of data sharing, data provenance, research reproducibility, licensing, attribution, privacy, and more, but our goal here is not to review… ▽ More This article offers a short guide to the steps scientists can take to ensure that their data and associated analyses continue to be of value and to be recognized. In just the past few years, hundreds of scholarly papers and reports have been written on questions of data sharing, data provenance, research reproducibility, licensing, attribution, privacy, and more, but our goal here is not to review that literature. Instead, we present a short guide intended for researchers who want to know why it is important to "care for and feed" data, with some practical advice on how to do that. △ Less

Submitted 9 January, 2014; originally announced January 2014.

Comments: Accepted in PLOS Computational Biology. This paper was written collaboratively, on the web, in the open, using Authorea. The living version of this article, which includes sources and history, is available at http://www.authorea.com/3410/

A measure of total research impact independent of time and discipline

Authors: Alberto Pepe, Michael J. Kurtz

Abstract: Authorship and citation practices evolve with time and differ by academic discipline. As such, indicators of research productivity based on citation records are naturally subject to historical and disciplinary effects. We observe these effects on a corpus of astronomer career data constructed from a database of refereed publications. We employ a simple mechanism to measure research output using au… ▽ More Authorship and citation practices evolve with time and differ by academic discipline. As such, indicators of research productivity based on citation records are naturally subject to historical and disciplinary effects. We observe these effects on a corpus of astronomer career data constructed from a database of refereed publications. We employ a simple mechanism to measure research output using author and reference counts available in bibliographic databases to develop a citation-based indicator of research productivity. The total research impact (tori) quantifies, for an individual, the total amount of scholarly work that others have devoted to his/her work, measured in the volume of research papers. A derived measure, the research impact quotient (riq), is an age independent measure of an individual's research ability. We demonstrate that these measures are substantially less vulnerable to temporal debasement and cross-disciplinary bias than the most popular current measures. The proposed measures of research impact, tori and riq, have been implemented in the Smithsonian/NASA Astrophysics Data System. △ Less

Submitted 10 September, 2012; originally announced September 2012.

Comments: 14 pages, 5 figures. PLoS ONE, in press

Do Linguistic Style and Readability of Scientific Abstracts affect their Virality?

Authors: Marco Guerini, Alberto Pepe, Bruno Lepri

Abstract: Reactions to textual content posted in an online social network show different dynamics depending on the linguistic style and readability of the submitted content. Do similar dynamics exist for responses to scientific articles? Our intuition, supported by previous research, suggests that the success of a scientific article depends on its content, rather than on its linguistic style. In this articl… ▽ More Reactions to textual content posted in an online social network show different dynamics depending on the linguistic style and readability of the submitted content. Do similar dynamics exist for responses to scientific articles? Our intuition, supported by previous research, suggests that the success of a scientific article depends on its content, rather than on its linguistic style. In this article, we examine a corpus of scientific abstracts and three forms of associated reactions: article downloads, citations, and bookmarks. Through a class-based psycholinguistic analysis and readability indices tests, we show that certain stylistic and readability features of abstracts clearly concur in determining the success and viral capability of a scientific article. △ Less

Submitted 19 March, 2012; originally announced March 2012.

Comments: Proceedings of the Sixth International AAAI Conference on Weblogs and Social Media (ICWSM 2012), 4-8 June 2012, Dublin, Ireland

How the Scientific Community Reacts to Newly Submitted Preprints: Article Downloads, Twitter Mentions, and Citations

Authors: Xin Shuai, Alberto Pepe, Johan Bollen

Abstract: We analyze the online response to the preprint publication of a cohort of 4,606 scientific articles submitted to the preprint database arXiv.org between October 2010 and May 2011. We study three forms of responses to these preprints: downloads on the arXiv.org site, mentions on the social media site Twitter, and early citations in the scholarly record. We perform two analyses. First, we analyze th… ▽ More We analyze the online response to the preprint publication of a cohort of 4,606 scientific articles submitted to the preprint database arXiv.org between October 2010 and May 2011. We study three forms of responses to these preprints: downloads on the arXiv.org site, mentions on the social media site Twitter, and early citations in the scholarly record. We perform two analyses. First, we analyze the delay and time span of article downloads and Twitter mentions following submission, to understand the temporal configuration of these reactions and whether one precedes or follows the other. Second, we run regression and correlation tests to investigate the relationship between Twitter mentions, arXiv downloads and article citations. We find that Twitter mentions and arXiv downloads of scholarly articles follow two distinct temporal patterns of activity, with Twitter mentions having shorter delays and narrower time spans than arXiv downloads. We also find that the volume of Twitter mentions is statistically correlated with arXiv downloads and early citations just months after the publication of a preprint, with a possible bias that favors highly mentioned articles. △ Less

Submitted 17 September, 2012; v1 submitted 11 February, 2012; originally announced February 2012.

Comments: 15 pages, 7 Figures, 3 Tables. PLoS One, in press

The ADS All-Sky Survey

Authors: Alberto Pepe, Alyssa Goodman, August Muench

Abstract: The ADS All-Sky Survey (ADSASS) is an ongoing effort aimed at turning the NASA Astrophysics Data System (ADS), widely known for its unrivaled value as a literature resource for astronomers, into a data resource. The ADS is not a data repository per se, but it implicitly contains valuable holdings of astronomical data, in the form of images, tables and object references contained within articles. T… ▽ More The ADS All-Sky Survey (ADSASS) is an ongoing effort aimed at turning the NASA Astrophysics Data System (ADS), widely known for its unrivaled value as a literature resource for astronomers, into a data resource. The ADS is not a data repository per se, but it implicitly contains valuable holdings of astronomical data, in the form of images, tables and object references contained within articles. The objective of the ADSASS effort is to extract these data and make them discoverable and available through existing data viewers. The resulting ADSASS data layer promises to greatly enhance workflows and enable new research by tying astronomical literature and data assets into one resource. △ Less

Submitted 16 November, 2011; originally announced November 2011.

One, None and One Hundred Thousand Profiles: Re-imagining the Pirandellian Identity Dilemma in the Era of Online Social Networks

Authors: Alberto Pepe, Spencer Wolff, Karen Van Godtsenhoven

Abstract: Uno, Nessuno, Centomila ("One, No One and One Hundred Thousand") is a classic novel by Italian playwright Luigi Pirandello. Published in 1925, it recounts the tragedy of Vitangelo Moscarda, a man who struggles to reclaim a coherent and unitary identity for himself in the face of an inherently social and multi-faceted world. What would Moscarda identity tragedy look like today? In this article we t… ▽ More Uno, Nessuno, Centomila ("One, No One and One Hundred Thousand") is a classic novel by Italian playwright Luigi Pirandello. Published in 1925, it recounts the tragedy of Vitangelo Moscarda, a man who struggles to reclaim a coherent and unitary identity for himself in the face of an inherently social and multi-faceted world. What would Moscarda identity tragedy look like today? In this article we transplant Moscarda's identity play from its offline setting to the contemporary arena of social media and online social networks. With reference to established theories on identity construction, performance, and self-presentation, we re-imagine how Moscarda would go about defending the integrity of his selfhood in the face of the discountenancing influences of the online world. △ Less

Submitted 15 September, 2011; originally announced September 2011.

Comments: An abridged version of this paper will be presented/performed under the title "Identity dilemmas on Facebook" at the Symposium on the Dynamics of the Internet and Society "A Decade in Internet Time" to be held Wednesday 21 - Saturday 24 September 2011 at the Oxford Internet Institute, University of Oxford, UK

Journal ref: One, None, One Hundred Thousand Profiles. Alberto Pepe, Spencer Wolff, Karen Van Godtsenhoven. First Monday. Volume 17, Number 4 - 2 April 2012

The use of microblogging for field-based scientific research

Authors: Alberto Pepe, Matthew S. Mayernik

Abstract: Documenting the context in which data are collected is an integral part of the scientific research lifecycle. In field-based research, contextual information provides a detailed description of scientific practices and thus enables data interpretation and reuse. For field data, losing contextual information often means losing the data altogether. Yet, documenting the context of distributed, collabo… ▽ More Documenting the context in which data are collected is an integral part of the scientific research lifecycle. In field-based research, contextual information provides a detailed description of scientific practices and thus enables data interpretation and reuse. For field data, losing contextual information often means losing the data altogether. Yet, documenting the context of distributed, collaborative, field-based research can be a significant challenge due to the unpredictable nature of real-world settings and to the high degree of variability in data collection methods and scientific practices of different researchers. In this article, we propose the use of microblogging as a mechanism to support collection, ingestion, and publication of contextual information about the variegated digital artifacts that are produced in field research. We perform interviews with scholars involved in field-based environmental and urban sensing research, to determine the extent of adoption of Twitter and similar microblogging platforms and their potential use for field-specific research applications. Based on the results of these interviews as well as participant observation of field activities, we present the design, development, and pilot evaluation of a microblogging application integrated with an existing data collection platform on a handheld device. We investigate whether microblogging accommodates the variable and unpredictable nature of highly mobile research and whether it represents a suitable mechanism to document the context of field research data early in the scientific information lifecycle. △ Less

Submitted 8 September, 2011; originally announced September 2011.

Comments: Proceedings of the 45th Hawaii International Conference on System Science (HICSS-45 2012)

The relationship between acquaintanceship and coauthorship in scientific collaboration networks

Authors: Alberto Pepe

Abstract: This article examines the relationship between acquaintanceship and coauthorship patterns in a multi-disciplinary, multi-institutional, geographically distributed research center. Two social networks are constructed and compared: a network of coauthorship, representing how researchers write articles with one another, and a network of acquaintanceship, representing how those researchers know each o… ▽ More This article examines the relationship between acquaintanceship and coauthorship patterns in a multi-disciplinary, multi-institutional, geographically distributed research center. Two social networks are constructed and compared: a network of coauthorship, representing how researchers write articles with one another, and a network of acquaintanceship, representing how those researchers know each other on a personal level, based on their responses to an online survey. Statistical analyses of the topology and community structure of these networks point to the importance of small-scale, local, personal networks predicated upon acquaintanceship for accomplishing collaborative work in scientific communities. △ Less

Submitted 22 August, 2011; originally announced August 2011.

Journal ref: Journal of the American Society for Information Science and Technology 62(11): 2121--2132 (2011)

Political protest Italian-style: The dissonance between the blogosphere and mainstream media in the promotion and coverage of Beppe Grillo's V-day

Authors: Alberto Pepe, Corinna di Gennaro

Abstract: We analyze the organization, promotion and public perception of V-day, a political rally that took place on September 8, 2007, to protest against corruption in the Italian Parliament. Launched by blogger Beppe Grillo, and promoted via a word of mouth mobilization on the Italian blogosphere, V-day brought close to one million Italians in the streets on a single day, but was mostly ignored by mainst… ▽ More We analyze the organization, promotion and public perception of V-day, a political rally that took place on September 8, 2007, to protest against corruption in the Italian Parliament. Launched by blogger Beppe Grillo, and promoted via a word of mouth mobilization on the Italian blogosphere, V-day brought close to one million Italians in the streets on a single day, but was mostly ignored by mainstream media. This article is divided into two parts. In the first part, we analyze the volume and content of online articles published by both bloggers and mainstream news sources from June 14 (the day V-day was announced) until September 15, 2007 (one week after it took place) . We find that the success of V-day can be attributed to the coverage of bloggers and small-scale local news outlets only, suggesting a strong grassroots component in the organization of the rally. We also find a dissonant thematic relationship between content published by blogs and mainstream media: while the majority of blogs analyzed promote V-day, major mainstream media sources critique the methods of information production and dissemination employed by Grillo. Based on this finding, in the second part of the study, we explore the role of Grillo in the organization of the rally from a network analysis perspective. We study the interlinking structure of the V-day blogosphere network, to determine its structure, its levels of heterogeneity, and resilience. Our analysis contradicts the hypothesis that Grillo served as a top-down, broadcast-like source of information. Rather, we find that information about V-day was transferred across heterogeneous nodes in a moderately robust and resilient core network of blogs. We speculate that the organization of V-day represents the very first case, in Italian history, of a political demonstration developed and promoted primarily via the use of social media on the web. △ Less

Submitted 28 March, 2011; originally announced March 2011.

Comments: 30 pages; First Monday. Volume 14, Number 12. 2009

Journal ref: First Monday. Volume 14, Number 12. 2009

The Dilated Triple

Authors: Marko A. Rodriguez, Alberto Pepe, Joshua Shinavier

Abstract: The basic unit of meaning on the Semantic Web is the RDF statement, or triple, which combines a distinct subject, predicate and object to make a definite assertion about the world. A set of triples constitutes a graph, to which they give a collective meaning. It is upon this simple foundation that the rich, complex knowledge structures of the Semantic Web are built. Yet the very expressiveness of… ▽ More The basic unit of meaning on the Semantic Web is the RDF statement, or triple, which combines a distinct subject, predicate and object to make a definite assertion about the world. A set of triples constitutes a graph, to which they give a collective meaning. It is upon this simple foundation that the rich, complex knowledge structures of the Semantic Web are built. Yet the very expressiveness of RDF, by inviting comparison with real-world knowledge, highlights a fundamental shortcoming, in that RDF is limited to statements of absolute fact, independent of the context in which a statement is asserted. This is in stark contrast with the thoroughly context-sensitive nature of human thought. The model presented here provides a particularly simple means of contextualizing an RDF triple by associating it with related statements in the same graph. This approach, in combination with a notion of graph similarity, is sufficient to select only those statements from an RDF graph which are subjectively most relevant to the context of the requesting process. △ Less

Submitted 6 June, 2010; originally announced June 2010.

Report number: LA-UR-08-03927 ACM Class: I.2.4

Journal ref: Emergent Web Intelligence: Advanced Semantic Technologies, Advanced Information and Knowledge Processing series, pages 3-16, ISBN:978-1-84996-076-2, Springer-Verlag, June 2010

Collaboration in sensor network research: an in-depth longitudinal analysis of assortative mixing patterns

Authors: Alberto Pepe, Marko A. Rodriguez

Abstract: Many investigations of scientific collaboration are based on statistical analyses of large networks constructed from bibliographic repositories. These investigations often rely on a wealth of bibliographic data, but very little or no other information about the individuals in the network, and thus, fail to illustrate the broader social and academic landscape in which collaboration takes place. I… ▽ More Many investigations of scientific collaboration are based on statistical analyses of large networks constructed from bibliographic repositories. These investigations often rely on a wealth of bibliographic data, but very little or no other information about the individuals in the network, and thus, fail to illustrate the broader social and academic landscape in which collaboration takes place. In this article, we perform an in-depth longitudinal analysis of a relatively small network of scientific collaboration (N = 291) constructed from the bibliographic record of a research center involved in the development and application of sensor network and wireless technologies. We perform a preliminary analysis of selected structural properties of the network, computing its range, configuration and topology. We then support our preliminary statistical analysis with an in-depth temporal investigation of the assortative mixing of selected node characteristics, unveiling the researchers' propensity to collaborate preferentially with others with a similar academic profile. Our qualitative analysis of mixing patterns offers clues as to the nature of the scientific community being modeled in relation to its organizational, disciplinary, institutional, and international arrangements of collaboration. △ Less

Submitted 11 December, 2009; v1 submitted 21 November, 2009; originally announced November 2009.

Comments: Scientometrics (In press)

Journal ref: Scientometrics. Volume 84, Number 3. 2010

Modeling public mood and emotion: Twitter sentiment and socio-economic phenomena

Authors: Johan Bollen, Alberto Pepe, Huina Mao

Abstract: Microblogging is a form of online communication by which users broadcast brief text updates, also known as tweets, to the public or a selected circle of contacts. A variegated mosaic of microblogging uses has emerged since the launch of Twitter in 2006: daily chatter, conversation, information sharing, and news commentary, among others. Regardless of their content and intended use, tweets often… ▽ More Microblogging is a form of online communication by which users broadcast brief text updates, also known as tweets, to the public or a selected circle of contacts. A variegated mosaic of microblogging uses has emerged since the launch of Twitter in 2006: daily chatter, conversation, information sharing, and news commentary, among others. Regardless of their content and intended use, tweets often convey pertinent information about their author's mood status. As such, tweets can be regarded as temporally-authentic microscopic instantiations of public mood state. In this article, we perform a sentiment analysis of all public tweets broadcasted by Twitter users between August 1 and December 20, 2008. For every day in the timeline, we extract six dimensions of mood (tension, depression, anger, vigor, fatigue, confusion) using an extended version of the Profile of Mood States (POMS), a well-established psychometric instrument. We compare our results to fluctuations recorded by stock market and crude oil price indices and major events in media and popular culture, such as the U.S. Presidential Election of November 4, 2008 and Thanksgiving Day. We find that events in the social, political, cultural and economic sphere do have a significant, immediate and highly specific effect on the various dimensions of public mood. We speculate that large scale analyses of mood can provide a solid platform to model collective emotive trends in terms of their predictive value with regards to existing social as well as economic indicators. △ Less

Submitted 8 November, 2009; originally announced November 2009.

Comments: submitted to WWW2010

Journal ref: Proceedings of the Fifth International AAAI Conference on Weblogs and Social Media (ICWSM 2011), 17-21 July 2011, Barcelona, Spain

From Artifacts to Aggregations: Modeling Scientific Life Cycles on the Semantic Web

Authors: Alberto Pepe, Matthew Mayernik, Christine L. Borgman, Herbert Van de Sompel

Abstract: In the process of scientific research, many information objects are generated, all of which may remain valuable indefinitely. However, artifacts such as instrument data and associated calibration information may have little value in isolation; their meaning is derived from their relationships to each other. Individual artifacts are best represented as components of a life cycle that is specific… ▽ More In the process of scientific research, many information objects are generated, all of which may remain valuable indefinitely. However, artifacts such as instrument data and associated calibration information may have little value in isolation; their meaning is derived from their relationships to each other. Individual artifacts are best represented as components of a life cycle that is specific to a scientific research domain or project. Current cataloging practices do not describe objects at a sufficient level of granularity nor do they offer the globally persistent identifiers necessary to discover and manage scholarly products with World Wide Web standards. The Open Archives Initiative's Object Reuse and Exchange data model (OAI-ORE) meets these requirements. We demonstrate a conceptual implementation of OAI-ORE to represent the scientific life cycles of embedded networked sensor applications in seismology and environmental sciences. By establishing relationships between publications, data, and contextual research information, we illustrate how to obtain a richer and more realistic view of scientific practices. That view can facilitate new forms of scientific research and learning. Our analysis is framed by studies of scientific practices in a large, multi-disciplinary, multi-university science and engineering research center, the Center for Embedded Networked Sensing (CENS). △ Less

Submitted 20 October, 2009; v1 submitted 14 June, 2009; originally announced June 2009.

Comments: 28 pages. To appear in the Journal of the American Society for Information Science and Technology (JASIST)

Journal ref: Journal of the American Society for Information Science and Technology (JASIST). Volume 61, Issue 3. 2010

Faith in the Algorithm, Part 1: Beyond the Turing Test

Authors: Marko A. Rodriguez, Alberto Pepe

Abstract: Since the Turing test was first proposed by Alan Turing in 1950, the primary goal of artificial intelligence has been predicated on the ability for computers to imitate human behavior. However, the majority of uses for the computer can be said to fall outside the domain of human abilities and it is exactly outside of this domain where computers have demonstrated their greatest contribution to in… ▽ More Since the Turing test was first proposed by Alan Turing in 1950, the primary goal of artificial intelligence has been predicated on the ability for computers to imitate human behavior. However, the majority of uses for the computer can be said to fall outside the domain of human abilities and it is exactly outside of this domain where computers have demonstrated their greatest contribution to intelligence. Another goal for artificial intelligence is one that is not predicated on human mimicry, but instead, on human amplification. This article surveys various systems that contribute to the advancement of human and social intelligence. △ Less

Submitted 1 March, 2009; originally announced March 2009.

Report number: LA-UR-09-00052

Journal ref: Proceedings of the AISB Symposium on Computing and Philosophy, The Society for the Study of Artificial Intelligence and Simulation of Behaviour, Edinburgh, Scotland, April 2009.

A Grateful Dead Analysis: The Relationship Between Concert and Listening Behavior

Authors: Marko A. Rodriguez, Vadas Gintautas, Alberto Pepe

Abstract: The Grateful Dead were an American band that was born out of the San Francisco, California psychedelic movement of the 1960s. The band played music together from 1965 to 1995 and is well known for concert performances containing extended improvisations and long and unique set lists. This article presents a comparative analysis between 1,590 of the Grateful Dead's concert set lists from 1972 to 1… ▽ More The Grateful Dead were an American band that was born out of the San Francisco, California psychedelic movement of the 1960s. The band played music together from 1965 to 1995 and is well known for concert performances containing extended improvisations and long and unique set lists. This article presents a comparative analysis between 1,590 of the Grateful Dead's concert set lists from 1972 to 1995 and 2,616,990 last.fm Grateful Dead listening events from August 2005 to October 2007. While there is a strong correlation between how songs were played in concert and how they are listened to by last.fm members, the outlying songs in this trend identify interesting aspects of the band and their fans 10 years after the band's dissolution. △ Less

Submitted 15 July, 2008; originally announced July 2008.

Report number: LA-UR-08-04421 ACM Class: K.4.0

Journal ref: First Monday, volume 14, number 1, ISSN:1396-0466, University of Illinois at Chicago Library, January 2009.

Between conjecture and memento: shaping a collective emotional perception of the future

Authors: Alberto Pepe, Johan Bollen

Abstract: Large scale surveys of public mood are costly and often impractical to perform. However, the web is awash with material indicative of public mood such as blogs, emails, and web queries. Inexpensive content analysis on such extensive corpora can be used to assess public mood fluctuations. The work presented here is concerned with the analysis of the public mood towards the future. Using an extens… ▽ More Large scale surveys of public mood are costly and often impractical to perform. However, the web is awash with material indicative of public mood such as blogs, emails, and web queries. Inexpensive content analysis on such extensive corpora can be used to assess public mood fluctuations. The work presented here is concerned with the analysis of the public mood towards the future. Using an extension of the Profile of Mood States questionnaire, we have extracted mood indicators from 10,741 emails submitted in 2006 to futureme.org, a web service that allows its users to send themselves emails to be delivered at a later date. Our results indicate long-term optimism toward the future, but medium-term apprehension and confusion. △ Less

Submitted 24 January, 2008; originally announced January 2008.

Comments: 6 pages. AAAI Spring Symposium on Emotion, Personality, and Social Behavior

On the relationship between the structural and socioacademic communities of a coauthorship network

Authors: Marko A. Rodriguez, Alberto Pepe

Abstract: This article presents a study that compares detected structural communities in a coauthorship network to the socioacademic characteristics of the scholars that compose the network. The coauthorship network was created from the bibliographic record of a multi-institution, interdisciplinary research group focused on the study of sensor networks and wireless communication. Four different community… ▽ More This article presents a study that compares detected structural communities in a coauthorship network to the socioacademic characteristics of the scholars that compose the network. The coauthorship network was created from the bibliographic record of a multi-institution, interdisciplinary research group focused on the study of sensor networks and wireless communication. Four different community detection algorithms were employed to assign a structural community to each scholar in the network: leading eigenvector, walktrap, edge betweenness and spinglass. Socioacademic characteristics were gathered from the scholars and include such information as their academic department, academic affiliation, country of origin, and academic position. A Pearson's $χ^2$ test, with a simulated Monte Carlo, revealed that structural communities best represent groupings of individuals working in the same academic department and at the same institution. A generalization of this result suggests that, even in interdisciplinary, multi-institutional research groups, coauthorship is primarily driven by departmental and institutional affiliation. △ Less

Submitted 18 July, 2008; v1 submitted 15 January, 2008; originally announced January 2008.

Report number: LA-UR-07-8339 ACM Class: H.3.7; G.2.2

Journal ref: Journal of Informetrics, volume 2, issue 3, pages 195-201, ISSN: 1751-1577, Elsevier, July 2008

Protocols for Scholarly Communication

Authors: Alberto Pepe, Joanne Yeomans

Abstract: CERN, the European Organization for Nuclear Research, has operated an institutional preprint repository for more than 10 years. The repository contains over 850,000 records of which more than 450,000 are full-text OA preprints, mostly in the field of particle physics, and it is integrated with the library's holdings of books, conference proceedings, journals and other grey literature. In order t… ▽ More CERN, the European Organization for Nuclear Research, has operated an institutional preprint repository for more than 10 years. The repository contains over 850,000 records of which more than 450,000 are full-text OA preprints, mostly in the field of particle physics, and it is integrated with the library's holdings of books, conference proceedings, journals and other grey literature. In order to encourage effective propagation and open access to scholarly material, CERN is implementing a range of innovative library services into its document repository: automatic keywording, reference extraction, collaborative management tools and bibliometric tools. Some of these services, such as user reviewing and automatic metadata extraction, could make up an interesting testbed for future publishing solutions and certainly provide an exciting environment for e-science possibilities. The future protocol for scientific communication should naturally guide authors towards OA publication and CERN wants to help reach a full open access publishing environment for the particle physics community and the related sciences in the next few years. △ Less

Submitted 1 November, 2006; originally announced November 2006.

Comments: 8 pages, to appear in Library and Information Systems in Astronomy V

Report number: CERN-OPEN-2006-053