-
Manual Labelling Artificially Inflates Deep Learning-Based Segmentation Performance on RGB Images of Closed Canopy: Validation Using TLS
Authors:
Matthew J. Allen,
Harry J. F. Owen,
Stuart W. D. Grieve,
Emily R. Lines
Abstract:
Monitoring forest dynamics at an individual tree scale is essential for accurately assessing ecosystem responses to climate change, yet traditional methods relying on field-based forest inventories are labor-intensive and limited in spatial coverage. Advances in remote sensing using drone-acquired RGB imagery combined with deep learning models have promised precise individual tree crown (ITC) segm…
▽ More
Monitoring forest dynamics at an individual tree scale is essential for accurately assessing ecosystem responses to climate change, yet traditional methods relying on field-based forest inventories are labor-intensive and limited in spatial coverage. Advances in remote sensing using drone-acquired RGB imagery combined with deep learning models have promised precise individual tree crown (ITC) segmentation; however, existing methods are frequently validated against human-annotated images, lacking rigorous independent ground truth. In this study, we generate high-fidelity validation labels from co-located Terrestrial Laser Scanning (TLS) data for drone imagery of mixed unmanaged boreal and Mediterranean forests. We evaluate the performance of two widely used deep learning ITC segmentation models - DeepForest (RetinaNet) and Detectree2 (Mask R-CNN) - on these data, and compare to performance on further Mediterranean forest data labelled manually. When validated against TLS-derived ground truth from Mediterranean forests, model performance decreased significantly compared to assessment based on hand-labelled from an ecologically similar site (AP50: 0.094 vs. 0.670). Restricting evaluation to only canopy trees shrank this gap considerably (Canopy AP50: 0.365), although performance was still far lower than on similar hand-labelled data. Models also performed poorly on boreal forest data (AP50: 0.142), although again increasing when evaluated on canopy trees only (Canopy AP50: 0.308). Both models showed very poor localisation accuracy at stricter IoU thresholds, even when restricted to canopy trees (Max AP75: 0.051). Similar results have been observed in studies using aerial LiDAR data, suggesting fundamental limitations in aerial-based segmentation approaches in closed canopy forests.
△ Less
Submitted 19 March, 2025; v1 submitted 18 March, 2025;
originally announced March 2025.
-
PointsToWood: A deep learning framework for complete canopy leaf-wood segmentation of TLS data across diverse European forests
Authors:
Harry J. F. Owen,
Matthew J. A. Allen,
Stuart W. D. Grieve,
Phill Wilkes,
Emily R. Lines
Abstract:
Point clouds from Terrestrial Laser Scanning (TLS) are an increasingly popular source of data for studying plant structure and function but typically require extensive manual processing to extract ecologically important information. One key task is the accurate semantic segmentation of different plant material within point clouds, particularly wood and leaves, which is required to understand plant…
▽ More
Point clouds from Terrestrial Laser Scanning (TLS) are an increasingly popular source of data for studying plant structure and function but typically require extensive manual processing to extract ecologically important information. One key task is the accurate semantic segmentation of different plant material within point clouds, particularly wood and leaves, which is required to understand plant productivity, architecture and physiology. Existing automated semantic segmentation methods are primarily developed for single ecosystem types, and whilst they show good accuracy for biomass assessment from the trunk and large branches, often perform less well within the crown. In this study, we demonstrate a new framework that uses a deep learning architecture newly developed from PointNet and pointNEXT for processing 3D point clouds to provide a reliable semantic segmentation of wood and leaf in TLS point clouds from the tree base to branch tips, trained on data from diverse mature European forests. Our model uses meticulously labelled data combined with voxel-based sampling, neighbourhood rescaling, and a novel gated reflectance integration module embedded throughout the feature extraction layers. We evaluate its performance across open datasets from boreal, temperate, Mediterranean and tropical regions, encompassing diverse ecosystem types and sensor characteristics. Our results show consistent outperformance against the most widely used PointNet based approach for leaf/wood segmentation on our high-density TLS dataset collected across diverse mixed forest plots across all major biomes in Europe. We also find consistently strong performance tested on others open data from China, Eastern Cameroon, Germany and Finland, collected using both time-of-flight and phase-shift sensors, showcasing the transferability of our model to a wide range of ecosystems and sensors.
△ Less
Submitted 6 March, 2025;
originally announced March 2025.
-
Benchmarking tree species classification from proximally-sensed laser scanning data: introducing the FOR-species20K dataset
Authors:
Stefano Puliti,
Emily R. Lines,
Jana Müllerová,
Julian Frey,
Zoe Schindler,
Adrian Straker,
Matthew J. Allen,
Lukas Winiwarter,
Nataliia Rehush,
Hristina Hristova,
Brent Murray,
Kim Calders,
Louise Terryn,
Nicholas Coops,
Bernhard Höfle,
Samuli Junttila,
Martin Krůček,
Grzegorz Krok,
Kamil Král,
Shaun R. Levick,
Linda Luck,
Azim Missarov,
Martin Mokroš,
Harry J. F. Owen,
Krzysztof Stereńczak
, et al. (8 additional authors not shown)
Abstract:
Proximally-sensed laser scanning offers significant potential for automated forest data capture, but challenges remain in automatically identifying tree species without additional ground data. Deep learning (DL) shows promise for automation, yet progress is slowed by the lack of large, diverse, openly available labeled datasets of single tree point clouds. This has impacted the robustness of DL mo…
▽ More
Proximally-sensed laser scanning offers significant potential for automated forest data capture, but challenges remain in automatically identifying tree species without additional ground data. Deep learning (DL) shows promise for automation, yet progress is slowed by the lack of large, diverse, openly available labeled datasets of single tree point clouds. This has impacted the robustness of DL models and the ability to establish best practices for species classification.
To overcome these challenges, the FOR-species20K benchmark dataset was created, comprising over 20,000 tree point clouds from 33 species, captured using terrestrial (TLS), mobile (MLS), and drone laser scanning (ULS) across various European forests, with some data from other regions. This dataset enables the benchmarking of DL models for tree species classification, including both point cloud-based (PointNet++, MinkNet, MLP-Mixer, DGCNNs) and multi-view image-based methods (SimpleView, DetailView, YOLOv5).
2D image-based models generally performed better (average OA = 0.77) than 3D point cloud-based models (average OA = 0.72), with consistent results across different scanning platforms and sensors. The top model, DetailView, was particularly robust, handling data imbalances well and generalizing effectively across tree sizes.
The FOR-species20K dataset, available at https://zenodo.org/records/13255198, is a key resource for developing and benchmarking DL models for tree species classification using laser scanning data, providing a foundation for future advancements in the field.
△ Less
Submitted 12 August, 2024;
originally announced August 2024.
-
Alya towards Exascale: Optimal OpenACC Performance of the Navier-Stokes Finite Element Assembly on GPUs
Authors:
Herbert Owen,
Dominik Ernst,
Thomas Gruber,
Oriol Lemkuhl,
Guillaume Houzeaux,
Lucas Gasparino,
Gerhard Wellein
Abstract:
This paper addresses the challenge of providing portable and highly efficient code structures for CPU and GPU architectures. We choose the assembly of the right-hand term in the incompressible flow module of the High-Performance Computational Mechanics code Alya, which is one of the two CFD codes in the Unified European Benchmark Suite. Starting from an efficient CPU-code and a related OpenACC-por…
▽ More
This paper addresses the challenge of providing portable and highly efficient code structures for CPU and GPU architectures. We choose the assembly of the right-hand term in the incompressible flow module of the High-Performance Computational Mechanics code Alya, which is one of the two CFD codes in the Unified European Benchmark Suite. Starting from an efficient CPU-code and a related OpenACC-port for GPUs we successively investigate performance potentials arising from code specialization, algorithmic restructuring and low-level optimizations.
We demonstrate that only the combination of these different dimensions of runtime optimization unveils the full performance potential on the GPU and CPU. Roofline-based performance modelling is applied in this process and we demonstrate the need to investigate new optimization strategies if a classical roofline limit such as memory bandwidth utilization is achieved, rather than stopping the process. The final unified OpenACC-based implementation boosts performance by more than 50x on an NVIDIA A100 GPU (achieving approximately 2.5 TF/s FP64) and a further factor of 5x for an Intel Icelake based CPU-node (achieving approximately 1.0 TF/s FP64).
The insights gained in our manual approach lays ground implementing unified but still highly efficient code structures for related kernels in Alya and other applications. These can be realized by manual coding or automatic code generation frameworks.
△ Less
Submitted 22 January, 2024;
originally announced March 2024.
-
Tracking before detection using partial orders and optimization
Authors:
Michael Robinson,
Michael Stein,
Henry S. Owen
Abstract:
This article addresses the problem of multi-object tracking by using a non-deterministic model of target behaviors with hard constraints. To capture the evolution of target features as well as their locations, we permit objects to lie in a general topological target configuration space, rather than a Euclidean space. We obtain tracker performance bounds based on sample rates, and derive a flexible…
▽ More
This article addresses the problem of multi-object tracking by using a non-deterministic model of target behaviors with hard constraints. To capture the evolution of target features as well as their locations, we permit objects to lie in a general topological target configuration space, rather than a Euclidean space. We obtain tracker performance bounds based on sample rates, and derive a flexible, agnostic tracking algorithm. We demonstrate our algorithm on two scenarios involving laboratory and field data.
△ Less
Submitted 22 January, 2024;
originally announced January 2024.
-
AI applications in forest monitoring need remote sensing benchmark datasets
Authors:
Emily R. Lines,
Matt Allen,
Carlos Cabo,
Kim Calders,
Amandine Debus,
Stuart W. D. Grieve,
Milto Miltiadou,
Adam Noach,
Harry J. F. Owen,
Stefano Puliti
Abstract:
With the rise in high resolution remote sensing technologies there has been an explosion in the amount of data available for forest monitoring, and an accompanying growth in artificial intelligence applications to automatically derive forest properties of interest from these datasets. Many studies use their own data at small spatio-temporal scales, and demonstrate an application of an existing or…
▽ More
With the rise in high resolution remote sensing technologies there has been an explosion in the amount of data available for forest monitoring, and an accompanying growth in artificial intelligence applications to automatically derive forest properties of interest from these datasets. Many studies use their own data at small spatio-temporal scales, and demonstrate an application of an existing or adapted data science method for a particular task. This approach often involves intensive and time-consuming data collection and processing, but generates results restricted to specific ecosystems and sensor types. There is a lack of widespread acknowledgement of how the types and structures of data used affects performance and accuracy of analysis algorithms. To accelerate progress in the field more efficiently, benchmarking datasets upon which methods can be tested and compared are sorely needed.
Here, we discuss how lack of standardisation impacts confidence in estimation of key forest properties, and how considerations of data collection need to be accounted for in assessing method performance. We present pragmatic requirements and considerations for the creation of rigorous, useful benchmarking datasets for forest monitoring applications, and discuss how tools from modern data science can improve use of existing data. We list a set of example large-scale datasets that could contribute to benchmarking, and present a vision for how community-driven, representative benchmarking initiatives could benefit the field.
△ Less
Submitted 19 December, 2022;
originally announced December 2022.
-
Investigating Expectation Violations in Mobile Apps
Authors:
Sherlock A. Licorish,
Helen E. Owen,
Bastin Tony Roy Savarimuthu,
Priyanka Patel
Abstract:
Information technology and software services are pervasive, occupying the centre of most aspects of contemporary societies. This has given rise to commonly expected norms and expectations around how such systems should work, appropriate penalties for violating these expectations, and more importantly, indicators of how to reduce the consequences of violations and sanctions. Evidence for expectatio…
▽ More
Information technology and software services are pervasive, occupying the centre of most aspects of contemporary societies. This has given rise to commonly expected norms and expectations around how such systems should work, appropriate penalties for violating these expectations, and more importantly, indicators of how to reduce the consequences of violations and sanctions. Evidence for expectation violations and ensuing sanctions exists in a range of portals used by individuals and groups to start new friendships, explore new ideas, and provide feedback for products and services. Therein lies insights that could lead to functional socio-technical systems, and general awareness and anticipations of human actions (and interactions) when using information technology and software services. However, limited previous work has examined such artifacts to provide these understandings. To contribute to such understandings and theoretical advancement we study expectation violations in mobile apps, considered among the most engaging socio-technical systems. We used content analysis and expectancy violation theory (EVT) and expectation confirmation theory (ECT) to explore the evidence and nature of sanctions in app reviews for a specific domain of apps. Our outcomes show that users respond to expectation violation with sanctions when their app does not work as anticipated, developers seem to target specific market niches when providing services in an app domain, and users within an app domain respond with similar sanctions. We contribute to the advancement of expectation violation theories, and we provide practical insights for the mobile app community.
△ Less
Submitted 6 January, 2022;
originally announced January 2022.
-
Heterogeneous CPU/GPU co-execution of CFD simulations on the POWER9 architecture: Application to airplane aerodynamics
Authors:
R. Borrell,
D. Dosimont,
M. Garcia-Gasulla,
G. Houzeaux,
O. Lehmkuhl,
V. Mehta,
H. Owen,
M. Vazquez,
G. Oyarzun
Abstract:
High fidelity Computational Fluid Dynamics simulations are generally associated with large computing requirements, which are progressively acute with each new generation of supercomputers. However, significant research efforts are required to unlock the computing power of leading-edge systems, currently referred to as pre-Exascale systems, based on increasingly complex architectures. In this paper…
▽ More
High fidelity Computational Fluid Dynamics simulations are generally associated with large computing requirements, which are progressively acute with each new generation of supercomputers. However, significant research efforts are required to unlock the computing power of leading-edge systems, currently referred to as pre-Exascale systems, based on increasingly complex architectures. In this paper, we present the approach implemented in the computational mechanics code Alya. We describe in detail the parallelization strategy implemented to fully exploit the different levels of parallelism, together with a novel co-execution method for the efficient utilization of heterogeneous CPU/GPU architectures. The latter is based on a multi-code co-execution approach with a dynamic load balancing mechanism. The assessment of the performance of all the proposed strategies has been carried out for airplane simulations on the POWER9 architecture accelerated with NVIDIA Volta V100 GPUs.
△ Less
Submitted 6 July, 2020; v1 submitted 12 May, 2020;
originally announced May 2020.