-
A Tightly Coupled IMU-Based Motion Capture Approach for Estimating Multibody Kinematics and Kinetics
Authors:
Hassan Osman,
Daan de Kanter,
Jelle Boelens,
Manon Kok,
Ajay Seth
Abstract:
Inertial Measurement Units (IMUs) enable portable, multibody motion capture (MoCap) in diverse environments beyond the laboratory, making them a practical choice for diagnosing mobility disorders and supporting rehabilitation in clinical or home settings. However, challenges associated with IMU measurements, including magnetic distortions and drift errors, complicate their broader use for MoCap. I…
▽ More
Inertial Measurement Units (IMUs) enable portable, multibody motion capture (MoCap) in diverse environments beyond the laboratory, making them a practical choice for diagnosing mobility disorders and supporting rehabilitation in clinical or home settings. However, challenges associated with IMU measurements, including magnetic distortions and drift errors, complicate their broader use for MoCap. In this work, we propose a tightly coupled motion capture approach that directly integrates IMU measurements with multibody dynamic models via an Iterated Extended Kalman Filter (IEKF) to simultaneously estimate the system's kinematics and kinetics. By enforcing kinematic and kinetic properties and utilizing only accelerometer and gyroscope data, our method improves IMU-based state estimation accuracy. Our approach is designed to allow for incorporating additional sensor data, such as optical MoCap measurements and joint torque readings, to further enhance estimation accuracy. We validated our approach using highly accurate ground truth data from a 3 Degree of Freedom (DoF) pendulum and a 6 DoF Kuka robot. We demonstrate a maximum Root Mean Square Difference (RMSD) in the pendulum's computed joint angles of 3.75 degrees compared to optical MoCap Inverse Kinematics (IK), which serves as the gold standard in the absence of internal encoders. For the Kuka robot, we observe a maximum joint angle RMSD of 3.24 degrees compared to the Kuka's internal encoders, while the maximum joint angle RMSD of the optical MoCap IK compared to the encoders was 1.16 degrees. Additionally, we report a maximum joint torque RMSD of 2 Nm in the pendulum compared to optical MoCap Inverse Dynamics (ID), and 3.73 Nm in the Kuka robot relative to its internal torque sensors.
△ Less
Submitted 12 May, 2025;
originally announced May 2025.
-
On the Connection Between Magnetic-Field Odometry Aided Inertial Navigation and Magnetic-Field SLAM
Authors:
Isaac Skog,
Manon Kok,
Gustaf Hendeby,
Chuan Huang,
Thomas Edridge
Abstract:
Magnetic-field simultaneous localization and mapping (SLAM) using consumer-grade inertial and magnetometer sensors offers a scalable, cost-effective solution for indoor localization. However, the rapid error accumulation in the inertial navigation process limits the feasible exploratory phases of these systems. Advances in magnetometer array processing have demonstrated that odometry information,…
▽ More
Magnetic-field simultaneous localization and mapping (SLAM) using consumer-grade inertial and magnetometer sensors offers a scalable, cost-effective solution for indoor localization. However, the rapid error accumulation in the inertial navigation process limits the feasible exploratory phases of these systems. Advances in magnetometer array processing have demonstrated that odometry information, i.e., displacement and rotation information, can be extracted from local magnetic field variations and used to create magnetic-field odometry-aided inertial navigation systems. The error growth rate of these systems is significantly lower than that of standalone inertial navigation systems. This study seeks an answer to whether a magnetic-field SLAM system fed with measurements from a magnetometer array can indirectly extract odometry information -- without requiring algorithmic modifications -- and thus sustain longer exploratory phases. The theoretical analysis and simulation results show that such a system can extract odometry information and indirectly create a magnetic field odometry-aided inertial navigation system during the exploration phases. However, practical challenges related to map resolution and computational complexity remain significant.
△ Less
Submitted 14 May, 2025; v1 submitted 6 March, 2025;
originally announced March 2025.
-
ECTIL: Label-efficient Computational Tumour Infiltrating Lymphocyte (TIL) assessment in breast cancer: Multicentre validation in 2,340 patients with breast cancer
Authors:
Yoni Schirris,
Rosie Voorthuis,
Mark Opdam,
Marte Liefaard,
Gabe S Sonke,
Gwen Dackus,
Vincent de Jong,
Yuwei Wang,
Annelot Van Rossum,
Tessa G Steenbruggen,
Lars C Steggink,
Liesbeth G. E. de Vries,
Marc van de Vijver,
Roberto Salgado,
Efstratios Gavves,
Paul J van Diest,
Sabine C Linn,
Jonas Teuwen,
Renee Menezes,
Marleen Kok,
Hugo Horlings
Abstract:
The level of tumour-infiltrating lymphocytes (TILs) is a prognostic factor for patients with (triple-negative) breast cancer (BC). Computational TIL assessment (CTA) has the potential to assist pathologists in this labour-intensive task, but current CTA models rely heavily on many detailed annotations. We propose and validate a fundamentally simpler deep learning based CTA that can be trained in o…
▽ More
The level of tumour-infiltrating lymphocytes (TILs) is a prognostic factor for patients with (triple-negative) breast cancer (BC). Computational TIL assessment (CTA) has the potential to assist pathologists in this labour-intensive task, but current CTA models rely heavily on many detailed annotations. We propose and validate a fundamentally simpler deep learning based CTA that can be trained in only ten minutes on hundredfold fewer pathologist annotations. We collected whole slide images (WSIs) with TILs scores and clinical data of 2,340 patients with BC from six cohorts including three randomised clinical trials. Morphological features were extracted from whole slide images (WSIs) using a pathology foundation model. Our label-efficient Computational stromal TIL assessment model (ECTIL) directly regresses the TILs score from these features. ECTIL trained on only a few hundred samples (ECTIL-TCGA) showed concordance with the pathologist over five heterogeneous external cohorts (r=0.54-0.74, AUROC=0.80-0.94). Training on all slides of five cohorts (ECTIL-combined) improved results on a held-out test set (r=0.69, AUROC=0.85). Multivariable Cox regression analyses indicated that every 10% increase of ECTIL scores was associated with improved overall survival independent of clinicopathological variables (HR 0.86, p<0.01), similar to the pathologist score (HR 0.87, p<0.001). We demonstrate that ECTIL is highly concordant with an expert pathologist and obtains a similar hazard ratio. ECTIL has a fundamentally simpler design than existing methods and can be trained on orders of magnitude fewer annotations. Such a CTA may be used to pre-screen patients for, e.g., immunotherapy clinical trial inclusion, or as a tool to assist clinicians in the diagnostic work-up of patients with BC. Our model is available under an open source licence (https://github.com/nki-ai/ectil).
△ Less
Submitted 24 January, 2025;
originally announced January 2025.
-
Rao-Blackwellized Particle Smoothing for Simultaneous Localization and Mapping
Authors:
Manon Kok,
Arno Solin,
Thomas B. Schön
Abstract:
Simultaneous localization and mapping (SLAM) is the task of building a map representation of an unknown environment while at the same time using it for positioning. A probabilistic interpretation of the SLAM task allows for incorporating prior knowledge and for operation under uncertainty. Contrary to the common practice of computing point estimates of the system states, we capture the full poster…
▽ More
Simultaneous localization and mapping (SLAM) is the task of building a map representation of an unknown environment while at the same time using it for positioning. A probabilistic interpretation of the SLAM task allows for incorporating prior knowledge and for operation under uncertainty. Contrary to the common practice of computing point estimates of the system states, we capture the full posterior density through approximate Bayesian inference. This dynamic learning task falls under state estimation, where the state-of-the-art is in sequential Monte Carlo methods that tackle the forward filtering problem. In this paper, we introduce a framework for probabilistic SLAM using particle smoothing that does not only incorporate observed data in current state estimates, but it also back-tracks the updated knowledge to correct for past drift and ambiguities in both the map and in the states. Our solution can efficiently handle both dense and sparse map representations by Rao-Blackwellization of conditionally linear and conditionally linearized models. We show through simulations and real-world experiments how the principles apply to radio (BLE/Wi-Fi), magnetic field, and visual SLAM. The proposed solution is general, efficient, and works well under confounding noise.
△ Less
Submitted 5 June, 2024; v1 submitted 6 June, 2023;
originally announced June 2023.
-
Tightly Integrated Motion Classification and State Estimation in Foot-Mounted Navigation Systems
Authors:
Isaac Skog,
Gustaf Hendeby,
Manon Kok
Abstract:
A framework for tightly integrated motion mode classification and state estimation in motion-constrained inertial navigation systems is presented. The framework uses a jump Markov model to describe the navigation system's motion mode and navigation state dynamics with a single model. A bank of Kalman filters is then used for joint inference of the navigation state and the motion mode. A method for…
▽ More
A framework for tightly integrated motion mode classification and state estimation in motion-constrained inertial navigation systems is presented. The framework uses a jump Markov model to describe the navigation system's motion mode and navigation state dynamics with a single model. A bank of Kalman filters is then used for joint inference of the navigation state and the motion mode. A method for learning unknown parameters in the jump Markov model, such as the motion mode transition probabilities, is also presented. The application of the proposed framework is illustrated via two examples. The first example is a foot-mounted navigation system that adapts its behavior to different gait speeds. The second example is a foot-mounted navigation system that detects when the user walks on flat ground and locks the vertical position estimate accordingly. Both examples show that the proposed framework provides significantly better position accuracy than a standard zero-velocity aided inertial navigation system. More importantly, the examples show that the proposed framework provides a theoretically well-grounded approach for developing new motion-constrained inertial navigation systems that can learn different motion patterns.
△ Less
Submitted 22 August, 2023; v1 submitted 16 May, 2023;
originally announced May 2023.
-
Observability of the relative motion from inertial data in kinematic chains
Authors:
Manon Kok,
Karsten Eckhoff,
Ive Weygers,
Thomas Seel
Abstract:
Real-time motion tracking of kinematic chains is a key prerequisite in the control of, e.g., robotic actuators and autonomous vehicles and also has numerous biomechanical applications. In recent years, it has been shown that, by placing inertial sensors on segments that are connected by rotational joints, the motion of that kinematic chain can be tracked accurately. These methods specifically avoi…
▽ More
Real-time motion tracking of kinematic chains is a key prerequisite in the control of, e.g., robotic actuators and autonomous vehicles and also has numerous biomechanical applications. In recent years, it has been shown that, by placing inertial sensors on segments that are connected by rotational joints, the motion of that kinematic chain can be tracked accurately. These methods specifically avoid using magnetometer measurements, which are known to be unreliable since the magnetic field at the different sensor locations is typically different. They rely on the assumption that the motion of the kinematic chain is sufficiently rich to assure observability of the relative pose. However, a formal investigation of this crucial requirement has not yet been presented, and no specific conditions for observability have so far been given. In this work, we present an observability analysis and show that the relative pose of the body segments is indeed observable under a very mild condition on the motion. We support our results by simulation studies, in which we employ a state estimator that neither uses magnetometer measurements nor additional sensors and does not impose assumptions on the accelerometer to measure only the direction of gravity, nor on the range of motion or degrees of freedom of the joints. We investigate the effect of the amount of excitation and of stationary periods in the data on the accuracy of the estimates. We then use experimental data from two mechanical joints as well as from a human gait experiment to validate the observability criterion in practice and to show that small excitation levels are sufficient for obtaining accurate estimates even in the presence of time periods during which the motion is not observable.
△ Less
Submitted 6 June, 2022; v1 submitted 4 February, 2021;
originally announced February 2021.
-
Sensor Artificial Intelligence and its Application to Space Systems -- A White Paper
Authors:
Anko Börner,
Heinz-Wilhelm Hübers,
Odej Kao,
Florian Schmidt,
Sören Becker,
Joachim Denzler,
Daniel Matolin,
David Haber,
Sergio Lucia,
Wojciech Samek,
Rudolph Triebel,
Sascha Eichstädt,
Felix Biessmann,
Anna Kruspe,
Peter Jung,
Manon Kok,
Guillermo Gallego,
Ralf Berger
Abstract:
Information and communication technologies have accompanied our everyday life for years. A steadily increasing number of computers, cameras, mobile devices, etc. generate more and more data, but at the same time we realize that the data can only partially be analyzed with classical approaches. The research and development of methods based on artificial intelligence (AI) made enormous progress in t…
▽ More
Information and communication technologies have accompanied our everyday life for years. A steadily increasing number of computers, cameras, mobile devices, etc. generate more and more data, but at the same time we realize that the data can only partially be analyzed with classical approaches. The research and development of methods based on artificial intelligence (AI) made enormous progress in the area of interpretability of data in recent years. With growing experience, both, the potential and limitations of these new technologies are increasingly better understood. Typically, AI approaches start with the data from which information and directions for action are derived. However, the circumstances under which such data are collected and how they change over time are rarely considered. A closer look at the sensors and their physical properties within AI approaches will lead to more robust and widely applicable algorithms. This holistic approach which considers entire signal chains from the origin to a data product, "Sensor AI", is a highly relevant topic with great potential. It will play a decisive role in autonomous driving as well as in areas of automated production, predictive maintenance or space research. The goal of this white paper is to establish "Sensor AI" as a dedicated research topic. We want to exchange knowledge on the current state-of-the-art on Sensor AI, to identify synergies among research groups and thus boost the collaboration in this key technology for science and industry.
△ Less
Submitted 9 June, 2020;
originally announced June 2020.
-
Sparse Magnetometer-free Inertial Motion Tracking -- A Condition for Observability in Double Hinge Joint Systems
Authors:
Karsten Eckhoff,
Manon Kok,
Sergio Lucia,
Thomas Seel
Abstract:
Inertial measurement units are commonly used in a growing number of application fields to track or capture motions of kinematic chains, such as human limbs, exoskeletons or robotic actuators. A major challenge is the presence of magnetic disturbances that result in unreliable magnetometer readings. Recent research revealed that this problem can be overcome by exploitation of kinematic constraints.…
▽ More
Inertial measurement units are commonly used in a growing number of application fields to track or capture motions of kinematic chains, such as human limbs, exoskeletons or robotic actuators. A major challenge is the presence of magnetic disturbances that result in unreliable magnetometer readings. Recent research revealed that this problem can be overcome by exploitation of kinematic constraints. While typically each segment of the kinematic chain is equipped with an IMU, a novel approach called sparse inertial motion tracking aims at infering the complete motion states from measurements of a reduced set of sensors. In the present contribution, we combine the magnetometer-free and the sparse approach for real-time motion tracking of double-hinge joint systems with non-parallel joint axes. Analyzing the observability of the system, we find a condition which assures that the relative orientations between all segments are uniquely determined by a kinematic constraint, which contains only the gyroscope readings. Furthermore, we propose a moving-horizon estimator and validate it in a simulation study of three movements with different degrees of excitation. The results of this study confirm all theoretical conjectures and demonstrate that magnetometer-free sparse inertial real-time motion tracking is feasible under precise and simple excitation conditions.
△ Less
Submitted 3 February, 2020;
originally announced February 2020.
-
A Fast and Robust Algorithm for Orientation Estimation using Inertial Sensors
Authors:
Manon Kok,
Thomas B. Schön
Abstract:
We present a novel algorithm for online, real-time orientation estimation. Our algorithm integrates gyroscope data and corrects the resulting orientation estimate for integration drift using accelerometer and magnetometer data. This correction is computed, at each time instance, using a single gradient descent step with fixed step length. This fixed step length results in robustness against model…
▽ More
We present a novel algorithm for online, real-time orientation estimation. Our algorithm integrates gyroscope data and corrects the resulting orientation estimate for integration drift using accelerometer and magnetometer data. This correction is computed, at each time instance, using a single gradient descent step with fixed step length. This fixed step length results in robustness against model errors, e.g. caused by large accelerations or by short-term magnetic field disturbances, which we numerically illustrate using Monte Carlo simulations. Our algorithm estimates a three-dimensional update to the orientation rather than the entire orientation itself. This reduces the computational complexity by approximately 1/3 with respect to the state of the art. It also improves the quality of the resulting estimates, specifically when the orientation corrections are large. We illustrate the efficacy of the algorithm using experimental data.
△ Less
Submitted 1 October, 2019;
originally announced October 2019.
-
Sensor Fusion for Magneto-Inductive Navigation
Authors:
Johan Wahlström,
Manon Kok,
Pedro Porto Buarque de Gusmao,
Traian E. Abrudan,
Niki Trigoni,
Andrew Markham
Abstract:
Magneto-inductive navigation is an inexpensive and easily deployable solution to many of today's navigation problems. By utilizing very low frequency magnetic fields, magneto-inductive technology circumvents the problems with attenuation and multipath that often plague competing modalities. Using triaxial transmitter and receiver coils, it is possible to compute position and orientation estimates…
▽ More
Magneto-inductive navigation is an inexpensive and easily deployable solution to many of today's navigation problems. By utilizing very low frequency magnetic fields, magneto-inductive technology circumvents the problems with attenuation and multipath that often plague competing modalities. Using triaxial transmitter and receiver coils, it is possible to compute position and orientation estimates in three dimensions. However, in many situations, additional information is available that constrains the set of possible solutions. For example, the receiver may be known to be coplanar with the transmitter, or orientation information may be available from inertial sensors. We employ a maximum a posteriori estimator to fuse magneto-inductive signals with such complementary information. Further, we derive the Cramer-Rao bound for the position estimates and investigate the problem of detecting distortions caused by ferrous material. The performance of the estimator is compared to the Cramer-Rao bound and a state-of-the-art estimator using both simulations and real-world data. By fusing magneto-inductive signals with accelerometer measurements, the median position error is reduced almost by a factor of two.
△ Less
Submitted 18 September, 2019;
originally announced September 2019.
-
Hepatic vessel segmentation using a reduced filter 3D U-Net in ultrasound imaging
Authors:
Bart R. Thomson,
Jasper Nijkamp,
Oleksandra Ivashchenko,
Ferdinand van der Heijden,
Jasper N. Smit,
Niels F. M. Kok,
Koert F. D. Kuhlmann,
Theo J. M. Ruers,
Matteo Fusaglia
Abstract:
Accurate hepatic vessel segmentation on ultrasound (US) images can be an important tool in the planning and execution of surgery, however proves to be a challenging task due to noise and speckle. Our method comprises a reduced filter 3D U-Net implementation to automatically detect hepatic vasculature in 3D US volumes. A comparison is made between volumes acquired with a 3D probe and stacked 2D US…
▽ More
Accurate hepatic vessel segmentation on ultrasound (US) images can be an important tool in the planning and execution of surgery, however proves to be a challenging task due to noise and speckle. Our method comprises a reduced filter 3D U-Net implementation to automatically detect hepatic vasculature in 3D US volumes. A comparison is made between volumes acquired with a 3D probe and stacked 2D US images based on electromagnetic tracking. Experiments are conducted on 67 scans, where 45 are used in training, 12 in validation and 10 in testing. This network architecture yields Dice scores of 0.740 and 0.781 for 3D and stacked 2D volumes respectively, comparing promising to literature and inter-observer performance (Dice = 0.879).
△ Less
Submitted 28 July, 2019;
originally announced July 2019.
-
Scalable Magnetic Field SLAM in 3D Using Gaussian Process Maps
Authors:
Manon Kok,
Arno Solin
Abstract:
We present a method for scalable and fully 3D magnetic field simultaneous localisation and mapping (SLAM) using local anomalies in the magnetic field as a source of position information. These anomalies are due to the presence of ferromagnetic material in the structure of buildings and in objects such as furniture. We represent the magnetic field map using a Gaussian process model and take well-kn…
▽ More
We present a method for scalable and fully 3D magnetic field simultaneous localisation and mapping (SLAM) using local anomalies in the magnetic field as a source of position information. These anomalies are due to the presence of ferromagnetic material in the structure of buildings and in objects such as furniture. We represent the magnetic field map using a Gaussian process model and take well-known physical properties of the magnetic field into account. We build local maps using three-dimensional hexagonal block tiling. To make our approach computationally tractable we use reduced-rank Gaussian process regression in combination with a Rao-Blackwellised particle filter. We show that it is possible to obtain accurate position and orientation estimates using measurements from a smartphone, and that our approach provides a scalable magnetic field SLAM algorithm in terms of both computational complexity and map storage.
△ Less
Submitted 10 June, 2018; v1 submitted 5 April, 2018;
originally announced April 2018.
-
Using Inertial Sensors for Position and Orientation Estimation
Authors:
Manon Kok,
Jeroen D. Hol,
Thomas B. Schön
Abstract:
In recent years, MEMS inertial sensors (3D accelerometers and 3D gyroscopes) have become widely available due to their small size and low cost. Inertial sensor measurements are obtained at high sampling rates and can be integrated to obtain position and orientation information. These estimates are accurate on a short time scale, but suffer from integration drift over longer time scales. To overcom…
▽ More
In recent years, MEMS inertial sensors (3D accelerometers and 3D gyroscopes) have become widely available due to their small size and low cost. Inertial sensor measurements are obtained at high sampling rates and can be integrated to obtain position and orientation information. These estimates are accurate on a short time scale, but suffer from integration drift over longer time scales. To overcome this issue, inertial sensors are typically combined with additional sensors and models. In this tutorial we focus on the signal processing aspects of position and orientation estimation using inertial sensors. We discuss different modeling choices and a selected number of important algorithms. The algorithms include optimization-based smoothing and filtering as well as computationally cheaper extended Kalman filter and complementary filter implementations. The quality of their estimates is illustrated using both experimental and simulated data.
△ Less
Submitted 10 June, 2018; v1 submitted 20 April, 2017;
originally announced April 2017.
-
A Scalable and Distributed Solution to the Inertial Motion Capture Problem
Authors:
Manon Kok,
Sina Khoshfetrat Pakazad,
Thomas B. Schön,
Anders Hansson,
Jeroen D. Hol
Abstract:
In inertial motion capture, a multitude of body segments are equipped with inertial sensors, consisting of 3D accelerometers and 3D gyroscopes. Using an optimization-based approach to solve the motion capture problem allows for natural inclusion of biomechanical constraints and for modeling the connection of the body segments at the joint locations. The computational complexity of solving this pro…
▽ More
In inertial motion capture, a multitude of body segments are equipped with inertial sensors, consisting of 3D accelerometers and 3D gyroscopes. Using an optimization-based approach to solve the motion capture problem allows for natural inclusion of biomechanical constraints and for modeling the connection of the body segments at the joint locations. The computational complexity of solving this problem grows both with the length of the data set and with the number of sensors and body segments considered. In this work, we present a scalable and distributed solution to this problem using tailored message passing, capable of exploiting the structure that is inherent in the problem. As a proof-of-concept we apply our algorithm to data from a lower body configuration.
△ Less
Submitted 18 August, 2016; v1 submitted 21 March, 2016;
originally announced March 2016.
-
Magnetometer calibration using inertial sensors
Authors:
Manon Kok,
Thomas B. Schön
Abstract:
In this work we present a practical algorithm for calibrating a magnetometer for the presence of magnetic disturbances and for magnetometer sensor errors. To allow for combining the magnetometer measurements with inertial measurements for orientation estimation, the algorithm also corrects for misalignment between the magnetometer and the inertial sensor axes. The calibration algorithm is formulat…
▽ More
In this work we present a practical algorithm for calibrating a magnetometer for the presence of magnetic disturbances and for magnetometer sensor errors. To allow for combining the magnetometer measurements with inertial measurements for orientation estimation, the algorithm also corrects for misalignment between the magnetometer and the inertial sensor axes. The calibration algorithm is formulated as the solution to a maximum likelihood problem and the computations are performed offline. The algorithm is shown to give good results using data from two different commercially available sensor units. Using the calibrated magnetometer measurements in combination with the inertial sensors to determine the sensor's orientation is shown to lead to significantly improved heading estimates.
△ Less
Submitted 14 July, 2016; v1 submitted 20 January, 2016;
originally announced January 2016.
-
Nonlinear state space smoothing using the conditional particle filter
Authors:
Andreas Svensson,
Thomas B. Schön,
Manon Kok
Abstract:
To estimate the smoothing distribution in a nonlinear state space model, we apply the conditional particle filter with ancestor sampling. This gives an iterative algorithm in a Markov chain Monte Carlo fashion, with asymptotic convergence results. The computational complexity is analyzed, and our proposed algorithm is successfully applied to the challenging problem of sensor fusion between ultra-w…
▽ More
To estimate the smoothing distribution in a nonlinear state space model, we apply the conditional particle filter with ancestor sampling. This gives an iterative algorithm in a Markov chain Monte Carlo fashion, with asymptotic convergence results. The computational complexity is analyzed, and our proposed algorithm is successfully applied to the challenging problem of sensor fusion between ultra-wideband and accelerometer/gyroscope measurements for indoor positioning. It appears to be a competitive alternative to existing nonlinear smoothing algorithms, in particular the forward filtering-backward simulation smoother.
△ Less
Submitted 16 September, 2015; v1 submitted 12 February, 2015;
originally announced February 2015.
