-
SceneRAG: Scene-level Retrieval-Augmented Generation for Video Understanding
Authors:
Nianbo Zeng,
Haowen Hou,
Fei Richard Yu,
Si Shi,
Ying Tiffany He
Abstract:
Despite recent advances in retrieval-augmented generation (RAG) for video understanding, effectively understanding long-form video content remains underexplored due to the vast scale and high complexity of video data. Current RAG approaches typically segment videos into fixed-length chunks, which often disrupts the continuity of contextual information and fails to capture authentic scene boundarie…
▽ More
Despite recent advances in retrieval-augmented generation (RAG) for video understanding, effectively understanding long-form video content remains underexplored due to the vast scale and high complexity of video data. Current RAG approaches typically segment videos into fixed-length chunks, which often disrupts the continuity of contextual information and fails to capture authentic scene boundaries. Inspired by the human ability to naturally organize continuous experiences into coherent scenes, we present SceneRAG, a unified framework that leverages large language models to segment videos into narrative-consistent scenes by processing ASR transcripts alongside temporal metadata. SceneRAG further sharpens these initial boundaries through lightweight heuristics and iterative correction. For each scene, the framework fuses information from both visual and textual modalities to extract entity relations and dynamically builds a knowledge graph, enabling robust multi-hop retrieval and generation that account for long-range dependencies. Experiments on the LongerVideos benchmark, featuring over 134 hours of diverse content, confirm that SceneRAG substantially outperforms prior baselines, achieving a win rate of up to 72.5 percent on generation tasks.
△ Less
Submitted 9 June, 2025;
originally announced June 2025.
-
Non-Hermitian sensing from the perspective of post-selected measurements
Authors:
Neng Zeng,
Tao Liu,
Keyu Xia,
Yu-Ran Zhang,
Franco Nori
Abstract:
By employing the Naimark dilation, we establish a fundamental connection between non-Hermitian quantum sensing and post-selected measurements. The sensitivity of non-Hermitian quantum sensors is determined by the effective quantum Fisher information (QFI), which incorporates the success probability of post-selection. We demonstrate that non-Hermitian sensors cannot outperform their Hermitian count…
▽ More
By employing the Naimark dilation, we establish a fundamental connection between non-Hermitian quantum sensing and post-selected measurements. The sensitivity of non-Hermitian quantum sensors is determined by the effective quantum Fisher information (QFI), which incorporates the success probability of post-selection. We demonstrate that non-Hermitian sensors cannot outperform their Hermitian counterpart when all information is harnessed, since the total QFI for the extended system constrains the effective QFI of the non-Hermitian subsystem. Moreover, we quantify the efficiency of non-Hermitian sensors with the ratio of the effective QFI to the total QFI, which can be optimized within the framework of post-selected measurements with minimal experimental trials. Our work provides a distinctive theoretical framework for investigating non-Hermitian quantum sensing and designing noise-resilient quantum metrological protocols.
△ Less
Submitted 12 May, 2025; v1 submitted 8 May, 2025;
originally announced May 2025.
-
SimWorld: A Unified Benchmark for Simulator-Conditioned Scene Generation via World Model
Authors:
Xinqing Li,
Ruiqi Song,
Qingyu Xie,
Ye Wu,
Nanxin Zeng,
Yunfeng Ai
Abstract:
With the rapid advancement of autonomous driving technology, a lack of data has become a major obstacle to enhancing perception model accuracy. Researchers are now exploring controllable data generation using world models to diversify datasets. However, previous work has been limited to studying image generation quality on specific public datasets. There is still relatively little research on how…
▽ More
With the rapid advancement of autonomous driving technology, a lack of data has become a major obstacle to enhancing perception model accuracy. Researchers are now exploring controllable data generation using world models to diversify datasets. However, previous work has been limited to studying image generation quality on specific public datasets. There is still relatively little research on how to build data generation engines for real-world application scenes to achieve large-scale data generation for challenging scenes. In this paper, a simulator-conditioned scene generation engine based on world model is proposed. By constructing a simulation system consistent with real-world scenes, simulation data and labels, which serve as the conditions for data generation in the world model, for any scenes can be collected. It is a novel data generation pipeline by combining the powerful scene simulation capabilities of the simulation engine with the robust data generation capabilities of the world model. In addition, a benchmark with proportionally constructed virtual and real data, is provided for exploring the capabilities of world models in real-world scenes. Quantitative results show that these generated images significantly improve downstream perception models performance. Finally, we explored the generative performance of the world model in urban autonomous driving scenarios. All the data and code will be available at https://github.com/Li-Zn-H/SimWorld.
△ Less
Submitted 18 March, 2025;
originally announced March 2025.
-
Quantum Noise Spectroscopy of Criticality in an Atomically Thin Magnet
Authors:
Mark E. Ziffer,
Francisco Machado,
Benedikt Ursprung,
Artur Lozovoi,
Aya Batoul Tazi,
Zhiyang Yuan,
Michael E. Ziebel,
Tom Delord,
Nanyu Zeng,
Evan Telford,
Daniel G. Chica,
Dane W. deQuilettes,
Xiaoyang Zhu,
James C. Hone,
Kenneth L. Shepard,
Xavier Roy,
Nathalie P. de Leon,
Emily J. Davis,
Shubhayu Chatterjee,
Carlos A. Meriles,
Jonathan S. Owen,
P. James Schuck,
Abhay N. Pasupathy
Abstract:
Dynamic critical fluctuations in magnetic materials encode important information about magnetic ordering in the associated critical exponents. Using nitrogen-vacancy centers in diamond, we implement $T_2$ (spin-decoherence) noise magnetometry to study critical dynamics in a 2D Van der Waals magnet CrSBr. By analyzing NV decoherence on time scales approaching the characteristic correlation time…
▽ More
Dynamic critical fluctuations in magnetic materials encode important information about magnetic ordering in the associated critical exponents. Using nitrogen-vacancy centers in diamond, we implement $T_2$ (spin-decoherence) noise magnetometry to study critical dynamics in a 2D Van der Waals magnet CrSBr. By analyzing NV decoherence on time scales approaching the characteristic correlation time $τ_c$ of critical fluctuations, we extract the critical exponent $ν$ for the correlation length. Our result deviates from the Ising prediction and highlights the role of long-range dipolar interactions in 2D CrSBr. Furthermore, analyzing the divergence of the correlation length suggests the possibility of 2D-XY criticality in CrSBr in a temperature window near $T_C$ where static magnetic domains are absent. Our work provides a first demonstration of $T_2$ noise magnetometry to quantitatively analyze critical scaling behavior in 2D materials.
△ Less
Submitted 15 August, 2024; v1 submitted 8 July, 2024;
originally announced July 2024.
-
Flattening Singular Values of Factorized Convolution for Medical Images
Authors:
Zexin Feng,
Na Zeng,
Jiansheng Fang,
Xingyue Wang,
Xiaoxi Lu,
Heng Meng,
Jiang Liu
Abstract:
Convolutional neural networks (CNNs) have long been the paradigm of choice for robust medical image processing (MIP). Therefore, it is crucial to effectively and efficiently deploy CNNs on devices with different computing capabilities to support computer-aided diagnosis. Many methods employ factorized convolutional layers to alleviate the burden of limited computational resources at the expense of…
▽ More
Convolutional neural networks (CNNs) have long been the paradigm of choice for robust medical image processing (MIP). Therefore, it is crucial to effectively and efficiently deploy CNNs on devices with different computing capabilities to support computer-aided diagnosis. Many methods employ factorized convolutional layers to alleviate the burden of limited computational resources at the expense of expressiveness. To this end, given weak medical image-driven CNN model optimization, a Singular value equalization generalizer-induced Factorized Convolution (SFConv) is proposed to improve the expressive power of factorized convolutions in MIP models. We first decompose the weight matrix of convolutional filters into two low-rank matrices to achieve model reduction. Then minimize the KL divergence between the two low-rank weight matrices and the uniform distribution, thereby reducing the number of singular value directions with significant variance. Extensive experiments on fundus and OCTA datasets demonstrate that our SFConv yields competitive expressiveness over vanilla convolutions while reducing complexity.
△ Less
Submitted 1 March, 2024;
originally announced March 2024.
-
Implementation Guidance for Wood Harvesting and Storage
Authors:
Ning Zeng,
Daniel Sanchez,
Erica Belmont,
Henry Hausmann
Abstract:
This implementation guidance focuses on carbon removal and sequestration via wood harvesting and storage (WHS), a process where woody biomass, with the embedded carbon, is stored for long timescales in shallow geologic storage. The engineering structure designed to ensure such durable storage by preventing biomass decomposition is called a Wood Vault.
This guidance contains the requirements for…
▽ More
This implementation guidance focuses on carbon removal and sequestration via wood harvesting and storage (WHS), a process where woody biomass, with the embedded carbon, is stored for long timescales in shallow geologic storage. The engineering structure designed to ensure such durable storage by preventing biomass decomposition is called a Wood Vault.
This guidance contains the requirements for a basic Wood Vault project, and is intended to aid project developers, verifiers, and registries in this space. It describes a set of requirements that govern the end-to-end process of carbon removal and sequestration. This includes carbon accounting, wood sourcing via wood residual (WR) utilization, Wood Vault construction and maintenance, as well as processes for monitoring, verification, and credit issuance. Carbon accounting requirements include baseline, or counterfactual specification, and full life cycle analysis (LCA) within a specified process boundary. For the vault itself, the guidance describes a buried vault with the burial chamber covered by a layer of low permeability material to create anoxic condition. Other types of vaults can also be used to adjust to local environmental, transport, and economic constraints. Monitoring and verification requirements include in-situ sensors, gas sampling, sample excavations, and site maintenance. This guidance also contemplates land ownership and legal assurances, as well as environmental and societal impact assessments.
The implementation guidance concludes with recommendations regarding auditing, certification and carbon credit issuance.
△ Less
Submitted 8 September, 2023;
originally announced September 2023.
-
PiML Toolbox for Interpretable Machine Learning Model Development and Diagnostics
Authors:
Agus Sudjianto,
Aijun Zhang,
Zebin Yang,
Yu Su,
Ningzhou Zeng
Abstract:
PiML (read $π$-ML, /`pai`em`el/) is an integrated and open-access Python toolbox for interpretable machine learning model development and model diagnostics. It is designed with machine learning workflows in both low-code and high-code modes, including data pipeline, model training and tuning, model interpretation and explanation, and model diagnostics and comparison. The toolbox supports a growing…
▽ More
PiML (read $π$-ML, /`pai`em`el/) is an integrated and open-access Python toolbox for interpretable machine learning model development and model diagnostics. It is designed with machine learning workflows in both low-code and high-code modes, including data pipeline, model training and tuning, model interpretation and explanation, and model diagnostics and comparison. The toolbox supports a growing list of interpretable models (e.g. GAM, GAMI-Net, XGB1/XGB2) with inherent local and/or global interpretability. It also supports model-agnostic explainability tools (e.g. PFI, PDP, LIME, SHAP) and a powerful suite of model-agnostic diagnostics (e.g. weakness, reliability, robustness, resilience, fairness). Integration of PiML models and tests to existing MLOps platforms for quality assurance are enabled by flexible high-code APIs. Furthermore, PiML toolbox comes with a comprehensive user guide and hands-on examples, including the applications for model development and validation in banking. The project is available at https://github.com/SelfExplainML/PiML-Toolbox.
△ Less
Submitted 19 December, 2023; v1 submitted 7 May, 2023;
originally announced May 2023.
-
Decreasing emissions and increasing sink capacity to support China in achieving carbon neutrality before 2060
Authors:
Pengfei Han,
Ning Zeng,
Wen Zhang,
Qixiang Cai,
Ruqi Yang,
Bo Yao,
Xiaohui Lin,
Guocheng Wang,
Di Liu,
Yongqiang Yu
Abstract:
In September 2020, President Xi Jinping announced that China strives to achieve carbon neutrality before 2060. This ambitious and bold commitment was well received by the global community. However, the technology and pathway are not so clear. Here, we conducted an extensive review covering more than 200 published papers and summarized the key technologies to achieve carbon neutrality. We projected…
▽ More
In September 2020, President Xi Jinping announced that China strives to achieve carbon neutrality before 2060. This ambitious and bold commitment was well received by the global community. However, the technology and pathway are not so clear. Here, we conducted an extensive review covering more than 200 published papers and summarized the key technologies to achieve carbon neutrality. We projected sectoral CO2 emissions for 2020-2050 based on our previous studies and published scenarios. We applied a medium sink scenario for terrestrial sinks due to the potential resource competition and included an ocean sink, which has generally not been included in previous estimates. We analyzed and revisited China's historical terrestrial carbon sink capacity from 1980-2020 based on multiple models and a literature review. To achieve neutrality, it is necessary to increase sink capacity and decrease emissions from many sources. On the one hand, critical measures to reduce emissions include decreasing the use of fossil fuels; substantially increasing the proportion of the renewable energy and nuclear energy. On the other hand, the capacity of future carbon sinks is projected to decrease due to the natural evolution of terrestrial ecosystems, and anthropogenic management practices are needed to increase sink capacity, including increasing the forest sinks through national ecological restoration projects and large-scale land greening campaigns; increasing wood harvesting and storage; and developing CCUS. This paper provides basic source and sink data,and established and promising new technologies for decreasing emissions and increasing sinks for use by the scientific community and policy makers.
△ Less
Submitted 17 December, 2023; v1 submitted 22 February, 2021;
originally announced February 2021.
-
Global to local impacts on atmospheric CO2 caused by COVID-19 lockdown
Authors:
Ning Zeng,
Pengfei Han,
Di Liu,
Zhiqiang Liu,
Tomohiro Oda,
Cory Martin,
Zhu Liu,
Bo Yao,
Wanqi Sun,
Pucai Wang,
Qixiang Cai,
Russell Dickerson,
Shamil Maksyutov
Abstract:
The world-wide lockdown in response to the COVID-19 pandemic in year 2020 led to economic slowdown and large reduction of fossil fuel CO2 emissions, but it is unclear how much it would reduce atmospheric CO2 concentration, and whether it can be observed. We estimated that a 7.9% reduction in emissions for 4 months would result in a 0.25 ppm decrease in the Northern Hemisphere CO2, an increment tha…
▽ More
The world-wide lockdown in response to the COVID-19 pandemic in year 2020 led to economic slowdown and large reduction of fossil fuel CO2 emissions, but it is unclear how much it would reduce atmospheric CO2 concentration, and whether it can be observed. We estimated that a 7.9% reduction in emissions for 4 months would result in a 0.25 ppm decrease in the Northern Hemisphere CO2, an increment that is within the capability of current CO2 analyzers, but is a few times smaller than natural CO2 variabilities caused by weather and the biosphere such as El Nino. We used a state-of-the-art atmospheric transport model to simulate CO2, driven by a new daily fossil fuel emissions dataset and hourly biospheric fluxes from a carbon cycle model forced with observed climate variability. Our results show a 0.13 ppm decrease in atmospheric column CO2 anomaly averaged over 50S-50N for the period February-April 2020 relative to a 10-year climatology. A similar decrease was observed by the carbon satellite GOSAT3. Using model sensitivity experiments, we further found that COVID, the biosphere and weather contributed 54%, 23%, and 23% respectively. This seemingly small change stands out as the largest sub-annual anomaly in the last 10 years. Measurements from global ground stations were analyzed. At city scale, on-road CO2 enhancement measured in Beijing shows reduction of 20-30 ppm, consistent with drastically reduced traffic during the lockdown. The ability of our current carbon monitoring systems in detecting the small and short-lasting COVID signal on the background of fossil fuel CO2 accumulated over the last two centuries is encouraging. The COVID-19 pandemic is an unintended experiment whose impact suggests that to keep atmospheric CO2 at a climate-safe level will require sustained effort of similar magnitude and improved accuracy and expanded spatiotemporal coverage of our monitoring systems.
△ Less
Submitted 24 October, 2020;
originally announced October 2020.
-
Nonequilibrium Green's function method for thermal transport in junctions
Authors:
Jian-Sheng Wang,
Nan Zeng,
Jian Wang,
Chee Kwan Gan
Abstract:
We present a detailed treatment of the nonequilibrium Green's function method for thermal transport due to atomic vibrations in nanostructures. Some of the key equations, such as self-energy and conductance with nonlinear effect, are derived. A self-consistent mean-field theory is proposed. Computational procedures are discussed. The method is applied to a number of systems including one-dimensi…
▽ More
We present a detailed treatment of the nonequilibrium Green's function method for thermal transport due to atomic vibrations in nanostructures. Some of the key equations, such as self-energy and conductance with nonlinear effect, are derived. A self-consistent mean-field theory is proposed. Computational procedures are discussed. The method is applied to a number of systems including one-dimensional chains, a benzene ring junction, and carbon nanotubes. Mean-field calculations of the Fermi-Pasta-Ulam model are compared with classical molecular dynamics simulations. We find that nonlinearity suppresses thermal transport even at moderately high temperatures.
△ Less
Submitted 9 January, 2007;
originally announced January 2007.
-
Nonequilibrium Green's function approach to mesoscopic thermal transport
Authors:
Jian-Sheng Wang,
Jian Wang,
Nan Zeng
Abstract:
We present a formulation of a nonequilibrium Green's function method for thermal current in nanojunction atomic systems with nonlinear interactions. This first-principle approach is applied to the calculation of the thermal conductance in carbon nanotube junctions. It is shown that nonlinearity already becomes important at low temperatures. Nonlinear interactions greatly suppress phonon transmis…
▽ More
We present a formulation of a nonequilibrium Green's function method for thermal current in nanojunction atomic systems with nonlinear interactions. This first-principle approach is applied to the calculation of the thermal conductance in carbon nanotube junctions. It is shown that nonlinearity already becomes important at low temperatures. Nonlinear interactions greatly suppress phonon transmission at room temperature. The peak of thermal conductance is found to be around 400K, in good agreement with experiments. High-order phonon scattering processes are important for diffusive heat transport.
△ Less
Submitted 1 May, 2006;
originally announced May 2006.
