-
Deep Learning with Pretrained 'Internal World' Layers: A Gemma 3-Based Modular Architecture for Wildfire Prediction
Authors:
Ayoub Jadouli,
Chaker El Amrani
Abstract:
Deep learning models, especially large Transformers, carry substantial "memory" in their intermediate layers -- an \emph{internal world} that encodes a wealth of relational and contextual knowledge. This work harnesses that internal world for wildfire occurrence prediction by introducing a modular architecture built upon Gemma 3, a state-of-the-art multimodal model. Rather than relying on Gemma 3'…
▽ More
Deep learning models, especially large Transformers, carry substantial "memory" in their intermediate layers -- an \emph{internal world} that encodes a wealth of relational and contextual knowledge. This work harnesses that internal world for wildfire occurrence prediction by introducing a modular architecture built upon Gemma 3, a state-of-the-art multimodal model. Rather than relying on Gemma 3's original embedding and positional encoding stacks, we develop a custom feed-forward module that transforms tabular wildfire features into the hidden dimension required by Gemma 3's mid-layer Transformer blocks. We freeze these Gemma 3 sub-layers -- thus preserving their pretrained representation power -- while training only the smaller input and output networks. This approach minimizes the number of trainable parameters and reduces the risk of overfitting on limited wildfire data, yet retains the benefits of Gemma 3's broad knowledge. Evaluations on a Moroccan wildfire dataset demonstrate improved predictive accuracy and robustness compared to standard feed-forward and convolutional baselines. Ablation studies confirm that the frozen Transformer layers consistently contribute to better representations, underscoring the feasibility of reusing large-model mid-layers as a learned internal world. Our findings suggest that strategic modular reuse of pretrained Transformers can enable more data-efficient and interpretable solutions for critical environmental applications such as wildfire risk management.
△ Less
Submitted 20 April, 2025;
originally announced April 2025.
-
Advanced Wildfire Prediction in Morocco: Developing a Deep Learning Dataset from Multisource Observations
Authors:
Ayoub Jadouli,
Chaker El Amrani
Abstract:
Wildfires pose significant threats to ecosystems, economies, and communities worldwide, necessitating advanced predictive methods for effective mitigation. This study introduces a novel and comprehensive dataset specifically designed for wildfire prediction in Morocco, addressing its unique geographical and climatic challenges. By integrating satellite observations and ground station data, we comp…
▽ More
Wildfires pose significant threats to ecosystems, economies, and communities worldwide, necessitating advanced predictive methods for effective mitigation. This study introduces a novel and comprehensive dataset specifically designed for wildfire prediction in Morocco, addressing its unique geographical and climatic challenges. By integrating satellite observations and ground station data, we compile essential environmental indicators such as vegetation health (NDVI), population density, soil moisture levels, and meteorological data aimed at predicting next-day wildfire occurrences with high accuracy. Our methodology incorporates state-of-the-art machine learning and deep learning algorithms, demonstrating superior performance in capturing wildfire dynamics compared to traditional models. Preliminary results show that models using this dataset achieve an accuracy of up to 90%, significantly improving prediction capabilities. The public availability of this dataset fosters scientific collaboration, aiming to refine predictive models and develop innovative wildfire management strategies. Our work not only advances the technical field of dataset creation but also emphasizes the necessity for localized research in underrepresented regions, providing a scalable model for other areas facing similar environmental challenges.
△ Less
Submitted 9 November, 2024;
originally announced November 2024.
-
Parallel Multi-path Feed Forward Neural Networks (PMFFNN) for Long Columnar Datasets: A Novel Approach to Complexity Reduction
Authors:
Ayoub Jadouli,
Chaker El Amrani
Abstract:
Traditional Feed-Forward Neural Networks (FFNN) and one-dimensional Convolutional Neural Networks (1D CNN) often encounter difficulties when dealing with long, columnar datasets that contain numerous features. The challenge arises from two primary factors: the large volume of data and the potential absence of meaningful relationships between features. In conventional training, large datasets can o…
▽ More
Traditional Feed-Forward Neural Networks (FFNN) and one-dimensional Convolutional Neural Networks (1D CNN) often encounter difficulties when dealing with long, columnar datasets that contain numerous features. The challenge arises from two primary factors: the large volume of data and the potential absence of meaningful relationships between features. In conventional training, large datasets can overwhelm the model, causing significant portions of the input to remain underutilized. As a result, the model may fail to capture the critical information necessary for effective learning, which leads to diminished performance.
To overcome these limitations, we introduce a novel architecture called Parallel Multi-path Feed Forward Neural Networks (PMFFNN). Our approach leverages multiple parallel pathways to process distinct subsets of columns from the input dataset. By doing so, the architecture ensures that each subset of features receives focused attention, which is often neglected in traditional models. This approach maximizes the utilization of feature diversity, ensuring that no critical data sections are overlooked during training.
Our architecture offers two key advantages. First, it allows for more effective handling of long, columnar data by distributing the learning task across parallel paths. Second, it reduces the complexity of the model by narrowing the feature scope in each path, which leads to faster training times and improved resource efficiency. The empirical results indicate that PMFFNN outperforms traditional FFNNs and 1D CNNs, providing an optimized solution for managing large-scale data.
△ Less
Submitted 8 November, 2024;
originally announced November 2024.
-
Enhancing Wildfire Forecasting Through Multisource Spatio-Temporal Data, Deep Learning, Ensemble Models and Transfer Learning
Authors:
Ayoub Jadouli,
Chaker El Amrani
Abstract:
This paper presents a novel approach in wildfire prediction through the integration of multisource spatiotemporal data, including satellite data, and the application of deep learning techniques. Specifically, we utilize an ensemble model built on transfer learning algorithms to forecast wildfires. The key focus is on understanding the significance of weather sequences, human activities, and specif…
▽ More
This paper presents a novel approach in wildfire prediction through the integration of multisource spatiotemporal data, including satellite data, and the application of deep learning techniques. Specifically, we utilize an ensemble model built on transfer learning algorithms to forecast wildfires. The key focus is on understanding the significance of weather sequences, human activities, and specific weather parameters in wildfire prediction. The study encounters challenges in acquiring real-time data for training the network, especially in Moroccan wildlands. The future work intends to develop a global model capable of processing multichannel, multidimensional, and unformatted data sources to enhance our understanding of the future entropy of surface tiles.
△ Less
Submitted 19 July, 2024;
originally announced July 2024.
-
SAT-CEP-monitor: An air quality monitoring software architecture combining complex event processing with satellite remote sensing
Authors:
Badr-Eddine Boudriki Semlali,
Chaker El Amrani,
Guadalupe Ortiz,
Juan Boubeta-Puig,
Alfonso Garcia-de-Prado
Abstract:
Air pollution is a major problem today that causes serious damage to human health. Urban areas are the most affected by the degradation of air quality caused by anthropogenic gas emissions. Although there are multiple proposals for air quality monitoring, in most cases, two limitations are imposed: the impossibility of processing data in Near Real-Time (NRT) for remote sensing approaches and the i…
▽ More
Air pollution is a major problem today that causes serious damage to human health. Urban areas are the most affected by the degradation of air quality caused by anthropogenic gas emissions. Although there are multiple proposals for air quality monitoring, in most cases, two limitations are imposed: the impossibility of processing data in Near Real-Time (NRT) for remote sensing approaches and the impossibility of reaching areas of limited accessibility or low network coverage for ground data approaches. We propose a software architecture that efficiently combines complex event processing with remote sensing data from various satellite sensors to monitor air quality in NRT, giving support to decision-makers. We illustrate the proposed solution by calculating the air quality levels for several areas of Morocco and Spain, extracting and processing satellite information in NRT. This study also validates the air quality measured by ground stations and satellite sensor data.
△ Less
Submitted 29 January, 2024;
originally announced January 2024.
