-
A Survey of the Self Supervised Learning Mechanisms for Vision Transformers
Authors:
Asifullah Khan,
Anabia Sohail,
Mustansar Fiaz,
Mehdi Hassan,
Tariq Habib Afridi,
Sibghat Ullah Marwat,
Farzeen Munir,
Safdar Ali,
Hannan Naseem,
Muhammad Zaigham Zaheer,
Kamran Ali,
Tangina Sultana,
Ziaurrehman Tanoli,
Naeem Akhter
Abstract:
Deep supervised learning models require high volume of labeled data to attain sufficiently good results. Although, the practice of gathering and annotating such big data is costly and laborious. Recently, the application of self supervised learning (SSL) in vision tasks has gained significant attention. The intuition behind SSL is to exploit the synchronous relationships within the data as a form…
▽ More
Deep supervised learning models require high volume of labeled data to attain sufficiently good results. Although, the practice of gathering and annotating such big data is costly and laborious. Recently, the application of self supervised learning (SSL) in vision tasks has gained significant attention. The intuition behind SSL is to exploit the synchronous relationships within the data as a form of self-supervision, which can be versatile. In the current big data era, most of the data is unlabeled, and the success of SSL thus relies in finding ways to utilize this vast amount of unlabeled data available. Thus it is better for deep learning algorithms to reduce reliance on human supervision and instead focus on self-supervision based on the inherent relationships within the data. With the advent of ViTs, which have achieved remarkable results in computer vision, it is crucial to explore and understand the various SSL mechanisms employed for training these models specifically in scenarios where there is limited labelled data available. In this survey, we develop a comprehensive taxonomy of systematically classifying the SSL techniques based upon their representations and pre-training tasks being applied. Additionally, we discuss the motivations behind SSL, review popular pre-training tasks, and highlight the challenges and advancements in this field. Furthermore, we present a comparative analysis of different SSL methods, evaluate their strengths and limitations, and identify potential avenues for future research.
△ Less
Submitted 20 March, 2025; v1 submitted 30 August, 2024;
originally announced August 2024.
-
Non-Hermitian Mode Cleaning in Periodically Modulated Multimode Fibers
Authors:
M. N. Akhter,
S. B. Ivars,
M. Botey,
R. Herrero,
K. Staliunas
Abstract:
We show that the simultaneous modulation of the propagation constant and of the gain/loss coefficient along the multimode fibers results in unidirectional coupling among the modes, which, depending on the modulation parameters, leads to the enhancement or reduction of the excitation of higher order transverse modes. In the latter case, effective mode-cleaning is predicted, in ideal case resulting…
▽ More
We show that the simultaneous modulation of the propagation constant and of the gain/loss coefficient along the multimode fibers results in unidirectional coupling among the modes, which, depending on the modulation parameters, leads to the enhancement or reduction of the excitation of higher order transverse modes. In the latter case, effective mode-cleaning is predicted, in ideal case resulting in single-mode spatially coherent output. The effect is semi-analytically predicted on a simplified Gaussian beam approximation and numerically proven by solving the wave propagation equation introducing the modulated potential.
△ Less
Submitted 23 November, 2022;
originally announced November 2022.
-
Insights into performance evaluation of com-pound-protein interaction prediction methods
Authors:
Adiba Yaseen,
Imran Amin,
Naeem Akhter,
Asa Ben-Hur,
Fayyaz Minhas
Abstract:
Motivation: Machine learning based prediction of compound-protein interactions (CPIs) is important for drug design, screening and repurposing studies and can improve the efficiency and cost-effectiveness of wet lab assays. Despite the publication of many research papers reporting CPI predictors in the recent years, we have observed a number of fundamental issues in experiment design that lead to o…
▽ More
Motivation: Machine learning based prediction of compound-protein interactions (CPIs) is important for drug design, screening and repurposing studies and can improve the efficiency and cost-effectiveness of wet lab assays. Despite the publication of many research papers reporting CPI predictors in the recent years, we have observed a number of fundamental issues in experiment design that lead to over optimistic estimates of model performance. Results: In this paper, we analyze the impact of several important factors affecting generalization perfor-mance of CPI predictors that are overlooked in existing work: 1. Similarity between training and test examples in cross-validation 2. The strategy for generating negative examples, in the absence of experimentally verified negative examples. 3. Choice of evaluation protocols and performance metrics and their alignment with real-world use of CPI predictors in screening large compound libraries. Using both an existing state-of-the-art method (CPI-NN) and a proposed kernel based approach, we have found that assessment of predictive performance of CPI predictors requires careful con-trol over similarity between training and test examples. We also show that random pairing for gen-erating synthetic negative examples for training and performance evaluation results in models with better generalization performance in comparison to more sophisticated strategies used in existing studies. Furthermore, we have found that our kernel based approach, despite its simple design, exceeds the prediction performance of CPI-NN. We have used the proposed model for compound screening of several proteins including SARS-CoV-2 Spike and Human ACE2 proteins and found strong evidence in support of its top hits. Availability: Code and raw experimental results available at https://github.com/adibayaseen/HKRCPI Contact: [email protected]
△ Less
Submitted 28 January, 2022;
originally announced February 2022.
-
Ovarian Cancer Prediction from Ovarian Cysts Based on TVUS Using Machine Learning Algorithms
Authors:
Laboni Akter,
Nasrin Akhter
Abstract:
Ovarian Cancer (OC) is type of female reproductive malignancy which can be found among young girls and mostly the women in their fertile or reproductive. There are few number of cysts are dangerous and may it cause cancer. So, it is very important to predict and it can be from different types of screening are used for this detection using Transvaginal Ultrasonography (TVUS) screening. In this rese…
▽ More
Ovarian Cancer (OC) is type of female reproductive malignancy which can be found among young girls and mostly the women in their fertile or reproductive. There are few number of cysts are dangerous and may it cause cancer. So, it is very important to predict and it can be from different types of screening are used for this detection using Transvaginal Ultrasonography (TVUS) screening. In this research, we employed an actual datasets called PLCO with TVUS screening and three machine learning (ML) techniques, respectively Random Forest KNN, and XGBoost within three target variables. We obtained a best performance from this algorithms as far as accuracy, recall, f1 score and precision with the approximations of 99.50%, 99.50%, 99.49% and 99.50% individually. The AUC score of 99.87%, 98.97% and 99.88% are observed in these Random Forest, KNN and XGB algorithms .This approach helps assist physicians and suspects in identifying ovarian risks early on, reducing ovarian malignancy-related complications and deaths.
△ Less
Submitted 30 August, 2021;
originally announced August 2021.
-
D-OccNet: Detailed 3D Reconstruction Using Cross-Domain Learning
Authors:
Minhaj Uddin Ansari,
Talha Bilal,
Naeem Akhter
Abstract:
Deep learning based 3D reconstruction of single view 2D image is becoming increasingly popular due to their wide range of real-world applications, but this task is inherently challenging because of the partial observability of an object from a single perspective. Recently, state of the art probability based Occupancy Networks reconstructed 3D surfaces from three different types of input domains: s…
▽ More
Deep learning based 3D reconstruction of single view 2D image is becoming increasingly popular due to their wide range of real-world applications, but this task is inherently challenging because of the partial observability of an object from a single perspective. Recently, state of the art probability based Occupancy Networks reconstructed 3D surfaces from three different types of input domains: single view 2D image, point cloud and voxel. In this study, we extend the work on Occupancy Networks by exploiting cross-domain learning of image and point cloud domains. Specifically, we first convert the single view 2D image into a simpler point cloud representation, and then reconstruct a 3D surface from it. Our network, the Double Occupancy Network (D-OccNet) outperforms Occupancy Networks in terms of visual quality and details captured in the 3D reconstruction.
△ Less
Submitted 28 April, 2021;
originally announced April 2021.
-
Decoy Selection for Protein Structure Prediction Via Extreme Gradient Boosting and Ranking
Authors:
Nasrin Akhter,
Gopinath Chennupati,
Hristo Djidjev,
Amarda Shehu
Abstract:
Identifying one or more biologically-active/native decoys from millions of non-native decoys is one of the major challenges in computational structural biology. The extreme lack of balance in positive and negative samples (native and non-native decoys) in a decoy set makes the problem even more complicated. Consensus methods show varied success in handling the challenge of decoy selection despite…
▽ More
Identifying one or more biologically-active/native decoys from millions of non-native decoys is one of the major challenges in computational structural biology. The extreme lack of balance in positive and negative samples (native and non-native decoys) in a decoy set makes the problem even more complicated. Consensus methods show varied success in handling the challenge of decoy selection despite some issues associated with clustering large decoy sets and decoy sets that do not show much structural similarity. Recent investigations into energy landscape-based decoy selection approaches show promises. However, lack of generalization over varied test cases remains a bottleneck for these methods. We propose a novel decoy selection method, ML-Select, a machine learning framework that exploits the energy landscape associated with the structure space probed through a template-free decoy generation. The proposed method outperforms both clustering and energy ranking-based methods, all the while consistently offering better performance on varied test-cases. Moreover, ML-Select shows promising results even for the decoy sets consisting of mostly low-quality decoys. ML-Select is a useful method for decoy selection. This work suggests further research in finding more effective ways to adopt machine learning frameworks in achieving robust performance for decoy selection in template-free protein structure prediction.
△ Less
Submitted 3 October, 2020;
originally announced October 2020.
-
Energy-aware virtual machine selection method for cloud data center resource allocation
Authors:
Nasrin Akhter,
Mohamed Othman,
Ranesh Kumar Naha
Abstract:
Saving energy is an important issue for cloud providers to reduce energy cost in a data center. With the increasing popularity of cloud computing, it is time to examine various energy reduction methods for which energy consumption could be reduced and lead us to green cloud computing. In this paper, our aim is to propose a virtual machine selection algorithm to improve the energy efficiency of a c…
▽ More
Saving energy is an important issue for cloud providers to reduce energy cost in a data center. With the increasing popularity of cloud computing, it is time to examine various energy reduction methods for which energy consumption could be reduced and lead us to green cloud computing. In this paper, our aim is to propose a virtual machine selection algorithm to improve the energy efficiency of a cloud data center. We are also presenting experimental results of the proposed algorithm in a cloud computing based simulation environment. The proposed algorithm dynamically took the virtual machines' allocation, deallocation, and reallocation action to the physical server. However, it depends on the load and heuristics based on the analysis placement of a virtual machine which is decided over time. From the results obtained from the simulation, we have found that our proposed virtual machine selection algorithm reduces the total energy consumption by 19% compared to the existing one. Therefore, the energy consumption cost of a cloud data center reduces and also lowers the carbon footprint. Simulation-based experimental results show that the proposed heuristics which are based on resource provisioning algorithms reduce the energy consumption of the cloud data center and decrease the virtual machine's migration rate.
△ Less
Submitted 20 December, 2018;
originally announced December 2018.
-
Evaluation of Energy-efficient VM Consolidation for Cloud Based Data Center - Revisited
Authors:
Nasrin Akhter,
Mohamed Othman,
Ranesh Kumar Naha
Abstract:
In this paper, a re-evaluation undertaken for dynamic VM consolidation problem and optimal online deterministic algorithms for the single VM migration in an experimental environment. We proceeded to focus on energy and performance trade-off by planet lab workload traces, which consists of a thousand Planetlab VMs with widespread simulation environments. All experiments are done in a simulated clou…
▽ More
In this paper, a re-evaluation undertaken for dynamic VM consolidation problem and optimal online deterministic algorithms for the single VM migration in an experimental environment. We proceeded to focus on energy and performance trade-off by planet lab workload traces, which consists of a thousand Planetlab VMs with widespread simulation environments. All experiments are done in a simulated cloud environment by the CloudSim simulation tool. A new paradigm of utility-oriented IT services is cloud computing, which offers a pay-as-you-go model. In recent years, there has been increasing interest among many users from business, scientific, engineering and educational territories in cloud computing. There is increasing concern that high energy consumption issues are a disadvantage for various institutions. However, so far too little attention has been given to the various methods to reduce energy consumption in cloud environments while ensuring performance. Besides the evaluation of energy-efficient data center management algorithms in the cloud, we proposed a further research directed toward the development of energy efficient algorithms. By the experimental evaluation of the current proposal for the competitive analysis of dynamic VM consolidation and optimal online deterministic algorithms for the single VM migration, we found different results for different algorithm combinations. Cloud-based data centers` consume massive energy, which has a negative effect on the environment and operational cost, this work contributes to the energy consumption reduction in the cloud environment.
△ Less
Submitted 15 December, 2018;
originally announced December 2018.
-
Extremal k-apex Trees for Randic Index
Authors:
Naveed Akhter,
Muhammad Kamran Jamil,
Ioan Tomescu
Abstract:
The Randic (connectivity) index is one of the most successful molecular descriptors in structure-property and structure-activity relationships studies. J. Gao found the sharp upper bound for the Randic index of apex trees. In this paper, we proved that k-apex trees are not regular graphs for k\ge2 and proposed a sharp upper bound for the Randic index of k-apex trees for k>1.
The Randic (connectivity) index is one of the most successful molecular descriptors in structure-property and structure-activity relationships studies. J. Gao found the sharp upper bound for the Randic index of apex trees. In this paper, we proved that k-apex trees are not regular graphs for k\ge2 and proposed a sharp upper bound for the Randic index of k-apex trees for k>1.
△ Less
Submitted 7 December, 2015;
originally announced December 2015.
-
Very high-energy observations of the two high-frequency peaked BL Lac objects 1ES 1218+304 and H 1426+428
Authors:
C. Mueller,
N. Akhter,
J. Ball,
D. A. Bramel,
J. Carson,
C. E. Covault,
D. Driscoll,
P. Fortin,
D. M. Gingrich,
D. S. Hanna,
A. Jarvis,
J. Kildea,
T. Lindner,
R. Mukherjee,
R. A. Ong,
K. Ragan,
R. A. Scalzo,
D. A. Williams,
J. Zweerink
Abstract:
We present results of very-high-energy gamma-ray observations (E > 160 GeV) of two high-frequency-peaked BL Lac (HBL) objects, 1ES 1218+304 and H 1426+428, with the Solar Tower Atmospheric Cherenkov Effect Experiment (STACEE). Both sources are very-high-energy gamma-ray emitters above 100 GeV, detected using ground-based Cherenkov telescopes. STACEE observations of 1ES 1218+304 and H 1426+428 did…
▽ More
We present results of very-high-energy gamma-ray observations (E > 160 GeV) of two high-frequency-peaked BL Lac (HBL) objects, 1ES 1218+304 and H 1426+428, with the Solar Tower Atmospheric Cherenkov Effect Experiment (STACEE). Both sources are very-high-energy gamma-ray emitters above 100 GeV, detected using ground-based Cherenkov telescopes. STACEE observations of 1ES 1218+304 and H 1426+428 did not produce detections; we present 99% CL flux upper limits for both sources, assuming spectral indices measured mostly at higher energies.
△ Less
Submitted 14 April, 2011;
originally announced April 2011.
-
Simulation and Performance Analysis of Adaptive Filtering Algorithms in Noise Cancellation
Authors:
Lilatul Ferdouse,
Nasrin Akhter,
Tamanna Haque Nipa,
Fariha Tasmin Jaigirdar
Abstract:
Noise problems in signals have gained huge attention due to the need of noise-free output signal in numerous communication systems. The principal of adaptive noise cancellation is to acquire an estimation of the unwanted interfering signal and subtract it from the corrupted signal. Noise cancellation operation is controlled adaptively with the target of achieving improved signal to noise ratio. Th…
▽ More
Noise problems in signals have gained huge attention due to the need of noise-free output signal in numerous communication systems. The principal of adaptive noise cancellation is to acquire an estimation of the unwanted interfering signal and subtract it from the corrupted signal. Noise cancellation operation is controlled adaptively with the target of achieving improved signal to noise ratio. This paper concentrates upon the analysis of adaptive noise canceller using Recursive Least Square (RLS), Fast Transversal Recursive Least Square (FTRLS) and Gradient Adaptive Lattice (GAL) algorithms. The performance analysis of the algorithms is done based on convergence behavior, convergence time, correlation coefficients and signal to noise ratio. After comparing all the simulated results we observed that GAL performs the best in noise cancellation in terms of Correlation Coefficient, SNR and Convergence Time. RLS, FTRLS and GAL were never evaluated and compared before on their performance in noise cancellation in terms of the criteria we considered here.
△ Less
Submitted 11 April, 2011;
originally announced April 2011.
-
Implementation of the Trigonometric LMS Algorithm using Original Cordic Rotation
Authors:
Nasrin Akhter,
Kaniz Fatema,
Lilatul Ferdouse,
Faria Khandaker
Abstract:
The LMS algorithm is one of the most successful adaptive filtering algorithms. It uses the instantaneous value of the square of the error signal as an estimate of the mean-square error (MSE). The LMS algorithm changes (adapts) the filter tap weights so that the error signal is minimized in the mean square sense. In Trigonometric LMS (TLMS) and Hyperbolic LMS (HLMS), two new versions of LMS algorit…
▽ More
The LMS algorithm is one of the most successful adaptive filtering algorithms. It uses the instantaneous value of the square of the error signal as an estimate of the mean-square error (MSE). The LMS algorithm changes (adapts) the filter tap weights so that the error signal is minimized in the mean square sense. In Trigonometric LMS (TLMS) and Hyperbolic LMS (HLMS), two new versions of LMS algorithms, same formulations are performed as in the LMS algorithm with the exception that filter tap weights are now expressed using trigonometric and hyperbolic formulations, in cases for TLMS and HLMS respectively. Hence appears the CORDIC algorithm as it can efficiently perform trigonometric, hyperbolic, linear and logarithmic functions. While hardware-efficient algorithms often exist, the dominance of the software systems has kept those algorithms out of the spotlight. Among these hardware- efficient algorithms, CORDIC is an iterative solution for trigonometric and other transcendental functions. Former researches worked on CORDIC algorithm to observe the convergence behavior of Trigonometric LMS (TLMS) algorithm and obtained a satisfactory result in the context of convergence performance of TLMS algorithm. But revious researches directly used the CORDIC block output in their simulation ignoring the internal step-by-step rotations of the CORDIC processor. This gives rise to a need for verification of the convergence performance of the TLMS algorithm to investigate if it actually performs satisfactorily if implemented with step-by-step CORDIC rotation. This research work has done this job. It focuses on the internal operations of the CORDIC hardware, implements the Trigonometric LMS (TLMS) and Hyperbolic LMS (HLMS) algorithms using actual CORDIC rotations. The obtained simulation results are highly satisfactory and also it shows that convergence behavior of HLMS is much better than TLMS.
△ Less
Submitted 21 April, 2011; v1 submitted 18 August, 2010;
originally announced August 2010.
-
STACEE Observations of 1ES 1218+304
Authors:
STACEE Collaboration,
R. Mukherjee,
N. Akhter,
J. Ball,
J. E. Carson,
C. E. Covault,
D. D. Driscoll,
P. Fortin,
D. M. Gingrich,
D. S. Hanna,
A. Jarvis,
J. Kildea,
T. Lindner,
C. Mueller,
R. A. Ong,
K. Ragan,
D. A. Williams,
J. Zweerink
Abstract:
We present the analysis and results of recent high-energy gamma-ray observations of the high energy-peaked BL Lac (HBL) object 1ES 1218+304 with the Solar Tower Atmospheric Cherenkov Effect Experiment (STACEE). 1ES 1218+304 is an X-ray bright HBL at a redshift z=0.182. It has been predicted to be a gamma-ray emitter above 100 GeV, detectable by ground-based Cherenkov telescopes. Recently this so…
▽ More
We present the analysis and results of recent high-energy gamma-ray observations of the high energy-peaked BL Lac (HBL) object 1ES 1218+304 with the Solar Tower Atmospheric Cherenkov Effect Experiment (STACEE). 1ES 1218+304 is an X-ray bright HBL at a redshift z=0.182. It has been predicted to be a gamma-ray emitter above 100 GeV, detectable by ground-based Cherenkov telescopes. Recently this source has been detected by MAGIC and VERITAS, confirming these predictions. STACEE's sensitivity to astrophysical sources at energies above 100 GeV allows it to explore high energy sources such as X-ray bright active galaxies and gamma-ray bursts. We present results from STACEE observations of 1ES 1218+304 in the 2006 and 2007 observing seasons.
△ Less
Submitted 22 October, 2007;
originally announced October 2007.
