Understanding the potential of spiking neural networks

Technologies used: Python, PyTorch, Nvidia CUDA, SpeechBrain

This project explores the capabilities of Spiking Neural Networks (SNNs) in providing lightweight solutions to sequential tasks involving text and speech. We showed that SNNs, due to their intrinsic memory provided by membrane potential, can be an efficient alternative to Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) networks for sequence processing, especially for certain edge applications. 

• Recent advances in training and input encoding techniques have brought SNNs up to the same level as top-performing Artificial Neural Networks (ANNs) in the realm of image classification. Nonetheless, these tasks don't entirely leverage the unique internal dynamics that SNNs offer. 

• Using Natural Language Processing (NLP) and Automatic Speech Recognition (ASR) as applications, we demonstrated that SNNs can be designed to process sequential workloads with extraordinary efficiency, requiring 10x less parameters compared to bi-directional LSTMs counterparts, while maintaining competitive performance.

• Furthermore, we illustrated that SNNs can be seamlessly combined with ANNs to form hybrid networks. This approach harnessed advantages of both types of networks and yielded impressive performances in terms of accuracy and efficiency.

Publications:

• B. Wickramasinghe, S. Chowdhury, A. Kosta, W. Ponghiran, K. Roy, “Unlocking the Potential of Spiking Neural Networks: Understanding the What, Why, and Where,” submitted to special issue on Advancing Machine Intelligence with Neuromorphic Computing, IEEE Transactions on Cognitive and Developmental Systems (under review, 2023).

Effective algorithms for open world continual learning (on going)

Technologies used: Python, PyTorch, Nvidia CUDA

This project involves understanding and developing algorithms for continual (or lifelong) learning using deep neural networks. Continual learning (CL), or the ability to acquire, process, and learn from new information without forgetting acquired knowledge, is an essential characteristic of artificial general intelligence. However, traditional neural networks do not possess the ability to adapt to such dynamic environments.   

• As the initial step, we formulated a comprehensive analysis of the latest continual learning techniques in deep learning, providing insights into the field’s trajectory and potential.

• We underscore the importance of open world CL, where an agent is required to identify previously unseen, unlabelled data without the help of human inputs (e.g., autonomous vehicles, robots). 

• To that effect, we develop a unified framework that leverages out-of-distribution detection to identify new data and continually accommodate that knowledge, while preserving old knowledge.

Publications:

• B. Wickramasinghe, G. Saha, K. Roy, “Continual Learning: A Review of Techniques, Challenges and Future Avenues,” submitted to IEEE Transactions on Artificial Intelligence (under review, 2023).

Dynamically adaptable deep learning framework for capturing spoofing artifacts in speech

Technologies used: Python, PyTorch, Nvidia CUDA, MATLAB

The project presents a unique spoofing attack detection framework that merges concepts from the mammalian auditory system with a deep learning model to improve speech feature extraction. The aim was to create a system capable of detecting replay attacks on automatic speaker verification systems by dynamically recognizing and capturing environmental and device artifacts introduced during replay.

• A Deep Neural Network (DNN) based adaptive controller was developed for dynamic artifact capture and enhancement using a filterbank, inspired by the intricate non-linear filtering in the mammalian cochlea.

• The adaptive controller was trained to generate quality factors (Q-factors) for the filters in the filterbank, which are then used to selectively amplify filter gains when replay artifacts are encountered.

• The amplification of filter outputs adjusts acoustic feature amplitudes, resulting in acoustic features containing amplified replay information for the DNN-based classifier to exploit.

• The adaptive controller and the spoofing detection system were optimized simultaneously, eliminating the need for Q-factors.

• Extensive experiments on replay attacks of variable quality showed that the adaptive model outperforms static filter-based methods, particularly for high-quality attacks.

Link: https://github.com/buddhiW/AdaptiveNet 

Publications:

• B. Wickramasinghe, E. Ambikairajah, V. Sethu, J. Epps, H. Li, T. Dang, "DNN controlled adaptive front-end for replay attack detection systems", Speech Communication, 154, p. 102973. doi:10.1016/j.specom.2023.102973.

Adaptive Filterbanks: A bio-inspired solution for replay spoofing attacks in speaker verification 

Technologies used: MATLAB

The project integrates non-linear adaptation, which increases the dynamic range in mammalian hearing, into a traditional parallel filterbank used for spoofing attack detection. 

• A standard linear filter model amplifies both low and high energy inputs with the same filter gain, which may overlook low energy components like undetectable replay artifacts.

• The project transformed the fixed-gain parallel filter bank into a time-variable one by coupling it with a second filter bank of second-order resonators, whose gains can be adjusted using their quality factor (Q-factor).

• Adaptive filter gains were used to boost these subtle artifacts while preserving high energy components, enhancing the model's sensitivity and allowing it to handle a wider dynamic range of inputs. This dynamic coupling created a new, multifaceted framework that can adapt to varying inputs over time.

• The new framework was rigorously tested, focusing on two primary aspects: its adaptive capabilities and its compression function.

• Gaussian Mixture Model (GMM) based classifiers were used to assess the framework's ability to detect replayed signals. Comparisons with traditional frameworks showed that incorporating input-dependent adaptation within the framework significantly improved performance in spoofing detection.

Publications:

• B. Wickramasinghe, E. Ambikairajah, J. Epps. "Biologically Inspired Adaptive-Q Filterbanks for Replay Spoofing Attack Detection." In Interspeech, pp. 2953-2957. 2019.

Leveraging spatial differentiation from auditory modeling to detect spoofing attacks

Technologies used: MATLAB

The project emphasizes the benefits of spatial differentiation in enhancing replay attack detection. Spatial differentiation is a technique known to boost auditory response sensitivity by mimicking fluid coupling in the mammalian cochlea's basilar membrane.

• Spatial differentiation was integrated into a parallel filterbank used for frequency band decomposition in replay attack detection.

• Detailed analyses of filter transfer functions and pole-zero diagrams confirmed the effectiveness of spatial differentiation in enhancing the transition bands of the filters in the filterbank.

• Feature domain analyses further confirmed that additional sharpening of filters led to a significant reduction in correlation between feature coefficients.

• Gaussian Mixture Model based experiments on replay attack detection, conducted both with and without spatial differentiation, showed that feature de-correlation, achieved through spatial differentiation, significantly improved detection performance.

• Spatial differentiation is a general method that can be used to enhance feature extraction in any speech-based application, as it does not rely on domain knowledge about the task.

Publications:

• B. Wickramasinghe, E. Ambikairajah, J. Epps, V. Sethu, H. Li, “Auditory Inspired Spatial Differentiation for Replay Spoofing Attack Detection,” in ICASSP 2019, pp. 6011-6015, 2019.

Speech demodulation techniques for identifying reverberation and noise in replayed speech 

Technologies used: MATLAB, Python, TensorFlow

The project conducted an in-depth exploration of Amplitude Modulation-Frequency Modulation (AM-FM) modeling approaches, revealing their potent application in enhancing the detection of replay attacks. 

• Frequency Domain Linear Prediction (FDLP) was examined as a method to capture the temporal envelope, which carries the AM details of resonances of a speech signal.

• Visualizations demonstrated that distortions in the temporal envelope reflect the impacts of recording devices, reverberation, and environmental noise on replayed signals.

• Multiple features were extracted from the temporal envelope and the FDLP residual and tested for their effectiveness in detecting replay attacks using DNN and GMM classifiers, demonstrating a significant ability to distinguish between genuine and replayed signals.

• Furthermore, we implemented an innovative technique that leverages the spectral centroid of a signal to compute its sub-band FM component, demonstrating competitive performance to that of traditional magnitude-based features .

• The fusion of AM and FM-derived features indicated that both AM and FM provide complementary information helpful in distinguishing genuine signals from replayed ones.

Publications:

• B. Wickramasinghe, S. Irtza, E. Ambikairajah, J. Epps, "Frequency Domain Linear Prediction Features for Replay Spoofing Attack Detection," Proc. Interspeech 2018, pp. 661-665, 2018.

• T. Gunendradasan, B. Wickramasinghe, P. N. Le, E. Ambikairajah, and J. Epps, "Detection of Replay-Spoofing Attacks Using Frequency Modulation Features," Proc. Interspeech 2018, pp. 636-640, 2018.

• Buddhi Wickramasinghe, Phu Ngoc Le, Eliathamby Ambikairajah, and Julien Epps, “Perceptual Frequency Scale-based Filters for Replay Spoofing Attack Detection,” 2018 IEEE International Conference on Information and Automation for Sustainability (ICIAFS), Colombo, Sri Lanka, 2018.

Mutual Information Analysis based power analysis attacks 

Technologies used: C, Nvidia CUDA, MATLAB

Side channel attack is a type of security attack on a cryptographic system where an attacker gains information from the physical implementation of the system, rather than the software or cryptographic vulnerabilities. Power analysis attack is a form of side channel attack that exploits the power consumed by a device when performing cryptographic operations. Mutual Information Analysis (MIA) based power analysis attack is a type of power analysis attacks. The primary goal of the project was to develop an efficient implementation of the MIA algorithm on a cryptographic device using CUDA C, demonstrating their practicality in real-world situations. We were able to develop a fast implementation which takes around 15 minutes using POSIX threads on CPU and around 2 minutes using CUDA C on GPU.  

Publications:

• Malin Prematilake, Buddhi Wickramasinghe, Olitha Vithanage, Hasindu Gamaarachchi, Roshan Ragel, “Accelerating Mutual Information Analysis based Power Analysis Attacks.” 2016 IEEE International Conference on Information and Automation for Sustainability, Galle, Sri Lanka, 2016.

Personalized Notice Board  

Technologies used: Python, OpenCV, Raspberry Pi

The primary goal of this project was to design and implement a unique personalized notice board system. This innovative system combines facial recognition techniques and customized content delivery to provide users with a tailored information experience. When a user stands in front of the notice board, the system uses facial recognition to identify the user and displays only notices that are relevant to them. Software framework of the project is based on Python and OpenCV. Project was built using Raspberry Pi2 development board. 

Web based simulator for ARM instruction architecture

Technologies used: HTML, PHP, and JavaScript

This project comprises a web-based ARM instruction set architecture simulator with a suite of functionality that includes the ability to upload and modify code, compile it, and simulate its execution. The simulator provides users with an in-depth view of memory operations and register performance during execution, and it also offers standard error messages. 

MIPS processor simulator using Verilog 

Technologies used: Verilog

An instruction Set Architecture (ISA) acts as an interface that defines the hardware operations which are available to the software. MIPS is an ISA which supports different types of instructions, namely R-type, load and store, conditional branch, etc.

This tool implements a basic MIPS based multi-cycle processor and controller using the Hardware Definition Language, Verilog. Different hardware components included in the data path of the processor have been implemented in dedicated modules which can be easily interconnected to form the complete data path. Using this implementation, one can easily get an understanding on how different components in the data path works with the help of testing modules.