Introduction

Speech enhancement aims to extract desired speech signals degraded by noise and interference. It serves as a crucial frontend module in applications, including human-machine interaction, including human-machine interaction, video conferencing, and hearing aids.With the rapid advancement of deep neural networks (DNNs), data driven speech enhancement systems have demonstrated remarkable performance, surpassing traditional rule-based signal processing methods.

In low-SNR conditions, speech enhancement techniques often fail to retain speech components. One approach to address this issue is the two-stage strategy. However, the high computational demands of neural networks make them unsuitable for deployment on edge devices for real-time applications.

DNN-based multi-channel speech enhancement approaches have made great progress in recent years, but the number of microphones required for these methods is often too large to enable practical deployment.

Recent works have focused on developing lightweight models that maintain competitive performance with reduced computational resources. For example, RNNoise, GTCRN, DPCRN. However, their speech enhancement performance in high noise level scenarios remains questionable.

Independent vector analysis (IVA) is a traditional blind source separation method, While DNN-based methods often outperform IVA, the latter remains effective in separating speech and noise even in low-SNR conditions

In this blog, we propose a lightweight hybrid dual-channel speech enhancement system that combines IVA with a modified dual-channel GTCRN to achieve effective speech enhancement in low-SNR conditions. IVA is employed as a coarse estimator to obtain preliminary separated speech and noise signals, subsequently, our modified GTCRN refines these estimates, effectively integrating both the original noisy mixture and the separated source information provided by IVA to further enhance speech quality.

Proposed Method

Figure 1. The framework of our proposed system

As depicted in Figure 1, our proposed system is built upon the GTCRN architecture. We introduce the dual-channel version of the model and incorporate several modifications to ensure the comprehensive utilization of diverse information. These modifications include feature selection, auxiliary information selection, masking approach selection, and the adoption of a dual encoder structure.

The system consists of several key components:

Figure 2. Feature selection module

Figure 3. Subband feature extraction module

Figure 4. Encoder module

Figure 5. Grouped temporal convolution block

Loss Function: Our model is trained on the hybrid loss function, which consists of scale-invariant signal-to-noise ratio (SISNR) loss and complex compressed mean-squared error (ccMSE) loss.

Experiments

Dataset: We generate the simulated dataset with the image method ,with dual-channel RIRs based on a linear array with two microphones placed 4 cm apart. The room size ranges from 3m×3m×2.5m to 10m×10m×3 m, and the reverberation time (RT60) ranges from 0.1 s to 0.4 s. The distance from the source to the array is randomly selected from {0.5 m, 1 m, 2 m, 3m}, with the direction of arrival (DOA) difference between the target speech and interference noise being greater than 5 °. We convolve the speech dataset from the DNS-3 challenge with these dual-channel RIRs to generate the simulated speech signals. The early reflection (50 ms) of the first channel is preserved as the training target. For the noise signals, we select data from the DNS-3 and DCASE datasets. During training, the SNR ranges from −10 dB to 0 dB. The validation set maintains this range of SNR, with 500 noisy-clean pairs in each case. For the test set, we set three SNR levels: −12.5 dB,−7.5 dB, and −2.5 dB, each with 500 noisy-clean pairs. All utterances are sampled at 16 kHz.

Evaluation metrics:
The evaluation is conducted using the objective metrics, including perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility (STOI). Additionally, DNN-based non-intrusive subjective metrics DNSMOS P.808 and DNSMOS P.835 are also employed.

Ablation study:
We conduct an ablation study on our modified GTCRN to evaluate the impact of various factors, including the use of speech and noise information from IVA, the type of feature, the type of masking approach, and the adoption of the dual-encoder, as shown in Table 1.
As seen in IDs 1 and 2, IDs 3 and 5, and IDs 4 and 6. It is clear that Masking 2 (IDs 2, 5, 6) outperforms Masking 1 (IDs 1, 3, 4), which can be attributed to the coarse estimation and speech preservation provided by IVA.
A comparison between ID-1 and ID-3 highlights the effectiveness of the LPS feature, which improves all metrics, indicating that phase information is not as crucial.
Compared to ID-5, the inclusion of the noise information provided by IVA (ID-6) leads to substantial improvements in nearly all metrics, despite a slight degradation in SIG is observed. This is because the two separated signals allow for a more comprehensive capture of the noisy mixture, while the speech channel focuses solely on the speech component.
Finally, the results for ID-7 exhibit noticeable declines across most metrics, despite a marginal improvement in STOI over ID-6. We attribute this to the computational limitations of our lightweight network, which restrict the ability to fully leverage the potential benefits of the dual-encoder framework.
Consequently, we select the best-performing method in the table (ID-6) for comparison with the baseline models.

Table 1. Results of the ablation study on the simulated validation set. BOLD indicates the best score in each metric

Results comparison with baselines:
Four methods are selected as baselines: (a) Aux-IVA, (b)GTCRN, (c) DC-GTCRN, a dual-channel version of GTCRN,and (d) DC-GTCRN-L, a larger-scale version of DC-GTCRN. The results of the simulated test set are presented in Table 2. By integrating Aux-IVA, our proposed method attains the highest scores, further validating the efficacy of auxiliary information and underscoring the superiority of our hybrid approach. The parameters and computational loads reported in the table include the Aux-IVA module, which contributes a negligible increase in parameters and only 0.20 MMACs per second per iteration.

Table 2. Results comparison with baselines on the simulated test set. BOLD indicates the best score in each metric

A set of typical audio samples is presented in Figure 6, clearly demonstrating that our proposed method excels in both speech preservation and noise suppression. Audio samples are available: https://github.com/Max1Wz/H-GTCRN.

Figure 6. Typical spectrograms

Conclusion

We propose a hybrid dual-channel speech enhancement system designed for low-SNR conditions, integrating IVA and a modified GTCRN. Aux-IVA acts as a coarse estimator, providing auxiliary information, while the GTCRN further refines the speech quality. Through various architecture modifications, both the original and auxiliary information are fully leveraged. With only a minimal increase in parameters and computational complexity, the proposed system effectively enhances speech. Experimental results validate its effectiveness.

View More

Close