Research on machine learning-based classification of EM signals has been actively conducted across various application domains [7, 9, 10]. For example, in [7], a traditional machine learning approach—using k-nearest neighbors to classify electronic devices based on harmonic radar responses—was employed, with feature extraction performed using FFT. This approach highlights the potential of harmonic radar-based EM analysis for the classification of electronic devices.

A 22-layer deep residual network for integrated circuit device identification and verification was introduced in [9] by leveraging raw EM signals captured using a near-field probe and an oscilloscope. The time-domain signals were reshaped into two-dimensional (2D) arrays for CNN processing, enabling automatic feature extraction. This approach demonstrated strong noise robustness and high classification accuracy, reinforcing the efficacy of deep learning in EM leakage signal analysis. Additionally, in [10], the researchers proposed a deep learning-based classification method based on the use of radar-based EM signals to distinguish between different objects, such as vehicles, pedestrians, and drones. This approach involved the range-Doppler map representation of radar signals, which were then processed using a CNN-based model. The results demonstrated the effectiveness of deep learning in handling active EM signals for object classification.

The findings from these studies not only demonstrate the feasibility of classification using EM signals but also highlight the importance of selecting appropriate models and feature extraction methods tailored to the characteristics of the dataset. However, despite these advancements in EM signal-based classification, a comprehensive review of previous studies indicates that to the best of our knowledge, no prior research has investigated vehicle classification based on EM leakage signals, nor has any study employed an approach that uses the audio signal features adopted in our experiments.

FFT is a foundational algorithm extensively used to transform signals from the time domain to the frequency domain. However, despite its computational efficiency, FFT operates under the assumption that the signal is stationary within the analysis window, thus limiting the ability of FFT to capture frequency variations over time. STFT addresses this limitation by segmenting the signal into short time frames, enabling localized frequency analysis. However, the fixed time resolution of STFT restricts its ability to adapt to varying frequency dynamics.

MEL is a time–frequency representation designed to align with human auditory perception that is widely employed in speech and music processing [18]. By applying the Mel scale to the frequency domain, the nonlinear perception of sound frequency can be easily captured. Further advancements, such as MFCC [19], have enabled the extraction of compact feature representations from the Mel-frequency scale, which have proven effective in machine learning applications, including speech recognition.

Overall, traditional feature extraction methods, such as FFT and STFT, are widely adopted for general signal analysis, while MEL and MFCC demonstrate superior performance as feature extraction methods in audio classification tasks. In this study, we extended the application of these audio signal processing methods to classify vehicles based on simulated EM leakage signals.

SpecAugment [20] is a widely used data augmentation technique in the field of speech signal processing. It masks 2D input data, such as spectrograms, in both the time and frequency domains to increase the diversity of training data. This approach has proven effective in improving the generalization performance of models and in preventing overfitting. SpecAugment can be applied to 2D features—MEL, STFT, and MFCC—by masking regions along both the time and frequency axes. Meanwhile, FFT, which is a one-dimensional (1D) frequency-domain feature, can be applied as a zero-masking operation to randomly selected frequency ranges.

Classification of EM signals is commonly performed using machine learning-based methods [21, 22]. In this study, we focused on two widely used models: random forest (RF) [23] and the light gradient boosting model (LGBM) [24].

RF [23] is an ensemble learning method that involves randomly selecting a feature from an entire set of features to create a single decision tree. This process is repeated several times until various decision trees are learned, after which the most frequent prediction is determined as the final prediction. Such an algorithm ensures diversity and helps avoid overfitting.

LGBM [24] is a gradient boosting model that uses tree-based learning algorithms to build powerful machine learning models. It follows a leaf-wise tree growth strategy instead of the traditional level-wise tree growth strategy. The key idea of LGBM lies in adopting new techniques, such as gradient-based one-side sampling and exclusive feature bundling, that can further improve training speed and reduce memory usage.

Residual connection [25] is a technique that addresses the vanishing gradient problem in neural networks and enables the effective propagation of low-level features from the early layers to the later layers. This approach operates by bypassing one or more layers through skip connections, thus adding the input of a layer directly to its output. By enabling a smoother gradient flow, residual connections enhance training stability and preserve the low-level features extracted in the initial layers. Originally proposed in the ResNet [25] architecture, this technique has been widely utilized in time series signal processing and related fields.

CNN [26] is a deep learning algorithm primarily employed for image recognition and pattern identification. This network employs convolutional layers to learn spatial patterns and features from the input data while pooling the layers to reduce spatial resolution and extract high-level features, thereby enhancing computational efficiency. This approach is effective for extracting diverse features from input data. CNNs have found application not only in computer vision [25–30] but also in various other domains, such as speech [31] and natural language processing [32, 33].

In this study, we conducted a comparative analysis of previously proposed CNN-based models to evaluate the effectiveness of our baseline models. The selected models include widely used architectures in image classification, such as VGG16 [28], InceptionV3 [29], ResNet50 [25], and MobileNetV3 [30], as well as Conformer [31], which has demonstrated strong performance in speech recognition tasks. VGG16 [28], published in 2014, achieved state-of-the-art (SOTA) performance on the ImageNet dataset at the time of its introduction. It is well known for its deep architecture and simplicity. InceptionV3, proposed in 2015, also achieved SOTA performance on ImageNet [34] by further improving classification accuracy and efficiency with its factorized convolutional structure. ResNet50 [25], introduced in the same year, set a new benchmark in ImageNet classification with its deep residual learning framework and remains widely used in deep learning applications. Meanwhile, MobileNetV3 [29], introduced in 2019, was optimized for mobile and edge devices, thereby prioritizing efficiency over accuracy, and thus did not achieve SOTA on ImageNet. Furthermore, Conformer [31], announced in 2020, established a new benchmark in speech recognition tasks, achieving SOTA performance on the LibriSpeech [35] dataset.

For this experiment, we focused on four key features: FFT, STFT, MFCC, and MEL. FFT and STFT were extracted from the EM leakage signals using the numpy (https://numpy.org) and librosa (https://librosa.org) tools. Notably, since the outputs of FFT and STFT were complex-valued, we extracted the magnitude spectra by taking the absolute values to convert the complex outputs into real-valued features. The MEL and MFCC were extracted from the EM leakage signals using the librosa tool. MEL spectrograms were computed based on the magnitude spectra obtained from STFT, while MFCCs were derived from the logarithmic MEL spectrograms using discrete cosine transform (DCT), thus ensuring that the values of both MEL and MFCC were real as well. Fig. 3 depicts visualizations of the FFT, STFT, MFCC, and MEL features acquired from the signal simulated for the DC converter in a Truck. Fig. 4 presents the MEL features extracted from the three single classes and six multi-class combinations, representing the nine classes considered in the experiments.

To prepare the data for the machine learning models, 128 × 14 MEL, MFCC, and STFT features were converted into 1D vectors of size 1,792, since the models require 1D input vectors. FFT features, on the other hand, were used in their original form as 1D vectors of size 2,100. For the deep learning models, we retained the original 128 × 14 size for the MEL, MFCC, and STFT features, while using the same 2,100-sized 1D vectors for FFT.

Additionally, we conducted experiments by fusing two or three features that demonstrated better performance, using them as inputs for the models. Each feature extraction technique was applied to the signal, following which all features were obtained with a uniform dimension of 128 × 14. Subsequently, feature fusion was performed by concatenating the features along the vertical axis. As an example, concatenation in the case of 3-feature fusion can be expressed as follows:

We employed two machine learning models—RF [4] and LGBM [5]. For RF, the following hyperparameters were considered: 150 decision trees, a random state of 42, a maximum depth of 12, and an entropy-based impurity criterion. Meanwhile, the LGBM classifier was trained based on the following hyperparameters: 100 estimators, 10 leaves, a maximum depth of 12, a learning rate of 0.01, and logloss as the evaluation metric.

For the experiments, we employed two CNN-based models: vanilla CNN and residual CNN. As depicted in Fig. 5, the architecture of a vanilla CNN model comprises three convolutional blocks, three fully connected (FC) layers, and a final softmax layer, thus forming the baseline architecture. Each convolutional block consists of two convolutional layers with 3 × 3 kernels, ReLU activation, and batch normalization to stabilize the training process and improve convergence. The convolutional layers are followed by max pooling layers of size 2 × 2, responsible for reducing spatial dimensions while retaining the essential features. The filter sizes in these convolutional blocks increase progressively from 32 to 64 and 128, enabling the network to capture complex features as it deepens. Following the convolutional layers, the feature maps are flattened and passed through the three FC layers with 256, 128, and 64 units, respectively. At each FC layer, ReLU activation is implemented to extract high-level representations. To prevent overfitting, dropout layers at a rate of 0.2 were inserted between the FC layers. The final output layer, implemented with softmax activation and consisting of 9 units, provides the class probabilities for the classification task. Notably, the same architecture was employed in experiments using FFT features, but with the 2D CNN layers replaced with 1D CNN layers, since FFT features are 1D vectors.

In addition to the vanilla CNN model described above, we experimented with a model incorporating residual connections. These connections were applied between the three convolutional blocks, enabling the model to simultaneously learn both raw and transformed feature representations. The residual CNN architecture is illustrated in Fig. 6.

Hyperparameters are key factors that affect the performance of deep learning models. Depending on their configurations, memory capacity, data learning time, and learning outcomes can change significantly. In this experiment, we were able to achieve improved performance by adjusting the hyperparameters, such as the learning rate, batch size, and epochs. The hyperparameter settings for this study are presented in Table 1.

For the model performance evaluation, we estimated the accuracy and micro-average of the precision, recall, F1-score by using a confusion matrix, as depicted in Table 2.

In the above matrix, true positive (TP) indicates a correctly predicted event, false positive (FP) refers to an incorrectly predicted event, true negative (TN) denotes a correctly predicted non-event, and false negative (FN) signifies an incorrectly predicted non-event. The accuracy, precision, recall, and F1-score are defined as follows:

To conduct the experiments, we employed a workstation equipped with an Intel Xeon Silver 4110 CPU featuring 16 cores and 32 threads running at 2.10 GHz with 64 GB RAM, along with five NVIDIA Tesla V100-PCIE GPUs, each with 32 GB of memory, amounting to a total of 160 GB of GPU memory.

Table 3 summarizes the baseline performance of the RF, LGBM, and CNN models on the test dataset trained using features extracted from EM leakage signals. The experiments were conducted using 10-fold cross-validation, with each experiment repeated 10 times. The results are presented in terms of mean and standard deviation.

Among the single features, STFT consistently achieved the highest accuracy across all models, reaching an accuracy of 92.85% ± 0.59% in the case of CNN. MEL also performed strongly, particularly with CNN, achieving an accuracy of 92.70% ± 0.52%. In contrast, FFT exhibited the lowest performance with RF and LGBM, attaining accuracies of 62.05% ± 0.43% and 68.60% ± 0.74%, respectively. However, in the case of CNN, FFT outperformed MFCC, achieving an accuracy of 91.23% ± 1.64%, compared to MFCC’s 87.37% ± 0.44%. These results highlight the significance of the features and demonstrate their varying performance across different models, where CNN is able to leverage FFT more effectively than MFCC.

For feature fusion, the 2-feature combination (STFT + MEL) consistently outperformed other combinations, with CNN achieving the highest accuracy of 92.90% ± 0.56%. In contrast, the 3-feature combination (STFT + MEL + MFCC) exhibited a slightly lower accuracy with CNN (90.56% ± 0.90%), thereby confirming that MFCC is not an appropriate feature for detecting EM leakage signals, as also demonstrated in the single feature experiment.

Overall, vanilla CNN outperformed RF and LGBM across all features, with both machine learning models exhibiting more sensitivity to feature selection. Moreover, FFT attained the lowest performance. These findings emphasize the importance of matching feature extraction methods to the learning model to optimize classification performance.

Table 4 presents the results of vehicle classification using the vanilla CNN and residual CNN models, while also comparing the classification accuracy with and without the application of SpecAugment across various features and feature combinations. When using SpecAugment, both models displayed a tendency toward improved accuracy across features. The most significant improvement was observed for the 2-feature combination in the vanilla CNN model, which achieved the highest improvement of 93.77% ± 0.70%, compared to 92.90% ± 0.56% without augmentation. Furthermore, among the single features, MFCC showed only minor improvements over its counterparts. These findings underscore the effectiveness of SpecAugment in enhancing the generalization ability of CNN-based models to generalize, particularly for high-performing features, such as STFT and MEL.

To examine the effect of residual connections, a comparison between the residual CNN and vanilla CNN was conducted, revealing that the application of residual connections generally leads to performance improvements in most cases, as evident from Table 4. Among the combinations, the 2-feature combination demonstrated the most notable improvement, achieving an increase in accuracy from 92.90% ± 0.56% to 93.40% ± 0.53%. Meanwhile, only a slight improvement was achieved for MEL, with its accuracy rising from 92.70% ± 0.52% to 92.93% ± 0.35%. In contrast, FFT experienced a decrease in performance, with its accuracy declining from 91.23% ± 1.64% to 90.62% ± 2.20%. These results imply that residual connections serve to enhance classification accuracy across various features, particularly for combinations such as STFT and MEL. While their impact on certain single features, such as FFT, may be less pronounced, residual connections can still be considered an effective method for optimizing performance through enhanced feature interactions.

Overall, the experimental results of the CNN model, as shown in Table 4, establish that applying both residual connections and SpecAugment yields better performance than the other cases across all features and feature combinations, except for the FFT feature.

Table 5 presents the precision, recall, and F1-score for each class when using the best-performing residual CNN model. The results highlight the model’s effectiveness in classification, with the highest performance achieved for the multi-class of SUV and CC, yielding a precision of 99.15%, recall of 99.57%, and F1-score of 99.36%. It is evident that the model tends to demonstrate strong performance for the classes containing SUV but experiences difficulty in accurately classifying the classes that include Truck. Overall, the average precision, recall, and F1-score across all classes were 94.79%, 94.41%, and 94.03%, respectively.

Fig. 7(a) presents the training and validation results for accuracy obtained from one of the 10 experiments conducted using the residual CNN model with 2-feature fusion and SpecAugment— the combination that demonstrated the best performance in Table 3—while the training and validation losses are presented in Fig. 7(b). It is observed that the accuracy and loss for the training dataset converge to 1 and 0, respectively, while both values for the validation dataset begin to oscillate after approximately 10 epochs. This indicates that the model is overfitting to the training dataset. Consequently, we chose the model with the highest validation accuracy for the final evaluation.

To evaluate the effectiveness of the proposed model, we trained and tested the various CNN-based deep learning models introduced in Section IV using the same SEML-V dataset employed in our experiments. For each model, the input features were interpolated to match the specific input requirements of the respective architectures. Table 6 presents the classification accuracies of VGG16, InceptionV3, ResNet50, MobileNetV3, and Conformer, along with the best performing residual CNN model. The residual CNN achieved the highest accuracy of 93.40%, demonstrating its effectiveness in feature extraction and classification. Among the other models, VGG16 achieved the best performance. In contrast, Conformer attained the lowest accuracy of 78.25%, which may be attributed to its complex architecture combining convolutional modules, feed-forward networks, and self-attention mechanisms.

Unlike conventional CNNs, such hybrid structures typically require both large-scale datasets and pre-training to converge effectively. On the contrary, models such as VGG16, InceptionV3, ResNet50, and MobileNetV3 generally showed higher accuracy when fine-tuned using pretrained weights from ImageNet, suggesting that fine-tuning enables more effective adaptation by leveraging the generalizable features learned from large-scale datasets. However, even without using pretrained weights, the residual CNN model achieved the highest accuracy among all models, be it the ones trained from scratch or the ones fine-tuned using pretrained weights. This result underscores the importance of selecting a model architecture that is well suited to the characteristics of the concerned dataset.

The strong performance of the residual CNN is attributed to its use of shortcut connections, which alleviates the degradation problem in deep networks by preserving gradient flow and enabling stable optimization. This structure also facilitates the preservation of low-level features across layers, allowing the model to effectively learn hierarchical representations, which contributes to improved classification performance. For these reasons, the residual CNN model demonstrates superior generalization compared to conventional CNN architectures. Overall, these findings indicate that robust feature preservation and stable training are essential for the high-performance classification of the SEML-V dataset.