DBS-NET: A Dual-Branch Network Integrating Supervised and Contrastive Self-Supervised Learning for Birdsong Classification
Source abstract
Abstract
Birdsong classification plays a crucial role in monitoring species distribution, population structure, and environmental changes. Existing methods typically use supervised learning to extract specific features for classification, but this may limit the generalization ability of the model and lead to generalization errors. Unsupervised feature extraction methods are an emerging approach that offers enhanced adaptability, particularly for handling unlabeled and diverse birdsong data. However, their drawback may bring additional time cost to downstream tasks, which may impact overall efficiency. To address these challenges, we propose DBS-NET, a Dual-Branch Network Model for birdsong classification. DBS-NET consists of two branches: a supervised branch (Res-iDAFF) and an unsupervised branch (based on a contrastive learning approach). We introduce an iterative dual-attention feature fusion (iDAFF) module in the backbone to enhance contextual feature extraction, and a linear residual classifier is exploited further improve classification accuracy. Additionally, to address class imbalance in the dataset, a weighted loss function is introduced to adjust the cross-entropy loss with optimized class weights. To improve training efficiency, the backbone networks of both branches share a portion of their weights, reducing the computational overhead. In the experiments on a self-built 30-class dataset and the Birdsdata dataset, the proposed method achieved accuracies of 97.54% and 97.09%, respectively, outperforming other supervised and unsupervised birdsong classification methods.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Animal Vocal Communication and Behavior · Amphibian and Reptile Biology · Species Distribution and Climate Change
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Big Data and Intelligent Engineering, Southwest Forestry University, Kunming 650224, China
Funding
National Natural Science Foundation of China
References (41)
Show source-provided references
1. (2024, May 20). IUCN Red List. The IUCN Red List of Threatened Species. Di sponí vel em. Aces so em. 2004; Volume 12. Available online: https://www.iucnredlist.org/.
2. Deep learning bird song recognition based on MFF-ScSEnet10.1016/j.ecolind.2023.110844 ↗
3. Sastry, S., Khanal, S., Dhakal, A., Huang, D., and Jacobs, N. (2023). BirdSAT: Cross-View Contrastive Masked Autoencoders for Bird Species Classification and Mapping. arXiv.10.1109/WACV57701.2024.00698 ↗
4. Birdsong classification based on multi-feature fusion10.1007/s11042-021-11396-9 ↗
5. Handcrafted features and late fusion with deep learning for bird sound classification10.1016/j.ecoinf.2019.05.007 ↗
6. (2023). A hierarchical birdsong feature extraction architecture combining static and dynamic modeling. Ecol. Indic., 150, 110258.10.1016/j.ecolind.2023.110258 ↗
7. Classification of birdsong spectrograms based on DR-ACGAN and dynamic convolution10.1016/j.ecoinf.2023.102250 ↗
8. BirdNET: A deep learning solution for avian diversity monitoring10.1016/j.ecoinf.2021.101236 ↗
9. Self-supervised learning: Generative or contrastive
10. Automatic large-scale classification of bird sounds is strongly improved by unsupervised feature learning10.7717/peerj.488 ↗
11. Acoustic animal identification using unsupervised learning10.1111/2041-210X.14103 ↗
12. Unsupervised classification to improve the quality of a bird song recording dataset10.1016/j.ecoinf.2022.101952 ↗
13. Boosting hyperspectral image classification with unsupervised feature learning
14. Bootstrap your own latent-a new approach to self-supervised learning
15. Zbontar, J., Jing, L., Misra, I., LeCun, Y., and Deny, S. (2021, January 18–24). Barlow twins: Self-supervised learning via redundancy reduction. Proceedings of the International Conference on Machine Learning, PMLR, Virtual.
16. He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R. (2020, January 13–19). Momentum contrast for unsupervised visual representation learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00975 ↗
17. Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. (2020, January 13–18). A simple framework for contrastive learning of visual representations. Proceedings of the International Conference on Machine Learning, PMLR, Virtual.
18. Chen, X., and He, K. (2021, January 20–25). Exploring simple siamese representation learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01549 ↗
19. Unsupervised learning of visual features by contrasting cluster assignments
20. Wang, T., and Isola, P. (2020, January 13–18). Understanding contrastive representation learning through alignment and uniformity on the hypersphere. Proceedings of the International Conference on Machine Learning, PMLR, Virtual.
21. Li, J., Zhou, P., Xiong, C., and Hoi, S.C. (2020). Prototypical contrastive learning of unsupervised representations. arXiv.
22. Wu, Z., Xiong, Y., Yu, S.X., and Lin, D. (2018, January 18–23). Unsupervised feature learning via non-parametric instance discrimination. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00393 ↗
23. Oord, A.V.D., Li, Y., and Vinyals, O. (2018). Representation learning with contrastive predictive coding. arXiv.
24. Self-organizing neural network that discovers surfaces in random-dot stereograms10.1038/355161a0 ↗
25. Hadsell, R., Chopra, S., and LeCun, Y. (2006, January 17–22). Dimensionality reduction by learning an invariant mapping. Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), New York, NY, USA.
26. Chen, X., Fan, H., Girshick, R., and He, K. (2020). Improved baselines with momentum contrastive learning. arXiv.
27. Li, C., Li, F., Zhang, L., Yang, A., Hu, Z., and He, M. (2023). Intrusion Detection for Industrial Control Systems Based on Improved Contrastive Learning SimCLR. Appl. Sci., 13.10.3390/app13169227 ↗
28. Self-supervised learning advanced plant disease image classification with SimCLR10.1007/s43674-023-00065-z ↗
29. Shi, Z., Chen, H., Liu, C., and Qiu, J. (2024, May 20). Metaformer Model with ArcFaceLoss and Contrastive Learning for SnakeCLEF2023 Fine-Grained Classification. CLEF (Working Notes). 2023; pp. 2137–2148. Available online: https://api.semanticscholar.org/CorpusID:264441643.
30. Automated Rib Fracture Detection on Chest X-Ray Using Contrastive Learning10.1007/s10278-023-00868-z ↗
31. Big self-supervised models are strong semi-supervised learners
32. Fixmatch: Simplifying semi-supervised learning with consistency and confidence
33. One-step progressive representation transfer learning for bird sound classification10.1016/j.apacoust.2023.109614 ↗
34. SILIC: A cross database framework for automatically extracting robust biodiversity information from soundscape recordings based on object detection and a tiny training dataset10.1016/j.ecoinf.2021.101534 ↗
35. Höchst, J., Bellafkir, H., Lampe, P., Vogelbacher, M., Mühling, M., Schneider, D., Lindner, K., Rösner, S., Schabo, D.G., and Farwig, N. (2022, January 17–19). Bird@ Edge: Bird Species Recognition at the Edge. Proceedings of the Networked Systems: 10th International Conference, NETYS 2022, Virtual Event.10.1007/978-3-031-17436-0_6 ↗
36. Xie, J., Towsey, M., Eichinski, P., Zhang, J., and Roe, P. (September, January 31). Acoustic feature extraction using perceptual wavelet packet decomposition for frog call classification. Proceedings of the 2015 IEEE 11th International Conference on e-Science, Munich, Germany.10.1109/eScience.2015.47 ↗
37. Priyadarshani, N., Marsland, S., Castro, I., and Punchihewa, A. (2016). Birdsong denoising using wavelets. PLoS ONE, 11.10.1371/journal.pone.0146790 ↗
38. Dai, Y., Gieseke, F., Oehmcke, S., Wu, Y., and Barnard, K. (2021, January 3–8). Attentional feature fusion. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.10.1109/WACV48630.2021.00360 ↗
39. Linderman, G.C., Rachh, M., Hoskins, J.G., Steinerberger, S., and Kluger, Y. (2017). Efficient algorithms for t-distributed stochastic neighborhood embedding. arXiv.
40. SDFIE-NET–A self-learning dual-feature fusion information capture expression method for birdsong recognition10.1016/j.apacoust.2024.110004 ↗
41. Kendall, A., Gal, Y., and Cipolla, R. (2018, January 18–23). Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.