Advancing Communication: A CNN-Powered Framework for Assamese Sign Language Recognition

Himangshu Chetia, Bidisha Bhuyan, Madhusmita Bhuyan, Chhaya Prasad, Chandana Dev

Abstract


This paper introduces an advanced method for recognizing Assamese Sign Language (ASL) using deep learning, specifically Convolutional Neural Networks (CNN). The proposed framework encompasses three primary phases: data preparation, model development, and real-time gesture recognition. A dataset of 10 static gestures representing distinct Assamese alphabets was created, utilizing OpenCV for frame capture and MediaPipe for hand landmark detection. The collected data underwent preprocessing, and the hand gesture images were further split into training and testing sets. A CNN model was then trained for gesture classification, yielding a perfect accuracy score of 100%, along with exceptional precision, recall, and F1 scores across all 10 categories. The findings demonstrate the CNN model's proficiency in extracting and learning key features of ASL gestures, enabling the system to accurately recognize and convert the signs into text in real-time. This real-time recognition ability holds significant potential for enhancing communication between the deaf and hearing communities in Assam. Future work will focus on expanding the dataset to incorporate dynamic gestures and optimizing the system for real-time deployment in mobile applications.

Keywords


Assamese Sign Language; Deep Learning; Convolutional Neural Networks; Gesture Recognition; Hand Landmark Detection

Full Text:

PDF

References


R. R. Verma, A. Konkimalla, A. Thakar, K. Sikka, A. C. Singh, and T. Khanna, “Prevalence of hearing loss in India,” National Medical Journal of India, vol. 34, no. 4, pp. 216–222, Jan. 2022, doi: 10.25259/NMJI_66_21.

V. Sharma, A. K. Gupta, A. Sharma, and S. Saini, “A unified approach for continuous sign language recognition and translation,” Int J Data Sci Anal, Apr. 2024, doi: https://doi.org/10.1007/s41060-024-00549-2.

S. Sharma and S. Singh, “A Spatio-Temporal Framework for Dynamic Indian Sign Language Recognition,” Wireless Pers Commun, vol. 132, no. 4, pp. 2527–2541, Oct. 2023, doi: 10.1007/s11277-023-10730-8.

A. Hussain, N. Saikia, and C. Dev, “Advancements in Indian Sign Language Recognition Systems: Enhancing Communication and Accessibility for the Deaf and Hearing Impaired,” Asian Journal of Electrical Sciences, vol. 12, no. 2, pp. 37–49, Dec. 2023, doi: 10.51983/ajes-2023.12.2.4132.

A. K. Sahoo, “Indian Sign Language Recognition Using Machine Learning Techniques,” Macromolecular Symposia, vol. 397, no. 1, p. 2000241, Jun. 2021, doi: 10.1002/masy.202000241.

J. Bora, S. Dehingia, A. Boruah, A. A. Chetia, and D. Gogoi, “Real-time Assamese Sign Language Recognition using MediaPipe and Deep Learning,” Procedia Computer Science, vol. 218, pp. 1384–1393, 2023, doi: 10.1016/j.procs.2023.01.117.

J. Rekha, J. Bhattacharya and S. Majumder, "Shape, texture and local movement hand gesture features for Indian Sign Language recognition," 3rd International Conference on Trendz in Information Sciences & Computing (TISC2011), Chennai, India, 2011, pp. 30-35, doi: 10.1109/TISC.2011.6169079.

S. Sharma and S. Singh, “Recognition of Indian Sign Language (ISL) Using Deep Learning Model,” Wireless Pers Commun, vol. 123, no. 1, pp. 671–692, Mar. 2022, doi: 10.1007/s11277-021-09152-1.

K. Shenoy, T. Dastane, V. Rao and D. Vyavaharkar, "Real-time Indian Sign Language (ISL) Recognition," 2018 9th International Conference on Computing, Communication and Networking Technologies (ICCCNT), Bengaluru, India, 2018, pp. 1-9, doi: 10.1109/ICCCNT.2018.8493808.

B. Sundar and T. Bagyammal, “American Sign Language Recognition for Alphabets Using MediaPipe and LSTM,” Procedia Computer Science, vol. 215, pp. 642–651, 2022, doi: 10.1016/j.procs.2022.12.066.

T.-W. Chong and B.-G. Lee, “American Sign Language Recognition Using Leap Motion Controller with Machine Learning Approach,” Sensors, vol. 18, no. 10, p. 3554, Oct. 2018, doi: 10.3390/s18103554.

A. Thongtawee, O. Pinsanoh, and Y. Kitjaidure, “A Novel Feature Extraction for American Sign Language Recognition Using Webcam,” in 2018 11th Biomedical Engineering International Conference (BMEiCON), Chiang Mai: IEEE, Nov. 2018, pp. 1–5. doi: 10.1109/BMEiCON.2018.8609933.

M. N. Huda, I. Alam and S A. Siddique, “Automated Bangla Sign Language Conversion System: Present and Future,” in Proceedings of the 2019 International Conference on Language Technologies for All (LT4All), Paris, UNESCO Headquarters, 5-6 December, 2019, pp. 61–65. Available: https://lt4all.elra.info/proceedings/lt4all2019/pdf/2019.lt4all-1.16.pdf

C. M. Sharma, K. Tomar, R. K. Mishra, and V. M. Chariar, “Indian sign language recognition using fine-tuned deep transfer learning model,” in Proceedings of the 1st International Conference on Innovation in Computer and Information Science (ICICIS 2021), Ganzhou, China: SCITEPRESS - Science and Technology Publications, 2021, pp. 63–68. doi: 10.5220/0010790300003167.

M. J. Hasan, S. K. Nahid Hasan, and K. S. Alam, “Deep Convolutional Neural Network-Based Bangla Sign Language Detection on a Novel Dataset,” in Machine Intelligence and Data Science Applications, V. Skala, T. P. Singh, T. Choudhury, R. Tomar, and Md. Abul Bashar, Eds., Singapore: Springer Nature, 2022, pp. 157–168. doi: 10.1007/978-981-19-2347-0_13

S. Das, Md. S. Imtiaz, N. H. Neom, N. Siddique and H. Wang, “A hybrid approach for Bangla sign language recognition using deep transfer learning model with random forest classifier,” Expert Systems with Applications, vol. 213, p. 118914, Mar. 2023, doi: 10.1016/j.eswa.2022.118914.

V. Sharma, A. K. Gupta, A. Sharma, and S. Saini, “A unified approach for continuous sign language recognition and translation,” Int J Data Sci Anal, Apr. 2024, doi: 10.1007/s41060-024-00549-2.

M. A. Hossen, A. Govindaiah, S. Sultana and A. Bhuiyan, "Bengali Sign Language Recognition Using Deep Convolutional Neural Network," 2018 Joint 7th International Conference on Informatics, Electronics & Vision (ICIEV) and 2018 2nd International Conference on Imaging, Vision & Pattern Recognition (icIVPR), Kitakyushu, Japan, 2018, pp. 369-373, doi: 10.1109/ICIEV.2018.8640962.




Copyright (c) 2025 Himangshu Chetia, Bidisha Bhuyan, Madhusmita Bhuyan, Chhaya Prasad, Chandana Dev

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.


Call for Paper for the upcoming issue
 
The journal welcomes publications of high-quality research papers, review papers, white papers, conference papers, etc. on theoretical developments and practical applications in the domain of Electrical and Electronics and its allied sciences.
 
Authors are solicited to contribute to the journal by submitting articles that illustrate original research works, short communications and review articles in the thrust areas of the journal as mentioned on the About page.
 
Submission and AJEEE Paper Template:
  • Download the AJEEE paper template and submit your paper at the PAPER SUBMISSION page.
 
ADBU Journal of Electrical and Electronics Engineering (AJEEE) - ISSN: 2582-0257 is an International peer-reviewed Open-Access Online journal in the English language that publishes scientific articles which contribute new novel experimentation and theoretical work in all areas of Electrical and Electronics Engineering and its applications.

* The views, interpretations and opinions expressed in the articles are those of the author(s) and should not be considered to reflect the opinions of the Editorial Board of this journal- AJEEE.

Creative Commons License