Published Paper


Design of an Iterative Validation Model for English to Chhattisgarhi Translation Using HMF-NMT SAPT and AER-Net in Multimodal Contexts

1 Mr. Rohan B. Kokate; 2 Dr. Anupa Sinha; 3 Dr. Shrikant V. Sonekar; 4 Mr. Umesh Samarth
India
Page: 1836-1853
Published on: 2025 December

Abstract

Most notable after NMT systems dealing with under-resourced languages like Chhattisgarhi are found ever demanding in function with audiovisual implementations concerning the language translations. Existing NMT models for English to Chhattisgarhi translation are predominantly unimodal and unable to maintain visual/cultural content context incapable of semantic, morphological, or idiomatic accuracy. As a result, the algorithms highly affect translations by viewing multimedia stimuli carrying complicated semantics over a cultural and temporal interface. This paper proposes an all-encompassing Multimodal Translation and Adaptive Tuning Framework (MM-TAT) that exploits the latest neural architectures, by exploiting text, images, and video inputs, to significantly improve translation quality. The system will comprise five novel modules: (HMF-NMT) Hierarchical Multimodal Fusion NMT, a module that aligns semantic features across text, images, and videos using cross-attention methods; TVA, a Temporal Visual Attention mechanism that aligns video-derived features with sentence-level semantics to ensure tense and aspect consistency; SAPT, a Syntax-Aware POS-Tagging Transformer that integrates grammatical constraints via dual-head attention and a syntactic transition matrix. AER-Net- Adaptive Error Recovery Network: utilizes transformer-based post-editing through human correction feedback; SCG-Net- Semiotic Concept Graph that injects cultural and idiomatic context through knowledge graph embeddings. The aforementioned modules together yield an improvement of 18.4 BLEU to 33.6, a TER low of 23.1, and 92.8% post-editing correction rate. Considering cultural phrases in English returns an accuracy of 84.7%, showcasing the model's ability to maintain socio-linguistic fidelity. This work sets a new benchmark for multimodal, real-world English-to-Chhattisgarhi translations in process.

 

PDF