Multispeaker Urdu speech synthesis with limited data neural training
Abstract
Traditional text-to-speech (TTS) systems have relied on concatenative or parametric approaches, which require a labor-intensive, complex rule-writing process involving feature engineering. These limitations compromise the naturalness of the voices produced and hinder scaling to new voices or languages. Modern end-to-end neural TTS systems have brought the technology to a new state of the art, generating more expressive, natural speech with greater flexibility. They often face slow inference, low reliability, and offer minimal control over speech characteristics. A complete feed-forward topology with a clear separation between linguistic encoding, acoustic modelling, and waveform synthesis across different modules is one of the proposals we can make to address this deficiency. Our proposed system features a phoneme-level BERT encoder to support multiple languages and connects to a Transformer-based acoustic model with convolutional and variance-prediction layers to generate efficient, high-fidelity spectral results. The vocoder reconstructs waveforms using vector-quantized representations and is trained with audio loss functions, thereby eliminating the need for complex Generative Adversarial Networks (GANs) or International Phonetic Alphabet (IPA)-based preprocessing. The arrangement eliminates reliance on language embeddings and tailor-made phonetic dictionaries, enabling fast, reliable synthesis as the system is expanded to include new voices or languages. The proposed system was developed and tested using UrduMSD, which includes detailed speaker information such as speaking time, gender, and the number of utterances. Both objective and subjective evaluations on a 152-hour corpus of 195 speakers yield strong results, achieving a Mean Opinion Score (MOS) of 2.94 and a Speaker Embedding Cosine Similarity (SECS) score of 2.63, demonstrating that the model produces clear, natural-sounding speech across multiple speakers and speaking styles