DEV Community
Follow
From screenplay to one multi-voice MP3: parsing scripts for a multi-speaker TTS API on Cloudflare Workers
The author developed a web application that generates multi-speaker audio from a single script, improving upon traditional text-to-speech limitations. The system translates loose text into a structured format for a sophisticated TTS model, accommodating acting directions and multiple characters. A Cloudflare Worker, D1 for data, and R2 for MP3 storage form the backend, utilizing Fish Audio's Drama 3 model. The input format resembles a radio script, using "Name: line" and square brackets for delivery directions. A key design choice defaults unnamed lines to a narrator, enhancing user experience.Parsing is handled by a regular expression and word-count heuristic to identify speakers and lines, with careful consideration for colons in prose. The application actively strips control tags from user input to prevent injection of unintended speaker changes. The payload construction maps characters to unique speaker indices, ensuring correct voice assignment. An auto-casting mechanism assigns voices to unassigned characters from a curated pool, prioritizing user choices and avoiding duplicates.A critical discovery involved the TTS model's silent fallback to a different model if the 'model' header is missing or incorrect, emphasizing the need for explicit pinning and documentation. Robust error handling includes rejecting tiny responses and never logging sensitive data like script text or API keys. Cost calculation is based on UTF-8 bytes of the exact payload, with conservative credit reservation and refunds for failures. Audio duration is estimated from file size and a pinned bitrate, avoiding the need for an audio decoder.The system uses a synchronous request model for simplicity, which is suitable for shorter scripts but has potential limitations for longer ones. A significant "gotcha" involved iOS not playing sample MP3s without proper Range request support (206 Partial Content responses). The author provides a checklist of best practices, including defaulting unnamed lines to a narrator, previewing cast lists, stripping control tags, pinning model headers, accurate cost estimation, pinning bitrate, and serving media with Range support. The project offers free trials to experience the multi-voice scenes.