I built a real-time pipeline that reads game subtitles and converts them into dynamic voice acting (OCR → TTS → RVC) [P]

Our take

I developed a real-time pipeline that transforms game subtitles into dynamic voice acting by integrating OCR, TTS, and RVC technologies. This desktop app captures subtitles from the screen, converts them into speech, and customizes the voice for each character. Key challenges included minimizing latency to around 0.3 seconds, avoiding repeated subtitle spam, and smoothly managing multiple voice models. I also explored innovative features like emotion-based voice alterations and real-time translation.

I've been experimenting with real-time pipelines that combine OCR + TTS + voice conversion, and I ended up building a desktop app that can "voice" game subtitles dynamically.

The idea is simple: - Capture subtitles from screen (OCR) - Convert them into speech (TTS) - Transform the voice per character (RVC)

But the hard parts were: - Avoiding repeated subtitle spam (similarity filtering) - Keeping latency low (~0.3s) - Handling multiple characters with different voice models without reloading - Running everything in a smooth pipeline (no audio gaps)

One thing that helped a lot was using a two-stage pipeline: While one sentence is playing, the next one is already processed in the background.

I also experimented with: - Emotion-based voice changes - Real-time translation (EN → TR) - Audio ducking (lowering game sound during speech)

I'm curious: How would you approach reducing latency further in a multi-model setup like this? Or is there a better alternative to RVC for real-time character voice conversion?

I built a real-time pipeline that reads game subtitles and converts them into dynamic voice acting (OCR → TTS → RVC) [P]

Related Articles