Clone Talking: Real‑Time Voice Cloning with Free API Tiers
Run the Clone Talking app locally with free‑tier APIs for real‑time voice cloning.
Clone the repo, set API keys, and start the dev server to test voice cloning.
Summary
Clone Talking is an open‑source web app that enables real‑time voice conversations with AI persona clones, achieving sub‑second latency without requiring any app installation.
The stack uses Whisper for speech‑to‑text, OpenRouter for LLM inference (access to Claude, GPT‑4, Llama, and more), and VoiSpark for text‑to‑speech voice cloning.
Audio is streamed bidirectionally over WebSocket, while the backend runs on Node.js with Express and is exposed via ngrok for local testing.
The frontend is built with Next.js and TailwindCSS, and the entire project is licensed under MIT.
To run the app locally you need four free‑tier API keys: OpenAI (Whisper), OpenRouter, VoiSpark, and ngrok.
The project emphasizes ethical voice cloning by using a TTS provider that explicitly permits synthetic voice generation within its terms of service.
Future enhancements include custom voice training, multi‑language support, conversation memory, and external knowledge‑base integration.
Clone Talking demonstrates how quickly a real‑time voice cloning solution can be built with minimal infrastructure and free‑tier services.
Key changes
- Real‑time voice processing with sub‑second latency
- Uses Whisper STT, OpenRouter LLM, and VoiSpark TTS
- WebSocket transport for low‑latency bidirectional audio
- Node.js + Express + ngrok backend
- Next.js + TailwindCSS frontend
- Requires four free‑tier API keys (OpenAI, OpenRouter, VoiSpark, ngrok)
- MIT license and ethical voice cloning
- Future features: custom voice training, multi‑language, memory, knowledge‑base integration