Function Introduction

Input a piece of voice, and generate a voice translated into another language based on the timbre of the original voice.

Processing Flow Description

🔄 Overall Processing Flow

Input Audio → Speech Recognition (STT) → Text Translation → Voice Cloning → Text-to-Speech (TTS) → Output Audio

The system will automatically complete end-to-end translation from audio to audio, maintaining the original or specified timbre characteristics.

Details of Processing Steps

📋 Five Core Steps

1️⃣ Initialization (initialization)

• Download and prepare the audio file
• Automatically crop the cloned audio to within 30 seconds

2️⃣ Speech Recognition (speech_to_text)

• Convert audio to text using OpenAI gpt-4o-transcribe

3️⃣ Translation (translation)

• Users can choose their own LLM model (by default, the claude-haiku-4-5-20251001 model is used) for intelligent translation.

4️⃣ Voice Cloning (voice_clone)

• Analyze and extract audio features

5️⃣ Text-to-Speech (text_to_speech)

• Generate target language audio using the cloned timbre
• Output high-quality audio files

Supplier Selection Logic

🎯 Automatic Selection Rules

The system will automatically select the best voice cloning supplier based on the target language:

Selection Priority

1. User Designation: If a supplier is specified and supports the target language, it will be used first.
2. Language Matching: For Chinese and English, index_tts2 is preferred; for other languages, Fish is used.

⚠️ Notes

• It is recommended that the duration of the cloned audio be 10-30 seconds; excessively long audio will be automatically clipped.
• The audio must be clear and free of noise to achieve the best results.
• Supported formats: mp3, wav