gpt-realtime

GPT-realtime is an advanced speech model released by OpenAI, which can be used in AI voice call applications. Its features include: the ability to understand complex instructions and precisely invoke tools; generating natural, fluent, and expressive speech, with two new voices—Marin and Cedar—alongside upgrades to the existing eight voices; capturing non-verbal cues like laughter; seamlessly switching languages mid-sentence; and flexibly adjusting tone based on the context.

Note:

To stream audio output in real time, please use a wss connection; online debugging is not supported

Source code: https://github.com/302ai/openai-realtime-console