Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adding text chunker in 11labs #1013

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Added `examples/foundational/26d-gemini-multimodal-live-text.py` which is
using Gemini as TEXT modality and using another TTS provider for TTS process.

- Added `text_chunker` as utils and using it in `ElevenLabsTTSService` to
support chunked text as per their doc which recommends sending text word by
word, also Possible fix of #983.

### Changed

- Modified `OpenAIAssistantContextAggregator` to support controlled completions
Expand Down
5 changes: 4 additions & 1 deletion src/pipecat/services/elevenlabs.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
from pipecat.services.ai_services import WordTTSService
from pipecat.services.websocket_service import WebsocketService
from pipecat.transcriptions.language import Language
from pipecat.utils.string import text_chunker

# See .env.example for ElevenLabs configuration needed
try:
Expand Down Expand Up @@ -407,7 +408,9 @@ async def run_tts(self, text: str) -> AsyncGenerator[Frame, None]:
self._started = True
self._cumulative_time = 0

await self._send_text(text)
for text_chunk in text_chunker(text):
# Ref: https://elevenlabs.io/docs/developer-guides/reducing-latency#3-use-the-input-streaming-websocket
await self._send_text(text_chunk)
await self.start_tts_usage_metrics(text)
except Exception as e:
logger.error(f"{self} error sending message: {e}")
Expand Down
19 changes: 19 additions & 0 deletions src/pipecat/utils/string.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
#

import re
import typing

ENDOFSENTENCE_PATTERN_STR = r"""
(?<![A-Z]) # Negative lookbehind: not preceded by an uppercase letter (e.g., "U.S.A.")
Expand All @@ -23,3 +24,21 @@
def match_endofsentence(text: str) -> int:
match = ENDOFSENTENCE_PATTERN.search(text.rstrip())
return match.end() if match else 0


def text_chunker(chunks: str) -> str:
"""Used during input streaming to chunk text blocks and set last char to space"""
splitters = (".", ",", "?", "!", ";", ":", "—", "-", "(", ")", "[", "]", "}", " ")
buffer = ""
for text in chunks:
if buffer.endswith(splitters):
yield buffer if buffer.endswith(" ") else buffer + " "
buffer = text
elif text.startswith(splitters):
output = buffer + text[0]
yield output if output.endswith(" ") else output + " "
buffer = text[1:]
else:
buffer += text
if buffer != "":
yield buffer + " "
Loading