Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support for Realtime API with Azure OpenAI #848

Merged
merged 11 commits into from
Oct 8, 2024
Merged

Support for Realtime API with Azure OpenAI #848

merged 11 commits into from
Oct 8, 2024

Conversation

davidzhao
Copy link
Member

No description provided.

@davidzhao davidzhao requested a review from a team October 5, 2024 17:21
@changeset-bot changeset-bot
Copy link

changeset-bot bot commented Oct 5, 2024 •
edited
Loading

🦋 Changeset detected

Latest commit: da3b054

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
livekit-plugins-openai Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@davidzhao davidzhao mentioned this pull request Oct 5, 2024
Merged
bcherry
bcherry approved these changes Oct 7, 2024
View reviewed changes
Copy link
Contributor

@bcherry bcherry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good - I didn't test this one. I think we should consider just making the example work automatically either way based on env var detection. I made that change in the JS one while testing it and it felt good to me. lmk what you think

@davidzhao
Copy link
Member Author

looks good - I didn't test this one. I think we should consider just making the example work automatically either way based on env var detection. I made that change in the JS one while testing it and it felt good to me. lmk what you think

sure, we can do that! which env var do you think we should switch on?

@bcherry
Copy link
Contributor

bcherry commented Oct 8, 2024

looks good - I didn't test this one. I think we should consider just making the example work automatically either way based on env var detection. I made that change in the JS one while testing it and it felt good to me. lmk what you think

sure, we can do that! which env var do you think we should switch on?

I went with endpoint in https://github.com/livekit/agents-js/pull/97/files#diff-8a3cd674863fd6f2a24d4dfc3f6b151e66623fb99bbb0738ca45b75a76ddf906R17

theomonnom
theomonnom approved these changes Oct 8, 2024
View reviewed changes
Copy link
Member

@theomonnom theomonnom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/realtime/realtime_model.py Outdated Show resolved Hide resolved
@theomonnom theomonnom merged commit a264627 into main Oct 8, 2024
6 of 8 checks passed
@theomonnom theomonnom deleted the azure-support branch October 8, 2024 20:07
@github-actions github-actions bot mentioned this pull request Oct 8, 2024
Merged
@theomonnom
Copy link
Member

IMO Azure shouldn't be implicitly used. Users must specify RealtimeModel.with_azure

davidzhao
davidzhao commented Oct 8, 2024
View reviewed changes
examples/multimodal_agent.py
# azure_deployment="<model-deployment>",
# azure_endpoint="wss://<endpoint>.openai.azure.com/", # or AZURE_OPENAI_ENDPOINT
# api_key="<api-key>", # or AZURE_OPENAI_API_KEY
# api_version="2024-10-01-preview", # or OPENAI_API_VERSION
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is azure specific

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah this is weird but this is what the official OpenAI sdk is using

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keeping the same env vars since this is what is being documented on their docs

@github-actions github-actions bot mentioned this pull request Oct 8, 2024
Open
@illgitthat illgitthat mentioned this pull request Oct 9, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@theomonnom theomonnom theomonnom approved these changes

@bcherry bcherry bcherry approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@davidzhao @bcherry @theomonnom