Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PromptClip with Gemini, output modified #9

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

Cheez22
Copy link

@Cheez22 Cheez22 commented Oct 18, 2024

This version of the code uses Google's Gemini. The output has been modified to improve video editing efficiency, turning PromptClip into more of a search engine for videos.

This version of the code uses Google's Gemini. The output has been modified to improve video editing efficiency, turning PromptClip into more of a search engine for videos.
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@Cheez22
Copy link
Author

Cheez22 commented Nov 3, 2024

@codeAshu
@ashish-spext

Hi, just bumping this PR, please let me know if there's anything else I need to do on my side.

@ashish-spext
Copy link
Collaborator

ashish-spext commented Nov 4, 2024

@Cheez22 Copying all the existing files into a new notebook as cells is not a good idea, as it unnecessarily duplicates code that could be reused from the existing files.

There is no need to create a new notebook file as well just for Gemini.

Ideal implementation: Users should be able to pass a Gemini LLM instance to the text_prompter, scene_prompter, and multimodal_prompter functions in video_prompter.py called from existing notebooks.

This approach will enable users to select their preferred LLM in the existing notebooks (PromptClip_spoken.ipynb, PromptClip_visual.ipynb, PromptClip_multimodal.ipynb, and others that may be added in the future) without having to create separate copies of each notebook for different LLMs.

Could you delete the Gemini file and, instead, try passing the Gemini LLM instance directly into the functions of the existing notebooks? This approach should unlock the support without needing a separate file.

If passing the Gemini LLM works correctly, there will be no further changes needed. However, if any fixes are required to enable this functionality, it would be appreciated if you could include those changes in this PR.

Intended use for Gemini
Intended for Gemini.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants