Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for migrating qdrant collections #5

Merged
merged 13 commits into from
Jul 24, 2024
Merged

Conversation

olirice
Copy link
Collaborator

@olirice olirice commented Jul 22, 2024

What kind of change does this PR introduce?

Adds a vec2pg qdrant migrate command for migrating qdrant collections to postgres

Tasks:

  • implement migrate command
  • enable API key passed as an env var
  • test using local docker setup
  • minimal docs in README
  • test on CI
  • complete docs
  • measure throughput on qdrant hosted

README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
Co-authored-by: Juan Carlos <[email protected]>
Co-authored-by: Juan Carlos <[email protected]>
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
@olirice olirice requested a review from gregnr July 24, 2024 19:06
Co-authored-by: Juan Carlos <[email protected]>
Co-authored-by: Juan Carlos <[email protected]>
Copy link

@gregnr gregnr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work 👍

postgresql://postgres.ahqsutirwnsocaaorimo:<Password>@aws-0-us-east-1.pooler.supabase.com:6543/postgres
```

Then we can call `vec2pg qdrant migrate` passing our values. You can supply all parameters directly to the CLI, but its a good idea to pass the Qdrant API Key (QDRANT_API_KEY) and Supabase connection string (POSTGRES_CONNECTION_STRING) as environment variables to avoid logging credentials to your shell's history.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Then we can call `vec2pg qdrant migrate` passing our values. You can supply all parameters directly to the CLI, but its a good idea to pass the Qdrant API Key (QDRANT_API_KEY) and Supabase connection string (POSTGRES_CONNECTION_STRING) as environment variables to avoid logging credentials to your shell's history.
Then we can call `vec2pg qdrant migrate` passing our values. You can supply all parameters directly to the CLI, but it's a good idea to pass the Qdrant API Key (QDRANT_API_KEY) and Supabase connection string (POSTGRES_CONNECTION_STRING) as environment variables to avoid logging credentials to your shell's history.

@olirice olirice merged commit 48d7c87 into main Jul 24, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants