[Bug]: qdrant vector store does not handle nested payloads

[olyashok]

Bug Description

function that parses the node ignores nested attributes:
def parse_to_query_result(self, response: List[Any]) -> VectorStoreQueryResult:

• node = metadata_dict_to_node(payload) does not do anything since payload metadata has no "_node_content" so it reverts to "deprecated legacy logic for backward compatibility"

since qdrant returns nested metadata as List
this line if isinstance(val, (str, int, float, type(None))):

probably hsould add a List to that

ignores that part of payload

also why do we have this - i had a metadata called doc_id and was expecting it back but it was stripped
metadata.pop("id", None)
metadata.pop("document_id", None)
metadata.pop("doc_id", None)
metadata.pop("ref_doc_id", None)

Version

0.9.22

Steps to Reproduce

try to query qdrant with nested metadata - it is not processed by vector_stores utils

Relevant Logs/Tracbacks

No response

[dosubot]

🤖

It's great to see that you've identified a potential issue with the handling of nested payloads in the qdrant vector store. Your detailed description of the problem and the proposed solution is very helpful. Encouraging users to contribute to the improvement of LlamaIndex is a fantastic way to help the community. If you're interested in contributing further, creating a pull request with your proposed fix would be a valuable addition to the project. Thank you for your dedication to improving LlamaIndex!

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[logan-markewich]

It's really intended to work best if there is a node_content field (I.e you built the index with llama-index)

Support beyond that is beta at best.

Those values are popped from the dict, because the function that transforms the node into a dict adds those (a symptom of some older backwards support)

[olyashok]

oh well - not every vector database in the world is built with llama.
but you can keep building under that assumption - see where that takes you

[logan-markewich]

Please make a PR if you see room for improvement, there is certainly 101 thongs to work on and make better :) anyways, probably will fix this at some point

[tslmy]

oh well - not every vector database in the world is built with llama.
but you can keep building under that assumption - see where that takes you

Maybe I'm misreading your tone here, but this doesn't sound helpful.

Version
0.9.22

You are using a software with version <1, which implies there are a lot of problems. It does not mean folks has ever built things under any assumption; it merely means many corners have been cut to make something you and I can use right away.

It's like talking to a toddler. If they make a grammatical mistake, you excuse them; you don't say "you can keep on with that baby talk and see whether that would land you with a Ph.D."

I myself also see imperfections regarding integrations between LlamaIndex with other components, even places where LlamaIndex went against conventions / standards (#9770), but I don't mindread.

Anyway, I wish you a happy new year, @olyashok .

[olyashok]

Obviously i want to help improve it - why else i would file bugs. 2 easy improvements in that class: * Add List to if type check so that List parameters also get passed through * Do not strip out common payload attributes like “id”, “doc_id” Kind regards Alex

…

________________________________ From: Ming ***@***.***> Sent: Sunday, December 31, 2023 2:12:08 AM To: run-llama/llama_index ***@***.***> Cc: olyashok ***@***.***>; Mention ***@***.***> Subject: Re: [run-llama/llama_index] [Bug]: qdrant vector store does not handle nested payloads (Issue #9766) oh well - not every vector database in the world is built with llama. but you can keep building under that assumption - see where that takes you Maybe I'm misreading your tone here, but this doesn't sound helpful. Version 0.9.22 You are using a software with version <1, which implies there are a lot of problems. It does not mean folks has ever built things under any assumption; it merely means many corners have been cut to make something you and I can use right away. It's like talking to a toddler. If they make a grammatical mistake, you excuse them; you don't say "you can keep on with that baby talk and see whether that would land you with a Ph.D." I myself also see imperfections regarding integrations between LlamaIndex with other components, even places where LlamaIndex went against conventions / standards (#9770<#9770>), but I don't mindread. Anyway, I wish you a happy new year, @olyashok<https://github.com/olyashok> . — Reply to this email directly, view it on GitHub<#9766 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AMBUT7WPXUUQVGMYNNGS65LYMEF4RAVCNFSM6AAAAABBHVXTWGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZSG42TAOBUGY>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[logan-markewich]

@olyashok maybe see if the above change makes sense

[olyashok]

@logan-markewich - this will do it. thx!