One or several metadata.jsonl were found, but not in the same directory or in a parent directory of

[mst272]

Describe the bug

ImageFolder with metadata.jsonl error. I downloaded liuhaotian/LLaVA-CC3M-Pretrain-595K locally from Hugging Face. According to the tutorial in https://huggingface.co/docs/datasets/image_dataset#image-captioning, only put images.zip and metadata.jsonl containing information in the same folder. However, after loading, an error was reported: One or several metadata.jsonl were found, but not in the same directory or in a parent directory of.

The data in my jsonl file is as follows:

{"id": "GCC_train_002448550", "file_name": "GCC_train_002448550.jpg", "conversations": [{"from": "human", "value": "\nProvide a brief description of the given image."}, {"from": "gpt", "value": "a view of a city , where the flyover was proposed to reduce the increasing traffic on thursday ."}]}

Steps to reproduce the bug

from datasets import load_dataset
image = load_dataset("imagefolder",data_dir='data/opensource_data')

Expected behavior

success

Environment info

datasets==3.2.0

[lhoestq]

Hmmm I double checked in the source code and I found a contradiction: in the current implementation the metadata file is ignored if it's not in the same archive as the zip image somehow:

datasets/src/datasets/packaged_modules/folder_based_builder/folder_based_builder.py

Lines 352 to 353 in caa705e

	if metadata_file_candidate
	is None # ignore metadata_files that are not inside archives

in the tests suite the metadata file is placed inside the archive:

datasets/tests/packaged_modules/test_imagefolder.py

Line 223 in caa705e

image_metadata_filename = archive_dir / "metadata.jsonl"

Thanks for reporting this issue, it seems the documentation is wrong and we never implemented the support for zip + metadata outside zip. We might rewrite part of this code soon though to make it more flexible, it can be a good occasion to fix this. In the meantime feel free to open a PR to fix the documentation if you'd like