Define YOLOWorld classnames on iOS at runtime

[hdi200]

Hey! Just wondering is it possible to just convert the base yolov8s-worldv2.pt to an ML Model and define classnames to look for on iOS itself, on device at runtime? Rather than pre-preparing the class_names in python? Would I need to integrate clip into my app somehow?

[UltralyticsAssistant]

👋 Hello @hdi200, thank you for your interest in Ultralytics 🚀!

For new users, we recommend checking out the Docs, which include many helpful Python and CLI usage examples. These may already answer some of your questions.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us diagnose the issue effectively. If you're exploring custom integration ❓, please share more details about your approach and any relevant logs or error messages.

Join the Ultralytics community where it suits you best. For real-time chat, head to Discord 🎧. Prefer in-depth discussions? Check out Discourse. You can also join our Subreddit to learn and share insights with the community.

Upgrade

Ensure you upgrade to the latest ultralytics package and include all requirements. Use a Python>=3.8 environment with PyTorch>=1.8:

pip install -U ultralytics

Environments

YOLO can be run in various environments, each with dependencies like CUDA and CUDNN already installed:

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. Check the GCP Quickstart Guide
• Amazon Deep Learning AMI. Visit the AWS Quickstart Guide
• Docker Image. See the Docker Quickstart Guide

Status

If this badge is green, all Ultralytics CI tests are currently passing. These tests verify correct operation across macOS, Windows, and Ubuntu for all YOLO Modes and Tasks every 24 hours and on each commit.

This is an automated response, and an Ultralytics engineer will assist you soon 😊.

[glenn-jocher]

@hdi200 yes, you can define class names at runtime on iOS by using the YOLO model with Core ML. Integrating CLIP is not necessary for this functionality.

[hdi200]

But I thought YOLOWorld worked differently, because it was designed for open-vocabulary detection, meaning it can recognize objects dynamically based on descriptive text prompts rather than being limited to a fixed set of predefined classes. So if I want that open vocab deletion to run on iOS natively but still want that open detection how can I do it when there are no predefined classes to start with?

[glenn-jocher]

@hdi200 yOLO-World supports open-vocabulary detection by using descriptive text prompts. For iOS, you can convert the model to Core ML and dynamically set these prompts at runtime to achieve open detection without predefined classes.

[hdi200]

I tried converting it to coreml for use in the sample app but I got an error that there were no class names provided (which makes sense) when I tried to dynamically set class
Names detection didn’t happen. I know I must be approaching this wrong, any advice on how I’d get this setup on iOS or where to look, really appreciate the help so far!!

[glenn-jocher]

To set up YOLO-World for open-vocabulary detection on iOS, ensure your Core ML model supports dynamic text prompts. You might need to modify the app to input these prompts at runtime. Check Apple's Core ML documentation for guidance on handling dynamic inputs.