Skip to content

Commit

Permalink
Merge pull request #583 from rhatdan/pull
Browse files Browse the repository at this point in the history
Fix ramalama run on docker to work correctly
  • Loading branch information
ericcurtin authored Jan 13, 2025
2 parents 7c6e643 + bdf24da commit cc1db3c
Show file tree
Hide file tree
Showing 2 changed files with 8 additions and 3 deletions.
2 changes: 1 addition & 1 deletion docs/ramalama.1.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Running in containers eliminates the need for users to configure the host system

RamaLama pulls AI Models from model registries. Starting a chatbot or a rest API service from a simple single command. Models are treated similarly to how Podman and Docker treat container images.

When both Podman and Docker are installed, RamaLama defaults to Podman, The `RAMALAMA_CONTAINER_ENGINE=docker` environment variable can override this behavior. When neither are installed RamaLama attempts to run the model with software on the local system.
When both Podman and Docker are installed, RamaLama defaults to Podman, The `RAMALAMA_CONTAINER_ENGINE=docker` environment variable can override this behaviour. When neither are installed RamaLama attempts to run the model with software on the local system.

Note:

Expand Down
9 changes: 7 additions & 2 deletions ramalama/model.py
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,11 @@ def setup_container(self, args):

if os.path.basename(args.engine) == "podman":
conman_args += ["--pull=newer"]
elif os.path.basename(args.engine) == "docker":
try:
run_cmd([args.engine, "pull", "-q", args.image], ignore_all=True)
except Exception: # Ignore errors, the run command will handle it.
pass

if sys.stdout.isatty() or sys.stdin.isatty():
conman_args += ["-t"]
Expand Down Expand Up @@ -201,9 +206,9 @@ def exec_model_in_container(self, model_path, cmd_args, args):
return False

if model_path and os.path.exists(model_path):
conman_args += [f"--mount=type=bind,src={model_path},destination={MNT_FILE},rw=false"]
conman_args += [f"--mount=type=bind,src={model_path},destination={MNT_FILE},ro"]
else:
conman_args += [f"--mount=type=image,src={self.model},destination={MNT_DIR},rw=false,subpath=/models"]
conman_args += [f"--mount=type=image,src={self.model},destination={MNT_DIR},ro,subpath=/models"]

# Make sure Image precedes cmd_args.
conman_args += [self._image(args)] + cmd_args
Expand Down

0 comments on commit cc1db3c

Please sign in to comment.