This is for running AI models with Ollama on CPU only. Read the short docs from https://hub.docker.com/r/ollama/ollama for Nvidia GPU and Radeon GPU instructions.

Models can be found on Ollama’s models page or Hugging Face.

Usage

# create and run the container
# use a named volume, ollama_data, to store downloaded models
podman run -d -v ollama_data:/root/.ollama -p 11434:11434 --name oc docker.io/ollama/ollama

# check ollama supported commands
podman exec -it oc ollama --help

# download a model
podman exec -it oc ollama pull qwen2.5-coder:7b
# and run it
podman exec -it oc ollama run qwen2.5-coder:7b

# list downloaded models
podman exec -it oc ollama list
# remove a model
podman exec -it oc ollama rm model_name

To see the podman’s disk usage for images, containers and volumes use:

podman system df -h

Cleanup

podman rm -f oc
podman rmi ollama
podman volume rm ollama_data

Want to share this post? Click to copy the link.