Learning Timeline
Key Insights
Limitations for Mac M-Series Users
Currently, Docker on Mac does not have direct access to M-series GPUs. Mac users are encouraged to run Ollama natively if maximum GPU performance is required.
Benefits of Isolation Techniques
Running AI in Docker is not only more secure (as the AI cannot access your personal file system), but it also simplifies model management without cluttering your main system libraries.
Step by Step
Preparing the Docker Environment for Ollama
- Install Docker Desktop on your operating system (Windows, Mac, or Linux).
- Open your terminal or the WSL (Ubuntu) application if you are using Windows.
- For Nvidia GPU users, install the 'Nvidia Container Toolkit' via terminal (using APT commands or by referring to Nvidia documentation) to enable GPU access for Docker.
- Ensure the Docker service is running smoothly before proceeding to the next step.
Running Ollama in a Secure Container
- Open your terminal/WSL.
- Enter the 'docker run' command including the '--gpus all' flag and volume settings to store persistent data.
- Set 'port forwarding' to 11434:11434 for API access.
- Use the security-opt flag to restrict container privileges, ensuring the AI process is completely isolated from the OS.
- Press 'Enter' to begin the image download process and launch the container.
- Type the 'docker ps' command in the terminal to verify that the Ollama container status is 'Up' or currently running.
Running AI Models (DeepSeek) in Docker
- Identify the name of the running Ollama container (usually named 'ollama').
- Type the 'docker exec -it' command followed by the container name and the command to run the model (e.g., deepseek-r1).
- Wait for the model download process to complete inside the container.
- Start an AI chat session directly within the isolated terminal.
- Open 'Task Manager' or a GPU monitoring tool to observe performance spikes, confirming that the AI is utilizing your GPU hardware rather than the CPU.
