1. Purpose

This SOP provides instructions for installing and configuring Ollama on a Linux server, enabling remote access, verifying the service, and downloading an AI model.

Recommended SOP Structure

1. Purpose
2. Scope
3. Prerequisites
4. Hardware Requirements
5. Operating System Requirements
6. GPU Verification
7. Install Ollama
8. Configure Ollama Service
9. Configure External Access
10. Configure Firewall
11. Download Models
12. Verify Installation
13. Service Administration
14. Model Management
15. Upgrading Ollama
16. Backup & Recovery
17. Security Hardening
18. Monitoring & Logging
19. Troubleshooting
20. References

Additional Sections

1. Hardware Requirements

2. Verify GPU

Before installing models, verify GPU availability.

NVIDIA Driver

nvidia-smi

Expected output:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 580.xx |
| Driver Version |
| CUDA Version |
+-----------------------------------------------------------------------------+

Verify CUDA

nvcc --version

or

cat /usr/local/cuda/version.txt

3. Verify Ollama Uses GPU

Run

ollama run gemma4:12b

Open another terminal

watch -n 1 nvidia-smi

The Ollama process should appear using GPU memory.

4. Configure External Access

Instead of editing the system service directly:

Create an override:

sudo systemctl edit ollama

Add

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"

Reload

sudo systemctl daemon-reload
sudo systemctl restart ollama

This approach survives package upgrades better than modifying the service file directly.

5. Configure Firewall

Ubuntu (UFW)

sudo ufw allow from 10.0.0.0/8 to any port 11434

or

sudo ufw allow from 192.168.0.0/16 to any port 11434

RedHat

sudo firewall-cmd \
--permanent \
--add-port=11434/tcp

sudo firewall-cmd --reload

Never expose port 11434 directly to the Internet.

6. Verify API

Health check

curl http://localhost:11434

List installed models

curl http://localhost:11434/api/tags

Generate text

curl http://localhost:11434/api/generate \
-d '{
  "model":"gemma4:12b",
  "prompt":"Hello"
}'

7. Download Models

Examples

ollama pull gemma4:12b
ollama pull qwen3:14b
ollama pull llama3.3:70b
ollama pull mistral:latest

List models

ollama list

Delete model

ollama rm gemma4:12b

8. Service Administration

Status

sudo systemctl status ollama

Restart

sudo systemctl restart ollama

Stop

sudo systemctl stop ollama

Start

sudo systemctl start ollama

Enable

sudo systemctl enable ollama

Logs

journalctl -u ollama -f

9. Monitoring

CPU

htop

GPU

watch -n1 nvidia-smi

Memory

free -h

Disk

df -h

10. Upgrade Ollama

curl -fsSL https://ollama.com/install.sh | sh

Restart

sudo systemctl restart ollama

Verify version

ollama --version

11. Backup

Models are stored under

/usr/share/ollama

or

~/.ollama

depending on installation.

Back up

tar czvf ollama-models.tar.gz ~/.ollama

12. Security Hardening

Recommended enterprise controls:

• Restrict API access to internal networks only.
• Require authentication through a reverse proxy (e.g., NGINX or Traefik).
• Use TLS for all remote connections.
• Log all API requests.
• Limit model download permissions to administrators.
• Regularly update Ollama and AI models.
• Monitor GPU utilization and disk consumption.
• Restrict SSH access using MFA.
• Maintain OS patching.
• Review service logs periodically.

13. Reverse Proxy Example (NGINX)

server {
    listen 443 ssl;
    server_name ollama.company.com;

    ssl_certificate /etc/ssl/certs/fullchain.pem;
    ssl_certificate_key /etc/ssl/private/key.pem;

    location / {
        proxy_pass http://127.0.0.1:11434;
    }
}

Benefits:

• HTTPS
• Authentication
• Rate limiting
• Centralized logging
• SSO integration (OIDC/SAML via an authentication gateway)

14. Enterprise Deployment Recommendations

For organizations running multiple AI workloads, consider:

• Dedicated GPU servers for Ollama inference
• Docker or Kubernetes deployment for easier lifecycle management
• Reverse proxy with Microsoft Entra ID or Google Workspace SSO
• Prometheus and Grafana for metrics and dashboards
• Centralized logging with ELK or OpenSearch
• Load balancing across multiple Ollama nodes
• Internal model registry and version control
• Integration with Open WebUI, LibreChat, or custom applications via the Ollama REST API
• Network segmentation and firewall policies to isolate AI infrastructure