Practical playbooks for cloud, security, compliance, and tech career growth.

How a Transformer Layer Maps to GPU Memory

The key is to think of vLLM GPU memory as a budget shared by several very different kinds of data. Some memory is essentially fixed when the model starts; some grows with request length and concurrency. For your vLLM workload, the mental model is: GPU VRAM │ ├── Model weights ← mostly fixed

How an LLM Works with an Inference Engine

A useful way to understand an LLM is to separate it into four pieces: architecture + weights + tokenizer + runtime configuration. An inference engine such as vLLM reads those pieces, constructs the neural network in GPU memory, loads the learned weights into it, allocates memory for inference, and then repeatedly runs the

SOP Benchmark vLLM Models Using LiveBench

1. Purpose This SOP describes how to use LiveBench to evaluate model quality through a production-style vLLM OpenAI-compatible API. Environment: ComponentConfigurationInference EnginevLLMAPIOpenAI-compatibleEndpointhttps://dev-va-vllm.eveon.comAuthenticationAPI Key / Bearer TokenBenchmarkLiveBenchInfrastructureAWS ALB → EC2 → Docker → vLLM LiveBench is primarily used to evaluate model quality, including reasoning, coding, mathematics, data analysis, language, and instruction following.

SOP Production vLLM Deployment on AWS EC2 with ALB

1. Purpose This SOP describes a secure and production-ready deployment of vLLM on Amazon Linux 2023, using an NVIDIA GPU-based EC2 G6 or newer instance and an AWS Application Load Balancer (ALB). The recommended architecture is: Client / Application | | HTTPS :443 v +-------------------------+ | AWS Application | | Load Balancer | | | | - TLS / ACM | | -

SOP - Deploying vLLM on Amazon Linux

Version: 1.0 Platform: Amazon Linux 2023 Deployment Method: Docker (Recommended) Audience: System Administrators, DevOps Engineers, AI/ML Engineers 1. Purpose This document describes the recommended procedure for deploying vLLM on Amazon Linux 2023 using the official AWS Deep Learning Container (DLC). Using the AWS DLC is the simplest and

AWS EC2 Instance GPU Types

AWS EC2 GPU Instance Comparison FeatureAmazon EC2 G5Amazon EC2 G6Amazon EC2 G7Processor2nd-generation AMD EPYC processorsNewer-generation processors optimized for GPU workloadsCustom Intel Xeon 6 processorsGPUUp to 8 NVIDIA A10G Tensor Core GPUsNVIDIA GPUs with enhanced memory bandwidthUp to 8 NVIDIA RTX PRO 4500 Blackwell Server Edition GPUsGPU Memory24 GB per GPU24

SOP Running and Configuring vLLM on Linux using Docker

Document Version: 1.0 Applies To: Ubuntu 22.04/24.04, Debian 12, RHEL/Rocky 9 (or equivalent Linux distributions) Model: google/gemma-4-12B-it (recommended) 1. Purpose This document describes the standard procedure for deploying a production-ready vLLM inference server running the Google Gemma 4 12B model using Docker Compose. The

SOP: Install Open WebUI on Linux Using Docker Compose

1. Purpose This SOP describes how to install and configure Open WebUI on a Linux server using Docker Compose. The deployment supports: * Connection to Ollama running on the Linux host * Persistent storage for users, chats, configuration, and uploaded files * Optional Microsoft Entra ID authentication * Optional Google authentication * Optional NVIDIA GPU

SOP: Ollama Installation and Configuration

1. Purpose This SOP provides instructions for installing and configuring Ollama on a Linux server, enabling remote access, verifying the service, and downloading an AI model. Recommended SOP Structure 1. Purpose 2. Scope 3. Prerequisites 4. Hardware Requirements 5. Operating System Requirements 6. GPU Verification 7. Install Ollama 8. Configure

Employee Onboarding and Offboarding Process

Below is a typical enterprise employee onboarding and offboarding process covering HR, IT, Security, Facilities, Finance, Legal, and the employee’s manager. 1. Employee Onboarding Process Purpose The onboarding process ensures a new employee, contractor, or intern is properly hired, verified, equipped, trained, and granted the right access before starting

IT Management Guide

1. Purpose This guide defines a minimal but effective IT management process for a startup company developing AI solutions. The goal is to establish practical IT operations, security, and compliance processes that support SOC 2 and ISO 27001 readiness without creating unnecessary enterprise overhead. This guide covers: * Microsoft 365 or

SOP — Create and Manage Microsoft Azure for Enterprise

Version 1.0 Date 2026-06-22 Domain mptwork.com Model Identity rooted in the existing Microsoft Entra tenant; hierarchy of Management Groups → Subscriptions → Resource Groups → Resources Consoles Azure portal portal.azure.com · Entra admin center entra.microsoft.com Companion SOPs M365/Entra — sop-subscribe-manage-m365-mptwork.md, sop-entra-id-authentication-sso-mptwork.md · Network/IPAM — enterprise-ipam-reference-architecture.md · AWS