Topic
platform-engineering
8 articles
Running a big LLM across multiple GPUs with vLLM
A plain-English guide to serving a model too big for one GPU, in two tracks: a runbook from download to serving with every flag and error explained, and a deep dive into how tensor, pipeline, and expert parallelism split the model, with measured numbers from a 235B model on four RTX PRO 6000 cards.
I Ran an AI SRE Copilot on My Own Hardware. Here Is What It Actually Does.
Running NudgeBee v1.4.0 end to end - a self-hosted AIOps platform behind AI-SRE, AI-FinOps, AI-K8sOps, and agentic automation - on a Mac, a kiac cluster, and a DGX Spark.
HAMi Dynamic MIG on RTX PRO 6000: A Live Kubernetes Test
Hands-on HAMi Dynamic MIG test on Kubernetes and RTX PRO 6000 Blackwell: setup commands, real allocations, mixed profiles, reclamation, and recovery.
Devin Outposts on Kubernetes: Why Your AI Agent Needs Your Cluster
Devin Outposts runs AI coding agent sessions as pods on your own Kubernetes cluster, with an open-source operator to manage the fleet.
How to Share GPUs in Kubernetes at Scale with HAMi (Software vGPU Slicing)
Share NVIDIA GPUs in Kubernetes with HAMi software vGPU slicing: memory and compute limits, Helm configuration, a verified PyTorch manifest, a real RTX PRO 6000 OOM test, and Prometheus monitoring.
Slicing GPUs in Kubernetes with NVIDIA Multi-Instance GPU (MIG)
GPU sharing in Kubernetes explained: time-slicing vs MPS vs MIG, every nvidia-smi command to enable and disable MIG on one GPU or eight, GPU Operator automation, pitfalls, and DCGM monitoring.
Controlling MCP Tools with agentgateway on Kubernetes (Part 1)
Run AI agents behind agentgateway on Kubernetes: route their LLM and MCP tool calls through one proxy, keep secrets out of the agent, and block tools by policy.
Platform Engineering Demystified - Navigating the Basics
In this blog post, we’ll demystify the basic concepts of Platform Engineering by understanding how it’s a natural and necessary evolution to DevOps.
