Blog
Meine eigene KI: Ein lokaler Chatbot als Praktikumsprojekt
Künstliche Intelligenz ist inzwischen Teil unseres Alltags. Viele Menschen verwenden Programme wie ChatGPT, ohne genau zu wissen, was im Hintergrund passiert oder wohin ihre eingegebenen Daten übertragen werden. In meinem Praktikumsprojekt wollte ich deshalb herausfinden, wie ein KI-Chatbot funktioniert – und einen eigenen Chatbot entwickeln, der direkt auf meinem Computer läuft.
Serving LLMs in Production: vLLM, KServe, and the Deployment Decisions That Matter
GPU memory pressure, batching inefficiencies, and autoscaling complexity make LLM serving fundamentally different from traditional ML. We compare three production deployment strategies for vLLM — standalone, KServe, and BentoML — then go deep on KServe with deployment patterns, canary rollouts, and its role in a broader LLMOps platform.
The AI Gateway Landscape: agentgateway, LiteLLM, Kong and Envoy AI Gateway
Throughout out recent events we have seen that AI Gateways have become the single most important entry point for companies.
Kubernetes GPU Entitlements with DRA and vCluster
Sharing GPUs between tenants can be challenging when different teams have different entitlements. This guide demonstrates how to use a combination of Kubernetes DRA and vCluster to provide one tenant with access to dedicated A100 capacity and another with access to a MIG slice, while ensuring that each tenant can only see the GPU classes they are permitted to use.
Multi-Tenant Observability with vCluster: Centralized Metrics and Logs Using Prometheus, Loki, and Grafana
Discover how platform teams can implement centralized metrics and logging for multi-tenant Kubernetes using vCluster. This article walks through such an architecture for private-node vClusters, showing how a centralized observability stack can serve many isolated tenant clusters, laying the foundation for scalable, production-ready multi-tenant observability.
Stop Burning Your LLM Budget: Cost-Efficient LLMOps on Kubernetes
As LLM initiatives mature from pilot to production, infrastructure costs frequently scale faster than the value they deliver. The culprit is often operational inefficiency: idle GPUs, uncontrolled storage growth, cold start latency, and unplanned network egress. We examine the principal cost drivers in LLMOps on Kubernetes and provide actionable best practices across observability, GPU efficiency, throughput tuning, storage governance, and network topology.
