KubeCon + CloudNativeCon NA 2024 Recap

Interesting though is that KubeCon + CloudNativeCon is in some way like Reply: you have many specialized tracks that focus on Platform Engineering and Observability (Liquid Reply), AI + ML (ML Reply), Data and Processing (Data Reply), Operations and Performance (Red Reply), Software Development Life Cycle (Comsysto Reply), Security (Spike Reply) and Cloud Native Experience and Connectivity (Storm/Cluster/Go Reply).

Open Source, Kubernetes, and Cloud Native are just the connectors for all of those topics.

But what are my takeaways from KubeCon?

Market + Products

• Internal Developer Platforms (IDP) have become omnipresent, I counted at least 10 products/startups - however, looking into the community Backstage is still the way to go
• AI Operations - AI-enhanced platforms to simplify operations, mainly with/on Kubernetes, because the standardization and unified specification the cloud native ecosystem brings make it easy for automated ops
• Observability - OTEL has become the only truth, every observability vendor has support for it, open source projects expect it as default and this was also the tenor of all the vendors showcasing on KubeCon

Talkaways (Takeaways)

• Backstage as Developer Portal has become the anchor point for many engineering centers - The new backend and frontend plugin systems allow a better utilization, LLMs are moving in as assistants and it even improves compliance regulations
• Platform Engineering for GenAI is the major enabler for any large-scale LLM-using company. Nvidia, Cloudera, Databricks, Intuit, and Reddit, to just name a few, are all built on Kubernetes based platforms, powered by Platform Engineers
  • Optimizing and autoscaling for LLM load balancing
  • Optimizing LLM performance with OTEL
  • Envoy becomes an AI Gateway
  • Shifting from cloud based ML Engineering to Kubernetes/Cloud Native ML Engineering simplifies architectures, automation and costs
  • GPU partitioning and cost transparency
  • Caching/KVs for LLM improve performance drastically
• Quantum-Safe applications in Kubernetes - sounds complicated, is dead simple. NIST will not allow encryption methods that are not quantum-proof in the future.
• Multi-Cloud, multi-cluster Management has evolved into a commodity

In a live stream discussion with my fellow CNCF Ambassadors Dotan and Viktor (watch on YouTube), we summarized this KubeCon also as the inflection point from this continuously innovating and young community to a mature and evolving community. This is mainly reflected by the successful enterprises showcasing their wins without dropping their learnings.

Interesting Open Source Tools and Announcements

• Kanister - application data management solution for high resiliency of stateful apps like DBs, presented on Kubecon in context of Vector DBs and globally scaling RDS
• ChRIS Project - FOSS project for computational research and medicine - something for our Life Science friends out there?
• Kueue - Kueue is a cloud-native job queueing system for batch, HPC, AI/ML, and similar applications in a Kubernetes cluster. But it goes far beyond that.
• Karmada - Karmada (Kubernetes Armada) is a Kubernetes management system that enables you to run your cloud-native applications across multiple Kubernetes clusters and clouds, with no changes to your applications.
• Ray - Ray manages, executes, and optimizes compute needs across AI workloads

Random Stuff

NVIDIAs Architecture for a Digital Human.

I wrote a book on Platform Engineering for Architects, Crafting modern platforms as a product, with Andi Grabner from Dynatrace and Hilliary Lipsig from Red Hat. Dynatrace was so kind to sponsor 400 books (in the picture you see only 60) for a book signing during the booth crawl.

My predictions

I guess in future KubeCons, we will see an increasing amount of data management (large-scaled DBs and Vector DBs) as well as Gen/AI Platforms with Kubernetes. From all the end users and market research, the message was clear: Platform Engineering and IDPs are the solid foundation for GenAI. Over 50% of corporations are using K8s for their GenAI platforms, while end users like Reddit, Bloomberg, or the Boston Children's Hospital only become better after using cloud-native platforms and releasing their cloud complexity.

Next KubeCon

The next KubeCon + CloudNativeCon will be in the week of the 1st of April 2025 in London. 

Platform Engineering for GenAI

If you want to learn more about how Platform Engineering becomes your reliable foundation for your GenAI/LLM workload, get in touch with us. We can show you some basic blue prints and pitfalls.