Databricks utilizes Kubernetes extensively for orchestrating containerized workloads and managing its infrastructure across multiple cloud providers. Here’s how they leverage Kubernetes:

• Compute Infrastructure: Databricks runs its compute infrastructure on AWS, Azure, and GCP, using Kubernetes to manage containerized services across these platforms.
• Custom OS Images: They develop and manage their own OS images that bootstrap cloud VMs into Kubernetes nodes, ensuring consistency and control over their environment.
• Cluster Management: Databricks operates thousands of Kubernetes clusters and manages millions of pods, using Kubernetes to orchestrate complex workloads efficiently.
• Cloud-Agnostic Abstractions: They create abstractions on top of different cloud-managed Kubernetes services (EKS, AKS, GKE) to simplify application deployment for their engineers.
• Kubernetes Operators: Databricks utilizes the Kubernetes operator pattern to manage complex infrastructure components, building custom operators to control various aspects of their environment.