Skip to content

Introduction

Since its inception in 2015, Kubernetes has been pretty successful at popularizing the idea of container clusters. Adoption has reached the point that many users are deploying their applications across multiple clusters and are struggling to make it work smoothly.

SIG-Multicluster is a Special Interest Group focused on solving common challenges related to the management of multiple Kubernetes clusters, and applications that exist therein.

Specifically, the SIG aims to find Kubernetes-native ways to:

  • Expose workloads from multiple clusters to each other
  • Share cluster metadata and its place relative to others
  • Generally break down the walls between clusters

The SIG is responsible for designing, discussing, implementing and maintaining APIs, tools and documentation related to multi-cluster administration and application management. This includes not only active automated approaches such as Cluster Federation, but also those that employ batch workflow-style continuous deployment systems like Spinnaker and others. Standalone building blocks for these and other similar systems (for example a cluster registry), and proposed changes to kubernetes core where appropriate will also be in scope.

Problem statement: why multicluster?

There are many reasons to want to run multiple clusters, including but not limited to:

  • Location

    • Latency: it can be necessary to deploy the application as close to the customers as possible.
    • Jurisdiction: it can be mandated to keep user data in-country.
    • Data gravity: data already exists in one provider, but it can be decided to run the application in another environment.

  • Isolation

    • Environment (e.g. dev, test, prod)
    • Performance isolation: a workload may consume too many resources, at the expense of other workloads.
    • Security isolation: sensitive data or untrusted code must be isolated in their own environments.
    • Organizational isolation: teams may have different management domains.
    • Cost isolation: multitenancy can greatly complexify billing management for different teams.

  • Reliability

    • Blast radius: an infrastructure or application incident in one cluster must not impact the whole system.
    • Infrastructure diversity: an underlying zone, region, or provider outage does not bring down the whole system.
    • Scale: the application is too big to fit in a single cluster.
    • Upgrade scope: some parts of the application may require an infrastructure upgrade, that may impact other parts of the application. Having multiple clusters can also avoid the need for in-place cluster upgrades.

Project charter

The following charter defines the scope and governance of the Multicluster Special Interest Group:

  • Support an out-of-tree ecosystem by defining API standards that leave plenty of room for flexibility in their ultimately third party implementation.

  • Protect the known multicluster experience from incompatible changes upstream to single clusters by working with other SIGs.

  • Advocate and provide feedback on proposals for extending the single cluster experience to multicluster i.e. network policy.

Approach

To meet the goals listed above, the SIG-Multicluster team has worked to define three different APIs:

  • About API: allows to uniquely identify clusters within a set of clusters (clusterset)
  • Multicluster Services API: allows to expose services across clusters which are part of a given clusterset.
  • Work API: allows to define the workloads to be deployed across clusters which are part of a given clusterset.
  • ClusterInventory API: simplifies the discovery and management of clusters, offering a comprehensive status overview for both applications and human operators. Work in progress.

To leave room for implementation, SIG-Multicluster does not focus on the implementation of the mechanisms that rely on those APIs. For example, no reference implementation is provided for a cluster registry or for service discovery itself.

Getting started

Whether you are a user interested in using the different APIs or an implementer interested in conforming to the APIs, the following resources will help give you the necessary background:

Watch the video

Contributing

If you are interested in contributing to SIG-Multicluster or building an implementation of one of our APIs, then don’t hesitate to get involved in SIG meetings, issues on projects, or new designs.