Infrastructure / ARTICLE

NVIDIA Kubernetes Engine Preview: Managed Kubernetes on Your Own GPU Infrastructure

NVKE preview runs a managed Kubernetes plane on NVIDIA GPU infrastructure with independent workload clusters, SPIFFE/SPIRE node identity, and NICo as the only compatible provider; API authentication is off by default and management-plane upgrades are not yet available.

What the preview includes

NVIDIA has released NVKE as a preview for running a managed Kubernetes service on self-owned NVIDIA GPU infrastructure. Platform teams install the management plane; each workload cluster receives its own Kubernetes control plane, etcd, and node pools. Customers access their clusters through the Kubernetes API and kubectl. The preview status means APIs, configuration, and behavior can change between releases, and the project does not recommend NVKE for production workloads.

The repository offers three starting points. A local quickstart evaluates NVKE without GPU hardware on simulated machines. It targets a Linux host or VM with amd64 or arm64, at least 6 CPUs, 12 GiB free memory, and 100 GiB free disk, and requires Docker, virtual TPM support, and the tools listed in the guide. Users follow the quickstart through workload-cluster creation and confirm that cluster nodes report Ready. MacOS users can use the local site guide, which runs the evaluation inside a Colima virtual machine. For a real installation, operators supply machines, networking, node images, and registries, and configure access to NVIDIA Infra Controller (NICo). NICo is the only compatible infrastructure provider at preview. The platform and version reference lists validated combinations and pinned dependencies.

Architecture and lifecycle

NVKE is described in the repository as a “self-hosting factory for Kubernetes clusters,” not a Kubernetes distribution or a Cluster API flavor. A temporary seed cluster stands up a management cluster, and the management plane on it provisions and lifecycle-manages many isolated workload clusters on demand. An installation has two kinds of Kubernetes cluster: the management cluster runs the NVKE management plane and infrastructure controllers, while workload clusters run customer applications. The management plane creates and operates workload clusters through NICo, Cluster API, and the node agent.

Workload cluster lifecycle operations include creating, scaling, and deleting node pools, upgrading Kubernetes, and replacing nodes. The node pool guide documents manual steps and preview limits on replacement and upgrades. Node identity is established through SPIFFE/SPIRE; nodes attest with their TPM before receiving join credentials. Per-cluster encryption of etcd data is available. A companion project, NVIDIA Kubernetes Recipes (NVKR), builds node images and NVKE components from pinned inputs. NVKE deploys those artifacts, while NVKR owns their build and release evidence.

Build and operational constraints

Building from source uses Go 1.27.3, the version pinned in go.work, and GNU Make 4 or later. Public Go modules must be reachable on the first build. Flox is optional and supplies the repository’s development tools; on macOS it also supplies GNU Make 4. make build writes nine binaries to bin/: the node agent, updater, controller, nvkectl CLI, a reboot controller, an OS image check tool, management engine, tenant API, and operator API. Building does not start a management plane or create a cluster. The node agent runs on Linux.

The preview needs explicit operator configuration. API authentication is off after installation, and the API uses plain HTTP. Operators should configure API authentication and protected access before admitting callers. The management cluster does not upgrade, scale, or replace itself, and there is no management-plane upgrade or backup/restore procedure. FDE node images are for evaluation and provide no data-at-rest confidentiality. Documentation covers bootstrapping the management cluster, installing management plane services, building node images, creating and accessing workload clusters, and observing cluster health. These constraints place NVKE at an evaluation stage: the components and workflows can be tested end to end, but the management plane’s own upgrade and recovery paths are not yet defined, so production use remains out of scope.

END
Feedback