OKE Node Manager Overview

OKE Node Manager

• Objectives: Enhance OCI Compute and VCN integration for OKE worker nodes with Kubernetes-native management and monitoring.

Features

• Node Monitoring & Operations: Collect health and performance indicators, perform Node operations, including OCA plugin state and GPU information.
• Tunables: Collect key health and performance indicators with optional repair automation.
• Network Tests: Run automated health checks on worker nodes.
• VNIC Attachments: Configure secondary VNIC attachments on worker nodes.
• Interfaces: Configure network interfaces on worker nodes.

Design

• Controller manager: A set of controllers deployed as a single container will watch for create/update/delete events of OKE-managed Kubernetes resources.
• Node Agent: Lightweight DaemonSet for Node network configuration, returns status and health information for the node, and enforces management rules.

Node Agent Options

Option: VCN-Native DaemonSet (target)

• Deploys a container on each worker node to perform tasks, including installing CNI plugin binaries and network configuration.
• Pros: efficient use of existing application with shared function, smaller footprints for service maintenance and user deployment.
• Cons: more impact to existing CNI codebase and application behavior.

Option: new DaemonSet (prototype)

• Background: A prototype exists to demonstrate aspects of node monitoring and control between a controller manager and node agent implementation.
• Proposal: Implement node agent logic in a new DaemonSet, deployed as a cluster add-on.
• Pros: fewer changes to existing components, lower design and implementation effort.
• Cons: higher overhead on worker nodes with multiple add-ons enabled, higher maintenance of added repos/release cycles.

Requirements

• Scale to 10,000 nodes.
• Minimize infrastructure deployment to the customer’s data plane.
• Minimize storage space, reads, and writes to etcd.
• Data can be up to 5 minutes old.
• No need for high availability.
• No need for real-time data.

Vocabulary

• Cloud Controller Manager (CCM): A Kubernetes master component responsible for cloud provider-specific work.
• Container Networking Interface (CNI): A Kubernetes interface for setting up pod networking.
• Custom Resource (CR): An instantiation of a Custom Resource Definition.
• Custom Resource Definition (CRD): A user-defined Kubernetes object definition to be reconciled by an operator.
• Kubernetes Control Plane: Consists of Kube apiserver, Kube controller manager, Kube scheduler, Cloud Controller Manager, and Proxymux.
• Kubernetes Manager Instance (KMI): Servers in the OKE Service tenancy that host the Kubernetes Control Plane.
• Instance Metadata (IMDS): An endpoint that provides metadata for the instance.
• OKE Management Plane: OKE Service that manages KMIs.
• Native Pod Networking (NPN): OKE-offered CNI, where pods talk to each other via native VCN constructs (VNIC's).
• Pod: The smallest unit of application within a Kubernetes cluster.
• Proxymux: A component in the KMI that is responsible for proxying traffic for Kubernetes endpoints that require a long-lived connection.
• Virtual Cloud Network (VCN): An overlay cloud network.

This quiz covers the features and components of OKE Node Manager, including node monitoring, tunables, and network tests. It also touches on design aspects like controller manager and node agent.