Kubernetes is shipped with many namespaces some of them are critical for Kubernetes in order to function correctly. Messing around in one of these namespaces can damage the Kubernetes system.
And these are:
default
the home of the homelesskube-system
The namespace for objects created by the Kubernetes systemkube-public
This namespace is created automatically and is readable by all users (including those not authenticated). This namespace is mostly reserved for cluster usage, in case that some resources should be visible and readable publicly throughout the whole cluster. This is useful for exposing any cluster information necessary to bootstrap components. It is primarily managed by Kubernetes itself.kube-node-lease
This namespace holds Lease objects associated with each node. Node leases allow the kubelet to send heartbeats so that the control plane can detect node failure.
All Kubernetes system's namespaces will be regenerated again even when you delete them accidentally that's what Kubernetes components will try to do.
But sometimes if you are unfortunate enough deleting namespaces might be stuck at the terminating stage giving no option to regenerate the namespace again.
So the below cases mention what is the importance of each namespace in order to know what the symptoms might look like.
What is the default
?
default
namespace is used as the default place of any objects you create with no namespace specified.
What is the kube-system
?
Kube-system
is the namespace for objects and service accounts with high-level privileges within Kubernetes. Utilization of the Kubernetes controller stems from this namespace, in other words we will have some issues with the controllers and maybe in deploying new pods/deployment.
not only that this namespace also contains other important objects such as kube-dns
,kube-proxy
,
kube-dns
is the authoritative name server for the cluster domain (cluster.local) and it resolves external names recursively. Short names that are not fully qualified, such as myservice, are completed first with local search paths. more details here & here
kube-proxy
manages the forwarding of traffic addressed to the virtual IP addresses (VIPs) of the cluster’s Kubernetes Service objects to the appropriate backend pods more details here & here
This means you will have trouble with resolving external/internal communications.
How about the kube-public
?
kube-public
contains a single ConfigMap object, cluster-info, that aids discovery and security bootstrap.
All the above namespaces if you tried to delete them the server will respond with
Error from server (Forbidden): namespaces "kube-public" is forbidden: this namespace may not be deleted
Expect the lucky kube-node-release
which was added in Kubernetes v1.14, would get deleted the same as any normal namespace.
What is kube-node-release
?
kube-node-lease
This namespace holds Lease objects associated with each node. Node leases allow the kubelet to send heartbeats so that the control plane (Node controller) can detect node failure.
So what would happen if we deleted kube-node-lease
? usually, Kubernetes will create another one with Lease object for each node, but sometimes the namespace removal gets stuck at the terminating
status.
Which by then we will have a node Lease with an outdated heartbeat which might tell the Node controller that this node is not reachable and impact the overall communication between the nodes.
How to fix namespace removal stuck at terminating
?
Of course, you can try to figure out why the namespace is stuck at terminating but sometimes you can't so here we go with the force delete
create a temp json file
kubectl get namespace <terminating-namespace> -o json >tmp.json
execute the below
$ kubectl proxy Starting to serve on 127.0.0.1:8001
- Edit your tmp.json file. Remove the Kubernetes value from the finalizers field and save the file.
- Make the below command to update the namespace
curl -k -H "Content-Type: application/json" -X PUT --data-binary @tmp.json http://127.0.0.1:8001/api/v1/namespaces/<terminating-namespace>/finalize
Your output will look something like the one below
{ "kind": "Namespace", "apiVersion": "v1", "metadata": { "name": "<terminating-namespace>", "selfLink": "/api/v1/namespaces/<terminating-namespace>/finalize", "uid": "b50c9ea4-ec2b-11e8-a0be-fa163eeb47a5", "resourceVersion": "1602981", "creationTimestamp": "2021-10-18T18:48:30Z", "deletionTimestamp": "2021-10-18T18:59:36Z" }, "spec": { }, "status": { "phase": "Terminating" } }
CoverPhoto by Scott Webb on Unsplash