Connect with us

Tech

Exclusive: Cordons and Drains: How to Prepare a Kubernetes Node for Maintenance – TalkOfNews.com

Published

on

Cordons and Drains: How to Prepare a Kubernetes Node for Maintenance

#Cordons #Drains #Prepare #Kubernetes #Node #Maintenance

Graphic with the Kubernetes logo

Kubernetes Nodes need occasional maintenance. You could be updating the Node’s kernel, resizing its compute resource in your cloud account, or replacing physical hardware components in a self-hosted installation.

Kubernetes cordons and drains are two mechanisms you can use to safely prepare for Node downtime. They allow workloads running on a target Node to be rescheduled onto other ones. You can then shutdown the Node or remove it from your cluster without impacting service availability.

Applying a Node Cordon

Cordoning a Node marks it as unavailable to the Kubernetes scheduler. The Node will be ineligible to host any new Pods subsequently added to your cluster.

Use the kubectl cordon command to place a cordon around a named Node:

$ kubectl cordon node-1
node/node-1 cordoned

Existing Pods already running on the Node won’t be affected by the cordon. They’ll remain accessible and will still be hosted by the cordoned Node.

You can check which of your Nodes are currently cordoned with the get nodes command:

$ kubectl get nodes
NAME       STATUS                     ROLES                  AGE   VERSION
node-1     Ready,SchedulingDisabled   control-plane,master   26m   v1.23.3

Cordoned nodes appear with the SchedulingDisabled status.

Draining a Node

The next step is to drain remaining Pods out of the Node. This procedure will evict the Pods so they’re rescheduled onto other Nodes in your cluster. Pods are allowed to gracefully terminate before they’re forcefully removed from the target Node.

Advertisement

Run kubectl drain to initiate a drain procedure. Specify the name of the Node you’re taking out for maintenance:

$ kubectl drain node-1
node/node-1 already cordoned
evicting pod kube-system/storage-provisioner
evicting pod default/nginx-7c658794b9-zszdd
evicting pod kube-system/coredns-64897985d-dp6lx
pod/storage-provisioner evicted
pod/nginx-7c658794b9-zszdd evicted
pod/coredns-64897985d-dp6lx evicted
node/node-1 evicted

The drain procedure first cordons the Node if you’ve not already placed one manually. It will then evict running Kubernetes workloads by safely rescheduling them to other Nodes in your cluster.

You can shutdown or destroy the Node once the drain’s completed. You’ve freed the Node from its responsibilities to your cluster. The cordon provides an assurance that no new workloads have been scheduled since the drain completed.

Ignoring Pod Grace Periods

Drains can sometimes take a while to complete if your Pods have long grace periods. This might not be ideal when you need to urgently take a Node offline. Use the --grace-period flag to override Pod termination grace periods and force an immediate eviction:

$ kubectl drain node-1 --grace-period 0

This should be used with care – some workloads might not respond well if they’re stopped without being offered a chance to clean up.

Solving Drain Errors

Drains can sometimes result in an error depending on the types of Pod that exist in your cluster. Here are two common issues with their resolutions.

1. “Cannot delete Pods not managed by ReplicationController, ReplicaSet, Job, or StatefulSet”

This message appears if the Node hosts Pods which aren’t managed by a controller. It refers to Pods that have been created as standalone objects, where they’re not part of a higher-level resource like a Deployment or ReplicaSet.

Kubernetes can’t automatically reschedule these “bare” Pods so evicting them will cause them to become unavailable. Either manually address these Pods before performing the drain or use the --force flag to permit their deletion:

$ kubectl drain node-1 --force

2. “Cannot Delete DaemonSet-managed Pods”

Pods that are part of daemon sets pose a challenge to evictions. DaemonSet controllers disregard the schedulable status of your Nodes. Deleting a Pod that’s part of a DaemonSet will cause it to immediately return, even if you’ve cordoned the Node. Drain operations consequently abort with an error to warn you about this behavior.

You can proceed with the eviction by adding the --ignore-daemonsets flag. This will evict everything else while overlooking any DaemonSets that exist.

$ kubectl drain node-1 --ignore-daemonsets

You might need to use this flag even if you’ve not created any DaemonSets yourself. Internal components within the kube-system namespace could be using DaemonSet resources.

Advertisement

Minimizing Downtime With Pod Disruption Budgets

Draining a Node doesn’t guarantee your workloads will remain accessible throughout. Your other Nodes will need time to honor scheduling requests and create new containers.

This can be particularly impactful if you’re draining multiple Nodes in a short space of time. Draining the first Node could reschedule its Pods onto the second Node, which is itself then deleted.

Pod disruption budgets are a mechanism for avoiding this situation. You can use them with Deployments, ReplicationControllers, ReplicaSets, and StatefulSets.

Objects that are targeted by a Pod disruption budget are guaranteed to have a specific number of accessible Pods at any given time. Kubernetes will block Node drains that would cause the number of available Pods to fall too low.

Here’s an example of a PodDisruptionBudget YAML object:

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: demo-pdb
spec:
  minAvailable: 4
  selector:
    matchLabels:
      app: my-app

This policy requires there be at least four running Pods with the app=my-app label. Node drains that would cause only three Pods to be schedulable will be prevented.

The level of disruption allowed is expressed as either the maxUnavailable or minAvailable field. Only one of these can exist in a single Pod Disruption Budget object. Each one accepts an absolute number of Pods or a percentage that’s relative to the total number of Pods at full availability:

  • minAvailable: 4 – Require at least four Pods to be available.
  • maxUnavailable: 50% – Allow up to half of the total number of Pods to be unavailable.

Overriding Pod Disruption Budgets

Pod disruption budgets are a mechanism that provide protection for your workloads. They shouldn’t be overridden unless you must immediately shutdown a Node. The drain command’s --disable-eviction flag provides a way to achieve this.

$ kubectl drain node-1 --disable-eviction

This circumvents the regular Pod eviction process. Pods will be directly deleted instead, ignoring any applied disruption budgets.

Bringing Nodes Back Up

Once you’ve completed your maintenance, you can power the Node back up to reconnect it to your cluster. You must then remove the cordon you created to mark the Node as schedulable again:

$ kubectl uncordon node-1
node/node-1 uncordoned

Kubernetes will begin to allocate new workloads to the Node, returning it to active service.

Summary

Maintenance of Kubernetes Nodes shouldn’t be attempted until you’ve drained existing workloads and established a cordon. These measures help you avoid unexpected downtime when servicing actively used Nodes.

Advertisement

Basic drains are often adequate if you’ve got capacity in your cluster to immediately reschedule your workloads to other Nodes. Use Pod disruption budgets in situations where consistent availability must be guaranteed. They let you guard against unintentional downtime when multiple drains are commenced concurrently.


Tech

Exclusive: The iPhone 14 Pro’s camera cutout might look a lot like a notch after all – TalkOfNews.com

Published

on

By

Weibo leak of iPhone 14 Pro design and packaging sticker

#iPhone #Pros #camera #cutout #lot #notch


Continue Reading

Tech

Exclusive: Now you can watch Paramount Plus and Showtime in a single app – TalkOfNews.com

Published

on

By

Now you can watch Paramount Plus and Showtime in a single app

#watch #Paramount #Showtime #single #app

You can now access the content packed into Paramount Plus and Showtime from a single app. First bundled as a subscription last September, Paramount and Showtime’s catalogs have been separated between two apps for viewers in the US, but viewers can now access the full roster of TV shows and films from the Paramount Plus app.

For new subscribers, the Paramount Plus and Showtime bundle is available at a discounted rate until October 2nd, at $7.99 per month for the Essential plan (which includes ad breaks) with Showtime and $12.99 per month for the Premium plan with Showtime that adds more sports, live broadcasts of your local CBS station, and downloads for offline viewing on mobile, and mostly removes ads, except on live TV and certain shows. After October 2nd, the regular pricing resumes with the combined Essential tier at $11.99 per month and the Premium plan for $14.99 monthly.

Paramount Plus as a standalone service will continue to be available on two subscription tiers without Showtime: the ad-supported Essential plan ($4.99 per month) and the ad-free Premium tier ($9.99 per month).

“This singular user experience streamlines sign-up and enhances discovery, and this lower price will allow more households to enjoy with exceptional entertainment offering,” Paramount Global Streaming president Tom Ryan said in a press release announcing the new service.

iPhone screen showing the paramount plus app highlighting the showtime series your honor

Paramount Plus and Showtime were previously bundled in two apps under one price.
Image: Paramount
Advertisement

In case you haven’t been streaming Star Trek and Halo TV shows for the past few years, here’s the rundown on what these apps are. Paramount Plus is the rebranded version of CBS All Access. Its parent company, ViacomCBS, rebranded to Paramount earlier this year to further focus its attention on streaming. In May, Paramount Plus reported that its subscriber count grew to almost 40 million during its first quarter, and in June, Paramount Plus said it planned “to commission” 150 international original series, adding more to its current roster.

Laptop screen with Paramount Plus on the screen highlighting the Showtime series Billions

The service will be available at a lower price until October 2nd.
Image: Paramount

In mid-August, the streaming service partnered with Walmart to bundle its service with Walmart Plus, and it has been included in other packages like this one with T-Mobile’s wireless service. These bundles don’t cost quite as much as cable once did, but pulling the offerings closer together is similar to Disney’s approach with the combination of Disney Plus, Hulu, and ESPN Plus, while HBO Max will be absorbed into a new combo app with Discovery Plus in 2023. Netflix, meanwhile, is going it alone but has plans to roll out an ad-supported streaming package, and has started to offer games as an added feature for your subscription fee.

Advertisement
Continue Reading

Tech

Exclusive: Who Should Be the Next James Bond? io9 Picks – TalkOfNews.com

Published

on

By

Who Should Be the Next James Bond? io9 Picks

#James #Bond #io9 #Picks

Tech. Science. Culture.

ShopSubscribe

Tech. Science. Culture.

AboutGizmodo AdvisorGizmodo Store

Explore our other sites

  • avclub
  • deadspin
  • gizmodo
  • jalopnik
  • jezebel
  • kotaku
  • lifehacker
  • quartz
  • theroot
  • thetakeout
  • theonion
  • theinventory

  • Off
  • English

Movies

Our editors offer their choices for which actor should follow in the footsteps of Daniel Craig.

By
Artem Golub
Continue Reading

Exclusive

Copyright © 2022 Talk Of News.