07. How the Kubernetes scheduler works

Hi, I'm Brendan and I'm going to talk to you today about how the scheduler in Kubernetes works.

First of all it's probably worth noting what the scheduler actually is.

Well, in Kubernetes you have a bunch of machines, and a single container that has a single pod that a user is asked the Kubernetes API to run.

Well now the scheduler's job is to figure out if we have a scheduler over here, it says where in which specific machine should this container actually execute.

Should I place it here, should I place it here, should I place it here or should I place it here? This is one of the more important jobs that any binary does in Kubernetes because without it the containers really would have no idea where they should be executing.

It's very similar to the fact that in most modern PC's, your laptop or your phone or anything else there's multiple cores that a program could execute on, and the operating system scheduler is responsible for determining when and where a particular program should execute.

So, let's take a look at how the scheduler works.

Well, ultimately when you create a pod that object exists in the Kubernetes API, but it doesn't have a node associated with it.

The API object for that pod doesn't have a machine that has been scheduled onto.

The scheduler is continually watching for pods that are in this condition.

It's continually watching for pods that have been created but haven't been successfully scheduled.

It is also continually watching the state of all of the machines.

Because of course where to schedule the container has a lot to do with the current existing state of a particular machine.

What it's thinking about when it makes a scheduling decision, is a mixture of two things.

There are predicates, and there are priorities.

Predicates are hard constraints.

They are things that cannot be violated.

It might be something like I need to run on a machine with at least four gigabytes of memory.

If a machine doesn't have four gigabytes of memory there is no way that that container can be scheduled there.

They eliminate machines entirely from consideration by the scheduler.

Priorities on the other hand are softer.

They say things like, "It would be nice if my application was spread across a large number of failure domains.

It would be nice if every machine in the cluster roughly had the same amount of workload assigned to it." So, priorities are soft constraints.

They can be violated but give you some sense of the badness or goodness of either satisfying or not satisfying the constraint.

The hard constraints are a mix of things that are system supplied and things that a user can supply.

So, for example a system supplied hard constraint might be what I was talking about earlier, the memory requirement.

When you create a pod and you say I need four gigabytes of memory that implies a hard requirement.

But likewise, when you create a pod you could say I want to have a node selector, and that node selector is SSD.

I want to only schedule onto machines that have a label of SSD that indicates that they have flash hard drive available to them.

In that case, this hard constraint is a node selector hard constraint that says only machines that have that label pass the predicate for this particular node selector.

That's specified by the end-user.

Now, in terms of soft constraints we mentioned one which is something like spreading.

Spreading is the desire in general for an application to be spread across multiple machines.

So, that if one machine happens to fail the entire application doesn't fail.

This is obviously not a requirement if I have no space, if I had absolutely no space to run three replicas of your application without putting two of them on the same machine, would you rather that the scheduler didn't run one replica? No, I think you'd rather than it placed two replicas onto a specific machine.

So that's why it's a soft constraint that can be violated.

Another example of a soft constraint might be using what's called a taint.

So, a machine can have a taint which says, "Slightly sick".

Maybe a machine software has detected that this machine seems like it might be going bad.

It's not quite so bad that we need to take it out of rotation, but we've noticed an increase number of error rate when talking to memory for example or we've noticed that there may be some bad sectors on one of the disks.

A soft constraint would say, "I would generally prefer that it's not a sick machine." But again, if this is the only place where you can possibly play something, it's okay with me.

Now the actual scoring associated with this priority can give you some notion of how the scheduler behaves.

So, if I prefer spreading, if spreading is worth more to me than avoiding a sick machine, then I'm going to actually place my application here, not here because I prefer spreading.

I'll take the fact that the machine is a little bit sick.

If on the other hand, I prefer to avoiding sick machines over my desire for spreading, well, then I'm going to not place the container onto the machine that has been marked sick.

And instead place the container over here where I violated the spreading constraint but I have satisfied the sickness constraint.

Ultimately, it's this mixture of predicates that eliminate whole machines from consideration and priorities which give you the relative value of the different machines that indicate where the container is actually going to be scheduled.

In fact, the way that this works is the first cut is; if I start out with a complete list of nodes, I'm going to filter those nodes by the predicates then I'm going to sort those filtered nodes by the priorities and then ultimately the number one node that comes out of this sort, is going to be the place where I schedule my container.

So, I hope that's given you a rough introduction to the role of the scheduler in Kubernetes, and how it figures out where to place the containers you create through the Kubernetes API.