K8s Basics
So we are back to continuing with k8s basics dropping media forge in between 🥲🙂
Linux Basics
Back to understanding linux basics for k8s. They include processes, ports, networking, DNS, file systems, permissions, env variables, systemd, services, logs
- Processes:
- A combination of program instructions, memory used, CPU execution state, open files, network connections, security identity, env variables is called a process
- Process handling in physical rack server
- CPU core execute process instruction
- RAM store process data
- Disk (HDD/SSD) store process file
- NIC/network card process network traffic (send and receives network packets)
- Linux kernel schedue CPU time between processes
-
Physical rack visualization
|-------------------------| | CPU | | RAM | | SSD/HDD | | NIC | | |----------------| | | | Linux Kernel | | | |----------------| | |-------------------------|- NIC
- Network interface card
- Hardware for network communication like via ethernet ports, fibre ports, wireless radio hardwood
- VIsualization
Nestjs app -----| | |-----Application v | Socket API -----| | v Linux Kernel Networking Stack | v NIC driver -----------------| | | v | NIC hardware |------- Hardware to communicate via sockets | | v | Ethernet cable/ network ----| - NIC
- Ports:
- Communication endpoint
- Use ports to decide which process receives incoming n/w traffic
- Kernel does forwarding of n/w from ports to its associated process
- Networking
- Just basic concepts of who connects by what
- Eg: Ethernet cables/fiber connects server, cables connect to switches, etc
- DNS (Domain Name System)
- Converts domain names to IP address, via which servers communicate
- File systems
- organize data on storage systems
- eg: SSD, HDD, etc
- Permissions
- Eg: -rwxr-x—
- Rules followed in permissions:
`- rwx r-x r--` | |-| |-| |-| | | | | | | | v | | | other permission | | v | | group permission | v | owner permission v Filetype - filetype
-: regular filed: directoryl: symbolic link
- permissions -
rread,wwrite,xexecute,-no permission - Numeric representation
r- 4w- 2x- 1rwx- 7r-x- 5---- 0- rwx r-x r--- 750- Can be used as
"chmod 750 filename"
- Env variable - As we know it
- systemd / services
systemd- linux service manager
- controls starting, stopping, restarting services, etc.
eg services-> nginx, ssh, docker, etc.command->"systemctl status"
- Logs - As we know it
Containers
Images, containers, layers, volumes, networks, entrypoints, env variables, multi stage builds.
- Images
- packaged filesystem
- metadata used to create container
- contains application files, libraries, runtime binaries, default config, startup command.
- readonly templates
- when container starts
- mount image filesystem
- add a writable layer
- copy files from readonly to writable files & run from there
- start process
- Container Runtime
- eg: docker, containerd
- creates containers (processes from image), configure namespaces/cgroup, mount filesystem, set networking, start process
- Container
- Process made from image
- Eg: running “docker run redis”..
- creates redis process
- creates isolated filesystem
- creates virtual n/w interface
- creates writable container layer
- Layers
- something with which images are built.
- each Dockerfile instruction creates a build tor file/file/folder, that is a new filesystem layer.
- Each tor file/folder created is immutable, and each of them is called a layer.
- Container stacks them using OverlayFS (a linux filesystem tech that allows multiple directories/filesystems to be stacked together into one unified filesystem view).
- Volumes
- Persistent storage for containers outside container lifecycle.
- Networks
- Docker creates virtual networking using:
- linux bridges
- Virtual Ethernet interfaces (Veth)
- NAT rules
- routing tables
- Linux bridge:
- virtual software switch to which containers connect to; called “docker0”
- created by docker.
- Visualisation
`container A` ---\ `container B` ----> `"docker0" bridge --> `Host NIC` --> `Internet` `container C` ---/- Part of virtual network interfaces
- Virtual Ethernet interfaces (Veth pair)
- created by docker during container start
- called as “virtual cable pair” (veth pair)
- part of virtual network interfaces.
- has one side inside container (
eth0), one side outside container (veth7abc123) - visualisation:
container eth0 <--- virtual cable ---> host veth xyz
- NAT rules (Network Address Translation)
- Networking Address Translation rules
- Rules/Instructions to how packets (that travel through sys) must be rewritten.
- This rewriting allows containers w/ private IPs to access host’s internet via host IP (translation).
- Rules exist because not every packet must be translated.
- Eg commands: SNAT, DNAT, MASQUERADE, REDIRECT.
- This is not just of docker but of host system, linux in general, VPN, etc too. (used everywhere for rewriting packet addresses)
- It is a subset rule among many other rules in linux iptables.
- ip-table
- tool/sys/networking/firewall utility.
- iptable defines what happens to packets.
- used for managing packet rules.
- eg:
iptables |--- filter rules |--- forwarding rules |--- logging rules |--- NAT rules- eg of iptable rules:
- Allow incoming SSH traffic
- Rewrite packet source IP.
- . . . etc
- Answers questions like allow packet?, block packet?, rewrite IP?, forward IP?, log packet?
- ip-table
- Routing Tables
- Answers questions like where to hop next, which network interface should transmit the packet, etc.
- decides where the packets go to.
- In terms of docker, this decides which packets go to that container related packets go to docker0 (linux bridge), while internet traffic go to router etc.
- Docker creates virtual networking using:
- Entry Points
- main command started when container launches.
- Eg: ENTRYPOINT [“node”, “server.js”]
- When the process marked as entrypoint (here it is “node server.js”), it becomes PID 1 process.
- Env Variables (same we know it)
- Multi stage builds
- We can have multiple stages in dockerfile by writing like this - “FROM xyz as abc”.
- Generally all commands in dockerfile creator makes one layer.
- All layers all together makes one stage.
- And this stage builds itself to create one image which is referenced by docker to run in the container.
- With multistage builds, only the final stage is referenced as the image created by the dockerfile
- During each stage in multistage builds, multiple stages (few stage images / temporary stages) are created (collection of layer files) and stored in disk. But only final stage image is exposed to docker & becomes the o/p.
- Benefit : smaller containers, faster deployments, reduced attack surface, lower memory/storage usage.
Purpose of K8s
Problem and their solutions via k8s
- Container crashes —> self healing
- w/o containers K8s, need to manually up stuff
- K8s continuously watches containers
- w/ containers K8s, RAM etc re-allocation, filesystem remounting, network interface recreation etc is taken care of.
- multiple server -> scheduling
- K8s simplifies scheduling pods into free servers w/o having to manually track it via scheduler
- Traffic routing -> services
- K8s services allow having stable traffic routing to healthy BE pods w/o manually having to handle it.
- Scaling -> Replica Set via Services
- Distribution of load across back servers automatically done by K8s w/o manual intervention under higher load via ReplicaSet
- Config management -> configmaps / Secrets
- Diff env config & secrets injected into container memory w/o rebuilding using K8s.
- Deployments -> Rolling updates
- Updates applications w/o downtime via K8s.
- Process: — start new pod, wait until healthy, stop old pod, repeat gradually.
- Service discovery -> DNS
- K8s provides internal DNS to communicate set of pods even when they move frequently b/w back servers.
- Process: — Application attempts to DNS lookup, req sent through DNS Virtual n/w, cluster DNS responds, app server receives current service IP.
- Resource Control -> Requests / limits
- To avoid applns from competing for hardware resources, we can set request limits & limits for cpu, memory, etc in k8s deployment manifest (or any other manifest that has deployment spec). This set hardware resource limits on containers (not on pod or node).
- Receiving K8s Architecture First
- Cluster
- group of physical/virtual servers working together to run applications
- Consists of control plne and worker nodes
- Control plane
- management brain of K8s.
- doesn’t normally run an appln container
- Responsibilities :- accept requests, store cluster state, make decisions, monitor cluster status, command worker nodes.
- API server
- front door of K8s (and through it)
- Everything talks to it :- kubectl, scheduler, controller manager, kubelet - Responsibilities — validate req, authenticate user, authorize action, store resource obj.
- etcd
- K8s db. Stores cluster state
- Also writes to physical storage devices (CRI, CSI, etc)
- scheduler
- decides which node the pod must go to.
- a process running on control plane
- Controller manager
- continuously check desired state vs actual state
- a process on control plane that keeps doing - observe, compare, correct (called as reconciliation loop).
- reason why K8s correct itself
- kubelet
- agent running on every worker node.
- receives instruction from control plane
- ensures containers are running
- Container runtime
- runtime that actually runs containers
- eg: containerd, CRI-O
- Process :- Pull required image, create filesystem, configure networking, start process
- kube proxy
- manage n/w rules for services.
- when traffic arrives at a service, kube proxy routes it to correct pod based on iptables / networking rules.
- Cluster
Most important K8s concept (Desired state reconciliation)
- Desired state reconciliation
- We declare desired state (eg: replicas 3) and K8s try to match it to reality via reconciliation loops
- Self healing
- basically reconciliation fixing failures
- cluster repairing itself
- Auto recreation
- K8s automatically recreates missing resources
- Eg: Someone manually deletes a pod. But desired state says it needs 3 replicas, & reality has only 2 pods, so controller immediately creates another pod.
- Controllers
- software that perform reconciliation
- Reconciliation loop :- controller continuously watching, comparing, acting & repeating
- A linux process running on control plane service
- Deployments
- stores desired appln state
(deployment is the instruction, controllers perform the instruction)
- stores desired appln state
- Scaling
- changing desired capacity
- eg: from replica: 3 to replica: 10
- Rollback
- change desired state to prev version
Learn Objects
- Phase 1 - Pods
- what:
- smallest deployable unit in K8s.
- The thing that k8s schedules into worker nodes
- contains >= 1 containers
- not a process itself.
- a K8s object that run & manages the process (aka :- container)
- pod & container :-
- pod = wrapper
- container = isolated process env
pod container A container B |-------------| |-------------| | process A | | process B | |-------------| |-------------|- a pod provides shared n/w, shared storage, shared lifecycle.
- multi containers Pods - pods can have multiple containers
- pod lifecycle Green
- Pending -> Pod created, image downloading, waiting for node.
- running -> containers started, app “running”
- Failed -> container crashed - pod failed
- logs — “kubectl logs my-pod”
- Exec into pod :
- “kubectl exec -it my-pod – sh”
- allows to start a command inside a running container.
- what:
- Phase 2 — Deployment
- Replica set - ensure specific no. of pods exist
- Scaling - Change no. of pod replicas
- Rolling updates - update application version gradually
- Rollback - revert back version
- Phase 3 — Service
- Service - Provides a reliable n/w endpoint for pods
- Cluster IP :
- IP for internal cluster communication.
- So that traffic remains inside cluster networking & no external traffic reaches it.
- Nodeport
— port used to expose a service through a worker node’s IP.
— can be accessed from outside cluster cluster IP
— common uses includes connecting external load balancers, for development & testing quickly.- expose app “1010 DNS or load balancers, etc.
- Eg for NodePort & Node IP. “http://Node-IP:30050”
- worker node listens to this TCP port.
- Load balancer
- for exposing services externally. & forward traffic to cluster nodes
- Service discovery
- Service discovery is done via service names
- It is done so that pods can move around w/o breaking communication
- A service can point to multiple pods like redis-service can have multiple replicas of pods, but all can be accessed by same name.
- kube-dns/coredns
— DNS server K8S
- just another pod running on worker node CPU’s & RAM
- Label selectors
— labels used for services to find pods
- Eg:
Pod "label: app: backend" Service "selector: app: backend"Now K8S matches them.
- labels are metadata stored in it.cd
- Eg:
- Phase 4 — Configmap & Secrets
- Env variable injection
- Eg: Config map contain env variables, containers can access them inside process env.[var-name]
- Mounted files
- Provide config as files instead of env variables
- Eg: Config map has app.conf. It is mounted in containers as /etc/config/app.conf
- to app “it looks like normal file on disk”
- (means file contents become visible inside filesystem namespace, just like a normal file).
- Separation of config from image
— container image is kept independent from configuration
— ie, when config changes per namespace like diff for prod & dev, the image of container is unaffected. - Secrets
- store sensitive config
- consumed as env variables or mounted files.
- Env variable injection
- Phase 5 — Ingress
- Reverse proxy
— receive req & forward it to execution
- Ingress controller: is a process running on worker node CPUs.
- Eg: user requests at https://example.com ingress receives it & forwards it to frontend-service.
- Routing
— a thing that reverse proxy does.
- deciding which service gets the incoming req.
- Domain name
— human readable names created from Service names instead of IP addresses.
- DNS resolves the domain to the ingress endpoint
- Traffic then flows through switches, routers, NICs, and into the cluster
- TLS termination — handle HTTPS encryption
— TLS = Transport layer security.
— Ingress terminates TLS (stops encrypted traffic & decrypts it by security)- it decrypts & forwards the req internally.
- NGINX ingress controller
- perform the ingress functionality based on ingress obj stored in etcd.
- Reverse proxy
— receive req & forward it to execution
- Phase 6 — Storage
- Volumes
— provide storage to pods
— Volume ultimately resides on SSD, NVMe (solid state device), NVMe, & HDD, n/w storage - Persistent Volume (PV) — declares/represents actual storage available to the cluster.
- Persistent Volume Claim (PVC)
- Storage requested by pods
- Eg: Pod requests “I need 50GB”. K8S find a matching PV & binds Pod & PV. Pod don’t care which disk provides the storage.
- Stateful Workloads
- Apps / Pods which are not stateless
- Eg: PostgreSQL, MySQL, MongoDB, redis
- When these pods restart, the process change from old to new container, but same volume/storage is used.
- Volumes
— provide storage to pods
- Phase 6 — Networking Fundamentals
- Pod net networking
- Every pod gets its own IP & each one communicates to other via these IPs.
- Packets of data need to travel through these to reach from one pod to another via network :-
Pod 1 - VETH (virtual n/w interface) - worker node NIC - switch - other worker node - destination pod.
- Service virtual IPs
- Eg: “backend-service”
- stable addresses for a group of pods.
- kube-proxy
— routing btw pods internally based on service names.
- it also configures networking rules on every worker node.
- CNI plugin
- Container Networking Interface
- Create Pod networking.
- Process of CNI working (what all CNI does)
- when a new pod starts
- CNI allocates an IP
- create network interface
- configure routing
- connect pod to cluster N/W.
- when a new pod starts
- Points to note:
- CNI (container n/w interface) is a specification, by a set of plugins.
- it creates/configures n/working for containers & pods.
- CNI plugin is a plugin that resides on every K8s node. It is an executable file. unlike API objects or controllers, which create/configure upon running the file.
- It doesn’t generally come with K8s cluster when we create one. A ductii controllers and such. A administrator generally need to set it up.
- Commonly installed CNI plugins :- Calico, Cilium, Flannel. Page 2(2)
- But many providers provide auto installs, many things for developers thus it may feel like built in.
- Providers that auto installs include :- minikube kind, EKS, GKE, AKS. DNS Resolution - convert name into IP addresses.
- Pod net networking
- Phase 7 - Resource management
- Requests :
- to declare minimum resources required (pod-level)
- eg:
requests: cpu: 500m memory: 512Mi - this means: pods node needs 0.5 CPU & 512 MB RAM.
- Limits :
- to declare maximum resources allowed.
- eg:
limits: cpu : 1 memory : 1Gi - which means node needs max 1 CPU & 1 GB RAM
For nodes, resources are allocated via “capacity” and “allocatable resources” To decide placement of pods in nodes, scheduler adds the requests & limits of pods/containers & then compares with node allocatable capacity
- cpu throttling
- Prevent containers from exceeding CPU limits
- Process :
- Developer sets CPU limit
limits : cpu : 1" - kubelet reads pod spec.
- Container runtime configures cgroups.
- runtime creates linux cgroup setting for container
- linux kernel tracks cpu usage by watching how much CPU time certain cgroup consumes.
- Container reaches its quota C like if cpu is 1; but container consumes 100ms of cpu time (quota) every 100 ms window (period).
- Kernel throttles.
- kernel blocks further cpu execution. Container waits until the next period begins. (means process is either killed nor pod is contacted. Process sleeps.)
- Developer sets CPU limit
- OOM killed
- out of memory killed
- purpose is to protect node from memory exhaustion
- When RAM usage exceeds by containers, kernel kills the containers process. Then the pod may be restarted
Note
cpu limit exceed -> process only steps (Throttle)
memory limit exceed -> process killed (oom)
- scheduling
- K8s scheduler choosing which node will run the pod.
- Node Pressure:
- Purpose is to detect overloaded nodes
- Common pressure types: Memory pressure, disk pressure, PID pressure.
- e.g. K8s servers have RAM = 128 GB, used RAM = 127 GB
- node enters memory pressure
- K8s may evict nodes to protect the node.
If no free nodes 10% full memory exists on the cluster, then the Page 5exists on the cluster, then the new pod goes into pending state (In case of fixed replicaset with stateless deployment pods)
- Requests :