| Table of Contents |
|---|
A. root 계정 패스워드 설정
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
[sooabia@k8s-node-1 ~]$ sudo passwd
Changing password for user root.
New password:
Retype new password:
passwd: all authentication tokens updated successfully.
[sooabia@k8s-node-1 ~]$ |
B. 방화벽 해제
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
[root@k8s-node-1 ~]# systemctl stop firewalld
[root@k8s-node-1 ~]# systemctl disable firewalld
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@k8s-node-1 ~]#
|
C. OS 업데이트
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
[root@k8s-node-1 ~]# yum -y update
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
epel/x86_64/metalink | 12 kB 00:00:00
* base: mirror.hostduplex.com
* epel: mirror.steadfastnet.com
* extras: mirror.fileplanet.com
* updates: mirror.fileplanet.com
base | 3.6 kB 00:00:00
epel | 3.2 kB 00:00:00
extras | 3.4 kB 00:00:00
google-cloud-compute/signature | 454 B 00:00:00
google-cloud-compute/signature | 1.8 kB 00:00:00 !!!
google-cloud-sdk/signature | 454 B 00:00:00
google-cloud-sdk/signature | 1.4 kB 00:00:00 !!!
updates | 3.4 kB 00:00:00
(1/10): epel/x86_64/group_gz | 88 kB 00:00:00
(2/10): epel/x86_64/updateinfo | 951 kB 00:00:00
(3/10): base/7/x86_64/group_gz | 166 kB 00:00:00
(4/10): epel/x86_64/primary | 3.6 MB 00:00:00
(5/10): extras/7/x86_64/primary_db | 204 kB 00:00:00
(6/10): google-cloud-compute/updateinfo | 1.2 kB 00:00:00
(7/10): google-cloud-sdk/primary | 54 kB 00:00:00
(8/10): google-cloud-compute/primary | 3.8 kB 00:00:00
(9/10): base/7/x86_64/primary_db | 5.9 MB 00:00:01
(10/10): updates/7/x86_64/primary_db | 6.0 MB 00:00:00
epel 12733/12733
google-cloud-compute 11/11
google-cloud-sdk 341/341
No packages marked for update
[root@k8s-node-1 ~]# |
D. Hosts 설정
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
[root@k8s-node-1 ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
10.128.0.3 k8s-node-1.us-central1-c.c.myk8s-219201.internal k8s-node-1 # Added by Google
10.128.0.2 k8s-master
10.128.0.4 k8s-node-2
169.254.169.254 metadata.google.internal # Added by Google
[root@k8s-node-1 ~]# |
E. Docker 17.03 설치
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
[root@k8s-node-1 ~]# curl https://releases.rancher.com/install-docker/17.03.sh | sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 15249 100 15249 0 0 32628 0 --:--:-- --:--:-- --:--:-- 32723
+ '[' centos = redhat ']'
+ sh -c 'yum install -y -q yum-utils'
+ sh -c 'yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo'
Loaded plugins: fastestmirror
adding repo from: https://download.docker.com/linux/centos/docker-ce.repo
grabbing file https://download.docker.com/linux/centos/docker-ce.repo to /etc/yum.repos.d/docker-ce.repo
repo saved to /etc/yum.repos.d/docker-ce.repo
+ '[' stable '!=' stable ']'
+ sh -c 'yum makecache fast'
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
epel/x86_64/metalink | 18 kB 00:00:00
* base: mirror.fileplanet.com
* epel: d2lzkl7pfhq30w.cloudfront.net
* extras: mirror.fileplanet.com
* updates: mirrors.sonic.net
base | 3.6 kB 00:00:00
docker-ce-stable | 2.9 kB 00:00:00
epel | 3.2 kB 00:00:00
extras | 3.4 kB 00:00:00
google-cloud-compute/signature | 454 B 00:00:00
google-cloud-compute/signature | 1.8 kB 00:00:00 !!!
google-cloud-sdk/signature | 454 B 00:00:00
google-cloud-sdk/signature | 1.4 kB 00:00:00 !!!
updates | 3.4 kB 00:00:00
docker-ce-stable/x86_64/primary_db | 17 kB 00:00:00
Metadata Cache Created
+ sh -c 'yum install -y -q --setopt=obsoletes=0 docker-ce-17.03.2.ce'
warning: /var/cache/yum/x86_64/7/docker-ce-stable/packages/docker-ce-selinux-17.03.3.ce-1.el7.noarch.rpm: Header V4
RSA/SHA512 Signature, key ID 621e9f35: NOKEY
Public key for docker-ce-selinux-17.03.3.ce-1.el7.noarch.rpm is not installed
Importing GPG key 0x621E9F35:
Userid : "Docker Release (CE rpm) <docker@docker.com>"
Fingerprint: 060a 61c5 1b55 8a7f 742b 77aa c52f eb6b 621e 9f35
From : https://download.docker.com/linux/centos/gpg
Re-declaration of type docker_t
Failed to create node
Bad type declaration at /etc/selinux/targeted/tmp/modules/400/docker/cil:1
/usr/sbin/semodule: Failed!
restorecon: lstat(/var/lib/docker) failed: No such file or directory
warning: %post(docker-ce-selinux-17.03.3.ce-1.el7.noarch) scriptlet failed, exit status 255
Non-fatal POSTIN scriptlet failure in rpm package docker-ce-selinux-17.03.3.ce-1.el7.noarch
+ '[' -d /run/systemd/system ']'
+ sh -c 'service docker start'
Redirecting to /bin/systemctl start docker.service
+ sh -c 'docker version'
Client:
Version: 17.03.2-ce
API version: 1.27
Go version: go1.7.5
Git commit: f5ec1e2
Built: Tue Jun 27 02:21:36 2017
OS/Arch: linux/amd64
Server:
Version: 17.03.2-ce
API version: 1.27 (minimum version 1.12)
Go version: go1.7.5
Git commit: f5ec1e2
Built: Tue Jun 27 02:21:36 2017
OS/Arch: linux/amd64
Experimental: false
If you would like to use Docker as a non-root user, you should now consider
adding your user to the "docker" group with something like:
sudo usermod -aG docker your-user
Remember that you will have to log out and back in for this to take effect!
WARNING: Adding a user to the "docker" group will grant the ability to run
containers which can be used to obtain root privileges on the
docker host.
Refer to https://docs.docker.com/engine/security/security/#docker-daemon-attack-surface
for more information.
[root@k8s-node-1 ~]#
##### docker.service enable ##########################################################################################
[root@k8s-node-1 ~]# systemctl status docker.service
● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled; vendor preset: disabled)
Active: active (running) since Mon 2018-10-15 01:12:31 UTC; 3min 56s ago
Docs: https://docs.docker.com
Main PID: 1287 (dockerd)
Tasks: 16
Memory: 20.9M
CGroup: /system.slice/docker.service
├─1287 /usr/bin/dockerd
└─1290 docker-containerd -l unix:///var/run/docker/libcontainerd/docker-containerd.sock --metrics-interval=0 --start-timeout 2m --state-dir /var/run/docker/libcontainerd/containerd -...
Oct 15 01:12:30 k8s-node-1 dockerd[1287]: time="2018-10-15T01:12:30.119275927Z" level=info msg="libcontainerd: new containerd process, pid: 1290"
Oct 15 01:12:31 k8s-node-1 dockerd[1287]: time="2018-10-15T01:12:31.188947120Z" level=info msg="Graph migration to content-addressability took 0.00 seconds"
Oct 15 01:12:31 k8s-node-1 dockerd[1287]: time="2018-10-15T01:12:31.190451809Z" level=info msg="Loading containers: start."
Oct 15 01:12:31 k8s-node-1 dockerd[1287]: time="2018-10-15T01:12:31.216932570Z" level=info msg="Firewalld running: true"
Oct 15 01:12:31 k8s-node-1 dockerd[1287]: time="2018-10-15T01:12:31.423924991Z" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option...d IP address"
Oct 15 01:12:31 k8s-node-1 dockerd[1287]: time="2018-10-15T01:12:31.649819895Z" level=info msg="Loading containers: done."
Oct 15 01:12:31 k8s-node-1 dockerd[1287]: time="2018-10-15T01:12:31.666467212Z" level=info msg="Daemon has completed initialization"
Oct 15 01:12:31 k8s-node-1 dockerd[1287]: time="2018-10-15T01:12:31.666511476Z" level=info msg="Docker daemon" commit=f5ec1e2 graphdriver=overlay version=17.03.2-ce
Oct 15 01:12:31 k8s-node-1 dockerd[1287]: time="2018-10-15T01:12:31.683106722Z" level=info msg="API listen on /var/run/docker.sock"
Oct 15 01:12:31 k8s-node-1 systemd[1]: Started Docker Application Container Engine.
Hint: Some lines were ellipsized, use -l to show in full.
[root@k8s-node-1 ~]# systemctl enable docker.service
Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service to /usr/lib/systemd/system/docker.service.
[root@k8s-node-1 ~]# |
F. iptable 사용 설정
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
[root@k8s-node-1 ~]# sysctl net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-iptables = 1 |
G. kubelet kubeadm kubectl 설치
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
[root@k8s-node-1 ~]# yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirror.fileplanet.com
* epel: d2lzkl7pfhq30w.cloudfront.net
* extras: mirror.fileplanet.com
* updates: mirrors.sonic.net
kubernetes/signature | 454 B 00:00:00
Retrieving key from https://packages.cloud.google.com/yum/doc/yum-key.gpg
Importing GPG key 0xA7317B0F:
Userid : "Google Cloud Packages Automatic Signing Key <gc-team@google.com>"
Fingerprint: d0bc 747f d8ca f711 7500 d6fa 3746 c208 a731 7b0f
From : https://packages.cloud.google.com/yum/doc/yum-key.gpg
Retrieving key from https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
kubernetes/signature | 1.4 kB 00:00:00 !!!
kubernetes/primary | 37 kB 00:00:00
kubernetes 263/263
Resolving Dependencies
--> Running transaction check
---> Package kubeadm.x86_64 0:1.12.1-0 will be installed
--> Processing Dependency: kubernetes-cni >= 0.6.0 for package: kubeadm-1.12.1-0.x86_64
--> Processing Dependency: cri-tools >= 1.11.0 for package: kubeadm-1.12.1-0.x86_64
---> Package kubectl.x86_64 0:1.12.1-0 will be installed
---> Package kubelet.x86_64 0:1.12.1-0 will be installed
--> Processing Dependency: socat for package: kubelet-1.12.1-0.x86_64
--> Running transaction check
---> Package cri-tools.x86_64 0:1.12.0-0 will be installed
===================================================================================================================
Installing:
kubeadm x86_64 1.12.1-0 kubernetes 7.2 M
kubectl x86_64 1.12.1-0 google-cloud-sdk 7.7 M
kubelet x86_64 1.12.1-0 kubernetes 19 M
Installing for dependencies:
cri-tools x86_64 1.12.0-0 kubernetes 4.2 M
kubernetes-cni x86_64 0.6.0-0 kubernetes 8.6 M
socat x86_64 1.7.3.2-2.el7 base 290 k
Transaction Summary
===================================================================================================================
Install 3 Packages (+3 Dependent packages)
Total download size: 47 M
Installed size: 237 M
Downloading packages:
(1/6): 53edc739a0e51a4c17794de26b13ee5df939bd3161b37f503fe2af8980b41a89-cri-tools-1.12.0-0. | 4.2 MB 00:00:00
(2/6): ed7d25314d0fc930c9d0bae114016bf49ee852b3c4f243184630cf2c6cd62d43-kubectl-1.12.1-0.x8 | 7.7 MB 00:00:00
(3/6): 9c31cf74973740c100242b0cfc8d97abe2a95a3c126b1c4391c9f7915bdfd22b-kubeadm-1.12.1-0.x8 | 7.2 MB 00:00:00
(4/6): socat-1.7.3.2-2.el7.x86_64.rpm | 290 kB 00:00:00
(5/6): fe33057ffe95bfae65e2f269e1b05e99308853176e24a4d027bc082b471a07c0-kubernetes-cni-0.6. | 8.6 MB 00:00:00
(6/6): c4ebaa2e1ce38cda719cbe51274c4871b7ccb30371870525a217f6a430e60e3a-kubelet-1.12.1-0.x8 | 19 MB 00:00:01
-------------------------------------------------------------------------------------------------------------------
Total 29 MB/s | 47 MB 00:00:01
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : socat-1.7.3.2-2.el7.x86_64 1/6
Installing : kubernetes-cni-0.6.0-0.x86_64 2/6
Installing : kubelet-1.12.1-0.x86_64 3/6
Installing : kubectl-1.12.1-0.x86_64 4/6
Installing : cri-tools-1.12.0-0.x86_64 5/6
Installing : kubeadm-1.12.1-0.x86_64 6/6
Verifying : cri-tools-1.12.0-0.x86_64 1/6
Verifying : kubectl-1.12.1-0.x86_64 2/6
Verifying : kubeadm-1.12.1-0.x86_64 3/6
Verifying : kubelet-1.12.1-0.x86_64 4/6
Verifying : kubernetes-cni-0.6.0-0.x86_64 5/6
Verifying : socat-1.7.3.2-2.el7.x86_64 6/6
Installed:
kubeadm.x86_64 0:1.12.1-0 kubectl.x86_64 0:1.12.1-0 kubelet.x86_64 0:1.12.1-0
Dependency Installed:
cri-tools.x86_64 0:1.12.0-0 kubernetes-cni.x86_64 0:0.6.0-0 socat.x86_64 0:1.7.3.2-2.el7
Complete!
[root@k8s-node-1 ~]#
##### kubelet.service enable ##########################################################################################
[root@k8s-node-1 ~]# systemctl status kubelet.service
● kubelet.service - kubelet: The Kubernetes Node Agent
Loaded: loaded (/etc/systemd/system/kubelet.service; disabled; vendor preset: disabled)
Drop-In: /etc/systemd/system/kubelet.service.d
└─10-kubeadm.conf
Active: active (running) since Mon 2018-10-15 01:39:02 UTC; 53s ago
Docs: https://kubernetes.io/docs/
Main PID: 1993 (kubelet)
Tasks: 13
Memory: 50.2M
CGroup: /system.slice/kubelet.service
└─1993 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/e...
Oct 15 01:39:38 k8s-node-1 kubelet[1993]: E1015 01:39:38.191358 1993 kubelet.go:2167] Container runtime ...lized
Oct 15 01:39:39 k8s-node-1 kubelet[1993]: E1015 01:39:39.576344 1993 azure_dd.go:147] failed to get azur...ode-1
Oct 15 01:39:43 k8s-node-1 kubelet[1993]: W1015 01:39:43.192516 1993 cni.go:188] Unable to update cni co...net.d
Oct 15 01:39:43 k8s-node-1 kubelet[1993]: E1015 01:39:43.192635 1993 kubelet.go:2167] Container runtime ...lized
Oct 15 01:39:44 k8s-node-1 kubelet[1993]: E1015 01:39:44.864940 1993 pod_workers.go:186] Error syncing pod 28...
Oct 15 01:39:48 k8s-node-1 kubelet[1993]: W1015 01:39:48.193770 1993 cni.go:188] Unable to update cni co...net.d
Oct 15 01:39:48 k8s-node-1 kubelet[1993]: E1015 01:39:48.193888 1993 kubelet.go:2167] Container runtime ...lized
Oct 15 01:39:49 k8s-node-1 kubelet[1993]: E1015 01:39:49.588158 1993 azure_dd.go:147] failed to get azur...ode-1
Oct 15 01:39:53 k8s-node-1 kubelet[1993]: W1015 01:39:53.194922 1993 cni.go:188] Unable to update cni co...net.d
Oct 15 01:39:53 k8s-node-1 kubelet[1993]: E1015 01:39:53.195060 1993 kubelet.go:2167] Container runtime ...lized
Hint: Some lines were ellipsized, use -l to show in full.
[root@k8s-node-1 ~]# systemctl enable kubelet.service
Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /etc/systemd/system/kubelet.service.
[root@k8s-node-1 ~]# |
H. k8s-master 에 node join
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
[root@k8s-node-1 ~]# kubeadm join 10.142.0.2:6443 --token po3nwb.cmo9lqkstc81sa9m --discovery-token-ca-cert-hash sha256:bc39404d9cb39eb63dd3df90778f200e87bf3bc0cc4221285616955de58a6a4a
[preflight] running pre-flight checks
[WARNING RequiredIPVSKernelModulesAvailable]: the IPVS proxier will not be used, because the following required kernel modules are not loaded: [ip_vs ip_vs_rr ip_vs_wrr ip_vs_sh nf_conntrack_ipv4] or no builtin kernel ipvs support: map[ip_vs_rr:{} ip_vs_wrr:{} ip_vs_sh:{} nf_conntrack_ipv4:{} ip_vs:{}]
you can solve this problem with following methods:
1. Run 'modprobe -- ' to load missing kernel modules;
2. Provide the missing builtin kernel ipvs support
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
[discovery] Trying to connect to API Server "10.142.0.2:6443"
[discovery] Created cluster-info discovery client, requesting info from "https://10.142.0.2:6443"
[discovery] Requesting info from "https://10.142.0.2:6443" again to validate TLS against the pinned public key
[discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "10.142.0.2:6443"
[discovery] Successfully established connection with API Server "10.142.0.2:6443"
[kubelet] Downloading configuration for the kubelet from the "kubelet-config-1.12" ConfigMap in the kube-system namespace
[kubelet] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[preflight] Activating the kubelet service
[tlsbootstrap] Waiting for the kubelet to perform the TLS Bootstrap...
[patchnode] Uploading the CRI Socket information "/var/run/dockershim.sock" to the Node API object "k8s-node-1" as an annotation
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the master to see this node join the cluster.
[root@k8s-node-1 ~]# |
I. k8s-master 에서 node 확인
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
[root@k8s-master ~]# kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
k8s-master Ready master 10m v1.12.1 10.142.0.2 <none> CentOS Linux 7 (Core) 3.10.0-862.14.4.el7.x86_64 docker://17.3.2
k8s-node-1 Ready <none> 5m59s v1.12.1 10.142.0.3 <none> CentOS Linux 7 (Core) 3.10.0-862.14.4.el7.x86_64 docker://17.3.2
k8s-node-2 Ready <none> 5m15s v1.12.1 10.142.0.4 <none> CentOS Linux 7 (Core) 3.10.0-862.14.4.el7.x86_64 docker://17.3.2
[root@k8s-master ~]# |