This document describes how to install Kubernetes on Openshift Origin 3.11 platform.

1) Operating System Configurations (All Servers)


# It is recommended that the following tools be installed on all servers.
sudo apt update
sudo apt install -y curl wget net-tools gnupg2 software-properties-common apt-transport-https ca-certificates

# The Apinizer user is created and authorized.
sudo adduser apinizer
sudo usermod -aG sudo apinizer

# Transactions are continued by switching to the user.
su - apinizer

# The firewall is turned off.
sudo systemctl stop ufw
sudo systemctl disable ufw

# Kubernetes, MongoDB and Elasticsearch jointly do not want the use of swap in the operating system. For that, let's disable swap.
# For operating system swap disabled operation.
sudo swapoff -a

# The swap line in the /etc/fstab file is deleted or commented so that incase of a reboot swap will not open.
# Then the file is closed (:wq)
sudo vi /etc/fstab
POWERSHELL

2) Docker Installation

2.1) Container Installation (Will be Done on All Openshift Servers)

Before proceeding to Kubernetes installation, the following steps are followed to prepare the system and install Docker.

#For the modules to be permanently installed 
sudo tee /etc/modules-load.d/k8s.conf <<EOF
overlay
br_netfilter
EOF

#For the modules to be installed on the running system 
sudo modprobe overlay
sudo modprobe br_netfilter
POWERSHELL

sysctl settings

sudo vi /etc/sysctl.d/k8s.conf
POWERSHELL

The first three lines here are mandatory, and the others can be changed according to the need.

net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward=1
net.ipv4.tcp_max_syn_backlog=40000
net.core.somaxconn=40000
net.core.wmem_default=8388608
net.core.rmem_default=8388608
net.ipv4.tcp_sack=1
net.ipv4.tcp_window_scaling=1
net.ipv4.tcp_fin_timeout=15
net.ipv4.tcp_keepalive_intvl=30
net.ipv4.tcp_tw_reuse=1
net.ipv4.tcp_moderate_rcvbuf=1
net.core.rmem_max=134217728
net.core.wmem_max=134217728
net.ipv4.tcp_mem=134217728 134217728 134217728
net.ipv4.tcp_rmem=4096 277750 134217728
net.ipv4.tcp_wmem=4096 277750 134217728
net.core.netdev_max_backlog=300000
YML

Docker installation is done with the following codes.

sudo apt update

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"

sudo apt update

sudo apt install -y containerd.io docker-ce docker-ce-cli

sudo mkdir -p /etc/systemd/system/docker.service.d

sudo tee /etc/docker/daemon.json <<EOF
{
  "insecure-registries" : [ "172.30.0.0/16" ],
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2"
}
EOF
POWERSHELL

Docker service is started along with the final settings.

sudo systemctl daemon-reload 
sudo systemctl restart docker
sudo systemctl enable docker

sudo groupadd docker

sudo gpasswd -a $USER docker
POWERSHELL

2.2) Kubernetes Installation (On Master)

wget https://github.com/openshift/origin/releases/download/v3.11.0/openshift-origin-client-tools-v3.11.0-0cbc58b-linux-64bit.tar.gz
#Uncompress downloaded file.
tar xvf openshift-origin-client-tools*.tar.gz

cd openshift-origin-client*/
sudo mv  oc kubectl  /usr/local/bin/

#Verify installation of OpenShift client utility.
oc version

sudo systemctl restart docker

oc cluster up --public-hostname=YOURHOSTIP

oc login -u system:admin

oc adm policy add-cluster-role-to-user cluster-admin developer

oc login
POWERSHELL


2.2.1) Bash Auto-Completion (Optional, On Any Openshift Master Server)

This process can speed up the writing of Openshift commands.

apt install bash-completion
source /usr/share/bash-completion/bash_completion
kubectl completion bash | sudo tee /etc/bash_completion.d/kubectl > /dev/null
POWERSHELL


2.2.2) Setting User Configuration of Kubectl Command on Openshift Master Server (On Openshift Master Servers)

Definitions are made for the user who will run the kubectl commands

mkdir -p $HOME/.kube
sudo chown -R $(id -u):$(id -g) $HOME/.kube
POWERSHELL

2.2.3) Install Kubernetes Network Plugin (On Openshift Master Servers)

In this guide, we will use the Flannel network add-on. You can choose other supported network add-ons. Flannel is a simple and easy way to configure a layer 3 network architecture for Kubernetes.

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
POWERSHELL

Important

If you did not use the value 10.244.0.0/16 as podCIDR while initializing the Master, you should download the above yaml file and edit the network settings here as well.


2.2.4) Installation Check (On Any Openshift Master Server )

If the Node created in addition to the Master can be seen when the following code is run on the Master, the installation has been completed successfully.

If it does not transition from NotReady to Ready status within two minutes, the problem should be investigated with the command 'kubectl describe node NODENAME'.

oc get node  

NAME         STATUS   ROLES    AGE   VERSION
localhost    Ready    <none>   5d    v1.11.0+d4cacc0     
BASH


2.2.5) Defining Openshift Permissions (On Openshift Master Servers)

By default, Openshift deploys with at least one RBAC configuration to protect your cluster data. Currently, Dashboard only supports login with Bearer Token. Follow the steps below in order.

vi service.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kube-system
YML

vi adminuser.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kube-system
YML 
kubectl apply -f service.yaml

kubectl apply -f adminuser.yaml

kubectl create clusterrolebinding permissive-binding --clusterrole=cluster-admin --user=admin --user=kubelet --group=system:serviceaccounts

kubectl create clusterrolebinding apinizer -n kube-system --clusterrole=cluster-admin --serviceaccount=kube-system:apinizer
POWERSHELL


2.3) DNS Test (Optional, On Any Openshift Master Server)


oc apply -f https://k8s.io/examples/admin/dns/dnsutils.yaml
POWERSHELL