Installation of Openshift on the OpenStack cloud
During last “Day of Learning” in Red Hat which was great opportunity for me to spent whole day on learning something new to me and I choose to learn a bit about installation and management of the Openshift cluster. This post is mostly note for myself from what I did during that training. I was using Openshift 4.6.1 and I installed it on the OpenStack based cloud.
Prerequisites
To install Openshift on OpenStack cloud You need to prepare few things.
clouds.yaml
This is file used by openshift installer (and OpenStack client too) to get credentials to the cloud. It can be downloaded from Horizon: Project -> API Access ->Download OpenStack RC File -> OpenStack clouds.yaml File. This file should be located in ~/.config/openstack/clouds.yaml on Your computer.
SSH key
You need to have ssh key without pass phrase. You can generate it with command:
ssh-keygen -t rsa -b 4096
and follow intructions displayed there.
OpenStack preparation
You need to have external network (at least in the base variant of the installation which I was doing) and 2 Floating IPs which will be used as API IP and IP for apps endpoint. You also need to have flavor with at least 4 vCPU. Openshift installer will ask You about those things.
DNS
Installer will try to connect to the openshift cluster using domain name so You should have domains:
api.<cluster_name>.<base_domain>. IN A <API_FIP_IP>
*.apps.<cluster_name>.<base_domain>. IN A <APPS_FIP_IP>
For the development or testing purpose You can set it in some local dns server, like e.g. dnsmasq if You don’t control DNS entries for used domain.
Openshift secrerts
You need to pull secrets. It can be downloaded from https://try.openshift.com where You should login with Your Red Hat Developer account.
Install config file
You can run installer and give all requested data in the interactive shell. But that isn’t very efficient if You are spawning clusters many times. So You can also create file install-config.yaml which will contain all information required by the installer.
$ cat ./my-first-cluster/install-config.yaml
apiVersion: v1
baseDomain: "skaplons.cluster"
clusterID: "7060401a-60bc-4c49-b6c8-b76f8f18b580"
compute:
- name: worker
platform: {}
replicas: 1
controlPlane:
name: master
platform: {}
replicas: 3
metadata:
name: "skaplons"
networking:
clusterNetworks:
- cidr: 10.128.0.0/14
hostSubnetLength: 9
serviceCIDR: 172.30.0.0/16
machineCIDR: 10.196.0.0/16
platform:
openstack:
cloud: "upshift"
externalNetwork: "provider_net_shared"
region: "regionOne"
computeFlavor: "ocp-master"
lbFloatingIP: "<API_FIP_IP>"
ingressFloatingIp: "<APPS_FIP_ID>"
pullSecret: 'pull secret from try.openshift.com should be here'
sshKey: 'public ssh key should be here'
Installation
When all that is ready You can download and run installer:
$ wget https://mirror.openshift.com/pub/openshift-v4/clients/ocp/latest/openshift-install-linux.tar.gz
$ ./openshift-install create cluster --dir=./my-first-cluster
Now You can get some coffee and wait - installation should take about 30-40 minutes.
Cluster verification
To interact with Openshift cluster You need to have tool called oc which can be downloaded from cloud.redhat.com:
$ wget https://mirror.openshift.com/pub/openshift-v4/clients/ocp/latest/openshift-client-linux.tar.gz
$ tar zxfv openshift-client-linux.tar.gz
Now You should be able to interact with new cluster
$ export KUBECONFIG=`pwd`/my-first-cluster/auth/kubeconfig
$ ./oc whoami
system:admin
./oc get nodes 130 ↵
NAME STATUS ROLES AGE VERSION
skaplons-mptzf-master-0 Ready master 46m v1.19.0+d59ce34
skaplons-mptzf-master-1 Ready master 47m v1.19.0+d59ce34
skaplons-mptzf-master-2 Ready master 46m v1.19.0+d59ce34
skaplons-mptzf-worker-0-dbfb5 Ready worker 34m v1.19.0+d59ce34
You can also get link to the web console
$ oc whoami --show-console
Cluster management
Now, when cluster is installed You can manage it. Nodes in openshift cluster are managed by machine API. To list nodes You can run command:
$ ./oc get machines -n openshift-machine-api
NAME PHASE TYPE REGION ZONE AGE
skaplons-mptzf-master-0 Running ocp-master regionOne nova 80m
skaplons-mptzf-master-1 Running ocp-master regionOne nova 80m
skaplons-mptzf-master-2 Running ocp-master regionOne nova 80m
skaplons-mptzf-worker-0-dbfb5 Running ocp-master regionOne nova 78m
You can also get much more details about specific node:
$ ./oc describe machine skaplons-mptzf-worker-0 -n openshift-machine-api
<cut output here>
Of course You can do much more like listing all cluster control plane, or list region name of the nodes etc. but I will not list all possible commands here. For example to list all nodes and their flavor You can use command:
$ ./oc get machines -n openshift-machine-api -o jsonpath='{range .items[*]}{"\n"}{.metadata.name}{"\t"}{.spec.providerSpec.value.flavor}{end}{"\n"}'
skaplons-mptzf-master-0 ocp-master
skaplons-mptzf-master-1 ocp-master
skaplons-mptzf-master-2 ocp-master
skaplons-mptzf-worker-0-dbfb5 ocp-master
Machines can be grouped in machinesets which can be listed with command:
$ ./os get machinesets -n openshift-machine-api
NAME DESIRED CURRENT READY AVAILABLE AGE
skaplons-mptzf-worker-0 1
Scalling workers in machineset
You can scale up/down nodes in the machineset with command like:
$ ./oc scale machineset skaplons-mptzf-worker-0 --replicas=2 -n openshift-machine-api
If You are scalling nodes down to e.g. 0 workes, worker may not be really deleted but stay in Deleting state until You will add other workers e.g. in different region becuase existing pods needs to be moved somewhere before worker will be deleted.
Using scalling You can also change flavor of workers used in the machineset. To do that You need to execute commands like
$ ./oc patch machineset skaplons-mptzf-worker-0 --type='merge' --patch='{"spec":{"template": {"spec": {"providerSpec": {"value": { "flavor": "ci.m5.large"}}}}}}' -n openshift-machine-api
machineset.machine.openshift.io/skaplons-mptzf-worker-0 patched
and after that if You scale up Your machineset, new nodes will be using new flavor. Existing nodes will still use old flavor.
Summary
Installation of the Openshift cluster is really very easy. Of course, You can use some public cloud providers, like AWS or Google Cloud instead of OpenStack base cloud (but why would You really? :)). Openshift can be also installed directly on bare metal but I didn’t try that way at all. As a next steps I will probably explore Openshift a bit more and learn more details internals of it. But for now I think it’s enough for one day of learning :)