KubeInit is an Ansible collection, to use it the only installation requirement is to have Python 3 and Ansible +2.9.

There are two ways of launching KubeInit, directly using the ansible-playbook command from the project’s source code, or by running it inside a container.


  • A fresh deployed server with enough RAM and disk space (120GB in RAM and 300GB in disk) and CentOS 8 (it should work also in Fedora/Debian/Ubuntu hosts).

  • We assume that the hypervisor node is called nyctea (defined in the inventory).

  • Have root passwordless access with certificates.

  • Adjust the inventory file to suit your needs.

  • Having podman installed in the machine where you are running ansible-playbook.

Installing dependencies

KubeInit calls additional Ansible collections that needs to be installed. If there are dependencies issues when executing the collection, install them by executing:

git clone
cd kubeinit
ansible-galaxy collection install --force --requirements-file kubeinit/requirements.yml

An example of a possible dependency issue is the following:

TASK [Configure the cluster service node] ***************************************************************************************************************
ERROR! couldn't resolve module/action 'community.general.docker_login'. This often indicates a misspelling, missing collection, or incorrect module path.

By default the KubeInit’s container image installs these requirements, this should only affect those executing directly the collection from the source code.

There is also needed to build and install the collection if its used directly from the repository.

# From the root directory in the repository, execute:
rm -rf ~/.ansible/collections/ansible_collections/kubeinit/kubeinit
ansible-galaxy collection build kubeinit --verbose --force --output-path releases/
ansible-galaxy collection install --force --force-with-deps releases/kubeinit-kubeinit-`cat kubeinit/galaxy.yml | shyaml get-value version`.tar.gz

Multiple hypervisors support

Currently, it is supported the deployment of multiple Kubernetes clusters in multiple hosts. While it is supported to deploy different Kubernetes distributions based in different guest OS, for example Vanilla Kubernetes that is based in CentOS and OKD based in Fedora CoreOS the operative system of the hosts must be the same, this means to have 3 hypervisors (chassis) based in Debian that will host any of the currently supported distribution.

Any deployment based on mixed versions of the Hypervisors OS is not supported. This is motivated on the potential failures due to mismatch in OVS/OVN and Kernel versions.

Kubeinit spec

Currently the inventory to deploy the clusters is dynamic, this means that it is configured based on the variables that are passed to the deployment command. The first hypervisor is denoted also as the bastion host or ‡.

In particular the kubeinit_spec variable will determine the amount of controller nodes, compute nodes, and hypervisors that will be used.

The current supported syntax of this variable is:


For example, combinations like okd-libvirt-1-2-1, k8s-libvirt-3-1-1, eks-libvirt-1-0-1 are all valid.


Group variables


Default variables related to Ansible parameters, common across all the supported scenarios.


Specify the Python interpreter.


Enable SSH pipelining.


Define the default SSH common arguments.


Determine if Ansible debug is currently enabled.


Number of attempts to connect. Ansible retries connections only if it gets an SSH error with a return code of 255. Any errors with return codes other than 255 indicate an issue with program execution.


Initial configuration variables related to the cluster parameters.


The default for the cluster name is {{ kubeinit_cluster_distro + ‘cluster’ }} You can override this by setting a specific value for cluster_name in the kubeinit_cluster_spec command line variable, e.g. -e kubeinit_cluster_spec=’{“cluster_name”:”mycluster”}’


The default for the cluster domain name


The default for the hypervisor naming pattern


The default for the controller naming pattern


The default for the compute naming pattern


Default variables related to general cluster parameters.


The default name we will use for the network


Default values for cluster node configuration.


The following list of maps will be used to build the inventory of cluster nodes needed by the kubeinit_spec. Additional specifications provided on the command line can be used to override any of these defaults, e.g.

-e cluster_nodes_spec=’[{“when_group”:”controller_nodes”,”disk”=”35G”}]’


Additional specifications provided on the command line can be used to override any of these defaults, e.g.

-e extra_nodes_spec=’[{“name”:”nova-compute”,”when_distro”:[“okd”],”os”:”centos”}]’


Additional specifications provided on the command line can be used to override any of these defaults, e.g.

-e service_nodes_spec=’[{“services”:[“apache”,”bind”,”dnsmasq”]}]’


The following environment specific variables can be used with the deployment command. These variables might be specific to each scenario.


Certificate default country.


Certificate default state.


Certificate default locality.


Certificate default organization.


Certificate default OU.


Default public DNS.


Default SSH key type.


General deployment network parameters.


The default for the cluster network name is kimgtnet0.

You can override this by setting a specific value for network_name in the kubeinit_network_spec command line variable, e.g. -e kubeinit_network_spec=’{“network_name”:”mynetwork”}’

The network name will be used to create a libvirt network for the cluster guest vms. The network cidr will set the range of addresses reserved for the cluster nodes. The gateway offset will be used to select the gateway address within the range, a negative offset starts at the end of the range, so for network=, gateway_offset=-2 will select and gateway_offset=1 will select as the address. The other offset attributes follow the same convention.


The default for the network CIDR


The default for the gateway offset


The default for the DHCP start lease offset


The default for the DHCP end lease offset


The common secrets supported across all the scenarios are handled using the following variables.


This variable contains, a secret placeholder.


The secret name represents a secret placeholder.


The envvar name works for a secret placeholder.


All the secrets will enable the deployment capabilities, this is a docs placeholder.

  - secret_name: This is the secret name.
    dict_varname: Where we will store the secret.
    dict_key: The secret value.
    env_varname: A default value from an environmental variable.

The ones that are currently used:

  • kubeinit-ci-ssh-key: Docs placeholder.

  • kubeinit-setup-ssh-key: Docs placeholder.

  • kubeinit-ssh-key: Docs placeholder.

  • dockerhub-username: Docs placeholder.

  • dockerhub-password: Docs placeholder.

  • openshift-pullsecret: Docs placeholder.

  • gitlab-runner-registration-token: Docs placeholder.

  • gitlab-runner-access-token: Docs placeholder.


The following einventory structure must be followed.

Directly executing the deployment playbook

The following example command will deploy a multi-master OKD 4.5 cluster with 1 worker node in a single command and in approximately 30 minutes.

git clone
cd kubeinit
ansible-playbook \
    -v --user root \
    -e kubeinit_spec=okd-libvirt-3-2-1 \
    -i ./kubeinit/inventory.yml \

After provisioning any of the scenarios, you should have your environment ready to go. To connect to the nodes from the hypervisor use the IP addresses from the inventory files.

Running the deployment command from a container

The whole process is explained in the HowTo’s. The following commands build a container image with the project inside of it, and then launches the container executing the ansible-playbook command with all the standard ansible-playbook parameters.

Note: When running the deployment from a container, nyctea can not be, it needs to be the hypervisor’s IP address. Also when running the deployment as a user different than root, the keys needs to be also updated.

Running from the GIT repository

git clone
cd kubeinit
podman build -t kubeinit/kubeinit .
podman run --rm -it \
    -v ~/.ssh/id_rsa:/root/.ssh/id_rsa:z \
    -v ~/.ssh/ \
    -v ~/.ssh/config:/root/.ssh/config:z \
    kubeinit/kubeinit \
        -v --user root \
        -e kubeinit_spec=okd-libvirt-3-2-1 \
        -i ./kubeinit/inventory.yml \

Running from a release

# Get the latest release tag
TAG=$(curl --silent "" | jq -r .tag_name)
podman run --rm -it \
    -v ~/.ssh/id_rsa:/root/.ssh/id_rsa:z \
    -v ~/.ssh/ \
    -v ~/.ssh/config:/root/.ssh/config:z \$TAG \
        -v --user root \
        -e kubeinit_spec=okd-libvirt-3-2-1 \
        -i ./kubeinit/inventory.yml \

Connecting to the cluster

Accessing the cluster resources internally

Once the deployment is finished the machine used for provisioning the cluster have access to both the pod services and instances created. The SSH keys are stored in the .ssh folder from the home folder of the user triggering the deployment. For instance:

# From the hypervisor node the user should
# have passwordless access to the services pod
[email protected] kubeinit]# ssh -i ~/.ssh/rkecluster_id_rsa [email protected]
Welcome to Ubuntu 20.10 (GNU/Linux 5.8.0-53-generic x86_64)
  System load:  0.0                Users logged in:               0
  Usage of /:   2.5% of 147.34GB   IPv4 address for docker0:
  Memory usage: 4%                 IPv4 address for enp1s0:
  Swap usage:   0%                 IPv4 address for vetha0e3a877:
  Processes:    186

# Get the cluster nodes
[email protected]:~# kubectl get nodes
NAME            STATUS   ROLES               AGE   VERSION
rke-controller-01   Ready    controlplane,etcd   11m   v1.19.3

# In the root folder there are files with some details about the deployment
# like the kubeconfig file, the container images used in the deployment,
# and the registry pull secret.
[email protected]:~# ls
cluster.rkestate  httpd.conf               kubeinit_deployment_images.txt  pullsecret.json      rke           snap
cluster.yml       kube_config_cluster.yml  pod_cidr                        registry-auths.json  service_cidr

# The cluster config file is also copied to the default folder.
[email protected]:~# ls .kube/config

Accessing the cluster resources externally

When the services deployment is executed one of the tasks in the kubeinit_bind role will create an external ingress script on the primary (bastion) hypervisor called

Running that script will create the required tunnels for all the service endpoints on the cluster.

A user should be able to fetch the kubeadmin password running a command like:

echo $(ssh [email protected] ssh [email protected] cat install_dir/auth/kubeadmin-password)

The console should be available at https://console-openshift-console.apps.okdcluster.kubeinit.local from the external machine if you have a similar configuration in the /etc/resolv.conf.

search okdcluster.kubeinit.local
nameserver <ip of nyctea>

Cleaning up the environment

Each time a cluster is deployed, all the previously created resources are removed. In case a user needs to remove the resources created by Kubeinit execute from the project’s root folder:

ansible-playbook \
    -v --user root \
    -e kubeinit_spec=okd-libvirt-3-2-1 \
    -e kubeinit_stop_after_task=task-cleanup-hypervisors \
    -i ./kubeinit/inventory.yml \

In this case the deployment will stop just after cleaning the hypervisors resources. If its required to remove all the guests in the hypervisors its also possible to add the following variable -e kubeinit_libvirt_destroy_all_guests=True