Kushal Das: Testing Kubernetes on Fedora Atomic using gotun

Kubernetes is one of the major components of the modern container technologies. Previously, using Tunir I worked on ideas to test it automatically on Fedora Atomic images. The new gotun provides a better way to setup and test a complex system like Kubernetes.

In the following example I have used Fedora Infra OpenStack to setup 3 instances, and then used the upstream contrib/ansible repo to do the real setup of the Kubernetes. I have two scripts in my local directory where the ansible directory also exists. First, createinventory.py, which creates the ansible inventory file, and also a hosts file with the right IP addresses. We push this file to every VM and copy to /etc/ using sudo. You can easily do this over ansible, but I did not want to change or add anything to the git repo, that is why I am doing it like this.

#!/usr/bin/env python

import json

data = None
with open("current_run_info.json") as fobj:
    data = json.loads(fobj.read())

user = data['user']
host1 = data['vm1']
host2 = data['vm2']
host3 = data['vm3']
key = data['keyfile']

result = """kube-master.example.com ansible_ssh_host={0} ansible_ssh_user={1} ansible_ssh_private_key_file={2}
kube-node-01.example.com ansible_ssh_host={3} ansible_ssh_user={4} ansible_ssh_private_key_file={5}
kube-node-02.example.com ansible_ssh_host={6} ansible_ssh_user={7} ansible_ssh_private_key_file={8}

[masters]
kube-master.example.com
 
[etcd:children]
masters
 
[nodes]
kube-node-01.example.com
kube-node-02.example.com
""".format(host1,user,key,host2,user,key,host3,user,key)
with open("ansible/inventory/inventory", "w") as fobj:
    fobj.write(result)

hosts = """127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
{0} kube-master.example.com
{1} kube-node-01.example.com
{2} kube-node-02.example.com
""".format(host1,host2,host3)

with open("./hosts","w") as fobj:
    fobj.write(hosts)

Then, I also have a runsetup.sh script, which runs the actual script inside the ansible directory.

#!/usr/bin/sh
cd ./ansible/scripts/
export ANSIBLE_HOST_KEY_CHECKING=False
./deploy-cluster.sh

The following the job definition creates the 3VM(s) on the Cloud.

---
BACKEND: "openstack"
NUMBER: 3

OS_AUTH_URL: "https://fedorainfracloud.org:5000/v2.0"
OS_TENANT_ID: "ID of the tenant"
OS_USERNAME: "user"
OS_PASSWORD: "password"
OS_REGION_NAME: "RegionOne"
OS_IMAGE: "Fedora-Atomic-25-20170124.1.x86_64.qcow2"
OS_FLAVOR: "m1.medium"
OS_SECURITY_GROUPS:
    - "ssh-anywhere-cloudsig"
    - "default"
    - "wide-open-cloudsig"
OS_NETWORK: "NETWORKID"
OS_FLOATING_POOL: "external"
OS_KEYPAIR: "kushal-testkey"
key: "/home/kdas/kushal-testkey.pem"

Then comes the final text file which contains all the actual test commands.

HOSTCOMMAND: ./createinventory.py
COPY: ./hosts vm1:./hosts
vm1 sudo mv ./hosts /etc/hosts
COPY: ./hosts vm2:./hosts
vm2 sudo mv ./hosts /etc/hosts
COPY: ./hosts vm3:./hosts
vm3 sudo mv ./hosts /etc/hosts
HOSTTEST: ./runsetup.sh
vm1 sudo kubectl get nodes
vm1 sudo atomic run projectatomic/guestbookgo-atomicapp
SLEEP 60
vm1 sudo kubectl get pods

Here I am using the old guestbook application, but you can choose to deploy any application for this fresh Kubernetes cluster, and then test if it is working fine or not. Please let me know in the comments what do you think about this idea. Btw, remember that the ansible playbook will take a long time to run.