Foreword:
---------
This is a set of Openstack Heat templates which creates a simple topology of
virtual machines to be used to deploy Kubernetes and Calico.

It consists of one master VM and 2 optional slave VMs. In the future it might
be possible to configure the number of slaves but for now it is fixed.


Prerequisites:
--------------
In order to run these templates, you need an Openstack deployment (at least
Ocata version, later is preferred), either a single node installation or
multinode).

The job of the Heat stacks is to spawn either 1 or 3 VMs which will form a
Kubernetes cluster. The base image is required to exist, by default the stacks
will expect a Glance image named "xenial" to exist.

It is required to upload an image prior to using the templates. Currently the
templates operate on the assumption that an Ubuntu Xenial cloud image will be
used, as such it installs the required packages using apt.

See the main control.sh script to start/stop the set of stacks and various
run-time options, like DPDK support.


Usage:
------
For a DPDK enabled deployment, it is usually necessary to pass an extra metadata
in the flavor (e.g. hw:mem_page_size=large). For DPDK usecase you also have to
create a host aggregate which has the pinned=true metadata and add the desired
compute nodes to this host aggregate.

For floating IP support, you need to specify the name of the external network,
otherwise the script will use the default "external".

Example of running the script on a DPDK deployment:
  has_dpdk=true external_net=ext_net ./control.sh start

The set of templates currently define three stacks, which can be skipped from
starting or stopping, if so desired. This makes it useful to skip deleting
the nets or for starting the setup using one host only (master). E.g:
  skip_k8s_net=1 ./control.sh stop
  skip_k8s_slaves=1 ./control.sh start

Networking:
-----------
Have a look at k8s_net.yaml for the network configurations.

Currently the Heat templates define 2 networks:
- k8s_mgmt_net: this is primarily used for sshing into the node, but it also
  serves as the access to the external network. Thus the floating IPs (which are
  activated by default) will be assigned to the ports from this network
- k8s_int_net: kubernetes internal network, which is used by the nodes to join
  the cluster.

Separating the traffic into two networks makes sense in an Openstack environment
by hiding the internal traffic from the outside world.
Thus, for accessing the services inside the clusters, it will be required to use
the floating IPs assigned to the Kubernetes servers.

In terms of CNI, there will be two additional networks involved, which are
defined in k8s_net.yaml. These networks are not visible from outside of the Heat
stacks, Kubernetes and Calico will encapsulate packets on these networks using
IP-in-IP. In fact, to Openstack these are virtual networks, the only reason to
have them in k8s_pod_net.yaml is to have a central view of all the network
parameters.
The two networks are described by Heat stack output variables, as follows:
- k8s_pod_net_cidr: the POD network, passed to kubeadm init --pod-network-cidr
- k8s_svc_net_cidr: the service network, passed to kubeadm init --service-cidr


Calico networking:
------------------
In terms of Calico, k8s_net.yaml defines yet another stack output variable:
- k8s_cluster_ip: corresponds to the etcd_endpoints parameter in calico.yaml


Network security:
-----------------
For the moment, for ease of operation, the stacks ports will have port security
disabled. It should be possible to enable it, and provide a set of security
groups rule to allow all TCP and UDP traffic for the internal network.


Cluster setup:
--------------
The clusters configures itself automatically and installs the base IEC platform
together with the needed resources for Helm. SEBA or other applications will
have to be installed manually afterwards.

For the K8s cluster setup, the Master VM will print the join command in a file
in /home/ubuntu/joincmd. Then the slave VMs will connect to the Master VM using
ssh and read the joincmd file.

All of these are achieved by using cloud-init scripts that run at startup. You
can follow the progress of the init scripts by looking at the console log, which
right now are very verbose.
After the setup is completed, you can look for the joincmd string in the output.


Using the cluster:
------------------
Once the setup is complete, you can login to the k8s_master VM. Use the Horizon
interface or ssh into the floating ip, using the default credentials:
ubuntu:ubuntu

A public key is also generated, and a private key saved in a file called
ak-key.pem but for now password logins are permitted for ease of operation.

Once logged into the master VM, you need to become root.
  sudo su -

From here it is possible to run the usual kubernetes and helm tools, thanks to
having the KUBECONFIG env variable exported through /root/.profile.

It is also possible to use kubernetes as non-root, in which case you need to
manually create ~/.kube/ and copy the kubernetes config:
  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

The most basic operation you can run is verifying the nodes in the cluster:
  kubectl get nodes
  kubcetl describe node k8s-master