+
## KNI installer
-This repository contains the installer for the Akraino KNI deployment. Along with the [https://gerrit.akraino.org/r/#/admin/projects/kni/templates](https://gerrit.akraino.org/r/#/admin/projects/kni/templates) repository, it will allow to deploy Edge sites in a declarative way.
+This repository contains knictl, the CLI tool for the Akraino KNI deployment. Along with blueprints [https://gerrit.akraino.org/r/#/admin/projects/kni/blueprint-pae](https://gerrit.akraino.org/r/#/admin/projects/kni/blueprint-pae) repository, it will allow to deploy Edge sites in a declarative way.
## Dependencies
make build
-This will produce the `knictl` binary that can be used to deploy a site.
+This will produce a `knictl` tarball file that can be extracted, and will contain the knictl binary, and another dependencies.
+
+## Prerequisites
+Before starting the deployment, some configuration steps are needed on the installer host. A directory on $HOME/.kni needs to be created, and this directory needs to contain:
+ - **pull-secret.json**: This needs to contain the pull secret that has been described in the previous step. Please copy the secret literally, without adding extra newlines, whitespaces, etc
+ - **id_rsa.pub**: This is a public ssh key, that will be used to access the nodes by SSH. If there is no key present there, a new ssh keypair will be generated by the CLI tool.
+
+In case of AWS deployments, create a $HOME/.aws directory, with a **credentials** file on it. This file needs to have the following content:
+[default]
+aws_access_key_id=xxxx
+aws_secret_access_key=xxxx
+
+Please look at [https://docs.openshift.com/container-platform/4.1/installing/installing_aws/installing-aws-account.html](https://docs.openshift.com/container-platform/4.1/installing/installing_aws/installing-aws-account.html)
+
+In the case of libvirt, a helper script can be executed to prepare the installer for acting as a virthost. You can execute **utils/prep_host.sh** script to properly configure your server to install OpenShift on libvirt.
+
+## Structure of a site
+In order to deploy a blueprint, you need to create a repository with a site. The site configuration is based in [kustomize](https://github.com/kubernetes-sigs/kustomize), and needs to use our blueprints as base, referencing that properly. A sample site can be seen on[https://github.com/yrobla/kni-site](https://github.com/yrobla/kni-site) . Site needs to have this structure:
+
+ ├── 00_install-config
+ │ ├── install-config.name.patch.yaml
+ │ ├── install-config.patch.yaml
+ │ ├── kustomization.yaml
+ │ └── site-config.yaml
+ ├── 01_cluster-mods
+ │ ├── kustomization.yaml
+ │ ├── manifests
+ │ └── openshift
+ ├── 02_cluster-addons
+ │ └── kustomization.yaml
+ └── 03_services
+ └── kustomization.yaml
+
+***00_install-config***
+This repository will contain the basic settings for the site, including the base blueprint/profile, and the site name/domain. The following files are needed:
+
+**kustomization.yaml**: key file, where it will contain a link to the used blueprint/profile, and a reference to the used patches to customize the site
+
+ bases:
+ - git::https://gerrit.akraino.org/r/kni/blueprint-pae.git//profiles/production.aws/00_install-config
+ patches:
+ - install-config.patch.yaml
+ patchesJson6902:
+ - target:
+ version: v1
+ kind: InstallConfig
+ name: cluster
+ path: install-config.name.patch.yaml
+ transformers:
+ - site-config.yaml
+The entry in bases needs to reference the blueprint being used (in this case blueprint-pae), and the profile install-config file (in this case production.aws/00_install-config). The other entries need to be just written literally.
+
+**install-config.patch.yaml** is a patch to modify the domain from the base blueprint. You need to customize with the domain you want to give to your site.
+
+ apiVersion: v1
+ kind: InstallConfig
+ metadata:
+ name: cluster
+ baseDomain: devcluster.openshift.com
+
+**install-config.name.patch.yaml** is a patch to modify the site name from the base blueprint. You need to customize with the name you want to give to your site.
+
+ - op: replace
+ path: "/metadata/name"
+ value: kni-site
+**site-config.yaml** needs to be a dummy file, copied literally:
+
+ apiVersion: kni.akraino.org/v1alpha1
+ kind: SiteConfig
+ metadata:
+ name: notImportantHere
+ config: {}
+
+***01_cluster_mods***
+This is the directory that will contain all the customizations for the basic cluster deployment. You could create patches for modifying number of masters/workers, network settings... everything that needs to be modified on cluster deployment time. It needs to have a basic **kustomization.yaml** file, that will reference the same level file for the blueprint. And you could create additional patches following kustomize syntax:
+
+ bases:
+ - git::https://gerrit.akraino.org/r/kni/blueprint-pae.git//profiles/production.aws/01_cluster-mods
+
+**02_cluster_addons** and **03_services** follow same structure as 01_cluster_mods, but in this case is for adding additional workloads after cluster deployment. They also need to have a **kustomization.yaml** file that references the file of the same level for the blueprint, and can include additional resources and patches.
## How to deploy
+The whole deployment workflow is based on knictl CLI tool that this repository is providing.
-There is a Makefile on the root directory of this project. In order to deploy
-you will need to use the following syntax:
-
- make deploy CREDENTIALS=<git_private_repo> SETTINGS=<path_to_sample_settings> BASE_REPO=<git_base_manifests_repo> BASE_PATH=<path_in_manifests_repo>
-
-**CREDENTIALS: Content of private repo**
-
-This repository is needed to store private credentials. It is recommended that
-you store those credentials on a private repo where only allowed people have
-access, where access to the repositories can be controlled by SSH keys.
-Each setting is stored in individual files in the repository:
-
-- ssh-pub-key # public key to be used for SSH access into the nodes
-- coreos-pull-secret # place the file that you created before
-- aws-access-key-id # just for AWS deploy, key id that you created in AWS
-- aws-secret-access-key # just for AWS deploy, secret for the key id
-
-The right path to clone a private repo is: git@github.com:repo_user/repo_name.git
-
-You also can use a local directory to store secrets in local deployments.
-You should create a directory with a known path, and your directory shall contain the individual files listed before. In that case no SSH key is needed. You can reference you local secrets repo by:
-
- CREDENTIALS=file://<path_to_secrets_repo>
-
-**BASE_REPO: Repository for the base manifests**
-
-This is the repository where the default manifest templates are stored. There is one specific folder for each blueprint and provider: aws/1-node, libvirt/1-node, etc... This can be any repository with the right templates, but for Akraino it currently defaults to github.com/redhat-nfvpe/kni-edge-base.git
-
-**BASE_PATH: Path inside base manifests**
-
-Once the manifests repository is being cloned, it will contain several folders with all the specific manifests for the blueprint. In order to choose a provider and footprint, the BASE_PATH needs to be specific. Current values are: libvirt/1-node, libvirt/3-node, aws/1-node, aws/3-node
-
-**SITE_REPO: Repository for the site manifests**
-
-Each site can have different manifests and settings. This needs to be a per-site repository where individual settings need to be stored, and where the generated manifests per site will be stored (TODO).
-
-**SETTINGS_PATH: Specific site settings**
-
-Once the site repository is being cloned, it needs to contain a settings.yaml file with the per-site settings. This needs to be the path inside the SITE_REPO where the settings.yaml is contained. A sample settings file can be seen at [https://gerrit.akraino.org/r/gitweb?p=kni/templates.git;a=blob_plain;f=libvirt/sample_settings.yaml;hb=refs/heads/master](https://gerrit.akraino.org/r/gitweb?p=kni/templates.git;a=blob_plain;f=libvirt/sample_settings.yaml;hb=refs/heads/master) for libvirt, and at [https://gerrit.akraino.org/r/gitweb?p=kni/templates.git;a=blob_plain;f=aws/sample_settings.yaml;hb=refs/heads/master](https://gerrit.akraino.org/r/gitweb?p=kni/templates.git;a=blob_plain;f=aws/sample_settings.yaml;hb=refs/heads/master) for AWS. Different settings file need to be created per site.
-
-## How to deploy for AWS
-
-Before starting the deploy, please read the following documentation to prepare
-the AWS account properly:
-[https://github.com/openshift/installer/blob/master/docs/user/aws/README.md](https://github.com/openshift/installer/blob/master/docs/user/aws/README.md)
-
-There are two different footprints for AWS: 1 master/1 worker, and 3 masters/3 workers. Makefile needs to be called with:
-
- make deploy CREDENTIALS=git@github.com:<git_user>/akraino-secrets.git SITE_REPO=git::https://gerrit.akraino.org/r/kni/templates.git SETTINGS_PATH=aws/sample_settings.yaml BASE_REPO=git::https://gerrit.akraino.org/r/kni/templates.git BASE_PATH=[aws/1-node|aws/3-node]
-
-The file will look like:
-
- settings:
- baseDomain: "<base_domain>"
- clusterName: "<cluster_name>"
- clusterCIDR: "10.128.0.0/14"
- clusterSubnetLength: 9
- machineCIDR: "10.0.0.0/16"
- serviceCIDR: "172.30.0.0/16"
- SDNType: "OpenShiftSDN"
- AWSRegion: "<aws_region_to_deploy>"
-
-Where:
-- `<base_domain>` is the DNS zone matching with the one created on Route53
-- `<cluster_name>` is the name you are going to give to the cluster
-- `<aws_region_to_deploy>` is the region where you want your cluster to deploy
-
-SETTINGS can be a path to local file, or an url, will be queried with curl.
-
-The make process will create the needed artifacts and will start the deployment of the specified cluster
-
-Please consider that, for security reasons. AWS has disabled the SSH access to
-nodes. If you need to SSH for any reason, please take a look at `Unable to SSH
-into Master Nodes` section from the following document:
-
-[https://github.com/openshift/installer/blob/master/docs/user/troubleshooting.md](https://github.com/openshift/installer/blob/master/docs/user/troubleshooting.md)
-
-
-## How to deploy for Libvirt
-
-Hardware: support of nested virtualization needed
-First of all, we need to prepare a host in order to configure libvirt, iptables, permissions, etc.
-This repository contains a bash script that will prepare the host for you.
-
- source utils/prep_host.sh
-
-The official documentation that describes these changes can be found in the following link:
-
-[https://github.com/openshift/installer/blob/master/docs/dev/libvirt-howto.md](https://github.com/openshift/installer/blob/master/docs/dev/libvirt-howto.md)
-
-Unfortunately, Libvirt is only for development purposes from the OpenShift perspective, so the binary is not compiled with the libvirt bits by default. The user will have to compile it by his/her own version with libvirt enabled. In order to build a binary, please look at section `Building and consuming your own installer`. As a summary, doing `make binary` will compile openshift-installer binay with libvirt support.
-
-There are two different footprints for libvirt: 1 master/1 worker, and 3 masters/3 workers. Makefile needs to be called with:
-
- make deploy CREDENTIALS=git@github.com:<git_user>/akraino-secrets.git SITE_REPO=git::https://gerrit.akraino.org/r/kni/templates.git SETTINGS_PATH=libvirt/sample_settings.yaml BASE_REPO=git::https://gerrit.akraino.org/r/kni/templates.git BASE_PATH=[libvirt/1-node|libvirt/3-node] INSTALLER_PATH=file:///${GOPATH}/bin/openshift-install
-
-A sample settings.yaml file has been created specifically for Libvirt targets. It needs to look like:
-
- settings:
- baseDomain: "<base_domain>"
- clusterName: "<cluster_name>"
- clusterCIDR: "10.128.0.0/14"
- clusterSubnetLength: 9
- machineCIDR: "10.0.0.0/16"
- serviceCIDR: "172.30.0.0/16"
- SDNType: "OpenShiftSDN"
- libvirtURI: "<libvirt_host_ip>"
-
-Where:
-- `<base_domain>` is the DNS zone matching with the entry created in /etc/NetworkManager/dnsmasq.d/openshift.conf during the libvirt-howto machine setup. (tt.testing by default)
-- `<cluster_name>` is the name you are going to give to the cluster
-- `<libvirt_host_ip>` is the host IP where libvirt is configured (i.e. qemu+tcp://192.168.122.1/system)
-
-The rest of the options are exactly the same as in an AWS deployment.
-
-There is a recording of the deployment of a Libvirt blueprint with 1 master and 1 worker. Please, see the following link to watch it:
-
-[https://www.youtube.com/watch?v=3mDb1cba8uU](https://www.youtube.com/watch?v=3mDb1cba8uU)
-
-## How to use the cluster
-
-After the deployment finishes, a `kubeconfig` file will be placed inside
-build/auth directory:
-
- export KUBECONFIG=./build/auth/kubeconfig
-
-Then cluster can be managed with oc. You can get the client on this link
-[https://mirror.openshift.com/pub/openshift-v4/clients/ocp/latest/](https://mirror.openshift.com/pub/openshift-v4/clients/ocp/latest/), and check for `openshift-client` files (there are versions for Linux, Mac and Windows). Once downloaded, uncompress it, give it execution permissions and move to the desired path. After that, regular `oc` commands can be executed over the cluster.
+ **1. Fetch requirements for a site.**
+You need to have a site repository with the structure described above. Then, first thing is to fetch the requirements needed for the blueprint that the site references. This is achieved by:
-## How to destroy the cluster
+ ./knictl fetch_requirements github.com/site-repo.git
+Where the first argument references a site repository, following [go-getter](https://github.com/hashicorp/go-getter) syntax.
+This will download the site repository, and will create a folder with the site name inside $HOME/.kni . It will also fetch all the binaries needed, and will store them inside $HOME/.kni/\$SITE_NAME/requirements folder.
-In order to destroy the running cluster, and clean up environment, just use
-`make clean` command.
+ **2. Prepare manifests for a site**
+Next step is to run a procedure to prepare all the manifests for deploying a site. This is achieved by applying kustomize on the site repository, combining that with the base manifests for the blueprint, and doing a merge with the manifests generated by the installer at runtime. This is achieved by the following command:
-## Building and consuming your own installer
+ ./knictl prepare_manifests $SITE_NAME
+This will generate a set of manifests ready to apply, and will be stored on $HOME/.kni/\$SITE_NAME/final_manifests folder.
+Along with manifests, a **profile.env** file has been created also in $HOME/.kni/\$SITE_NAME folder. It includes environment vars that can be sourced before deploying the cluster. Current vars that can be exported are:
-The openshift-installer binaries are published on [https://mirror.openshift.com/pub/openshift-v4/clients/ocp/latest/](https://mirror.openshift.com/pub/openshift-v4/clients/ocp/latest/).
-For faster deploy, you can grab the installer from the upper link. However, there may be situations where you need to compile your own installer (such as the case of libvirt), or when you need a newer version.
-You can build the binary following the instructions on [https://github.com/openshift/installer](https://github.com/openshift/installer) - Quick Start section, or you can use the provided target from our project.
-The binary can be produced with the following command:
+ - OPENSHIFT_INSTALL_RELEASE_IMAGE_OVERRIDE : used when a new image is wanted, instead of the default one
+ - TF_VAR_libvirt_master_memory, TF_VAR_libvirt_master_vcpu. Used in the libvirt case, to define the memory and CPU for the vms.
- make binary
+ **3. Deploy the cluster**
+Before starting the deployment, it is recommended to source the env vars from profile.env . You can achieve it with:
-It accepts an additional INSTALLER_GIT_TAG parameter that allows to specify the installer version that you want to build. Once built, the `openshift-install` is placed in the build directory. This binary can be copied into $GOPATH/bin for easier use. Once generated, the new installer can be used with an env var:
+ source $HOME/.kni/\$SITE_NAME/profile.env
+Then, you need to deploy the cluster using the generated manifests. This can be achieved with:
- export INSTALLER_PATH=http://<url_to_binary>/openshift-install
+ $HOME/.kni/$SITE_NAME/requirements/openshift-install create cluster --dir=$HOME/.kni/$SITE_NAME/final_manifests
-Or pass it as a parameter to make command.
+ This will deploy a cluster based on the specified manifests. You can learn more about how to manage cluster deployment and how to interact with it on [https://docs.openshift.com/container-platform/4.1/welcome/index.html](https://docs.openshift.com/container-platform/4.1/welcome/index.html)
-## Customization: use your own manifests
+ **4. Apply workloads**
+After the cluster has been generated, the extra workloads that have been specified in manifests (like kubevirt), need to be applied. This can be achieved by:
-openshift-installer is also able to produce manifests, that end users can modify
-and deploy a cluster with the modified settings. New manifests can be generated
-with:
+ ./knictl apply_workloads $SITE_NAME
- /path/to/openshift-install create manifests
+This will execute kustomize on the site manifests and will apply the output to the cluster.
+After that, the site deployment can be considered as finished.
-This will generate a pair of folders: manifests and openshift. Those manifests
-can be modified with the desired values. After that this code can be executed to
-generate a new cluster based on the modified manifests:
+ **5. Destroy site**
+When needed, the site can be destroyed with the openshift-install command, using the following syntax:
- /path/to/openshift-install create cluster
\ No newline at end of file
+ $HOME/.kni/\$SITE_NAME/requirements/openshift-install destroy cluster --dir $HOME/.kni/\$SITE_NAME/final_manifests
Code Review / kni / installer.git / commitdiff