Upstream sonobuoy lacks 2 images for AArch64 architecture, so build
our own multiarch amd64 (x86_64) + arm64 (AArch64) images for:
- sonobuoy-plugin-systemd-logs (tag version 'latest');
- kube-conformance (tag version 'v1.11');
The new images will be pushed to Dockerhub as:
- akraino/validation:sonobuoy-plugin-systemd-logs-latest
- akraino/validation:kube-conformance-v1.11
Since upstream Dockerfiles require small changes for AArch64 support,
implement a new patching mechanism via new make targets in
`docker/build.mk`:
- .submodules_init
* initializes the (newly added) git submodules in this repository
and pulls the pinned-down commit for each submodule;
- .submodules_patched
* Applies patches found in the `patches` and/or `patches/<arch>`
subdirectories for each submodule (using `git am -3`), so we can
apply AArch64-specific changes only on arm64 platforms;
- .submodules_clean
* Un-applies patches by resetting each git submodule to its
pinned-down commit;
To add support for pre-build actions like submodules init + patching,
turn `.build` make target into a double-colon target and extend it
for the two new containers accordingly.
To prevent tox verify failures, adjust ui .eslintrc and tox.ini to
remove the git submodules first (second one being a temporary
workaround until ci-management allows configuring verify jobs with
git submodule handling disabled).
JIRA: VAL-24
Change-Id: Ibfeb3551e44e82fdade2886985ca4577b7d32522
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
!/.coafile
!/ui/src/main/webapp/.eslintrc
+# git submodule patching mechanism local artifacts
+.submodules_*
--- /dev/null
+[submodule "sonobuoy-plugin-systemd-logs"]
+ path = docker/sonobuoy-plugin-systemd-logs/sonobuoy-plugin-systemd-logs
+ url = https://github.com/heptio/sonobuoy-plugin-systemd-logs
+ branch = master
+[submodule "kube-conformance"]
+ path = docker/kube-conformance/kube-conformance
+ url = https://github.com/heptio/kube-conformance
+ branch = master
The postgresql container
-=================
+========================
Building and pushing the container
----------------------------------
.. code-block:: console
- make postgresql-build [ REGISTRY=<dockerhub_registry> NAME=<image_name>]
+ make postgresql-build [ REGISTRY=<dockerhub_registry> NAME=<image_name>]
To both build and push the container, use the command:
.. code-block:: console
- make postgresql [ REGISTRY=<dockerhub_registry> NAME=<image_name>]
+ make postgresql [ REGISTRY=<dockerhub_registry> NAME=<image_name>]
Using the container
-------------------
+
If you want to deploy the container, you can run the corresponding deploy.sh script with the appropriate parameters.
Example:
.. code-block:: console
+
./deploy.sh POSTGRES_PASSWORD=password
The ui container
-=================
+================
Building and pushing the container
----------------------------------
.. code-block:: console
- make ui-build [ REGISTRY=<dockerhub_registry> NAME=<image_name>]
+ make ui-build [ REGISTRY=<dockerhub_registry> NAME=<image_name>]
To both build and push the container, use the command:
.. code-block:: console
- make ui [ REGISTRY=<dockerhub_registry> NAME=<image_name>]
+ make ui [ REGISTRY=<dockerhub_registry> NAME=<image_name>]
Using the container
-------------------
+
If you want to deploy the container, you can run the corresponding deploy.sh script with the appropriate parameters.
Note, that you must also build and run the postgresql container for a functional UI.
Example:
.. code-block:: console
+
./deploy.sh postgres_db_user_pwd=password jenkins_url=http://192.168.2.2:8080 jenkins_user_name=name jenkins_user_pwd=jenkins_pwd jenkins_job_name=job1 nexus_results_url=https://nexus.akraino.org/content/sites/logs proxy_ip=172.28.40.9 proxy_port=3128
+
+The kube-conformance container
+==============================
+
+Building and pushing the container
+----------------------------------
+
+To build just the kube-conformance container, use the command:
+
+.. code-block:: console
+
+ make kube-conformance-build [ REGISTRY=<dockerhub_registry> NAME=<image_name>]
+
+To both build and push the container, use the command:
+
+.. code-block:: console
+
+ make kube-conformance [ REGISTRY=<dockerhub_registry> NAME=<image_name>]
+
+Using the container
+-------------------
+
+This is a standalone container able to launch Kubernetes end-to-end tests,
+for the purposes of conformance testing.
+
+It is a thin wrapper around the `e2e.test` binary in the upstream Kubernetes
+distribution, which drops results in a predetermined location for use as a
+[Heptio Sonobuoy](https://github.com/heptio/sonobuoy) plugin.
+
+To learn more about conformance testing and its Sonobuoy integration, read the
+[conformance guide](https://github.com/heptio/sonobuoy/blob/master/docs/conformance-testing.md).
+
+Example:
+
+.. code-block:: console
+
+ docker run -ti akraino/validation:kube-conformance-v1.11
+
+By default, the container will run the `run_e2e.sh` script. If you want to
+enter the container, add */bin/sh* at the end of the command above
+
+Normally, this conainer is not used directly, but instead leveraged via
+sonobuoy.
+
+The sonobuoy-plugin-systemd-logs container
+==========================================
+
+Building and pushing the container
+----------------------------------
+
+To build just the sonobuoy-plugin-systemd-logs container, use the command:
+
+.. code-block:: console
+
+ make sonobuoy-plugin-systemd-logs-build [ REGISTRY=<dockerhub_registry> NAME=<image_name>]
+
+To both build and push the container, use the command:
+
+.. code-block:: console
+
+ make sonobuoy-plugin-systemd-logs [ REGISTRY=<dockerhub_registry> NAME=<image_name>]
+
+Using the container
+-------------------
+
+This is a simple standalone container that gathers log information from
+systemd, by chrooting into the node's filesystem and running `journalctl`.
+
+This container is used by [Heptio Sonobuoy](https://github.com/heptio/sonobuoy)
+for gathering host logs in a Kubernetes cluster.
+
+Example:
+
+.. code-block:: console
+
+ docker run -ti akraino/validation:sonobuoy-plugin-systemd-logs-latest
+
+By default, the container will run the `get_systemd_logs.sh` script. If you
+want to enter the container, add */bin/sh* at the end of the command above.
+
+Normally, this conainer is not used directly, but instead leveraged via
+sonobuoy.
# limitations under the License. #
##############################################################################
-
# declare the variables
REGISTRY ?= akraino
NAME ?= validation
TAG_PRE ?= $(notdir $(CURDIR))
TAG_VER ?= latest
DOCKERFILE ?= Dockerfile
+WORK_DIR ?= $(CURDIR)
MTOOL ?= $(dir $(realpath $(lastword $(MAKEFILE_LIST))))/manifest-tool
+# git submodule & patch locations for upstream patching before build
+GIT_ROOT := $(shell git rev-parse --show-toplevel)
+GIT_DIR := $(shell git rev-parse --git-dir)
+PATCH_DIR := $(shell pwd)/patches
+AKRAINO_TAG := akraino-validation
+SHELL = /bin/sh
+PATCHES = $(shell find $(PATCH_DIR) $(PATCH_DIR)/$$(uname -m) \
+ -maxdepth 1 -name '*.patch' 2> /dev/null)
+
+export GIT_COMMITTER_NAME?=Akraino Validation
+export GIT_COMMITTER_EMAIL?=validation@akraino.org
+
# get the architecture of the host
HOST_ARCH = amd64
ifeq ($(shell uname -m), aarch64)
sudo chmod +x $@
.PHONY: .build
-.build:
+.build::
+ cd $(WORK_DIR) && \
docker build \
-t $(REGISTRY)/$(NAME):$(TAG_PRE)-$(HOST_ARCH)-$(TAG_VER) \
-f $(DOCKERFILE) \
--platforms linux/amd64,linux/arm64 \
--template $(REGISTRY)/$(NAME):$(TAG_PRE)-ARCH-$(TAG_VER) \
--target $(REGISTRY)/$(NAME):$(TAG_PRE)-$(TAG_VER)
+
+.submodules_init:
+ cd $(GIT_ROOT) && git submodule update --init 2>/dev/null; \
+ touch $@
+
+.submodules_patched: .submodules_init $(PATCHES)
+ $(MAKE) .submodules_clean
+ @cd $(GIT_ROOT) && git submodule -q foreach ' \
+ if [ $$name = $$(basename $(TAG_PRE)) ] && [ -n "$(PATCHES)" ]; then \
+ git tag $(AKRAINO_TAG)-upstream && \
+ git checkout -q -b akraino-validation && \
+ echo "`tput setaf 2`-- patching $$name`tput sgr0`";\
+ git am -3 --ignore-whitespace --patch-format=mbox \
+ --committer-date-is-author-date $(PATCHES) && \
+ git tag $(AKRAINO_TAG) || exit 1; \
+ fi'; \
+ touch $@
+
+.PHONY: .submodules_clean
+.submodules_clean:
+ @cd $(GIT_ROOT) && git submodule -q foreach ' \
+ git am -q --abort > /dev/null 2>&1; \
+ git checkout -q -f $(AKRAINO_TAG)-upstream > /dev/null 2>&1; \
+ git branch -q -D akraino-validation > /dev/null 2>&1; \
+ git tag | grep $(AKRAINO_TAG) | xargs git tag -d > /dev/null 2>&1; \
+ git reset -q --hard HEAD; \
+ git clean -xdff'
+ @rm -f .submodules_patched
--- /dev/null
+##############################################################################
+# Copyright (c) 2019 AT&T, ENEA AB, Nokia and others #
+# #
+# Licensed under the Apache License, Version 2.0 (the "License"); #
+# you maynot use this file except in compliance with the License. #
+# #
+# You may obtain a copy of the License at #
+# http://www.apache.org/licenses/LICENSE-2.0 #
+# #
+# Unless required by applicable law or agreed to in writing, software #
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT #
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. #
+# See the License for the specific language governing permissions and #
+# limitations under the License. #
+##############################################################################
+
+export WORK_DIR?=$(TAG_PRE)
+export TAG_VER?=v1.11
+
+.PHONY: .build
+.build:: .submodules_patched
+ $(MAKE) -C $(TAG_PRE) e2e.test kubectl ginkgo
+
+.PHONY: all
+all: .push_image .push_manifest
+
+.PHONY: build
+build: .build
+
+include ../build.mk
--- /dev/null
+Subproject commit 32d9e2a4edd5c354c97da0d1bd308473bf4f0c89
--- /dev/null
+From: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
+Date: Tue, 25 Jun 2019 17:59:13 +0200
+Subject: [PATCH] Dockerfile, Makefile: Add AArch64 support
+
+Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
+---
+ Dockerfile | 6 ++++--
+ Makefile | 8 +++++---
+ 2 files changed, 9 insertions(+), 5 deletions(-)
+
+diff --git a/Dockerfile b/Dockerfile
+index 289ac88..102685d 100644
+--- a/Dockerfile
++++ b/Dockerfile
+@@ -13,10 +13,12 @@
+ # See the License for the specific language governing permissions and
+ # limitations under the License.
+
+-FROM buildpack-deps:jessie-scm
++FROM arm64v8/buildpack-deps:jessie-scm
+ MAINTAINER Timothy St. Clair "tstclair@heptio.com"
+
+-RUN apt-get update && apt-get -y --no-install-recommends install \
++RUN echo 'Acquire::Check-Valid-Until "0";' > /etc/apt/apt.conf.d/10no--check-valid-until \
++ && sed -i -e '/security/s/^/#/g' -e 's|deb.debian.org/debian|snapshot.debian.org/archive/debian/20190301T035241Z|g' /etc/apt/sources.list \
++ && apt-get update && apt-get -y --no-install-recommends install \
+ ca-certificates \
+ && rm -rf /var/cache/apt/* \
+ && rm -rf /var/lib/apt/lists/*
+diff --git a/Makefile b/Makefile
+index 4335cfd..bdaa52c 100644
+--- a/Makefile
++++ b/Makefile
+@@ -29,6 +29,8 @@ is_root=$(filter 0,$(shell id -u))
+ DOCKER?=$(if $(or $(in_docker_group),$(is_root)),docker,sudo docker)
+ DIR := ${CURDIR}
+
++export KUBERNETES_SERVER_ARCH=arm64
++
+ .PHONY: all container getbins clean
+
+ all: container
+@@ -47,9 +49,9 @@ _cache/.getbins.$(kube_version_full).timestamp:
+ KUBERNETES_DOWNLOAD_TESTS=true \
+ KUBERNETES_SKIP_CONFIRM=true ./kubernetes/cluster/get-kube-binaries.sh
+ mv _cache/$(kube_version_full)/kubernetes/cluster ./
+- mv _cache/$(kube_version_full)/kubernetes/platforms/linux/amd64/e2e.test ./
+- mv _cache/$(kube_version_full)/kubernetes/platforms/linux/amd64/ginkgo ./
+- mv _cache/$(kube_version_full)/kubernetes/platforms/linux/amd64/kubectl ./
++ mv _cache/$(kube_version_full)/kubernetes/platforms/linux/arm64/e2e.test ./
++ mv _cache/$(kube_version_full)/kubernetes/platforms/linux/arm64/ginkgo ./
++ mv _cache/$(kube_version_full)/kubernetes/platforms/linux/arm64/kubectl ./
+ touch $@
+
+ container: e2e.test kubectl ginkgo
--- /dev/null
+##############################################################################
+# Copyright (c) 2019 AT&T, ENEA AB, Nokia and others #
+# #
+# Licensed under the Apache License, Version 2.0 (the "License"); #
+# you maynot use this file except in compliance with the License. #
+# #
+# You may obtain a copy of the License at #
+# http://www.apache.org/licenses/LICENSE-2.0 #
+# #
+# Unless required by applicable law or agreed to in writing, software #
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT #
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. #
+# See the License for the specific language governing permissions and #
+# limitations under the License. #
+##############################################################################
+
+export WORK_DIR?=$(TAG_PRE)
+
+.PHONY: .build
+.build:: .submodules_patched
+
+.PHONY: all
+all: .push_image .push_manifest
+
+.PHONY: build
+build: .build
+
+include ../build.mk
--- /dev/null
+From: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
+Date: Tue, 25 Jun 2019 17:45:29 +0200
+Subject: [PATCH] Dockerfile: AArch64 support
+
+Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
+---
+ Dockerfile | 6 ++++--
+ 1 file changed, 4 insertions(+), 2 deletions(-)
+
+diff --git a/Dockerfile b/Dockerfile
+index 06d872c..687d541 100644
+--- a/Dockerfile
++++ b/Dockerfile
+@@ -13,10 +13,12 @@
+ # See the License for the specific language governing permissions and
+ # limitations under the License.
+
+-FROM buildpack-deps:jessie-scm
++FROM arm64v8/buildpack-deps:jessie-scm
+ MAINTAINER Ken Simon "ken@heptio.com"
+
+-RUN apt-get update && apt-get -y --no-install-recommends install \
++RUN echo 'Acquire::Check-Valid-Until "0";' > /etc/apt/apt.conf.d/10no--check-valid-until \
++ && sed -i -e '/security/s/^/#/g' -e 's|deb.debian.org/debian|snapshot.debian.org/archive/debian/20190301T035241Z|g' /etc/apt/sources.list \
++ && apt-get update && apt-get -y --no-install-recommends install \
+ ca-certificates \
+ && rm -rf /var/cache/apt/* \
+ && rm -rf /var/lib/apt/lists/*
--- /dev/null
+Subproject commit f49e35d220772879aa057da7d103af35a21439b6
wget
unzip
bash
+ rm
passenv = http_proxy HTTP_PROXY https_proxy HTTPS_PROXY no_proxy NO_PROXY
install_command = python -m pip install --no-cache-dir {opts} {packages}
requests~=2.21.0
nodeenv
commands =
+ rm -rf docker/kube-conformance/kube-conformance docker/sonobuoy-plugin-systemd-logs/sonobuoy-plugin-systemd-logs
nodeenv -p
npm install --global write-good eslint complexity-report@2.0.0-alpha
wget --timeout=120 -q https://github.com/pmd/pmd/releases/download/pmd_releases/6.15.0/pmd-bin-6.15.0.zip
unzip -q pmd-bin-6.15.0.zip
- bash -c "export PATH=$PATH:$PWD/pmd-bin-6.15.0/bin/ && coala --non-interactive"
\ No newline at end of file
+ bash -c "export PATH=$PATH:$PWD/pmd-bin-6.15.0/bin/ && coala --non-interactive"
{
"globals": {
- "angular": 1,
+ "angular": writable,
"console": 1,
"confirm":1,
"localStorage":1,
"window":1
}
-}
\ No newline at end of file
+}
Code Review / validation.git / commitdiff