Add Robot test for running Tempest test suite 25/1125/6
authorJuha Kosonen <juha.kosonen@nokia.com>
Fri, 5 Jul 2019 11:36:05 +0000 (14:36 +0300)
committerJuha Kosonen <juha.kosonen@nokia.com>
Tue, 27 Aug 2019 12:39:55 +0000 (15:39 +0300)
Runs the test set defined by RefStack [1]. Creates OpenStack resources
needed for running Tempest tests if required.

Blacklist mechanism for excluding test cases from execution is
supported.

[1] https://refstack.openstack.org/api/v1/guidelines/2019.06/tests?target=platform&type=required&alias=true&flag=false

JIRA: VAL-36

Change-Id: I30e3f3a084e028c7ec75f1254319448ebace72a9
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
docker/README.rst
docker/openstack/Dockerfile [new file with mode: 0644]
docker/openstack/Makefile [new file with mode: 0644]
docker/openstack/pip-requirements.txt [new file with mode: 0644]
tests/openstack/tempest/blacklist.txt [new file with mode: 0644]
tests/openstack/tempest/tempest.resource [new file with mode: 0644]
tests/openstack/tempest/tempest.robot [new file with mode: 0644]

index 7179bee..1deab89 100644 (file)
@@ -275,6 +275,46 @@ want to enter the container, add */bin/sh* at the end of the command above.
 Normally, this conainer is not used directly, but instead leveraged via
 sonobuoy.
 
+The openstack container
+=======================
+
+Building and pushing the container
+----------------------------------
+
+To build just the openstack container, use the command:
+
+.. code-block:: console
+
+    make openstack-build [ REGISTRY=<dockerhub_registry> NAME=<image_name>]
+
+To both build and push the container, use the command:
+
+.. code-block:: console
+
+    make openstack [ REGISTRY=<dockerhub_registry> NAME=<image_name>]
+
+Using the container
+-------------------
+
+The openstack image is meant to be ran from a server that has access to the
+openstack deployment (jenkins slave, jumpserver, etc).
+
+Before running the image, copy openstack deployment environment variables
+(openrc) to a local folder (e.g. /root/openrc).
+
+Container needs to be started with the openrc file mounted. Optionally, test
+cases can be excluded from execution via a mounted blacklist file.
+
+The results folder can be mounted as well; this way the logs are
+stored on the local server.
+
+.. code-block:: console
+
+    docker run -ti -v /home/jenkins/openrc:/root/openrc \
+    -v /home/jenkins/blacklist.txt:/opt/akraino/validation/tests/openstack/tempest/blacklist.txt \
+    -v /home/jenkins/openstack_results:/opt/akraino/results/ \
+    akraino/validation:openstack-latest
+
 The helm container
 ==================
 
diff --git a/docker/openstack/Dockerfile b/docker/openstack/Dockerfile
new file mode 100644 (file)
index 0000000..95bb2e6
--- /dev/null
@@ -0,0 +1,54 @@
+##############################################################################
+# Copyright (c) 2019 AT&T, ENEA AB, Nokia and others                         #
+#                                                                            #
+# Licensed under the Apache License, Version 2.0 (the "License");            #
+# you maynot use this file except in compliance with the License.            #
+#                                                                            #
+# You may obtain a copy of the License at                                    #
+#       http://www.apache.org/licenses/LICENSE-2.0                           #
+#                                                                            #
+# Unless required by applicable law or agreed to in writing, software        #
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT  #
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.           #
+# See the License for the specific language governing permissions and        #
+# limitations under the License.                                             #
+##############################################################################
+
+# ref: https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#use-multi-stage-builds
+FROM python:3.6-alpine3.9 as build
+
+ARG REFSTACK_TARGET=2019.06
+
+# Install dependencies
+COPY pip-requirements.txt /wheels/requirements/pip-requirements.txt
+RUN apk --no-cache add --update \
+        gcc \
+        git \
+        libc-dev \
+        libffi \
+        libffi-dev \
+        make \
+        openssl-dev \
+        wget
+
+# Build binaries
+WORKDIR /wheels
+RUN pip3 install wheel
+RUN pip3 wheel -r /wheels/requirements/pip-requirements.txt
+RUN git clone https://gerrit.akraino.org/r/validation /opt/akraino/validation
+RUN wget -q -O /tmp/test_list.txt \
+    "https://refstack.openstack.org/api/v1/guidelines/$REFSTACK_TARGET/tests?target=platform&type=required&alias=true&flag=false"
+
+# Copy binaries in the final container and install requirements
+FROM python:3.6-alpine3.9
+COPY --from=build /wheels /wheels
+COPY --from=build /opt/akraino/validation /opt/akraino/validation
+COPY --from=build /tmp/test_list.txt /opt/akraino/validation/tests/openstack/tempest/test_list.txt
+
+RUN pip3 install -r /wheels/requirements/pip-requirements.txt \
+                 -f /wheels && \
+     rm -rf /wheels && \
+     rm -rf /root/.cache/pip/*
+
+# Install blueval dependencies
+RUN pip install -r /opt/akraino/validation/bluval/requirements.txt
diff --git a/docker/openstack/Makefile b/docker/openstack/Makefile
new file mode 100644 (file)
index 0000000..a5b4099
--- /dev/null
@@ -0,0 +1,23 @@
+##############################################################################
+# Copyright (c) 2019 AT&T, ENEA AB, Nokia and others                         #
+#                                                                            #
+# Licensed under the Apache License, Version 2.0 (the "License");            #
+# you maynot use this file except in compliance with the License.            #
+#                                                                            #
+# You may obtain a copy of the License at                                    #
+#       http://www.apache.org/licenses/LICENSE-2.0                           #
+#                                                                            #
+# Unless required by applicable law or agreed to in writing, software        #
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT  #
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.           #
+# See the License for the specific language governing permissions and        #
+# limitations under the License.                                             #
+##############################################################################
+
+.PHONY: all
+all: .push_image .push_manifest
+
+.PHONY: build
+build: .build
+
+include ../build.mk
diff --git a/docker/openstack/pip-requirements.txt b/docker/openstack/pip-requirements.txt
new file mode 100644 (file)
index 0000000..301c618
--- /dev/null
@@ -0,0 +1,3 @@
+python-tempestconf
+robotframework
+tempest
diff --git a/tests/openstack/tempest/blacklist.txt b/tests/openstack/tempest/blacklist.txt
new file mode 100644 (file)
index 0000000..e69de29
diff --git a/tests/openstack/tempest/tempest.resource b/tests/openstack/tempest/tempest.resource
new file mode 100644 (file)
index 0000000..d5720f0
--- /dev/null
@@ -0,0 +1,76 @@
+##############################################################################
+# Copyright (c) 2019 AT&T Intellectual Property.                             #
+# Copyright (c) 2019 Nokia.                                                  #
+#                                                                            #
+# Licensed under the Apache License, Version 2.0 (the "License");            #
+# you maynot use this file except in compliance with the License.            #
+#                                                                            #
+# You may obtain a copy of the License at                                    #
+#       http://www.apache.org/licenses/LICENSE-2.0                           #
+#                                                                            #
+# Unless required by applicable law or agreed to in writing, software        #
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT  #
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.           #
+# See the License for the specific language governing permissions and        #
+# limitations under the License.                                             #
+##############################################################################
+
+
+*** Settings ***
+Library            BuiltIn
+Library            Collections
+Library            OperatingSystem
+Library            Process
+Library            String
+
+
+*** Variables ***
+${REPORTDIR}       ${LOG_PATH}/${SUITE_NAME.replace(' ','_')}
+${WORKSPACE}       sut
+${WORKDIR}         ${TEMPDIR}/tempest-run/${WORKSPACE}
+
+
+*** Keywords ***
+Setup Workspace
+   Create Directory    ${WORKDIR}
+   ${result}=          Run Process  tempest  init  --name  ${WORKSPACE}  ${WORKDIR}
+   Should Be Equal As Integers  ${result.rc}  0
+
+Remove Workspace
+   Run Process         tempest  workspace  remove  --name  ${WORKSPACE}
+   Remove Directory    ${WORKDIR}  recursive=True
+
+Create Tempest Configuration File
+   ${env}=             Get Deployment Credential Variables
+   ${result}=          Run Process  discover-tempest-config
+   ...                     --create
+   ...                     --out  ${WORKDIR}/etc/tempest.conf
+   ...                     DEFAULT.log_dir  ${WORKDIR}/logs
+   ...                     DEFAULT.log_file  tempest.log
+   ...                     oslo_concurrency.lock_path  ${WORKDIR}/tempest_lock
+   ...                     auth.use_dynamic_credentials  true
+   ...                   env=${env}
+   Should Be Equal As Integers  ${result.rc}  0
+
+Get Deployment Credential Variables
+   ${env}=             Get Environment Variables
+   ${openrc}=          Get File  /root/openrc
+   ${str}=             Get Lines Matching Regexp  ${openrc}  ^export .*
+   @{lines}=           Split To Lines  ${str}
+   :FOR  ${line}  IN  @{lines}
+   \   ${str}=             Remove String Using Regexp  ${line}  ^export${SPACE}
+   \   ${key}  ${value}    Split String  ${str}  separator==
+   \   Set To Dictionary   ${env}  ${key}=${value}
+   Should Not Be Empty  ${env}
+   [Return]            ${env}
+
+Run Tempest Refstack Tests
+   ${result}=          Run Process  tempest  run
+   ...                     --workspace  ${WORKSPACE}
+   ...                     --load-list  ${CURDIR}/test_list.txt
+   ...                     --blacklist-file  ${CURDIR}/blacklist.txt
+   ...                     --concurrency  4
+   ...                   stdout=${WORKDIR}/logs/tempest_run.log
+   ...                   stderr=STDOUT
+   Copy Files          ${WORKDIR}/logs/*  ${REPORTDIR}/
+   Should Contain      ${result.stdout}  - Failed: 0
diff --git a/tests/openstack/tempest/tempest.robot b/tests/openstack/tempest/tempest.robot
new file mode 100644 (file)
index 0000000..11f71fa
--- /dev/null
@@ -0,0 +1,30 @@
+##############################################################################
+# Copyright (c) 2019 AT&T Intellectual Property.                             #
+# Copyright (c) 2019 Nokia.                                                  #
+#                                                                            #
+# Licensed under the Apache License, Version 2.0 (the "License");            #
+# you maynot use this file except in compliance with the License.            #
+#                                                                            #
+# You may obtain a copy of the License at                                    #
+#       http://www.apache.org/licenses/LICENSE-2.0                           #
+#                                                                            #
+# Unless required by applicable law or agreed to in writing, software        #
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT  #
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.           #
+# See the License for the specific language governing permissions and        #
+# limitations under the License.                                             #
+##############################################################################
+
+
+*** Settings ***
+Documentation     Tempest is the integration Test Suite for validating
+...               OpenStack APIs and deployment.
+Resource          tempest.resource
+Suite Setup       Run Keywords  Setup Workspace
+                  ...           Create Tempest Configuration File
+Suite Teardown    Remove Workspace
+
+
+*** Test Cases ***
+Tempest Refstack Test
+   Run Tempest Refstack Tests