From: Juha Kosonen Date: Fri, 5 Jul 2019 11:36:05 +0000 (+0300) Subject: Add Robot test for running Tempest test suite X-Git-Tag: 2.0.0~44^2 X-Git-Url: https://gerrit.akraino.org/r/gitweb?a=commitdiff_plain;h=b4b3a0e950a01f3ed9eedda7480cffecc989bdf1;p=validation.git Add Robot test for running Tempest test suite Runs the test set defined by RefStack [1]. Creates OpenStack resources needed for running Tempest tests if required. Blacklist mechanism for excluding test cases from execution is supported. [1] https://refstack.openstack.org/api/v1/guidelines/2019.06/tests?target=platform&type=required&alias=true&flag=false JIRA: VAL-36 Change-Id: I30e3f3a084e028c7ec75f1254319448ebace72a9 Signed-off-by: Juha Kosonen --- diff --git a/docker/README.rst b/docker/README.rst index 7179bee..1deab89 100644 --- a/docker/README.rst +++ b/docker/README.rst @@ -275,6 +275,46 @@ want to enter the container, add */bin/sh* at the end of the command above. Normally, this conainer is not used directly, but instead leveraged via sonobuoy. +The openstack container +======================= + +Building and pushing the container +---------------------------------- + +To build just the openstack container, use the command: + +.. code-block:: console + + make openstack-build [ REGISTRY= NAME=] + +To both build and push the container, use the command: + +.. code-block:: console + + make openstack [ REGISTRY= NAME=] + +Using the container +------------------- + +The openstack image is meant to be ran from a server that has access to the +openstack deployment (jenkins slave, jumpserver, etc). + +Before running the image, copy openstack deployment environment variables +(openrc) to a local folder (e.g. /root/openrc). + +Container needs to be started with the openrc file mounted. Optionally, test +cases can be excluded from execution via a mounted blacklist file. + +The results folder can be mounted as well; this way the logs are +stored on the local server. + +.. code-block:: console + + docker run -ti -v /home/jenkins/openrc:/root/openrc \ + -v /home/jenkins/blacklist.txt:/opt/akraino/validation/tests/openstack/tempest/blacklist.txt \ + -v /home/jenkins/openstack_results:/opt/akraino/results/ \ + akraino/validation:openstack-latest + The helm container ================== diff --git a/docker/openstack/Dockerfile b/docker/openstack/Dockerfile new file mode 100644 index 0000000..95bb2e6 --- /dev/null +++ b/docker/openstack/Dockerfile @@ -0,0 +1,54 @@ +############################################################################## +# Copyright (c) 2019 AT&T, ENEA AB, Nokia and others # +# # +# Licensed under the Apache License, Version 2.0 (the "License"); # +# you maynot use this file except in compliance with the License. # +# # +# You may obtain a copy of the License at # +# http://www.apache.org/licenses/LICENSE-2.0 # +# # +# Unless required by applicable law or agreed to in writing, software # +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # +# See the License for the specific language governing permissions and # +# limitations under the License. # +############################################################################## + +# ref: https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#use-multi-stage-builds +FROM python:3.6-alpine3.9 as build + +ARG REFSTACK_TARGET=2019.06 + +# Install dependencies +COPY pip-requirements.txt /wheels/requirements/pip-requirements.txt +RUN apk --no-cache add --update \ + gcc \ + git \ + libc-dev \ + libffi \ + libffi-dev \ + make \ + openssl-dev \ + wget + +# Build binaries +WORKDIR /wheels +RUN pip3 install wheel +RUN pip3 wheel -r /wheels/requirements/pip-requirements.txt +RUN git clone https://gerrit.akraino.org/r/validation /opt/akraino/validation +RUN wget -q -O /tmp/test_list.txt \ + "https://refstack.openstack.org/api/v1/guidelines/$REFSTACK_TARGET/tests?target=platform&type=required&alias=true&flag=false" + +# Copy binaries in the final container and install requirements +FROM python:3.6-alpine3.9 +COPY --from=build /wheels /wheels +COPY --from=build /opt/akraino/validation /opt/akraino/validation +COPY --from=build /tmp/test_list.txt /opt/akraino/validation/tests/openstack/tempest/test_list.txt + +RUN pip3 install -r /wheels/requirements/pip-requirements.txt \ + -f /wheels && \ + rm -rf /wheels && \ + rm -rf /root/.cache/pip/* + +# Install blueval dependencies +RUN pip install -r /opt/akraino/validation/bluval/requirements.txt diff --git a/docker/openstack/Makefile b/docker/openstack/Makefile new file mode 100644 index 0000000..a5b4099 --- /dev/null +++ b/docker/openstack/Makefile @@ -0,0 +1,23 @@ +############################################################################## +# Copyright (c) 2019 AT&T, ENEA AB, Nokia and others # +# # +# Licensed under the Apache License, Version 2.0 (the "License"); # +# you maynot use this file except in compliance with the License. # +# # +# You may obtain a copy of the License at # +# http://www.apache.org/licenses/LICENSE-2.0 # +# # +# Unless required by applicable law or agreed to in writing, software # +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # +# See the License for the specific language governing permissions and # +# limitations under the License. # +############################################################################## + +.PHONY: all +all: .push_image .push_manifest + +.PHONY: build +build: .build + +include ../build.mk diff --git a/docker/openstack/pip-requirements.txt b/docker/openstack/pip-requirements.txt new file mode 100644 index 0000000..301c618 --- /dev/null +++ b/docker/openstack/pip-requirements.txt @@ -0,0 +1,3 @@ +python-tempestconf +robotframework +tempest diff --git a/tests/openstack/tempest/blacklist.txt b/tests/openstack/tempest/blacklist.txt new file mode 100644 index 0000000..e69de29 diff --git a/tests/openstack/tempest/tempest.resource b/tests/openstack/tempest/tempest.resource new file mode 100644 index 0000000..d5720f0 --- /dev/null +++ b/tests/openstack/tempest/tempest.resource @@ -0,0 +1,76 @@ +############################################################################## +# Copyright (c) 2019 AT&T Intellectual Property. # +# Copyright (c) 2019 Nokia. # +# # +# Licensed under the Apache License, Version 2.0 (the "License"); # +# you maynot use this file except in compliance with the License. # +# # +# You may obtain a copy of the License at # +# http://www.apache.org/licenses/LICENSE-2.0 # +# # +# Unless required by applicable law or agreed to in writing, software # +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # +# See the License for the specific language governing permissions and # +# limitations under the License. # +############################################################################## + + +*** Settings *** +Library BuiltIn +Library Collections +Library OperatingSystem +Library Process +Library String + + +*** Variables *** +${REPORTDIR} ${LOG_PATH}/${SUITE_NAME.replace(' ','_')} +${WORKSPACE} sut +${WORKDIR} ${TEMPDIR}/tempest-run/${WORKSPACE} + + +*** Keywords *** +Setup Workspace + Create Directory ${WORKDIR} + ${result}= Run Process tempest init --name ${WORKSPACE} ${WORKDIR} + Should Be Equal As Integers ${result.rc} 0 + +Remove Workspace + Run Process tempest workspace remove --name ${WORKSPACE} + Remove Directory ${WORKDIR} recursive=True + +Create Tempest Configuration File + ${env}= Get Deployment Credential Variables + ${result}= Run Process discover-tempest-config + ... --create + ... --out ${WORKDIR}/etc/tempest.conf + ... DEFAULT.log_dir ${WORKDIR}/logs + ... DEFAULT.log_file tempest.log + ... oslo_concurrency.lock_path ${WORKDIR}/tempest_lock + ... auth.use_dynamic_credentials true + ... env=${env} + Should Be Equal As Integers ${result.rc} 0 + +Get Deployment Credential Variables + ${env}= Get Environment Variables + ${openrc}= Get File /root/openrc + ${str}= Get Lines Matching Regexp ${openrc} ^export .* + @{lines}= Split To Lines ${str} + :FOR ${line} IN @{lines} + \ ${str}= Remove String Using Regexp ${line} ^export${SPACE} + \ ${key} ${value} Split String ${str} separator== + \ Set To Dictionary ${env} ${key}=${value} + Should Not Be Empty ${env} + [Return] ${env} + +Run Tempest Refstack Tests + ${result}= Run Process tempest run + ... --workspace ${WORKSPACE} + ... --load-list ${CURDIR}/test_list.txt + ... --blacklist-file ${CURDIR}/blacklist.txt + ... --concurrency 4 + ... stdout=${WORKDIR}/logs/tempest_run.log + ... stderr=STDOUT + Copy Files ${WORKDIR}/logs/* ${REPORTDIR}/ + Should Contain ${result.stdout} - Failed: 0 diff --git a/tests/openstack/tempest/tempest.robot b/tests/openstack/tempest/tempest.robot new file mode 100644 index 0000000..11f71fa --- /dev/null +++ b/tests/openstack/tempest/tempest.robot @@ -0,0 +1,30 @@ +############################################################################## +# Copyright (c) 2019 AT&T Intellectual Property. # +# Copyright (c) 2019 Nokia. # +# # +# Licensed under the Apache License, Version 2.0 (the "License"); # +# you maynot use this file except in compliance with the License. # +# # +# You may obtain a copy of the License at # +# http://www.apache.org/licenses/LICENSE-2.0 # +# # +# Unless required by applicable law or agreed to in writing, software # +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # +# See the License for the specific language governing permissions and # +# limitations under the License. # +############################################################################## + + +*** Settings *** +Documentation Tempest is the integration Test Suite for validating +... OpenStack APIs and deployment. +Resource tempest.resource +Suite Setup Run Keywords Setup Workspace + ... Create Tempest Configuration File +Suite Teardown Remove Workspace + + +*** Test Cases *** +Tempest Refstack Test + Run Tempest Refstack Tests