3 months agoos:vuls:Update vuls version 01/3601/1 master
DANIEL STOICA [Fri, 26 Jun 2020 12:19:49 +0000 (15:19 +0300)]
os:vuls:Update vuls version

Update vuls version to v0.10.0

In vuls v0.10.0 was deprecated the '-ssh-config' vuls argument and was
moved in config.toml

Signed-off-by: DANIEL STOICA <>
Change-Id: Ia4c0357ec25725a0a2602c07f83edb397c203a04

3 months agoos: vuls: Add CentOS 8 support 97/3597/2
DANIEL STOICA [Thu, 25 Jun 2020 11:14:48 +0000 (14:14 +0300)]
os: vuls: Add CentOS 8 support

Add fetch command for oval database for CentOS 8 in

Add a centos version verification and get the vuls result for desired os
version in validation/tests/os/vuls/vuls.robot

Signed-off-by: DANIEL STOICA <>
Change-Id: I4431a471c6c978719cdb4a4c23099c0c9925c9df

3 months agoos: vuls: Pin to latest stable release 95/3595/1
Alexandru Avadanii [Wed, 24 Jun 2020 15:44:40 +0000 (17:44 +0200)]
os: vuls: Pin to latest stable release

Pin vuls & its related tools (go-cve-dictionary, goval-dictionary,
gost) to the currently latest stable version/tag.

Note that this does not pin other tools, e.g. LTP, lynis.


Change-Id: I2736d9a97cae5c1aa2b6275aae9ce88f182c4e18
Signed-off-by: Alexandru Avadanii <>
4 months agoMerge "Revert "Add a line to change loglevel""
Alexandru Avadanii [Wed, 17 Jun 2020 17:19:28 +0000 (17:19 +0000)]
Merge "Revert "Add a line to change loglevel""

4 months agoRevert "Add a line to change loglevel" 33/3333/2
Alexandru Avadanii [Wed, 17 Jun 2020 15:23:28 +0000 (15:23 +0000)]
Revert "Add a line to change loglevel"

This reverts commit a159ef011182e6c93e1d000d69cfb5493797f6ef.

Reason for revert: obsolete by a new global debug flag

Change-Id: I0414851d8901362f44ee37220dd847a5054a5135
Signed-off-by: Alexandru Avadanii <>
4 months agoMerge "Add a new variable Robot loglevel"
Alexandru Avadanii [Wed, 17 Jun 2020 15:27:08 +0000 (15:27 +0000)]
Merge "Add a new variable Robot loglevel"

4 months agoMerge "Remove inactive committers"
Tapio Tallgren [Wed, 17 Jun 2020 15:08:03 +0000 (15:08 +0000)]
Merge "Remove inactive committers"

4 months agoMerge "Use docker host network for containers"
Alexandru Avadanii [Wed, 10 Jun 2020 22:28:08 +0000 (22:28 +0000)]
Merge "Use docker host network for containers"

4 months agoMerge "Add a line to change loglevel"
Alexandru Avadanii [Wed, 10 Jun 2020 17:34:27 +0000 (17:34 +0000)]
Merge "Add a line to change loglevel"

4 months agoRemove inactive committers 62/3562/1
Tapio Tallgren [Wed, 10 Jun 2020 12:33:43 +0000 (15:33 +0300)]
Remove inactive committers

Both Juha and Cristina have new responsibilites and will no longer
contribute to this project.

Signed-off-by: Tapio Tallgren <>
Change-Id: I5c3dab86ceb0e9f672c61fd90bf521f496f1bb17

4 months agoAdd a new variable Robot loglevel 61/3561/2
Tapio Tallgren [Wed, 10 Jun 2020 11:54:15 +0000 (14:54 +0300)]
Add a new variable Robot loglevel

After this change, Robot will be run at the specified loglevel. This
helps in debugging by setting the loglevel to DEBUG or TRACE. By default
the loglevel is not changed.

Signed-off-by: Tapio Tallgren <>
Change-Id: I5292a75ea5454f9e1d471804bcfb9081fbcee418

Signed-off-by: Tapio Tallgren <>
Change-Id: I203ba34d1ada264f15cfa0733ae32a11fb7dec5e

4 months agoEnable mandatory tests for REC 56/3556/4
DANIEL STOICA [Tue, 9 Jun 2020 15:05:28 +0000 (18:05 +0300)]
Enable mandatory tests for REC

Enable mandatory tests for REC and do the rest of them optional

Sync the REC mandatory tests list with IEC:
- conformance, lynis, vuls and kube-hunter are mandatory for
  all projects where applicable;
- hp_baremetal and ltp are currently failing and are not part
  of mandatory release criteria, so they are marked as

Signed-off-by: DANIEL STOICA <>
Change-Id: Ie9b66ab28ab1a9f7ba070717bc1996e1590c7750

4 months agoUse docker host network for containers 50/3550/1
Igor D.C [Sat, 6 Jun 2020 07:14:03 +0000 (07:14 +0000)]
Use docker host network for containers

If vuls or lynis are told to test the same host where
they're running (and it's only reachable via localhost) then
having the containers in the host network will provide
reachability to localhost so vuls/lynis can run their tests.

Change-Id: I70c6975676facb6de41790928a5d810a43e273d6
Signed-off-by: Igor D.C <>
4 months agoMerge "Add ICN Bluval support"
Tapio Tallgren [Wed, 3 Jun 2020 12:31:42 +0000 (12:31 +0000)]
Merge "Add ICN Bluval support"

4 months agoAdd a line to change loglevel 43/3543/1
Tapio Tallgren [Wed, 3 Jun 2020 12:29:09 +0000 (15:29 +0300)]
Add a line to change loglevel

This does not change the default loglevel but makes it easy to change it
when debugging

Signed-off-by: Tapio Tallgren <>
Change-Id: I4c3c1919fe4c555d51aaa6c8c139af30a7f8c7a2

5 months agoFix sshpass and openssh-client install issue 93/3493/1
DANIEL STOICA [Wed, 20 May 2020 14:38:38 +0000 (17:38 +0300)]
Fix sshpass and openssh-client install issue

Sshpass and openssh-client are installed in os build container, but must
be installed in os container

Signed-off-by: DANIEL STOICA <>
Change-Id: I4329ac98ece075a9c9bf1b36ae710fd68aac8150

5 months agoMerge "Fix ssh cleanup issue for vuls test"
Alexandru Avadanii [Thu, 14 May 2020 12:19:56 +0000 (12:19 +0000)]
Merge "Fix ssh cleanup issue for vuls test"

5 months agoMerge "Fix argument issue"
Alexandru Avadanii [Thu, 14 May 2020 12:19:47 +0000 (12:19 +0000)]
Merge "Fix argument issue"

5 months agoMerge "bluval-eliot.yaml added for ELIOT BP"
Tapio Tallgren [Wed, 13 May 2020 14:25:00 +0000 (14:25 +0000)]
Merge "bluval-eliot.yaml added for ELIOT BP"

5 months agoFix argument issue 32/3432/1
DANIEL STOICA [Wed, 13 May 2020 11:42:03 +0000 (14:42 +0300)]
Fix argument issue

Change argument -p in -P to synchronize validation/bluval/
with ci-management/jjb/shell/ arguments
Signed-off-by: DANIEL STOICA <>
Change-Id: I53ee0751e924cc0f9c382364658eecd540848aa5

5 months agoFix ssh cleanup issue for vuls test 21/3421/1
DANIEL STOICA [Tue, 12 May 2020 14:00:05 +0000 (17:00 +0300)]
Fix ssh cleanup issue for vuls test

Make ssh cleanup only if the ssh_keyfile was created

Signed-off-by: DANIEL STOICA <>
Change-Id: I685c12bbd0a374a7a6381dfbe5eb53c36c076062

5 months agoChange go version for vuls tests 18/3418/2
DANIEL STOICA [Tue, 12 May 2020 11:12:26 +0000 (14:12 +0300)]
Change go version for vuls tests

Change go version in go1.14 because vuls no longer supports verison

Change GOPATH because go1.14 don't accept to set GOPATH in go/src

Signed-off-by: DANIEL STOICA <>
Change-Id: I2df1857ab0b58879051f027320ac95b353ed58ad

5 months agoSupport password-based ssh auth 95/3395/6
DANIEL STOICA [Wed, 29 Apr 2020 11:12:18 +0000 (14:12 +0300)]
Support password-based ssh auth

Add password ssh authentification in robot tests scripts
Add a password variable in tests/variable.yaml


Signed-off-by: DANIEL STOICA <>
Change-Id: I04145b32621dae49485cb8640d80932345957826

5 months agobluval-eliot.yaml added for ELIOT BP 98/3398/1
Srinivasan Selvam [Mon, 4 May 2020 13:20:37 +0000 (18:50 +0530)]
bluval-eliot.yaml added for ELIOT BP

Signed-off-by: Srinivasan Selvam <>
Change-Id: I084f0355ff3bc51f354c3293e3fed9cbbe70cf2c

5 months agoMerge "k8s conformance: Sanitize major/minor version str"
Cristina Pauna [Mon, 27 Apr 2020 08:17:00 +0000 (08:17 +0000)]
Merge "k8s conformance: Sanitize major/minor version str"

5 months agok8s conformance: Sanitize major/minor version str 87/3387/1
Alexandru Avadanii [Fri, 24 Apr 2020 14:49:52 +0000 (16:49 +0200)]
k8s conformance: Sanitize major/minor version str

Since we use `kubectl version` to determine K8s client/server version
strings, the major/minor semver components might include some custom
suffixes, e.g. '1.13+abcdef' would be split into '1' and '13+abcdef'.
Avoid including the custom semver suffixes into the kube-conformance
image tag name by sanitizating the major/minor version strings.


Change-Id: Iac4f719e4e982b90b50a23e5bcaeff4bda9fe104
Signed-off-by: Alexandru Avadanii <>
5 months agohardware: bios_version rework without dmidecode 84/3384/2
Alexandru Avadanii [Thu, 23 Apr 2020 17:45:39 +0000 (19:45 +0200)]
hardware: bios_version rework without dmidecode

Instead of relying on `dmidecode` for reading and parsing various
machine information that we test using the `hardware` layer testcases,
read and parse the same information directly from sysfs:
- product name by simply dumping:
- BIOS revision number (not to be confused with BIOS version number
  which is already available via sysfs /sys/class/dmi/...) by parsing
  /sys/firmware/dmi/tables/DMI according to the SMBIOS Reference
  Specification [1];
This mechanism assumes the target nodes support SMBIOS.

While at it, relax the matching condition for disk device names in
`lsblk` output, as well as the number of processors expected in `lscpu`
output; make the block device name regexp configurable via a new
variable named `blk_dev_regexp` in tests/variables.yaml.



Change-Id: Iaa3be16e7f56aa672304861c57c659f3cfb19f4e
Signed-off-by: Alexandru Avadanii <>
6 months agoblucon: Add argument '-p' for pulling latest tag 82/3382/2
Alexandru Avadanii [Wed, 22 Apr 2020 13:14:47 +0000 (15:14 +0200)]
blucon: Add argument '-p' for pulling latest tag

If `--pull` or `-p` argument is passed to blucon, it will now run a
`docker pull` command prior to launching the testcases with `docker
This should prevent the testcases from using stale docker images if
already present on the machine.


Change-Id: Id4b7947a0de7abf07f69812c6deef71919b9041a
Signed-off-by: Alexandru Avadanii <>
6 months agoMerge "docker: kube-conformance: Enforce upstream tag"
Cristina Pauna [Tue, 21 Apr 2020 13:48:30 +0000 (13:48 +0000)]
Merge "docker: kube-conformance: Enforce upstream tag"

6 months Allow validation/results dir override 72/3372/2
Alexandru Avadanii [Sat, 18 Apr 2020 16:42:08 +0000 (18:42 +0200)] Allow validation/results dir override

When running in CI, the validation git repository is cloned in the
job workspace, which usually has a custom directory name matching the
job name (e.g. bluval-daily-master) instead of the previously hardcoded
'validation' directory name.
Also, the results dir should not be a sibling of the validation git repo
directory when running in CI, but a subdirectory of the said dir.

To accomodate this, allow overriding both previously hardcoded directory
paths via environment variables (VALIDATION_DIR, RESULTS_DIR).

While at it, add a new '-t' argument for specifying the Docker tag to be
used for the validation docker images (previously hardcoded to 'latest')
in order to support running docker images associated to a specific
Validation release tag (e.g. 3.0.0).


Change-Id: I4d3c10881de28c64bcca05ff23aa1025f67a1f5e
Signed-off-by: Alexandru Avadanii <>
6 months agobluval: Add Jenkins log-parser rules 68/3368/9
Alexandru Avadanii [Fri, 17 Apr 2020 15:50:49 +0000 (17:50 +0200)]
bluval: Add Jenkins log-parser rules

Instead of creating the log-parser rules on the fly from the CI job,
add the rules to the current repository, allowing the Validation
committers to tweak them independently (not through ci-management

While at it, extend and improve the previous rules by:
- raising a warning instead of an error for robot FAIL messages (which
  will later be changed via ci-management to _not_ fail the CI job run);
- separating the test suites into log parsing sections;
- raising an info for PASS robot messages (so they are listed in the
  log-parser links section for better readability);
- fail the CI job only if critical tests failed;

For logparser section grouping to work as expected, fix one output
buffering issue that used to print the issued commands after their
output (and also fix another output buffering related issue to keep
things uniform).


Change-Id: I1506d84a44926e80913562f67f24885a98f94df7
Signed-off-by: Alexandru Avadanii <>
6 months agoAdd ICN Bluval support 70/3370/1
Igor DC [Thu, 16 Apr 2020 22:48:19 +0000 (22:48 +0000)]
Add ICN Bluval support

Signed-off-by: Igor DC <>
Change-Id: I2282455442504216d5ada78ef856d31565230417

6 months agodocker: kube-conformance: Enforce upstream tag 65/3365/1
Alexandru Avadanii [Thu, 16 Apr 2020 13:04:22 +0000 (15:04 +0200)]
docker: kube-conformance: Enforce upstream tag

Stop using environment TAG_VER when tagging kube-conformance images,
as they should be tagged with the upstream (fixed) tag, currently
'v1.16' (instead of the validation release tag, e.g. '3.0.0').


Change-Id: I1ea7e7395e7a796c7eb22fcdad41ba92c8d555ac
Signed-off-by: Alexandru Avadanii <>
6 months agoMerge "Change the criticality of OS security test" 3.0.0
Alexandru Avadanii [Tue, 14 Apr 2020 10:38:58 +0000 (10:38 +0000)]
Merge "Change the criticality of OS security test"

6 months agoChange the criticality of OS security test 43/3343/7
DANIEL STOICA [Tue, 7 Apr 2020 13:19:27 +0000 (16:19 +0300)]
Change the criticality of OS security test


Added a checker for the lynis and vuls tests reports, if
vulnerabilities are detected the tests fail with non-critical tag

Signed-off-by: DANIEL STOICA <>
Change-Id: I8399379aa60696cf6ce4c2aee1c2dcfb5f965fbc

6 months agodocker: os: Allow caching vuls db between builds 40/3340/3
Your Name [Mon, 6 Apr 2020 16:06:43 +0000 (16:06 +0000)]
docker: os: Allow caching vuls db between builds

Vuls database files are quite large and take a lot of time to download,
occasionally leading to out-of-memory issues on certain Jenkins slave
build nodes.

To overcome this limitation, allow the build process to cache its data
between subsequent runs:
- if the build host machine (i.e. build server) already has a file at
  that file will be included in the Docker build context and extracted
  during build (Dockerfile ADD implicitly does that for .tar.gz files),
  then brought up to date and re-archived;
- to enable vuls DB caching, one can create an (initially) empty tar.gz
  file at the expected location on the host (build) machine using:
  $ make -C docker/os .init_db_cache
  This is a one time (manual) job, `make build` or equivalent calls will
  ensure the db.tar.gz file is kept up to date afterwards;

Implementation quirks:
- getting the updated db.tar.gz from the os docker image requires us to
  spawn the image as a running container (since Docker does not allow
  extracting files from the image directly), so we handle this step as
  a post-docker-build step via another double-colon `.build` make
- since we want to perform both a pre-docker-build (copying build
  server persistent db.tar.gz to the current build dir, i.e. inside the
  Docker build context) and a post-docker-build step (copying back the
  updated db.tar.gz from the os docker image to the build server
  persistent location), we leverage double-colon Makefile targets for
  * pre-docker-build .build (from docker/os/Makefile before the include
  ../ statement) is executed first;
  * .build from ../ (common for all Docker builds in validation)
  is then normally executed, resulting in a tagged docker image;
  * post-docker-build .build (from docker/os/Makefile after the include
  statement) is executed last;

While at it, fix some related issues:
- fix build issue caused by a new (changed upstream) dependency on
  python 'cryptography' package >= 2.5 (which would require SSL and FFI
  libs inside the destionation container where pip tries to compile the
  new requirement) by adding the new requirement to
  pip-requirements.txt, so we also create a wheel for it in the build
- reduce verbosity of `go-cve-dictionary fetchnvd` to
  eliminate some non-printable characters from the job output;
- reduce verbosity of wget dot progress;
- add timestamp inside os build container to force Docker cache
  invalidation of database fetch/update step (to avoid missing upstream
  changes masked by Docker cache);
- split build container vuls database fetch into 2 separate steps
  covering tool download/installation, respectively fetch/update, so we
  leverage the Docker cache for the first step while always invalidating
  the cache for the second;
- `gost fetch` should use a number of threads comparable with the number
  of available processors;


Change-Id: I41d7bec0e72c92da1596abd67e2c1306ef9ffffa
Signed-off-by: Alexandru Avadanii <>
6 months agoEnable kube-hunter tests and fix layer order 39/3339/3
DANIEL STOICA [Fri, 3 Apr 2020 13:32:28 +0000 (16:32 +0300)]
Enable kube-hunter tests and fix layer order

Signed-off-by: DANIEL STOICA <>
Depends-On: I4d36305d9ffd6fb7688002b6cb72a18baed77803
Change-Id: I5c19b527ec86be0a60b7c31fcd579f0802e9874d

6 months agoMerge "Change the criticality of kube-hunter test"
Juha Kosonen [Fri, 3 Apr 2020 11:40:50 +0000 (11:40 +0000)]
Merge "Change the criticality of kube-hunter test"

6 months agoChange the criticality of kube-hunter test 13/3313/2
Juha Kosonen [Wed, 1 Apr 2020 07:28:30 +0000 (10:28 +0300)]
Change the criticality of kube-hunter test

Vulnerabilities discovered by kube-hunter does not impact to overall
result of test suite execution.

Change-Id: If3593412e68f397258de0fe7dcf86f102ae4d0ce
Signed-off-by: Juha Kosonen <>
6 months agoMerge "Enable vuls and lynis tests in CI"
Cristina Pauna [Tue, 31 Mar 2020 07:13:14 +0000 (07:13 +0000)]
Merge "Enable vuls and lynis tests in CI"

6 months agoEnable vuls and lynis tests in CI 06/3306/7
DANIEL STOICA [Tue, 24 Mar 2020 12:46:22 +0000 (14:46 +0200)]
Enable vuls and lynis tests in CI

 - added the os layer in validation/bluval/bluval-iec.yaml
 - made change in vuls robot script to sincronize the path


Signed-off-by: DANIEL STOICA <>
Depends-On: I9e5349a0e3e3a1e7331a5f5b4b4a8d5a1b01aa51
Change-Id: Ibb16f76405f5b43797e0d859db0681759adf5e18

6 months agoAdd path to kube-hunter job manifest file 08/3308/1
Juha Kosonen [Fri, 27 Mar 2020 08:55:58 +0000 (10:55 +0200)]
Add path to kube-hunter job manifest file

Change-Id: I3977be79d96c6122f571efff72d40495e420e767
Signed-off-by: Juha Kosonen <>
7 months agoUpdate Tempest test case list 00/3300/1
Juha Kosonen [Thu, 19 Mar 2020 10:47:44 +0000 (12:47 +0200)]
Update Tempest test case list

- RefStack 2019.06 -> 2019.11

Signed-off-by: Juha Kosonen <>
Change-Id: Icec93538cae9dfdb460b069e6918133ce1a593ae

7 months agoMerge "Add Vuls security test for ubuntu and centos"
Juha Kosonen [Thu, 5 Mar 2020 08:59:14 +0000 (08:59 +0000)]
Merge "Add Vuls security test for ubuntu and centos"

7 months agoAdd Vuls security test for ubuntu and centos 55/2255/8
Daniel Stoica [Mon, 24 Feb 2020 16:15:15 +0000 (18:15 +0200)]
Add Vuls security test for ubuntu and centos


Signed-off-by: Daniel Stoica <>
Change-Id: If81d5ecb68bea40ae179823d655311260e04fc56

7 months agoFix os layer build failures 66/2266/1
Juha Kosonen [Mon, 2 Mar 2020 12:21:31 +0000 (14:21 +0200)]
Fix os layer build failures

Following changes implemented:

- pkg-config is required for building recent version of LTP [1]
- do not try to re-create vuls directory



Signed-off-by: Juha Kosonen <>
Change-Id: Ib6050dead7e296306ed3f860797fdf6b3944fb43

7 months agoMerge "Add Vuls security test for x86_64"
Cristina Pauna [Mon, 24 Feb 2020 13:32:49 +0000 (13:32 +0000)]
Merge "Add Vuls security test for x86_64"

7 months agoAdd Vuls security test for x86_64 77/2177/12
Daniel Stoica [Tue, 7 Jan 2020 11:33:53 +0000 (13:33 +0200)]
Add Vuls security test for x86_64

Signed-off-by: Daniel Stoica <>
Change-Id: I6aa553d50fc33327ee893382aec0eee605a94eb1

8 months agoAdd Robot test for running kube-hunter 46/2246/1
Juha Kosonen [Thu, 20 Feb 2020 14:21:12 +0000 (16:21 +0200)]
Add Robot test for running kube-hunter

Integrated to k8s layer as an optional test.


Change-Id: I0d11a3aa9438a7fe3dbe52a1358af5ddcaabff61
Signed-off-by: Juha Kosonen <>
8 months agoUpdate k8s layer base container image 45/2245/1
Juha Kosonen [Thu, 20 Feb 2020 10:29:27 +0000 (12:29 +0200)]
Update k8s layer base container image

Starting from v1.16.5 building e2e test requires Go 1.13.4 or
greater. Layer image updated followingly:

Go 1.12.9 -> 1.13.8
Python 3.6.10 -> 3.7.6


Signed-off-by: Juha Kosonen <>
Change-Id: I551b573fce9e461d9523f07056cea4ae6be854dc

8 months agoMerge "Update committers after people move to new jobs"
Tapio Tallgren [Fri, 31 Jan 2020 12:15:36 +0000 (12:15 +0000)]
Merge "Update committers after people move to new jobs"

8 months agoUpdate committers after people move to new jobs 14/2214/2
Tapio Tallgren [Wed, 29 Jan 2020 10:57:54 +0000 (12:57 +0200)]
Update committers after people move to new jobs

Miguel has not been active in the project since last summer and
will not contribute anymore.
Naga has changed email address and will use a different user id
from now on.

Signed-off-by: Tapio Tallgren <>
Change-Id: I93f34debfd996ca51501a48e4ed5b4edf7dfdf79

8 months agoAdd read permission for test logs file 11/2211/3
Daniel Stoica [Mon, 27 Jan 2020 16:36:08 +0000 (18:36 +0200)]
Add read permission for test logs file

Signed-off-by: Daniel Stoica <>
Change-Id: Ic6a26f14cce46dde019357c0adf8f9b4cd9790cf

9 months ago[UI] Messages in Keywords 07/2207/1
Ioakeim Samaras [Wed, 22 Jan 2020 01:45:45 +0000 (17:45 -0800)]
[UI] Messages in Keywords

Display of messages of Robot keywords
is supported.

Signed-off-by: Ioakeim Samaras <>
Change-Id: I588d72a37ba347a0f9979d517532ecfefe67d99f

9 months agoMerge "Add PTL to INFO.yaml"
Juha Kosonen [Mon, 20 Jan 2020 10:14:09 +0000 (10:14 +0000)]
Merge "Add PTL to INFO.yaml"

9 months agoAdd PTL to INFO.yaml 62/2162/6
LF Jenkins CI [Tue, 17 Dec 2019 19:09:57 +0000 (19:09 +0000)]
Add PTL to INFO.yaml

Automatically generated INFO.yaml with PTL added in.

Change-Id: Icf0dece424586f38b4ea030281f10eb9c6600834
Signed-off-by: lf-jobbuilder <>
9 months agoMerge "Replace logging with services layer"
Juha Kosonen [Wed, 8 Jan 2020 17:05:54 +0000 (17:05 +0000)]
Merge "Replace logging with services layer"

9 months agoRun Docker Bench parallel on all nodes 79/2179/1
Juha Kosonen [Wed, 8 Jan 2020 12:51:51 +0000 (14:51 +0200)]
Run Docker Bench parallel on all nodes

Execute Docker Bench against all target node parallel instead of
testing them one by one. This way the total execution time does not
depend on the number of nodes to be tested.

Change-Id: I9a8005f76ebe86c659ec82debe26a55940917028
Signed-off-by: Juha Kosonen <>
9 months agoRemove docker-binary mount. Use container binary 76/2176/4
Alexandru Antone [Tue, 7 Jan 2020 10:44:41 +0000 (12:44 +0200)]
Remove docker-binary mount. Use container binary

The docker client binary mounted from the host is not compatible
with Alpine Linux for certain host distros.

Signed-off-by: Alexandru Antone <>
Change-Id: Iec10dea423adc32f2b9f651043e30a2aa02c7d76

9 months agoMerge "Add validaton tests for KNI"
Cristina Pauna [Tue, 7 Jan 2020 08:54:06 +0000 (08:54 +0000)]
Merge "Add validaton tests for KNI"

9 months agoMerge "[REC-74] openstack bluval integration"
Deepak Kataria [Fri, 3 Jan 2020 18:47:51 +0000 (18:47 +0000)]
Merge "[REC-74] openstack bluval integration"

10 months agoPrevent python to write .pyc files on import 69/2169/1
Juha Kosonen [Fri, 20 Dec 2019 10:39:51 +0000 (12:39 +0200)]
Prevent python to write .pyc files on import


Change-Id: I3bf6dd55cb510e2d9a2704369a1bd0599b48eb63
Signed-off-by: Juha Kosonen <>
10 months ago[VAL-93] Fix docker binary execution error 68/2168/1
Alexandru Antone [Thu, 19 Dec 2019 15:17:10 +0000 (17:17 +0200)]
[VAL-93] Fix docker binary execution error

The docker image was missing required by
the docker binary.
Fixed by adding libc6-compat.

Signed-off-by: Alexandru Antone <>
Change-Id: I50f583bb71ee27b08cfc8af6f175b8671b2d8361

10 months agoMerge "Fix for missing dependencies"
Juha Kosonen [Thu, 19 Dec 2019 06:51:07 +0000 (06:51 +0000)]
Merge "Fix for missing dependencies"

10 months agoFix for missing dependencies 65/2165/3
Alexandru Antone [Wed, 18 Dec 2019 12:04:21 +0000 (14:04 +0200)]
Fix for missing dependencies

Also switch to python:3.6-alpine3.9

Signed-off-by: Alexandru Antone <>
Change-Id: I615c30b107cdd64d24b19b4c2a52f2c157c4bdc6

10 months agoRun Redfish test framework parallel on all nodes 66/2166/1
Juha Kosonen [Wed, 18 Dec 2019 12:45:38 +0000 (14:45 +0200)]
Run Redfish test framework parallel on all nodes

Execute Redfish test framework tool against all target node parallel
instead of testing them one by one. This way the total execution time
does not depend on the number of nodes to be tested.

Change-Id: I2f1fbfd715aa63ccbe71e6d976404b0f21be344c
Signed-off-by: Juha Kosonen <>
10 months ago[VAL-80] Lynis 19/2119/3
Naga Sugguna [Wed, 11 Dec 2019 20:57:07 +0000 (20:57 +0000)]
[VAL-80] Lynis

Signed-off-by: Naga Sugguna <>
Change-Id: I8fd644e28c8d255b9365f726151933a9bcb74f03

10 months ago[REC-74] openstack bluval integration 02/1602/6
Naga Sugguna [Tue, 17 Sep 2019 19:42:53 +0000 (19:42 +0000)]
[REC-74] openstack bluval integration

Change-Id: I5dd973f8c8dba742e74843a6cd21aa0306ccd71b
Signed-off-by: Naga Sugguna <>
10 months ago[VAL-87] Bluval Containerization 89/2089/14
Naga Sugguna [Tue, 3 Dec 2019 07:21:14 +0000 (07:21 +0000)]
[VAL-87] Bluval Containerization

Intention of this change is to simplify BluVal installation

Now steps should be
1. clone the project
2. Update variables.yaml & volumes.yaml
3. ./ -l <hardware> -o -n host <bluprint name> builds the docker image takes few minutes first time
later on it should be quick.

Change-Id: I4b00e00baecc6d7eb81595d1d8866af67e2c53cf
Signed-off-by: Naga Sugguna <>
10 months agoMerge "Convention over configuration"
Naga Sugguna [Thu, 12 Dec 2019 17:07:26 +0000 (17:07 +0000)]
Merge "Convention over configuration"

10 months agoReplace logging with services layer 12/2112/1
valentin.radulescu [Tue, 10 Dec 2019 09:18:57 +0000 (01:18 -0800)]
Replace logging with services layer


Signed-off-by: valentin.radulescu <>
Change-Id: Ib3933fe83b3beb0aa984cc8f158b1478e683aa49

10 months ago[VAL-92] OS layer image UTF-8 issue 96/2096/1 2.0.1
Naga Sugguna [Thu, 5 Dec 2019 20:21:58 +0000 (20:21 +0000)]
[VAL-92] OS layer image UTF-8 issue

Signed-off-by: Naga Sugguna <>
Change-Id: I79db72c4124c12994823e13fb83176dda09fa37b

10 months agoMerge "Do not run all LTP test suites"
Cristina Pauna [Wed, 4 Dec 2019 16:46:15 +0000 (16:46 +0000)]
Merge "Do not run all LTP test suites"

10 months ago[UI] Ignore malformed results 87/2087/1
Ioakeim Samaras [Tue, 3 Dec 2019 01:56:30 +0000 (17:56 -0800)]
[UI] Ignore malformed results

Validation results are ignored when
there are no robot test results
associated with them

Signed-off-by: Ioakeim Samaras <>
Change-Id: Ia27c20022105e202e24f0281b7a7f69a6212fcc4

10 months agoDo not run all LTP test suites 75/2075/1
Juha Kosonen [Thu, 28 Nov 2019 14:03:07 +0000 (16:03 +0200)]
Do not run all LTP test suites


Change-Id: I705b8b20c61988be57c61e3cd7e1b60900a38f24
Signed-off-by: Juha Kosonen <>
11 months agoFix the log file name of LTP tests 03/2003/1
Juha Kosonen [Tue, 19 Nov 2019 08:35:53 +0000 (10:35 +0200)]
Fix the log file name of LTP tests

Change-Id: Ie8a0a068a0ac86503df781be0f4921558543fb01
Signed-off-by: Juha Kosonen <>
11 months agoFix LTP test case failures 97/1997/1
Juha Kosonen [Mon, 18 Nov 2019 12:37:34 +0000 (14:37 +0200)]
Fix LTP test case failures

- untar ltp to / with sudo
- output 'y' when test case expects user intervention
- set read access to output/results


Change-Id: Ia2268fcfb6a9720d051de88a1561456539ec5b03
Signed-off-by: Juha Kosonen <>
11 months ago[UI] Embed files inside image 90/1990/1
Ioakeim Samaras [Fri, 15 Nov 2019 20:20:15 +0000 (12:20 -0800)]
[UI] Embed files inside image

The files 'server.xml' and 'index.jsp'
are embedded inside the UI docker
image during build stage

Signed-off-by: Ioakeim Samaras <>
Change-Id: I056fbb28a2e58630061ea85d8846ebe08f302998

11 months agoMerge "LTP installed, executed, cleaned on target node" 2.0.0
Cristina Pauna [Fri, 15 Nov 2019 09:14:06 +0000 (09:14 +0000)]
Merge "LTP installed, executed, cleaned on target node"

11 months agoMerge "[k8s][conformance] Check sonobuoy container runs"
Cristina Pauna [Fri, 15 Nov 2019 09:13:15 +0000 (09:13 +0000)]
Merge "[k8s][conformance] Check sonobuoy container runs"

11 months ago[os] Fix python version 81/1981/1
Cristina Pauna [Thu, 14 Nov 2019 14:28:33 +0000 (16:28 +0200)]
[os] Fix python version

The python version we currently use in all our containers is 3.6
Also make sure this version is the default one.


Signed-off-by: Cristina Pauna <>
Change-Id: I414adf82f436c21abeb7d11d13b82beab8cc384b

11 months ago[k8s][conformance] Check sonobuoy container runs 69/1969/2
Cristina Pauna [Wed, 13 Nov 2019 13:03:54 +0000 (15:03 +0200)]
[k8s][conformance] Check sonobuoy container runs

Sometimes the sonobuoy container doesn't get into running state before
we check for its status. Added a check to make sure that sonobuoy
container is running before checking for status.


Signed-off-by: Cristina Pauna <>
Change-Id: I299083a2150b144714661a57533f3e5aab556cb0

11 months ago[UI] Handle malformed results 72/1972/1
Ioakeim Samaras [Wed, 13 Nov 2019 21:18:51 +0000 (13:18 -0800)]
[UI] Handle malformed results

When malformed results are stored
in Nexus, the UI ignores them.

Signed-off-by: Ioakeim Samaras <>
Change-Id: I83265825adb36a544618a3003fde56f4d0ccbe70

11 months agoAdd validaton tests for KNI 14/1914/2
Yolanda Robla [Tue, 5 Nov 2019 08:16:29 +0000 (09:16 +0100)]
Add validaton tests for KNI

Create new tests to be run on KNI blueprints.
They are based on kubernetes, so are are adding
hardware, os and k8s tests.

Signed-off-by: Yolanda Robla <>
Change-Id: I96defca94796dc16d32ef69286381428daeee4ed

11 months agoSelect kube-conformance version dynamically 33/1933/1
Juha Kosonen [Thu, 7 Nov 2019 13:10:50 +0000 (15:10 +0200)]
Select kube-conformance version dynamically

Use a version conformorming server version.

Change-Id: Ib2d82c810fe82da3825f0d1b63345d1fa0baf6ff
Signed-off-by: Juha Kosonen <>
11 months agoLTP installed, executed, cleaned on target node 52/1752/10
Naga Sugguna [Wed, 9 Oct 2019 21:16:34 +0000 (21:16 +0000)]
LTP installed, executed, cleaned on target node

Jira: [VAL-66]

LTP testcase fixed to run on cluster master node.
LTP prebuilt binary is installed and cleaned.
LTP output and results downloaded for every testcase

Change-Id: Id3b96f97418034b6099fb11a472a285889159cc4
Signed-off-by: Naga Sugguna <>
11 months agoConvention over configuration 91/1891/9
Naga Sugguna [Thu, 31 Oct 2019 20:56:12 +0000 (20:56 +0000)]
Convention over configuration

Jira: [VAL-83] is taking varibles.yaml and writing updated version to

varibles_updated.yaml is passed to robot framework.
OperatingSystem.Get Environment is better than $HOME.
Each container automatically removed
Each test suite has a debug.log by default

Signed-off-by: Naga Sugguna <>
Change-Id: Iae95d1ee12c8c5e44949be3faf5b0fc9fc40266e

11 months agoMerge "Update conformance test to support k8s 1.16"
Naga Sugguna [Wed, 6 Nov 2019 17:05:33 +0000 (17:05 +0000)]
Merge "Update conformance test to support k8s 1.16"

11 months agoMerge "Add python3 to os container"
Cristina Pauna [Wed, 6 Nov 2019 15:24:34 +0000 (15:24 +0000)]
Merge "Add python3 to os container"

11 months agoMerge "Default value to SSH_KEYFILE"
Naga Sugguna [Tue, 5 Nov 2019 20:37:00 +0000 (20:37 +0000)]
Merge "Default value to SSH_KEYFILE"

11 months agoMerge "Use a more generic kubectl command to check status"
Naga Sugguna [Tue, 5 Nov 2019 20:17:46 +0000 (20:17 +0000)]
Merge "Use a more generic kubectl command to check status"

11 months agoUse a more generic kubectl command to check status 99/1899/3
valentin.radulescu [Fri, 1 Nov 2019 15:14:26 +0000 (08:14 -0700)]
Use a more generic kubectl command to check status

Use "kubectl version" instead of "kubectl get pods
--all-namespaces" to check status.

Signed-off-by: valentin.radulescu <>
Change-Id: Ie7549f9af0060aa7e13e363f0a3b289e290a0bdf

11 months agoUpdate conformance test to support k8s 1.16 19/1919/1
Juha Kosonen [Tue, 5 Nov 2019 10:57:55 +0000 (12:57 +0200)]
Update conformance test to support k8s 1.16

- Sonobuoy v0.16.1
- kube-conformance 1.16


Change-Id: I63bf582466c6447326c67a0c4faffdcab496acad
Signed-off-by: Juha Kosonen <>
11 months agoUpdate eslint config 17/1917/1
Cristina Pauna [Tue, 5 Nov 2019 08:57:20 +0000 (10:57 +0200)]
Update eslint config

When runing the validation of the patces, the eslint verification
throws erros like: "Error: ESLint configuration of global 'console' in
ui/src/main/webapp/.eslintrc is invalid:"

Update the config file so that these errors are not thrown anymore.

Signed-off-by: Cristina Pauna <>
Change-Id: I951bc0370b6599e14fcdc2f68b45d0684b4e2b1b

11 months agoAdd python3 to os container 15/1915/1
Cristina Pauna [Tue, 5 Nov 2019 08:06:13 +0000 (10:06 +0200)]
Add python3 to os container


Signed-off-by: Cristina Pauna <>
Change-Id: I0a3e7d3f4f9c7f1e89e84624ca23234cfab5a4c1

11 months agoAdd LTP tests build 63/1863/5
Daniel Stoica [Mon, 28 Oct 2019 13:02:14 +0000 (15:02 +0200)]
Add LTP tests build

How to run the ltp tests:
 - copy ltp.tar.gz on the cluster
 - untar ltp.tat.gz in /opt/ltp
 - run /opt/ltp/runtest


Signed-off-by: Daniel Stoica <>
Change-Id: Ia5a719653b1e0ff89dcb77320a71ead6e247fbff

11 months agoDefault value to SSH_KEYFILE 89/1889/1
Naga Sugguna [Thu, 31 Oct 2019 20:22:39 +0000 (20:22 +0000)]
Default value to SSH_KEYFILE

Jira: VAL-75

Signed-off-by: Naga Sugguna <>
Change-Id: I389bb17cb08e2be3f94f2e5a378e655fca0f9288

11 months agoAdd a new parameter to blucon script 72/1872/2
valentin.radulescu [Tue, 29 Oct 2019 15:52:12 +0000 (08:52 -0700)]
Add a new parameter to blucon script

Jira: [VAL-72]
Add a parameter which specifies container network.

Signed-off-by: valentin.radulescu <>
Change-Id: I9defc4a122db4c4f6309d4a98f29500849eeeeec

11 months agoMerge "Add iec tests"
Juha Kosonen [Mon, 28 Oct 2019 12:59:13 +0000 (12:59 +0000)]
Merge "Add iec tests"

11 months agoAdd iec tests 20/1820/2
valentin.radulescu [Tue, 22 Oct 2019 13:18:51 +0000 (06:18 -0700)]
Add iec tests

Increase timeout to allow sonobuoy containers to start.

Signed-off-by: valentin.radulescu <>
Change-Id: I82998aaa2b1a04dc99a05a2cf5144df19bf21110