ci-management.git
9 days agoFix: Add caret to escape Jenkins vars 14/4414/1 master
Eric Ball [Wed, 15 Sep 2021 19:41:59 +0000 (12:41 -0700)]
Fix: Add caret to escape Jenkins vars

Change-Id: I0e5369667f8d0b17f4e470ccdadf0ffc07f66362
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
4 weeks agoFix: Add blank lines to npm and pip managed config 01/4401/1
Anil Belur [Tue, 24 Aug 2021 00:03:35 +0000 (10:03 +1000)]
Fix: Add blank lines to npm and pip managed config

The blank lines are required as a part of JCASC config
or Jobs fail on file not found exception.

Issue-ID: LF-Jira RELENG-3890
Change-Id: Ia8455dcbfcb311b49a0895737866d477bb0c831e
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 weeks agoFix: Add jenkins-init-script to managed files 95/4395/1
Eric Ball [Thu, 12 Aug 2021 22:32:05 +0000 (15:32 -0700)]
Fix: Add jenkins-init-script to managed files

Change-Id: I3117f16f789597b9e020b013f9f03ad7e07fd3db
Issue-ID: LF-Jira RELENG-3737
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
6 weeks agoChore: Update global-jjb to v0.65.2 94/4394/1
Eric Ball [Thu, 12 Aug 2021 18:09:42 +0000 (11:09 -0700)]
Chore: Update global-jjb to v0.65.2

Change-Id: Id350a2d145d8b6d58df6289ed5e9e3a54c853e16
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
7 weeks agoFeat: Add JCASC managed config files 85/4385/2
Eric Ball [Tue, 3 Aug 2021 19:25:08 +0000 (12:25 -0700)]
Feat: Add JCASC managed config files

Change-Id: I89b80cd83c7cb26fbb767ec71a40355e3bc6d819
Issue-ID: LF-Jira RELENG-3737
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
2 months agoUpdate global-jjb to v0.65.1 77/4377/1
Eric Ball [Tue, 20 Jul 2021 20:25:29 +0000 (13:25 -0700)]
Update global-jjb to v0.65.1

Change-Id: I94d5d7c600c10346e2415b2f5d03148473f00d2b
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
2 months agoMerge "Automation adds pred-vanet-mec.yaml"
Eric Ball [Wed, 14 Jul 2021 21:15:56 +0000 (21:15 +0000)]
Merge "Automation adds pred-vanet-mec.yaml"

2 months agoDrop AVAILABILITY_ZONE 71/4371/1
Mohammed Naser [Wed, 14 Jul 2021 17:03:38 +0000 (13:03 -0400)]
Drop AVAILABILITY_ZONE

By default, OpenStack will schedule you into the right availablity
zone that has capacity, hard coding this can potentially result in
not being able to provision new virtual machines.

This patch drops the hard-wired value and lets Nova on the
scheduling size pick the correct availability zone.

Signed-off-by: Mohammed Naser <mnaser@vexxhost.com>
Change-Id: Ibd3759a5f41af9ffa8f652dd47a388c5572c33d6

2 months agoAutomation adds pred-vanet-mec.yaml 59/4259/2
LF Jenkins CI [Tue, 13 Jul 2021 19:52:13 +0000 (19:52 +0000)]
Automation adds pred-vanet-mec.yaml

Change-Id: I6191b5e94eac6b58d73694fbd3ad678448137543
Signed-off-by: lf-jobbuilder <releng+lf-jobbuilder@linuxfoundation.org>
3 months agoMove install of "rename" into ARM conditional 19/4319/1
Paul Carver [Wed, 9 Jun 2021 15:36:40 +0000 (11:36 -0400)]
Move install of "rename" into ARM conditional

In order to fix the ARM build it was necessary to install the "rename"
command, but that was done via apt which does not exist on CentOS (the
ARM build is done on Ubuntu, the AMD build on CentOS) and the "rename"
command is not required for the AMD build. This change fixes that
mistake.

Signed-off-by: Paul Carver <pcarver@att.com>
Change-Id: I49755198a5774a611ae5977d7f17645461d08302

3 months agoAutomation adds cassini.yaml 56/4256/2
LF Jenkins CI [Tue, 1 Jun 2021 20:59:22 +0000 (20:59 +0000)]
Automation adds cassini.yaml

Change-Id: I450c2f3256c162ec1bef7adbd553583172eff7e8
Signed-off-by: lf-jobbuilder <releng+lf-jobbuilder@linuxfoundation.org>
4 months agoAutomation adds security.yaml 53/4253/2
LF Jenkins CI [Mon, 17 May 2021 21:04:17 +0000 (21:04 +0000)]
Automation adds security.yaml

Change-Id: I505491bfce68338ec933a9010a58fb7094a8dc46
Signed-off-by: lf-jobbuilder <releng+lf-jobbuilder@linuxfoundation.org>
4 months agoChange weekly crons to "@weekly", hourly to daily 45/4245/1
Eric Ball [Thu, 13 May 2021 22:01:05 +0000 (15:01 -0700)]
Change weekly crons to "@weekly", hourly to daily

@weekly will provide automatic load balancing from Jenkins, rather
than running all jobs at the same time. As for docker jobs, hourly
is simply more frequent than is necessary, putting an unnecessary
strain on resources.

Change-Id: Ib370637c561d67a7ac3befe2eb3889c3c2b68b69
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
4 months agoUpdate global-jjb to v0.63.1 44/4244/1
Eric Ball [Thu, 13 May 2021 21:20:45 +0000 (14:20 -0700)]
Update global-jjb to v0.63.1

Change-Id: If958365a5076c57797d018b2ffad6e113037180d
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
5 months agoReplace ubuntu1604 with ubuntu1804 04/4204/3
Eric Ball [Thu, 1 Apr 2021 19:18:04 +0000 (12:18 -0700)]
Replace ubuntu1604 with ubuntu1804

Replace the agent and all scripts that reference it.

Issue: LF-Jira RELENG-3483
Change-Id: Iffe5d0a2a0a2dc50ccb20f5d32a9bcc3c5b7fcca
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
5 months agoUpdate packer jobs with CentOS 8 & Ubuntu 18.04 98/4198/1
Eric Ball [Tue, 30 Mar 2021 22:04:45 +0000 (15:04 -0700)]
Update packer jobs with CentOS 8 & Ubuntu 18.04

The top priority is to remove all Ubuntu 16.04 jobs, as the OS is EOL.
This also includes updates to global-jjb and common-packer.

Issue: LF-Jira RELENG-3483
Change-Id: Ifd1705cb5401814231ff2cb1d5fbdce0901cd3ef
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
6 months agoDisable iec jobs 83/4183/2
Eric Ball [Fri, 12 Mar 2021 21:03:47 +0000 (13:03 -0800)]
Disable iec jobs

The iec jobs are hanging, and never recover or timeout. They need to
be disabled until they have been fixed by the IEC team.

Change-Id: Ib3aeebda81a009a5801e091eeaeed41bfe8ad9a7
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
7 months agoMerge "Fix broken ARM64 REC build"
Eric Ball [Fri, 12 Feb 2021 18:02:20 +0000 (18:02 +0000)]
Merge "Fix broken ARM64 REC build"

7 months agoFix broken ARM64 REC build 54/4154/2
Paul Carver [Thu, 4 Feb 2021 15:55:38 +0000 (15:55 +0000)]
Fix broken ARM64 REC build

The REC ARM64 build has been broken for a while, likely due to the
change of Jenkins builder nodes from Ubuntu 16.04 to 18.04. A number of
fixes are needed, not all known at this time. This change is a start at
fixing them.

1) Force installation of libvirt-daemon-system because the start of
   libvirt is failing when it used to succeed.

2) Install "rename" command. This was present in Ubuntu 16.04, missing
   in 18.04 and back again in 20.04. The Arm build hosts were changed
   from 16.04 to 18.04 a while back and the build has been broken ever
   since (due to multiple issues).

Change-Id: I1cf7ed62bd796bb20da21c9205487c0c6a079e85
Signed-off-by: Paul Carver <pcarver@att.com>
7 months agoAutomation adds tami.yaml 34/4134/2
LF Jenkins CI [Fri, 5 Feb 2021 01:57:15 +0000 (01:57 +0000)]
Automation adds tami.yaml

Change-Id: I960b163805b1bbab60f1f52519fb2ace639d92ef
Signed-off-by: lf-jobbuilder <releng+lf-jobbuilder@linuxfoundation.org>
7 months agoUpdate global-jjb to v0.60.5 60/4060/1
Eric Ball [Tue, 26 Jan 2021 17:56:53 +0000 (09:56 -0800)]
Update global-jjb to v0.60.5

Change-Id: I0e88feb16957619791acf213e60a6cd2ecbd624b
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
8 months agoMerge "Update common-packer to v0.7.1"
Andrew Grimberg [Tue, 29 Dec 2020 21:52:46 +0000 (21:52 +0000)]
Merge "Update common-packer to v0.7.1"

8 months agoUpdate common-packer to v0.7.1 14/4014/1
Eric Ball [Tue, 29 Dec 2020 21:34:40 +0000 (13:34 -0800)]
Update common-packer to v0.7.1

This fix is needed to resume packer builds, which are currently
broken.

Issue: LF-Jira RELENG-3344
Change-Id: I490c3d0ef8c59ae981232de0edc5dcaf13238b26
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
8 months agoReplace python-decoratortools per error message 77/3977/5
Paul Carver [Tue, 8 Dec 2020 18:31:36 +0000 (13:31 -0500)]
Replace python-decoratortools per error message

According to the error message during the ARM64 verify job,
python-decoratortools is not available but the package
python-peak.util.decorators replaces it.

See
https://jenkins.akraino.org/job/ta-infra-ansible-arm64-master-verify/47/console
for details of the error.

Additionally, there seems to be an error due to a conflict between the
Ubuntu 18.04 of the builder VM vs the CentOS packages we're trying to
build. It seems the conflict didn't occur on the older Ubuntu 16.04
builder VMs that we used to use. Since the builder VMs get thrown away
and we don't really care about them after we finish building, we'll
force overwrite the Ubuntu version of '/usr/bin/annotate-output' with
the CentOS copy that is coming from the CentOS RPM that we care about.

This change also fixes a harmlessly incorrect redirection of stderr.

Finally, the build is failing due to missing lxml and urllib3 modules
for Python. It's not clear where these were coming from before the issue
mentioned above, but they are needed so this change adds them.

Signed-off-by: Paul Carver <pcarver@att.com>
Change-Id: Ib4e446190f718ef326a02087dc88fbfd8bdd293e

9 months agoAdd GCP job for KNI Industrial Edge 27/3927/2
Ricardo Noriega [Fri, 20 Nov 2020 13:56:11 +0000 (14:56 +0100)]
Add GCP job for KNI Industrial Edge

Signed-off-by: Ricardo Noriega <rnoriega@redhat.com>
Change-Id: Ia1211399f40233d816b7d122af7a236d4270a40a

10 months agoUse SCM 'gerrit' choosing strategy for kni jobs 96/3896/1
James E. Blair [Thu, 12 Nov 2020 21:38:45 +0000 (13:38 -0800)]
Use SCM 'gerrit' choosing strategy for kni jobs

The KNI jobs appear to be building the master branch even after
fetching proposed changes to gerrit.  Change the choosing strategy
from 'default' to 'gerrit' to correct this.

See https://plugins.jenkins.io/gerrit-trigger/ for docs.

Change-Id: I028cde257415b2eb297cfa100f68d109df3fac71
Signed-off-by: James E. Blair <jeblair@redhat.com>
10 months agoAdd job for blueprint-management-hub 86/3886/3
Ricardo Noriega [Fri, 6 Nov 2020 17:39:43 +0000 (18:39 +0100)]
Add job for blueprint-management-hub

  Also adapt PAE job to new script

Signed-off-by: Ricardo Noriega <rnoriega@redhat.com>
Change-Id: I71566641949b401501b6248ca466de699aa49a93

10 months agoMerge "Don't run instance collection on deploy jobs"
Eric Ball [Thu, 5 Nov 2020 22:24:00 +0000 (22:24 +0000)]
Merge "Don't run instance collection on deploy jobs"

10 months agoDon't run instance collection on deploy jobs 02/3802/6
James E. Blair [Mon, 19 Oct 2020 21:47:47 +0000 (14:47 -0700)]
Don't run instance collection on deploy jobs

The deploy jobs are run on dedicated hosts that don't have the
pre-requisites needed by the lf-infra-publishers (eg, facter,
xmlstarlet, click).  This change adds the contents of the
lf-infra-publish macro except for these post-build scripts:

  - lf-infra-sysstat
  - lf-infra-package-listing
  - lf-infra-capture-instance-metadata

And uses a modified log ship builder (Nexus only, not S3)
with fewer dependencies.

Signed-off-by: James E. Blair <jeblair@redhat.com>
Change-Id: Ia7603a70fbe1993d154d2bbafa50026be1627599

11 months agoAdd arm64 builder with aarch64_{build,dev} labels 03/3803/1
Eric Ball [Wed, 21 Oct 2020 00:52:45 +0000 (17:52 -0700)]
Add arm64 builder with aarch64_{build,dev} labels

Change-Id: I688c595d3d226481adb45b0e5eef5624e2d1c964
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
11 months agoAdd ubuntu18.04-arm64 to packer builds 98/3798/1
Eric Ball [Thu, 15 Oct 2020 20:48:21 +0000 (13:48 -0700)]
Add ubuntu18.04-arm64 to packer builds

Issue: LF-Jira RELENG-3240
Change-Id: Icf980d0269d1f521c0631145aa1a56ee341b0959
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
11 months agoUpdate global-jjb to 0.57.4,common-packer to 0.7.0 97/3797/1
Eric Ball [Thu, 15 Oct 2020 20:46:16 +0000 (13:46 -0700)]
Update global-jjb to 0.57.4,common-packer to 0.7.0

Change-Id: Icfe153e0ed559007827788dbebd6512c1a6799bd
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
11 months agoUse kni-cloud node for cloud jobs 94/3794/1
James E. Blair [Wed, 14 Oct 2020 16:33:53 +0000 (09:33 -0700)]
Use kni-cloud node for cloud jobs

And kni-baremetal for baremetal jobs.  The two long-lived jenkins
nodes prd-kni-baremetal-1 and -2 are different; one has cloud
credentials, the other is for bare metal deployments.  Use the
more specific kni-cloud and kni-baremetal labels (already present
on the nodes) to select them for the appropriate jobs.

Signed-off-by: James E. Blair <jeblair@redhat.com>
Change-Id: I20c9601934e8e9b5bdb89c56ef168985891c4fdd

11 months agoRemove KNI job that is no longer needed 93/3793/1
Ricardo Noriega [Tue, 13 Oct 2020 10:22:40 +0000 (12:22 +0200)]
Remove KNI job that is no longer needed

Signed-off-by: Ricardo Noriega <rnoriega@redhat.com>
Change-Id: Ib9d6e92d8df11c3b3fc02b4a382b770db3029658

11 months agoChange labels for new Jenkins slaves 79/3779/2
Ricardo Noriega [Thu, 1 Oct 2020 11:31:26 +0000 (13:31 +0200)]
Change labels for new Jenkins slaves

Signed-off-by: Ricardo Noriega <rnoriega@redhat.com>
Change-Id: I2412be59f6150c481fa987d0bfc745b00a1c1868

12 months agoUpdate global-jjb to the latest v0.57.3 05/3705/3
Anil Belur [Thu, 10 Sep 2020 04:36:09 +0000 (14:36 +1000)]
Update global-jjb to the latest v0.57.3

Update defaults with the required inputs for the openstack-cron
job email notifications.

Change-Id: Ic9c3dc0272abdc4d817fc32bf4da45a7979ae966
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
13 months agoRemove unused cloud builder configs & packer jobs 64/3664/1
Eric Ball [Thu, 6 Aug 2020 21:10:52 +0000 (14:10 -0700)]
Remove unused cloud builder configs & packer jobs

Since these are not being used anywhere in ci-management, they no
longer need to be maintained. This will simplify maintenance, and the
builders can always be re-added if they become necessary again.

Change-Id: I04a0dffc2672eb84958b6279081b858d4f2f8cf5
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
13 months agoFix instance volume to a size that works 63/3663/1
Andrew Grimberg [Wed, 5 Aug 2020 23:50:18 +0000 (16:50 -0700)]
Fix instance volume to a size that works

Change-Id: I41093f9d84db3a9d95becf7a9d73cb2baad40ab5
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
13 months agoUpdate cloud images to use a volume and v2 flavors 62/3662/1
Andrew Grimberg [Wed, 5 Aug 2020 23:19:39 +0000 (16:19 -0700)]
Update cloud images to use a volume and v2 flavors

Change-Id: Ife2ba1f36406a39a9c60d4a13ffed125d13825c7
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
13 months agoClone missing iec repo if not found 24/3624/2
Ciprian Barbu [Thu, 16 Jul 2020 06:20:15 +0000 (09:20 +0300)]
Clone missing iec repo if not found

Make sure iec repo is present for rec clusters.

At the same time, make sure K8S_MASTER_IP is exported in the ssh session
shell when running the install script

Signed-off-by: Ciprian Barbu <ciprian.barbu@enea.com>
Change-Id: I962359fc954198912faf589bee20c6f528b4875d

14 months agoFix KNI CI jobs 08/3608/2
Ricardo Noriega [Thu, 2 Jul 2020 10:53:44 +0000 (12:53 +0200)]
Fix KNI CI jobs

Currently the repo is pointing to github, not gerrit
so refs are not working.

Signed-off-by: Ricardo Noriega <rnoriega@redhat.com>
Change-Id: Iedf209eb326bd643e9ebe4b3b57448ac683b92e7

15 months agoMerge "iec: fuel: Map new UNH baremetal POD to IEC xDFs"
Eric Ball [Wed, 24 Jun 2020 20:46:45 +0000 (20:46 +0000)]
Merge "iec: fuel: Map new UNH baremetal POD to IEC xDFs"

15 months agoiec, rec: Abstract out common seba_on_arm usecase 81/3581/9
Alexandru Avadanii [Wed, 17 Jun 2020 12:52:01 +0000 (14:52 +0200)]
iec, rec: Abstract out common seba_on_arm usecase

- move seba_on_arm IEC usecase to its own macros/templates files;
- enable seba_on_arm install/test jobs for REC deployments running
  on aarch64 baremetal nodes;
- replace dashes with underscores in jjb variables that need to be
  handled by the jinja2 parser, since jinja2 does not allow dashes
  in variable names;

While at it, add vim temporary files ('*.sw?') to .gitignore.

Change-Id: I62ae4ebfd4955fb484876e3a63c8f02a77e90b38
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
15 months agoiec: fuel: Map new UNH baremetal POD to IEC xDFs 87/3587/1
Alexandru Avadanii [Thu, 18 Jun 2020 16:30:46 +0000 (18:30 +0200)]
iec: fuel: Map new UNH baremetal POD to IEC xDFs

Change-Id: Ibdb345e3eb9fc12be4782414c7f5fa51c333f0d4
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
15 months agoGracefully stop the remote-installer container 65/3565/3
Alexandru Antone [Thu, 11 Jun 2020 13:52:48 +0000 (16:52 +0300)]
Gracefully stop the remote-installer container

Force killing the container with 'rm -f' bypasses the SIGTERM handler
defined in the container's entry point shell script which would
gracefully stop the nfs-server before exiting.

This leaves TCP port 2049 bound on some hosts and subsequent runs of
the remote-installer will fail to properly start NFS shares.

Signed-off-by: Alexandru Antone <Alexandru.Antone@enea.com>
Change-Id: I56d77ffabdb901504193c5e5d6b7d0f73d52488d

15 months agovalidation: Enable validation for REC in UNH lab 55/3555/2
Alexandru Avadanii [Tue, 9 Jun 2020 12:25:45 +0000 (14:25 +0200)]
validation: Enable validation for REC in UNH lab

Add new jobs for running the validation jobs on the REC POD in UNH
lab, similar to running validation jobs on the IEC PODs in Enea lab.

The new job(s) will upload the logs to nexus.akraino.org in the
'unh' lab silo.

Change-Id: I99fcc4854ead367fad8d203eb448908b229d6781
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
15 months agoMerge "validation:Fix jobs cleanup issue with permissions"
Eric Ball [Tue, 26 May 2020 20:51:10 +0000 (20:51 +0000)]
Merge "validation:Fix jobs cleanup issue with permissions"

16 months agovalidation:Fix jobs cleanup issue with permissions 86/3486/3
DANIEL STOICA [Tue, 19 May 2020 09:05:39 +0000 (12:05 +0300)]
validation:Fix jobs cleanup issue with permissions

Change the owner for the kube directory, because might be
owned by root, causing permission issues when Jenkins tries to
perform the workspace cleanup

Signed-off-by: DANIEL STOICA <daniel.stoica@enea.com>
Change-Id: I6fedecb633ab8ce78b03a9d8b9bec8ce9926bfcb

16 months agoRevert "Add signing to Akraino deploy templates" 85/3485/1
Eric Ball [Mon, 18 May 2020 23:05:47 +0000 (16:05 -0700)]
Revert "Add signing to Akraino deploy templates"

This reverts commit 0904fdd5e0e3b113ee90e77c646a7f156d878762.
We are seeing issues with signing very large objects (specifically,
the ISOs being pushed by TA). Until we find a solution for this,
the change needs to be reverted so that it does not cause any further
interruption to normal work.

Change-Id: Ib79b2bbc9aaca92eaf60a2a836bba5e84487ba89
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
16 months agoMerge "update 5g-mec-cloud-gaming jobs to Jenkins"
Eric Ball [Mon, 18 May 2020 23:04:11 +0000 (23:04 +0000)]
Merge "update 5g-mec-cloud-gaming jobs to Jenkins"

16 months agoAdd S3_BUCKET environment variable 84/3484/3
Houa Yang [Mon, 18 May 2020 21:55:01 +0000 (16:55 -0500)]
Add S3_BUCKET environment variable

Add S3_BUCKET variable to address "unrecognized macro" error.
Although the job passes, the error is confusing and unsightly.
The S3_BUCKET environment variable is needed for log shipping to s3.

Signed-off-by: Houa Yang <hyang@contractor.linuxfoundation.org>
Change-Id: I90af7149ab9f9e3e69491f8b37add6cfef54a826

16 months agoupdate 5g-mec-cloud-gaming jobs to Jenkins 36/3436/7
feng [Wed, 13 May 2020 16:25:25 +0000 (00:25 +0800)]
update 5g-mec-cloud-gaming jobs to Jenkins

Change-Id: Ibb074147debc66f4182cf0f74a51e3db644ae90a

16 months agoupdate 5g-mec-cloud-gaming jobs 11/3411/2
feng [Mon, 11 May 2020 01:16:24 +0000 (09:16 +0800)]
update 5g-mec-cloud-gaming jobs

Change-Id: I88bde4a3492c4de9ab99e71cbe0d031247a8c651

16 months agoMerge "Add signing to Akraino deploy templates"
Eric Ball [Fri, 8 May 2020 21:10:09 +0000 (21:10 +0000)]
Merge "Add signing to Akraino deploy templates"

16 months agovalidation:Fix jobs cleanup issue with permissions 07/3407/2
DANIEL STOICA [Fri, 8 May 2020 13:10:00 +0000 (16:10 +0300)]
validation:Fix jobs cleanup issue with permissions

Transfer only the ~/.kube/config file for k8s tests, since the rest
of contents of the "~/.kube/" directory (e.g. http-cache) might be
owned by root, causing permission issues when Jenkins tries to
perform the workspace cleanup. Since only the config file is actually
needed for running the k8s tests, skip other files/directories in
"~/.kube/"

Signed-off-by: DANIEL STOICA <daniel.stoica@enea.com>
Change-Id: Ic3d6823bd45069e66eb085a8c09383461f517d12

16 months agoMerge "Support password-based ssh auth"
Eric Ball [Wed, 6 May 2020 22:44:16 +0000 (22:44 +0000)]
Merge "Support password-based ssh auth"

16 months agoSupport password-based ssh auth 96/3396/7
DANIEL STOICA [Wed, 29 Apr 2020 11:39:15 +0000 (14:39 +0300)]
Support password-based ssh auth

Add ssh password to the run_bluval.sh script command

JIRA: VAL-106

Signed-off-by: DANIEL STOICA <daniel.stoica@enea.com>
Depends-On: I04145b32621dae49485cb8640d80932345957826
Change-Id: I0f2974cd21d59e5b6ad85be37a0e71d064ec70ab

16 months agoMerge "iec: Add robot parsing for SEBA tests"
Eric Ball [Mon, 4 May 2020 22:21:46 +0000 (22:21 +0000)]
Merge "iec: Add robot parsing for SEBA tests"

16 months agoMerge "Automation adds pcei.yaml"
Eric Ball [Mon, 4 May 2020 21:28:54 +0000 (21:28 +0000)]
Merge "Automation adds pcei.yaml"

16 months agoUpdate authentication URI endpoint 99/3399/1
Eric Ball [Mon, 4 May 2020 21:13:17 +0000 (14:13 -0700)]
Update authentication URI endpoint

Change at the hosting provider seems to have caused a problem with the
authentication URI. Removing the trailing '/' fixes the problem.

Change-Id: I7078d528672fb65cc1b68e735cdf25cc11c8e88b
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
16 months agoAutomation adds pcei.yaml 28/3328/2
LF Jenkins CI [Mon, 4 May 2020 19:08:57 +0000 (19:08 +0000)]
Automation adds pcei.yaml

Change-Id: I34ca6643323313df6087de9177d3caae8c1eb6c2
Signed-off-by: lf-jobbuilder <releng+lf-jobbuilder@linuxfoundation.org>
16 months agoAdd signing to Akraino deploy templates 93/3393/3
Eric Ball [Wed, 29 Apr 2020 01:17:53 +0000 (18:17 -0700)]
Add signing to Akraino deploy templates

Since Akraino projects do not follow standard release procedures, we
will simply sign the artifacts before pushing them to Nexus, just to
provide signatures showing that all artifacts come from LF infra.

Change-Id: Ia3478900acd02d419fe49c25b1902b1a3459f73d
Issue-ID: LF-Jira RELENG-2819
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
16 months agoAutomation adds 5g-mec-cloud-gaming.yaml 25/3325/2
LF Jenkins CI [Wed, 29 Apr 2020 20:28:52 +0000 (20:28 +0000)]
Automation adds 5g-mec-cloud-gaming.yaml

Change-Id: I287f0d3682819fe7867bfdff22941eea7d00dc32
Signed-off-by: lf-jobbuilder <releng+lf-jobbuilder@linuxfoundation.org>
16 months agoiec: Add robot parsing for SEBA tests 89/3389/1
Alexandru Avadanii [Mon, 27 Apr 2020 10:37:43 +0000 (12:37 +0200)]
iec: Add robot parsing for SEBA tests

Leverage the Jenkins Robot publisher for parsing the SEBA-in-a-box
test results.

While at it, disable the IEC destroy phase for baremetal deployments,
since aarch64 baremetal PODs are not shared by multiple installers the
same way virtual PODs are, hence there's no need for the in-between jobs
cleanup.

Change-Id: I264a5e4a917ca7e97c48dec18864e48e3637ab8e
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
16 months agoAdded ealt-edge repo verify job 88/3388/1
Srinivasan Selvam [Mon, 27 Apr 2020 07:51:52 +0000 (13:21 +0530)]
Added ealt-edge repo verify job

Signed-off-by: Srinivasan Selvam <srinivasan.s.n@huawei.com>
Change-Id: Iaaabeaa5cb1ba4213d24a84a55e5b5fb6e9bc25f

17 months agoMerge "Remove libvirt provider jobs"
Eric Ball [Fri, 24 Apr 2020 23:40:24 +0000 (23:40 +0000)]
Merge "Remove libvirt provider jobs"

17 months agovalidation: iec: Pull docker images before run 83/3383/1
Alexandru Avadanii [Wed, 22 Apr 2020 13:56:31 +0000 (15:56 +0200)]
validation: iec: Pull docker images before run

Add a new 'PULL' job paramater for validation-<lab>-* jobs, as well
as 'bluval-*' jobs; set it to 'false' by default and 'true' for IEC
validation jobs since most IEC jobs run on static Jenkins slaves where
the validation docker images might already be present from previous
runs.

JIRA: VAL-107

Depends-on: Id4b7947a0de7abf07f69812c6deef71919b9041a

Change-Id: Ic2f690d6627f6ae284a27727f3c98fe03960cbda
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
17 months agoMerge "iec: view: Stop filtering out the sonar job"
Eric Ball [Tue, 21 Apr 2020 21:26:23 +0000 (21:26 +0000)]
Merge "iec: view: Stop filtering out the sonar job"

17 months agobluval: Extend job description with BP, lab, layer 77/3377/1
Alexandru Avadanii [Tue, 21 Apr 2020 16:28:42 +0000 (18:28 +0200)]
bluval: Extend job description with BP, lab, layer

bluval jobs are shared across different blueprints, labs and layers,
so display some useful information in the job history overview pane.

Change-Id: I38e7c700774765066524a99da1e5786c5fec1b58
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
17 months agoiec: view: Stop filtering out the sonar job 76/3376/1
Alexandru Avadanii [Tue, 21 Apr 2020 14:09:16 +0000 (16:09 +0200)]
iec: view: Stop filtering out the sonar job

The 'iec-sonar' job should be displayed under the IEC Jenkins view,
similar to 'iec-tox-verify-master'.

Change-Id: Icfcc0f516f638c59713601fa77e8088178980875
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
17 months agovalidation: Use blucon.sh if python3 not available 73/3373/2
Alexandru Avadanii [Sat, 18 Apr 2020 18:42:56 +0000 (20:42 +0200)]
validation: Use blucon.sh if python3 not available

The Validation project supports running the test suites using nothing
more than Docker as a prerequisite (i.e. if no python3 is available),
so leverage `blucon.sh` now that it supports overriding the validation
git repository path and results dir path.

For most slaves, the new method is merely a fail-safe, since they
already have python3.

JIRA: VAL-95

Depends-on: I4d3c10881de28c64bcca05ff23aa1025f67a1f5e

Change-Id: I08e89300637862e40344131532ec40efd5f68501
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
17 months agovalidation: bluval: Use own repo logparser rules 69/3369/2
Alexandru Avadanii [Fri, 17 Apr 2020 15:59:35 +0000 (17:59 +0200)]
validation: bluval: Use own repo logparser rules

Instead of creating the logparser rules.txt file on the fly, rely on
the project's git repository providing it.

While at it, add and configure the 'robot' publisher in order to enable
Jenkins dashboard integration and display some nice summaries directly
in the 'blueval' view (using the 'Robot Results + Duration Trend'
column), as well as on the job's overview webpage.

JIRA: VAL-104

Depends-on: I1506d84a44926e80913562f67f24885a98f94df7

Change-Id: I971ae6f8949caac8debc1aee0f35e0b35b8f65e6
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
17 months agoMerge "iec-sonar: Disable git submodule cloning"
Eric Ball [Thu, 16 Apr 2020 19:01:42 +0000 (19:01 +0000)]
Merge "iec-sonar: Disable git submodule cloning"

17 months agoMerge "iec: seba_on_arm: fail job on error"
Eric Ball [Thu, 16 Apr 2020 18:59:54 +0000 (18:59 +0000)]
Merge "iec: seba_on_arm: fail job on error"

17 months agoiec-sonar: Disable git submodule cloning 66/3366/1
Alexandru Avadanii [Thu, 16 Apr 2020 14:37:27 +0000 (16:37 +0200)]
iec-sonar: Disable git submodule cloning

SonarQube should be limited to scanning the IEC repository contents
and not its (massive) collection of git submodules which include various
upstream components (including the full K8s source code, among other
large code bases).

Although we apply some inclusion/exclusion filtering as part of
SonarQube configuration, it seems that the large collection of git
submodules interferes with log collection and upload; hence disable
cloning git submodules before running SonarQube.
This is an attempt at fixing the ever-hanging iec-sonar job in CI.

Change-Id: If6b9b205d8f1924ca8180787a92415494461d5b7
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
17 months agoiec: seba_on_arm: fail job on error 64/3364/1
Alexandru Avadanii [Thu, 16 Apr 2020 12:52:40 +0000 (14:52 +0200)]
iec: seba_on_arm: fail job on error

Previously, seba_on_arm install/test jobs used to always return a zero
exit code, marking the job successful even when it failed.
Add `set -e` so the job is marked as failed when errors occur.

Change-Id: Id67ac0695255e475f1e6147266ac7051d3567913
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
17 months agovalidation: Only run tagged docker build once 60/3360/2
Alexandru Avadanii [Wed, 15 Apr 2020 19:11:34 +0000 (21:11 +0200)]
validation: Only run tagged docker build once

Previously, we used to manually change the release tag for the Docker
container image(s) built for the validation project, let the build run
once, then create another change that disabled the job to prevent
further (useless) rebuilds happening each night.

Instead of triggering the docker build job for the release (tag) using
cron, switch to triggering the job based on gerrit merge events
affecting the ci-management repository and only if the said merge event
affects the 'validation-docker.yaml' configuration file (the one where
the release tag is defined/updated).

Change-Id: Ia4493b558ee7d1319c42a04690aa1c7580eb3731
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
17 months agovalidation: Release tag 3.0.0 59/3359/4
Alexandru Avadanii [Wed, 15 Apr 2020 16:16:07 +0000 (18:16 +0200)]
validation: Release tag 3.0.0

Add jobs that build docker container images based on [1] and tag them
accordingly.

[1] https://gerrit.akraino.org/r/gitweb?p=validation.git;h=refs/tags/3.0.0

Change-Id: Idc98e28dcb2cc99ffba5c7dafbcd86dfc761a874
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
17 months agoMerge "iec: Stop upstream job if IEC installation fails"
Eric Ball [Wed, 15 Apr 2020 18:17:19 +0000 (18:17 +0000)]
Merge "iec: Stop upstream job if IEC installation fails"

17 months agoMerge "validation, iec: Rework common parameters list"
Eric Ball [Wed, 15 Apr 2020 18:16:07 +0000 (18:16 +0000)]
Merge "validation, iec: Rework common parameters list"

17 months agovalidation, iec: Rework common parameters list 58/3358/1
Alexandru Avadanii [Wed, 15 Apr 2020 14:35:40 +0000 (16:35 +0200)]
validation, iec: Rework common parameters list

Instead of abusing yaml anchors, use jjb inheritance of parameter
lists.

While at it, fix IEC Compass default value of private key, which
should be empty (Compass uses user/pass auth).

Change-Id: I5528b6ce4dcf648ec416194bfcbb88bddc1c8c39
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
17 months agoiec: Stop upstream job if IEC installation fails 57/3357/1
Alexandru Avadanii [Wed, 15 Apr 2020 13:35:29 +0000 (15:35 +0200)]
iec: Stop upstream job if IEC installation fails

Previously, we were using a single `trigger-builds` builder, which as
a consequence led to running all downstream projects even if the first
one (installing the IEC platform) failed.

Switch to using 2 different `trigger-builds` builders so:
- if IEC installation fails, no other downstream jobs are executed and
  the upstream (parent) job is marked as failed;
- if any other downstream job (except SEBA install/test jobs) fails,
  the upstream (parent) job is marked as failed/unstable, but all other
  downstream jobs are still executed (so we can still install/test SEBA
  and destroy the IEC installation at the end even if the 'validation'
  job failed);

Change-Id: I946c5bc9f2cc35afcbeb64d954beb10f3281fc11
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
17 months agoFix SCM macro usage of refspec, branches 56/3356/4
Alexandru Avadanii [Tue, 14 Apr 2020 13:55:55 +0000 (15:55 +0200)]
Fix SCM macro usage of refspec, branches

- fix ignored refspec SCM parameter (hardcoded to empty string):
  s/refspec: ''/refspec: '$GERRIT_REFSPEC'/g
- fix IEC SCM configuration regression (broken in previous commit):
  s/branches: '$GERRIT_BRANCH'/branch: 'GERRIT_BRANCH'/g
- add missing lf-infra-parameters macro to bluval job;
- use lf-infra-gerrit-scm macro in IEC templates (now that submodule
  recursive cloning parametrization is supported);
- retire 'validation-job-parameters' parameters macro, which is only
  an alias for 'lf-infra-parameters';
- rework validation SCM handling for refs/tags/...;

JIRA: VAL-103

Change-Id: I94957fb02afdfe8a01330e768dc5fde15f5ea05d
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
17 months agoMerge "iec: exclude seba_on_arm from virtual PODs"
Eric Ball [Wed, 15 Apr 2020 01:46:39 +0000 (01:46 +0000)]
Merge "iec: exclude seba_on_arm from virtual PODs"

17 months agoMerge "Fix k8s tests run in CI"
Eric Ball [Wed, 15 Apr 2020 01:46:09 +0000 (01:46 +0000)]
Merge "Fix k8s tests run in CI"

17 months agoiec: Add jobs for testing usecases 52/3352/3
Ciprian Barbu [Fri, 10 Apr 2020 14:17:30 +0000 (17:17 +0300)]
iec: Add jobs for testing usecases

Also rework the iec-seba-on-arm.sh script to reuse the existing
facilities which deal with installer and OS specifics

JIRA: IEC-14

Signed-off-by: Ciprian Barbu <ciprian.barbu@enea.com>
Change-Id: Ida04b93030751b4dbdfc929107d0f30d644974fa

17 months agoiec: exclude seba_on_arm from virtual PODs 51/3351/2
Alexandru Avadanii [Fri, 10 Apr 2020 12:47:06 +0000 (14:47 +0200)]
iec: exclude seba_on_arm from virtual PODs

Virtual PODs are slow and often cause issues with the SEBA on arm
deployments, raising misleading errors in our testing.

Since now we have a baremetal POD that will cover SEBA testing,
exclude the flaky virtual installations for SEBA.

Change-Id: I147bdd13d5d213837fefb25e8d299bb91e60ebf5
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
17 months agoFix k8s tests run in CI 54/3354/2
DANIEL STOICA [Tue, 14 Apr 2020 08:28:14 +0000 (11:28 +0300)]
Fix k8s tests run in CI

Enable k8s config file transfer if layer is empty

JIRA: IEC-41

Signed-off-by: DANIEL STOICA <daniel.stoica@enea.com>
Change-Id: Iac7772326271c811efd714bc9f66309581ff4dc1

17 months agoMerge "iec: aarch64 baremetal POD: map name, cron trigger"
Eric Ball [Tue, 14 Apr 2020 06:05:54 +0000 (06:05 +0000)]
Merge "iec: aarch64 baremetal POD: map name, cron trigger"

17 months agoRemove libvirt provider jobs 53/3353/1
Yolanda Robla [Mon, 13 Apr 2020 14:04:25 +0000 (16:04 +0200)]
Remove libvirt provider jobs

With the addition of virtual baremetal, we do not
need to use libvirt jobs anymore, they were buggy
and faulty so let's retire them.

Signed-off-by: Yolanda Robla <yroblamo@redhat.com>
Change-Id: I5fd196c6328b86f7ca9451b1c19e0fb75fb70289

17 months agoiec: aarch64 baremetal POD: map name, cron trigger 42/3342/8
Alexandru Avadanii [Tue, 7 Apr 2020 10:34:34 +0000 (12:34 +0200)]
iec: aarch64 baremetal POD: map name, cron trigger

IEC Fuel deploy scripts need a mapping between the Jenkins node (slave)
name and its lab/pod naming scheme, i.e. for the new baremetal POD
`ubuntu1804-dev-96c-256g-1` we will use `arm-baremetal1` for PDF/IDF
filenames.

While at it, update related jobs:
- extend ci-management static slave init jobs to also generate jobs
  for the new AArch64 baremetal POD label (aarch64_baremetal_dev).
- exclude Ubuntu 16.04 / CentOS from Fuel baremetal job matrix since
  baremetal provisioning in Fuel only supports Ubuntu 18.04 currently;
- schedule IEC Type 2 Fuel baremetal deployment every day @ 01:00;
- fix GERRIT_REFSPEC job paramater being ignored for IEC jobs;

JIRA: IT-19425
JIRA: IT-19445

Change-Id: I1e1debb7ad61e6aac7a6304ecc2bd5fd6e9fe3ce
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
17 months agovalidation: Move docker build job to aarch64_build 49/3349/1
Alexandru Avadanii [Thu, 9 Apr 2020 11:23:12 +0000 (13:23 +0200)]
validation: Move docker build job to aarch64_build

Previously, validation docker build jobs used to run on 'aarch64_dev'
labeled slaves, which are also used by IEC installers (compass, fuel)
that interfere with the machine's Docker service (usually by restarting
the Docker service when the install/deploy is started; which
unfortunately overlaps quite frequently with the validation docker build
job). Of course, this is also possible due to the 'aarch64_dev' nodes
having more than 2 executors (currently 3).

To avoid this, we could add build-blockers to both IEC install jobs and
validation docker build jobs, but that would create unnecessary cross
dependencies between IEC and validation projects.
Instead, move the validation Docker build to nodes servicing the label
'aarch64_build', which only have 2 executors and currently only run TA
RPM/ISO build jobs that do not interfere in a destructive way with the
Docker service on the machine.

JIRA: VAL-101

Change-Id: Iebc57a6a710b96ee6dcb04ea5e98243af3b44fa8
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
17 months agoChange tests to run in virtual baremetal 47/3347/1
Yolanda Robla [Wed, 8 Apr 2020 09:28:13 +0000 (11:28 +0200)]
Change tests to run in virtual baremetal

For daily tests, change the job to run in
virtual baremetal, in order to speed up and
preserve hardware.

Signed-off-by: Yolanda Robla <yroblamo@redhat.com>
Change-Id: I780544fab4e2144a1d7b3f4d1ae72f632b4e3132

17 months agoRun test for multiple layers on IEC blueprint 38/3338/3
DANIEL STOICA [Fri, 3 Apr 2020 13:25:06 +0000 (16:25 +0300)]
Run test for multiple layers on IEC blueprint

Initially only the k8s layer was tested for IEC.
With the introduction of more mandatory tests, allow testing
for multiple layers, by not specifying any particular layer.

Signed-off-by: DANIEL STOICA <daniel.stoica@enea.com>
Depends-On: I5c19b527ec86be0a60b7c31fcd579f0802e9874d
Change-Id: I4d36305d9ffd6fb7688002b6cb72a18baed77803

17 months agoEnable vuls and lynis tests in CI 07/3307/7
DANIEL STOICA [Tue, 24 Mar 2020 13:30:23 +0000 (15:30 +0200)]
Enable vuls and lynis tests in CI

 - created the ssh_key_dir and put the ssh_key
 - set host and username variable in validation/tests/variables.yaml
 - set ssh_key_dir and custom_variables_file path in validatin/bluval/volumes.yaml

Signed-off-by: DANIEL STOICA <daniel.stoica@enea.com>
Depends-On: Ibb16f76405f5b43797e0d859db0681759adf5e18
Change-Id: I9e5349a0e3e3a1e7331a5f5b4b4a8d5a1b01aa51

18 months agoMerge "Enable installation of TA in CI"
Eric Ball [Wed, 18 Mar 2020 17:06:16 +0000 (17:06 +0000)]
Merge "Enable installation of TA in CI"

18 months agoMerge "Fix installation bug on Ubuntu 18.04"
Eric Ball [Tue, 17 Mar 2020 16:29:48 +0000 (16:29 +0000)]
Merge "Fix installation bug on Ubuntu 18.04"

18 months agoEnable installation of TA in CI 75/2275/24
Cristina Pauna [Wed, 4 Mar 2020 10:00:59 +0000 (12:00 +0200)]
Enable installation of TA in CI

For each REC POD, the CD job is tied to a specific jenkins slave
that has access to the POD's lab.

For now, only UNH Ampere1 POD is added and user_config.yaml is stored
on the slave at:
  file:///data/rec_pod/user_config.yaml

Also for CD deployments, admin_user_password in user_config.yaml is
required to be the hash for "admin" to be able to pull post deployment
logs from the first controller.

Jenkins slave package requirements:
  - docker
  - jq

Co-authored-by: Cristina Pauna <cristina.pauna@enea.com>
Co-authored-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>>
Signed-off-by: Alexandru Antone <Alexandru.Antone@enea.com>
Change-Id: Iece4330c3242385c33d8a1daa092f78ce5d46cc8

18 months agoFix installation bug on Ubuntu 18.04 93/2293/2
Jingzhao.Ni [Thu, 5 Mar 2020 10:03:22 +0000 (18:03 +0800)]
Fix installation bug on Ubuntu 18.04

When deploying VMs on Ubuntu 18.04, the compass can not resolve the
host ip address correctly. It will cause the compass4nfv deploying
failed.
In this patch, it gets compass4nfv code from iecedge repository.
Some modification has been done for fixing those issues on specific
branch.

Signed-off-by: Jingzhao.Ni <Jingzhao.Ni@arm.com>
Change-Id: I86faffee4473ed2f6f36594a2b14be44f7b77191

18 months agoMerge "Increase timeout of libvirt build to 120"
Eric Ball [Thu, 5 Mar 2020 15:31:01 +0000 (15:31 +0000)]
Merge "Increase timeout of libvirt build to 120"