2 Akraino Blueprint Validation UI
8 This project contains the source code of the Akraino Blueprint Validation UI. It is based on the ONAP portal SDK, version 2.4.0. It should be noted that the copyright of all the files of the aforementioned project that were left intact, has not been changed.
10 This UI consists of the front-end and back-end parts.
12 The front-end part is based on HTML, CSS, and AngularJS technologies. The back-end part is based on Spring MVC and Apache Tomcat technologies.
14 Based on these instructions, a user can provide the prerequisites, compile the source code and deploy the UI.
19 The blueprint validation UI aims to be hosted by LF servers and will be exposed using public IP and domain names.
21 It provides a user-friendly way for displaying blueprints validation test results. Based on these results, the status of a blueprint can be determined (mature, incubation state, etc.).
23 In specific, the purpose of the UI is twofold:
25 1) Support full control loop of producing results. In this mode, the UI must be connected with a Jenkins instance capable of running blueprint validation tests.
26 It will enable the user to define a blueprint for validation using its name, version, layer, desired lab and desired timeslot. This data constitutes a submission. It should be noted that the blueprint family is derived from the blueprint name.
27 Also, the UI will have the ability to track the lifecycle of a submission. A submission state can be one of the following: submitted, waiting, running and completed. The implementation vehicle for this action is the REST API of Jenkins.
28 Moreover, the UI must be connected with a mariadb instance and the Nexus server where the results are stored.
29 Then, it will be able to trigger the appropriate job in Jenkins and receive the corresponding results from Nexus.
30 Note that it makes no difference whether the Jenkins instance is the community one or a private one.
31 2) Partial control of producing results. In this mode, the UI must be connected with a mariadb instance and the Nexus server where the results are stored.
32 Every blueprint owner is responsible of executing tests and storing results in Nexus using his/her own Jenkins instance. The UI only retrieves results from Nexus and displays them.
34 In both modes, user authentication, authorization and accounting (AAA) will be supported in order to control access to resources, enforce policies on these resources and audit their usage.
39 In order for the blueprint validation UI to be functional, the following items are taken for granted:
41 - An appropriate mariadb instance is up and running (look at the Database subsection).
42 This prerequisite concerns both of the UI modes.
44 - The available labs for blueprint validation execution are defined by the corresponding lab owners (look at the Database subsection). It is their responsibility to publish them. Currently, this data is statically stored in the blueprint validation UI mariadb database. In order for a lab owner to update them, he/her must update the corresponding table entries. This inconvenience will be handled in the future.
45 This prerequisite concerns only the full control loop mode.
47 - The available timeslots for blueprint validation execution of every lab are defined by the corresponding lab owners (look at the Database subsection). It is their responsibility to publish them. Currently, this data is statically stored in the blueprint validation UI mariadb database. In order for a lab owner to update them, he/her must update the corresponding table entries. This inconvenience will be handled in the future.
48 This prerequisite concerns only the full control loop mode.
50 - The data of the lab silos (i.e. which silo is used by a lab in order to store results in Nexus) is stored in the mariadb database (look at the Database subsection). It is the blueprint owner's responsibility to publish it. Currently, this data is statically stored in the blueprint validation UI mariadb database. In order for a blueprint owner to update it, he/her must update the corresponding table entries. This inconvenience will be handled in the future.
51 This prerequisite concerns only the full control loop mode.
53 - The data of available blueprints (i.e. blueprint name) is stored in the mariadb database (look at the Database subsection). It is the blueprint owner's responsibility to publish it. Currently, this data is statically stored in the blueprint validation UI mariadb database. In order for a blueprint owner to update it, he/her must update the corresponding table entries. This inconvenience will be handled in the future.
54 This prerequisite concerns only the full control loop mode.
56 - The data of an available blueprint instance for validation (i.e. version, layer and description of the layer) is stored in the mariadb database (look at the Database subsection). It is the blueprint owner's responsibility to publish it. Currently, this data is statically stored in the blueprint validation UI mariadb database. In order for a blueprint owner to update it, he/her must update the corresponding table entries. This inconvenience will be handled in the future.
57 This prerequisite concerns only the full control loop mode.
59 - A Jenkins instance exists capable of executing blueprint validation tests on the specified lab and storing the results to Nexus server (look at the Jenkins configuration subsection).
60 This prerequisite concerns only the full control loop mode.
62 - A Nexus server exists where all the blueprint validation results are stored (look at the Nexus subsection).
63 This prerequisite concerns both of the UI modes.
65 - The whole installation and deployment of a blueprint and its corresponding blueprint family components (i.e. the appropriate edge cloud stack with its combination of infrastructure hardware components, OS, K8s, software, etc) are already performed in the appropriate lab.
66 Recall that multiple labs can be used for a specific blueprint validation. Also, it is the responsibility of the blueprint submitter to ensure that the edge validation and community CI labs can support comprehensive validation of the blueprint and cover all use case characteristics.
67 This prerequisite concerns both of the UI modes.
75 .. code-block:: console
77 git clone "https://gerrit.akraino.org/r/validation"
84 In order to setup the development environment, the following tools are needed:
90 Execute the commands below in order to install these tools (note that the PROXY_IP and PROXY_PORT variables must be substituted with the ones that are used by the hosting operating system)
92 If the host is behind a proxy, define this proxy using the following commands:
94 .. code-block:: console
95 sudo touch /etc/apt/apt.conf.d/proxy.conf
96 sudo sh -c 'echo "Acquire::http::proxy \"http://<PROXY_IP>:<PROXY_PORT>/\";" >> /etc/apt/apt.conf.d/proxy.conf'
97 sudo sh -c 'echo "Acquire::https::proxy \"https://<PROXY_IP>:<PROXY_PORT>/\";" >> /etc/apt/apt.conf.d/proxy.conf'
98 sudo sh -c 'echo "Acquire::ftp::proxy \"ftp://<PROXY_IP>:<PROXY_PORT>/\";" >> /etc/apt/apt.conf.d/proxy.conf'
100 export http_proxy=http://<PROXY_IP>:<PROXY_PORT>
101 export https_proxy=http://<PROXY_IP>:<PROXY_PORT>
103 Install jdk and maven using the following commands:
105 .. code-block:: console
106 sudo apt install default-jdk
107 sudo apt install maven
109 If the host is behind a proxy, configure this proxy for maven:
111 .. code-block:: console
112 nano ~/.m2/settings.xml
113 <Paste the following lines>
115 <settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
118 <active>true</active>
119 <protocol>http</protocol>
120 <host><PROXY_IP></host>
121 <port><PROXY_PORT></port>
122 <nonProxyHosts>127.0.0.1|localhost</nonProxyHosts>
126 <active>true</active>
127 <protocol>https</protocol>
128 <host><PROXY_IP></host>
129 <port><PROXY_PORT></port>
130 <nonProxyHosts>127.0.0.1|localhost</nonProxyHosts>
135 <Save and exit from nano>
137 Install docker using the following commands:
139 .. code-block:: console
140 sudo apt install docker.io
142 sudo gpasswd -a $USER docker
145 If the host is behind a proxy, configure docker to use this proxy:
147 .. code-block:: console
148 mkdir /etc/systemd/system/docker.service.d
149 sudo nano /etc/systemd/system/docker.service.d/http-proxy.conf
150 <Paste the following lines>
153 Environment="HTTP_PROXY=http://<PROXY_IP>:<PROXY_PORT>/"
155 <Save and exit from nano>
157 sudo systemctl daemon-reload
158 sudo systemctl restart docker
160 Install mySQL client:
162 .. code-block:: console
163 sudo apt install mysql-client
167 A mariadb database instance is needed for both modes of the UI with the appropriate databases and tables in order for the back-end system to store and retrieve data.
169 The pom.xml file supports the creation of an appropriate docker image for development purposes. The initialization scripts reside under the db-scripts directory.
171 Also, a script has been developed, namely validation/docker/mariadb/deploy.sh which easily deploys the container. This script accepts the following items as input parameters:
173 CONTAINER_NAME, name of the container, default value is akraino-validation-mariadb
174 MARIADB_ROOT_PASSWORD, the desired mariadb root user password, this variable is required
175 MARIADB_AKRAINO_PASSWORD, the desired mariadb akraino user password, this variable is required
176 UI_ADMIN_PASSWORD, the desired Blueprint Validation UI password for the admin user, this variable is required
177 UI_AKRAINO_PASSWORD, the desired Blueprint Validation UI password for the akraino user, this variable is required
178 REGISTRY, registry of the mariadb image, default value is akraino
179 NAME, name of the mariadb image, default value is validation
180 TAG_PRE, first part of the image version, default value is mariadb
181 TAG_VER, last part of the image version, default value is latest
182 MARIADB_HOST_PORT, port on which mariadb is exposed on host, default value is 3307
184 Currently, two users are supported by the UI, namely admin (full privileges) and akraino (limited privileges). Their passwords must be defined in the database.
186 In order to build and deploy the image using only the required parameters, the below instructions should be followed:
188 The mariadb root password, mariadb akraino user password (currently the UI connects to the database using the akraino user), the UI admin password and the UI akraino password should be configured using the appropriate variables and the following commands should be executed:
190 .. code-block:: console
193 mvn docker:build -Ddocker.filter=akraino/validation:dev-mariadb-latest
195 ./deploy.sh TAG_PRE=dev-mariadb MARIADB_ROOT_PASSWORD=<mariadb root user password> MARIADB_AKRAINO_PASSWORD=<mariadb akraino user password> UI_ADMIN_PASSWORD=<UI admin user password> UI_AKRAINO_PASSWORD=<UI akraino user password>
196 mysql -p<MARIADB_AKRAINO_PASSWORD> -uakraino -h <IP of the mariadb container> < ../../ui/db-scripts/examples/initialize_db_example.sql
198 In order to retrieve the IP of the mariadb container, the following command should be executed:
200 .. code-block:: console
202 docker inspect <name of the mariadb container>
204 Furthermore, the TAG_PRE variable should be defined because the default value is 'mariadb' (note that the 'dev-mariadb' is used for development purposes - look at pom.xml file).
206 If the database must be re-deployed (it is assumed that the corresponding mariadb container has been stopped and deleted) while the persistent storage already exists (currently, the 'akraino-validation-mariadb' docker volume is used), a different approach should be used after the image building process.
208 To this end, another script has been developed, namely validation/docker/mariadb/deploy_with_existing_storage.sh which easily deploys the container. This script accepts the following as input parameters:
210 CONTAINER_NAME, the name of the container, default value is akraino-validation-mariadb
211 REGISTRY, the registry of the mariadb image, default value is akraino
212 NAME, the name of the mariadb image, default value is validation
213 TAG_PRE, the first part of the image version, default value is mariadb
214 TAG_VER, the last part of the image version, default value is latest
215 MARIADB_HOST_PORT, the port on which mariadb is exposed on host, default value is 3307
217 In order to deploy the image using only the required parameters and the existing persistent storage, the below instructions should be followed:
219 The mariadb root user password should be configured using the appropriate variable and the following commands should be executed:
221 .. code-block:: console
223 cd validation/docker/mariadb
224 ./deploy_with_existing_persistent_storage.sh TAG_PRE=dev-mariadb
226 Finally, if the database must be re-deployed (it is assumed that the corresponding mariadb container has been stopped and deleted) and the old persistent storage must be deleted, the used docker volume should be first deleted (note that all database's data will be lost).
228 To this end, after the image build process, the following commands should be executed:
230 .. code-block:: console
232 docker volume rm akraino-validation-mariadb
233 cd validation/docker/mariadb
234 ./deploy.sh TAG_PRE=dev-mariadb MARIADB_ROOT_PASSWORD=<root user password> MARIADB_AKRAINO_PASSWORD=<mariadb akraino user password> UI_ADMIN_PASSWORD=<UI admin user password> UI_AKRAINO_PASSWORD=<UI akraino user password>
235 mysql -p<MARIADB_AKRAINO_PASSWORD> -uakraino -h <IP of the mariadb container> < ../../ui/db-scripts/examples/initialize_db_example.sql
237 In the context of the full control loop mode, the following tables must be initialized with appropriate data:
239 - lab (here every lab owner should store the name of the lab)
240 - timeslot (here every lab owner should register the available timeslots that can be used for blueprint validation test execution)
241 - silo (here every lab owner should register the silo which is used for storing results in Nexus, for example for AT&T lab the value is 'att-blu-val')
242 - blueprint (here every blueprint owner should register the name of the blueprint)
243 - blueprint_instance_for_validation (here every blueprint owner should register the blueprint instances for validation, i.e. version, layer and description of a layer)
245 The following file can be used for initializing the aforementioned data (as it was performed in the above example using the 'mysql -p<MARIADB_AKRAINO_PASSWORD> -uakraino -h <IP of the mariadb container> < ../../ui/db-scripts/examples/initialize_db_example.sql' command):
247 db-scripts/examples/initialize_db_example.sql
249 Some of this data is illustrated below (refer to 'org.akraino.validation.ui.data' package for more info regarding available values):
251 .. code-block:: console
254 id:1, lab:0 (0 stands for AT&T)
257 id:1 , start date and time: 'now', duration: null, lab: 1
260 id:1, silo: 'att-blu-val', lab: 1
266 id: 2, blueprint_id: 3 (i.e. REC), version: "latest", layer: 0 (i.e. Hardware), layer_description: "AT&T Hardware"
268 It should be noted that currently the start date and time and the duration of the timeslot are not taken into account by the UI (see limitation section). Therefore, a user should define 'now' and null respectively for their content.
270 Based on this data, the UI enables the user to select an appropriate blueprint instance for validation.
272 Currently, this data cannot be retrieved dynamically by the UI (see limitations subsection). For this reason, in cases of new data, a user should define new entries in this database.
274 For example, if a user wants to define a new lab with the following data:
278 the following file should be created:
282 SET FOREIGN_KEY_CHECKS=1;
284 insert into lab values(2, 2);
286 2 stands for community lab. Refer to 'org.akraino.validation.ui.data' package for more info.
288 Then, the following command should be executed:
290 .. code-block:: console
292 mysql -p<MARIADB_AKRAINO_PASSWORD> -uakraino -h <IP of the mariadb container> < ./dbscript.sql
294 For example, if a user wants to define a new timeslot with the following data:
296 start date and time:'now', duration: 0, lab: AT&T
298 the following file should be created:
302 SET FOREIGN_KEY_CHECKS=1;
304 insert into timeslot values(2, 'now', null, 1);
306 1 is the id of the AT&T lab.
308 Then, the following command should be executed:
310 .. code-block:: console
312 mysql -p<MARIADB_AKRAINO_PASSWORD> -uakraino -h <IP of the mariadb container> < ./dbscript.sql
314 For example, if a user wants to define a new silo with the following data:
316 silo: 'community-blu-val', lab: AT&T
318 the following file should be created:
322 SET FOREIGN_KEY_CHECKS=1;
324 insert into silo values(2, 'community-blu-val', 2);
326 2 is the id of the community lab.
328 Then, the following command should be executed:
330 .. code-block:: console
332 mysql -p<MARIADB_AKRAINO_PASSWORD> -uakraino -h <IP of the mariadb container> < ./dbscript.sql
334 Furthermore, if a user wants to define a new blueprint, namely "newBlueprint" and a new instance of this blueprint with the following data:
336 version: "latest", layer: 2 (i.e. K8s), layer_description: "K8s with High Availability Ingress controller"
338 the following file should be created:
342 SET FOREIGN_KEY_CHECKS=1;
344 insert into blueprint (blueprint_id, blueprint_name) values(4, 'newBlueprint');
345 insert into blueprint_instance (blueprint_instance_id, blueprint_id, version, layer, layer_description) values(6, 4, 'latest', 2, 'K8s with High Availability Ingress controller');
347 Then, the following command should be executed:
349 .. code-block:: console
351 mysql -p<MARIADB_AKRAINO_PASSWORD> -uakraino -h <IP of the mariadb container> < ./dbscript.sql
353 The UI will automatically retrieve this new data and display it to the user.
355 - Jenkins Configuration
357 Recall that for full control loop, a Jenkins instance is needed capable of executing blueprint validation tests to the specified lab. The Blueprint validation UI will trigger job executions in that instance.
359 It should be noted that it is not the UI responsibility to deploy a Jenkins instance.
361 Furthermore, this instance must have the following option enabled: "Manage Jenkins -> Configure Global Security -> Prevent Cross Site Request Forgery exploits".
363 Also, currently, the corresponding Jenkins job should accept the following as input parameters: "SUBMISSION_ID", "BLUEPRINT", "VERSION", "LAYER", "LAB" and "UI_IP".
364 The "SUBMISSION_ID" and "UI_IP" parameters (i.e. IP address of the UI host machine-this is needed by the Jenkins instance in order to send back Job completion notification) are created and provided by the back-end part of the UI.
365 The "BLUEPRINT", "VERSION", "LAYER" and "LAB" parameters are configured by the UI user.
367 Moreover, as the Jenkins notification plugin (https://wiki.jenkins.io/display/JENKINS/Notification+Plugin) seems to ignore proxy settings, the corresponding Jenkins job must be configured to execute the following commands at the end (Post-build Actions)
373 All the blueprint validation results are stored in Nexus server for both modes of the UI.
375 It should be noted that it is not the UI responsibility to deploy a Nexus server.
377 These results must be available in the following url:
379 https://nexus.akraino.org/content/sites/logs/<lab_silo>/<Blueprint name>/<Blueprint version>/<timestamp>/results/<layer>/<name_of_the_test_suite>
381 where <lab_silo> is the silo used by a lab for storing results in Nexus (for example 'att-blu-val'), <Blueprint name> is the name of the blueprint, <Blueprint version> the the blueprint version, <timestamp> is the timestamp used for producinf the results, <layer> is the blueprint layer and <name_of_the_test_suite> is the name of the corresponding test suite.
383 Below, an example URL is illustrated
385 https://nexus.akraino.org/content/sites/logs/att-blu-val/rec/master/20190611-132818/results/hardware/bios_version/
387 Moreover, the results should be stored in the 'output.xml' file and placed in the aforementioned URL using the following format:
394 .. code-block:: console
402 The pom.xml file supports the building of an appropriate container image using the produced war file. Also, a script has been developed, namely validation/docker/ui/deploy.sh which easily deploys the container.
404 This script accepts the following as input parameters:
406 CONTAINER_NAME, the name of the contaner, default value is akraino-validation-ui
407 DB_IP_PORT, the IP and port of the maridb instance, this variable is required
408 MARIADB_AKRAINO_PASSWORD, the mariadb akraino user password, this variable is required
409 REGISTRY, the registry of the mariadb image, default value is akraino
410 NAME, the name of the mariadb image, default value is validation
411 TAG_PRE, the first part of the image version, default value is ui
412 TAG_VER, the last part of the image version, default value is latest
413 JENKINS_URL, the URL of the Jenkins instance (http or https must be defined), this variable is required
414 JENKINS_USERNAME, the Jenkins user name, this variable is required
415 JENKINS_USER_PASSWORD, the Jenkins user password, this variable is required
416 JENKINS_JOB_NAME, the name of Jenkins job capable of executing the blueprint validation tests, this variable is required
417 NEXUS_PROXY, the needed proxy in order for the Nexus server to be reachable, default value is none
418 JENKINS_PROXY, the needed proxy in order for the Jenkins server to be reachable, default value is none
420 In order to build the image using only the required parameters, the following data is needed:
422 - The mariadb akraino user password (look at the Database subsection)
423 - The IP and port of the mariadb
425 - The Jenkins username and password
426 - The name of Jenkins Job
428 Then, the following commands can be executed in order to build and deploy the UI container:
430 .. code-block:: console
433 mvn docker:build -Ddocker.filter=akraino/validation:dev-ui-latest
435 ./deploy.sh TAG_PRE=dev-ui DB_IP_PORT=<IP and port of the mariadb> MARIADB_AKRAINO_PASSWORD=<mariadb akraino password> JENKINS_URL=<http://jenkinsIP:port> JENKINS_USERNAME=<Jenkins user> JENKINS_USER_PASSWORD=<Jenkins password> JENKINS_JOB_NAME=<Jenkins job name>
437 The content of the DB_IP_PORT can be for example '172.17.0.3:3306'.
439 Furthermore, the TAG_PRE variable should be defined as the default value is 'ui' (note that the 'dev-ui' is used for development purposes - look at pom.xml file).
441 If no proxy exists, the proxy ip and port variables should not be defined.
443 The UI should be available in the following url:
445 http://localhost:8080/bluvalui/
447 Note that the deployment uses the network host mode, so the 8080 must be available on the host.
455 - The UI has been tested using Chrome and Firefox browsers.
456 - The back-end part of the UI does not take into account the start date and time and duration of the configured timeslot. It immediately triggers the corresponding Jenkins Job.
457 - Results data manipulation (filtering, graphical representation, indexing in time order, etc) is not supported.
458 - The silos, labs, and the available blueprints and timeslots must be manually configured in the mariadb database.