From: Ralf Mueller Date: Wed, 24 Apr 2019 10:09:13 +0000 (+0300) Subject: Initial version X-Git-Url: https://gerrit.akraino.org/r/gitweb?p=ta%2Fremote-installer.git;a=commitdiff_plain;h=f9adb9143ef94b16ae16941652e75deccad506ef Initial version Change-Id: I6197be5766e32f3fe70fc3f65243a1cb7032ec16 Signed-off-by: Ralf Mueller --- diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..d645695 --- /dev/null +++ b/LICENSE @@ -0,0 +1,202 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/README b/README new file mode 100644 index 0000000..80a4098 --- /dev/null +++ b/README @@ -0,0 +1,32 @@ +Manually build the image. + +NAME="remote-installer" +docker build \ + --network=host \ + --no-cache \ + --force-rm \ + --build-arg HTTP_PROXY="${http_proxy}" \ + --build-arg HTTPS_PROXY="${https_proxy}" \ + --build-arg NO_PROXY="${no_proxy}" \ + --build-arg http_proxy="${http_proxy}" \ + --build-arg https_proxy="${https_proxy}" \ + --build-arg no_proxy="${no_proxy}" \ + --tag remote-installer \ + --file docker-build/remote-installer/Dockerfile . + +Run the image with root passwd root (default), mount the starter's $HOME/tmp to /mnt and exports it!!! Webserver is available at port 8080. + +NAME="remote-installer" +docker run --detach --rm -e PW='root' --volume $HOME/tmp/:/opt/remoteinstaller --publish 443:443 -p 2049:2049 -p 15101:15101 --privileged $EXTRA --name "$NAME" "$NAME" +# Get container IP: +docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' $NAME + +# configuration options +# HOST_ADDR - Mandatory parameter with the external IP of the host, which runs the docker +# Usually the domain controller. +# PW - root password. +# API_PORT - IPv4 port used by the remote installer API. +# HTTPS_PORT - + +# Sometimes it is a good idea to remove unsued images +docker image rm $(docker image ls |grep none |awk -F ' ' '{print $3}') diff --git a/docker-build/remote-installer/Dockerfile b/docker-build/remote-installer/Dockerfile new file mode 100644 index 0000000..0c06de4 --- /dev/null +++ b/docker-build/remote-installer/Dockerfile @@ -0,0 +1,113 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +FROM centos:7.6.1810 +MAINTAINER Ralf Mueller + +ENV \ +ETC_REMOTE_INST="/etc/remoteinstaller" \ +PW="root" \ +API_PORT="15101" \ +API_LISTEN_ADDR="0.0.0.0" \ +HTTPS_PORT="443" \ +HOST_ADDR="127.0.0.1" \ +STARTUP="/etc/remoteinstaller/startup.sh" \ +CA_CERT="cacert.pem" \ +CLIENT_CERT="clientcert.pem" \ +CLIENT_KEY="clientkey.pem" \ +SERVER_CERT="servercert.pem" \ +SERVER_KEY="serverkey.pem" \ +INSTALLER_MOUNT="/opt/remoteinstaller" + +ENV IMAGES_STORE="$INSTALLER_MOUNT/images" +ENV IMAGES_HTML="/var/www/lighttpd/images" + +RUN mkdir -p "$INSTALLER_MOUNT" + +# prepare for basic systemd services +RUN yum -y install systemd epel-release; yum clean all \ +&& (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == systemd-tmpfiles-setup.service ] || rm -f $i; done) \ +&& rm -f /lib/systemd/system/multi-user.target.wants/* \ +&& rm -f /etc/systemd/system/*.wants/* \ +&& rm -f /lib/systemd/system/local-fs.target.wants/* \ +&& rm -f /lib/systemd/system/sockets.target.wants/*udev* \ +&& rm -f /lib/systemd/system/sockets.target.wants/*initctl* \ +&& rm -f /lib/systemd/system/basic.target.wants/* \ +&& rm -f /lib/systemd/system/anaconda.target.wants/* \ +\ +# Services for the workload \ +&& yum install -y iproute wget openssh-server lighttpd nfs-utils \ +python-setuptools python2-eventlet python-routes PyYAML \ +python-netaddr pexpect net-tools tcpdump \ +ipmitool \ +# mod_ssl \ +&& systemctl enable sshd \ +&& systemctl enable lighttpd \ +&& systemctl enable nfs-server \ +&& echo "$IMAGES_STORE" "*(rw,sync,no_root_squash,no_all_squash)" >>/etc/exports + + +# lighthttpd configuration + +RUN sed -i 's/server.use-ipv6 = "enable"/server.use-ipv6 = "disable"/' /etc/lighttpd/lighttpd.conf \ +&& echo $'\n\ +# SSL configuration\n\ +ssl.engine = "enable"\n\ +ssl.privkey = "/opt/remoteinstaller/certificates/serverkey.pem"\n\ +ssl.pemfile = "/opt/remoteinstaller/certificates/servercert.pem"\n\ +ssl.ca-file = "/opt/remoteinstaller/certificates/cacert.pem"\n\ +ssl.verifyclient.activate = "enable"\n\ +ssl.verifyclient.enforce = "enable"\n\ +' >> /etc/lighttpd/lighttpd.conf \ +&& mkdir -p "$IMAGES_HTML" + + +# Install hw-detector from LF +RUN wget -O x.tgz 'https://gerrit.akraino.org/r/gitweb?p=ta/hw-detector.git;a=snapshot;h=HEAD;sf=tgz' \ +&& tar -xzf x.tgz \ +&& rm -f x.tgz \ +&& pushd hw-detector*/src \ +&& python setup.py install \ +&& popd \ +&& rm -rf hw-detector* + + +# Install remote-installer to image +COPY src "$INSTALLER_MOUNT" +RUN pushd "$INSTALLER_MOUNT" \ +&& python setup.py install \ +&& rm -rf * \ +&& popd + +RUN mkdir -p "$ETC_REMOTE_INST" + +RUN echo '#!/bin/bash' >>$STARTUP \ +&& echo 'printenv >/etc/remoteinstaller/environment' >>$STARTUP \ +&& echo mkdir /run/systemd/system >>$STARTUP \ +&& echo nohup /usr/lib/systemd/systemd --system '&>/dev/null &' >>$STARTUP \ +&& echo "echo -e \"\$PW\n\$PW\n\n\" |passwd" >>$STARTUP \ +&& echo mount -o bind "$IMAGES_STORE" "$IMAGES_HTML" >>$STARTUP \ +&& echo 'sed -i "s/server.port = 80/server.port = $HTTPS_PORT/" /etc/lighttpd/lighttpd.conf' >>$STARTUP \ +# && echo "echo \\\$SERVER[\\\"sockets\\\"] == \\\"0.0.0.0:\$HTTPS_PORT {}\\\" >> /etc/lighttpd/lighttpd.conf" >>$STARTUP \ +&& echo python /lib/python2.7/site-packages/remoteinstaller-1.0-py2.7.egg/remoteinstaller/server/server.py \ + -H \$API_LISTEN_ADDR -P \$API_PORT -S \$HOST_ADDR \ + -C \$SERVER_CERT -K \$SERVER_KEY -c \$CLIENT_CERT -k \$CLIENT_KEY -A \$CA_CERT -d \ + >>$STARTUP \ +&& echo 'while [ false ]; do sleep 5 ;done' >>$STARTUP \ +&& chmod +x $STARTUP + +ENTRYPOINT ["/etc/remoteinstaller/startup.sh"] + +# CMD [ "arg1" ] + diff --git a/remote-installer.spec b/remote-installer.spec new file mode 100644 index 0000000..b57991e --- /dev/null +++ b/remote-installer.spec @@ -0,0 +1,50 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +Name: remote-installer +Version: %{_version} +Release: 1%{?dist} +Summary: Contains components for the remote-installer +Group: %{_platform_group} +License: %{_platform_licence} +Source0: %{name}-%{version}.tar.gz +Vendor: %{_platform_vendor} + +BuildArch: noarch + +# BuildRequires: docker + +%description +Contains components for the remote-installer + +%prep + +%build +docker build \ + --network=host \ + --no-cache \ + --force-rm \ + --build-arg HTTP_PROXY="${http_proxy}" \ + --build-arg HTTPS_PROXY="${https_proxy}" \ + --build-arg NO_PROXY="${no_proxy}" \ + --build-arg http_proxy="${http_proxy}" \ + --build-arg https_proxy="${https_proxy}" \ + --build-arg no_proxy="${no_proxy}" \ + --tag remote-installer \ + --file docker-build/remote-installer/Dockerfile . + +# Here hould be some registry but it should be handled by a Jenkis job +docker image save remote-installer >remote-installer-image.tar + +%files diff --git a/scripts/build.sh b/scripts/build.sh new file mode 100755 index 0000000..dcfc234 --- /dev/null +++ b/scripts/build.sh @@ -0,0 +1,61 @@ +#!/bin/bash +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +cd "$(dirname "$0")"/.. + +TAR_IMAGE="remote-installer.tar" + +help() +{ + echo -e "$(basename $0) [-hs]" + echo -e " -h display this help" + echo -e " -s save image as tar to $TAR_IMAGE" + echo + echo -e "Proxy configuration is taken from environment variables" + echo -e "http_proxy, https_proxy and no_proxy" +} + +while getopts "hs" arg; do + case $arg in + h) + help + exit 0 + ;; + s) + SAVE_IMAGE="yes" + ;; + esac +done + +docker build \ + --network=host \ + --no-cache \ + --force-rm \ + --build-arg HTTP_PROXY="${http_proxy}" \ + --build-arg HTTPS_PROXY="${https_proxy}" \ + --build-arg NO_PROXY="${no_proxy}" \ + --build-arg http_proxy="${http_proxy}" \ + --build-arg https_proxy="${https_proxy}" \ + --build-arg no_proxy="${no_proxy}" \ + --tag remote-installer \ + --file docker-build/remote-installer/Dockerfile . + + +# could be compressed but it's only used until there is an registry +if [ -n "$SAVE_IMAGE" ] +then + echo -e "Creating image tar ball at : $(dirname "$0")/$TAR_IMAGE" + docker image save remote-installer >"$(dirname "$0")"/"$TAR_IMAGE" +fi diff --git a/scripts/start.sh b/scripts/start.sh new file mode 100755 index 0000000..ca3b7e4 --- /dev/null +++ b/scripts/start.sh @@ -0,0 +1,91 @@ +#!/bin/bash +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +set -e + +cd "$(dirname "$0")"/.. + +API_PORT="15101" +BASE_DIR="" +CONT_NAME="remote-installer" +EXT_IP="" +HTTPS_PORT="443" +IMG_NAME="remote-installer" +ROOT_PW="root" + +error() +{ + echo "ERROR : $1" + [ -z $2 ] || help + exit 1 +} + +help() +{ + echo -e "$(basename $0) [-h -a -c -i -r -s ] -b -e " + echo -e " -h display this help" + echo -e " -a rest API port, default $API_PORT" + echo -e " -c container name, default $CONT_NAME" + echo -e " -b base directory, which contains images, certificates, etc." + echo -e " -e external ip address of the docker" + echo -e " -i secure https port, default $IMG_NAME" + echo -e " -p root password, default $ROOT_PW" + echo -e " -s secure https port, default $HTTPS_PORT" +} + +while getopts "ha:b:e:s:c:p:i:" arg; do + case $arg in + h) + help + exit 0 + ;; + b) + BASE_DIR="$OPTARG" + ;; + e) + EXT_IP="$OPTARG" + ;; + s) + HTTPS_PORT="$OPTARG" + ;; + a) + API_PORT="$OPTARG" + ;; + c) + CONT_NAME="$OPTARG" + ;; + i) + IMG_NAME="$OPTARG" + ;; + *) + error "Unknow argument!" showhelp + ;; + esac +done + +[ -n "$EXT_IP" ] || error "No external IP defined!" showhelp +[ -n "$BASE_DIR" ] || error "No base directory defined!" showhelp + +cont_id="$(docker run --detach --rm --privileged \ + --env API_PORT="$API_PORT" \ + --env HOST_ADDR="$EXT_IP" \ + --env HTTPS_PORT="$HTTPS_PORT" \ + --env PW="$ROOT_PW" \ + --volume "$BASE_DIR":/opt/remoteinstaller --publish "$HTTPS_PORT":"$HTTPS_PORT" -p 2049:2049 -p "$API_PORT":"$API_PORT" --name "$CONT_NAME" "$IMG_NAME")" \ + || error "failed to start container" + +echo -e "Container successfully started" +echo -e "ID : $cont_id" +echo -e "IP : $(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$cont_id")" diff --git a/scripts/stop.sh b/scripts/stop.sh new file mode 100755 index 0000000..b48b9b1 --- /dev/null +++ b/scripts/stop.sh @@ -0,0 +1,55 @@ +#!/bin/bash +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +set -e + +cd "$(dirname "$0")"/.. + +CONT_NAME="remote-installer" + +error() +{ + echo "ERROR : $1" + [ -z $2 ] || help + exit 1 +} + +help() +{ + echo -e "$(basename $0) [-h -a -c ]" + echo -e " -h display this help" + echo -e " -c container name or ID, default $CONT_NAME" +} + +while getopts "hc:" arg; do + case $arg in + h) + help + exit 0 + ;; + c) + CONT_NAME="$OPTARG" + ;; + *) + error "Unknow argument!" showhelp + ;; + esac +done + +docker container stop "$CONT_NAME" \ + || error "failed to stop container $CONT_NAME" + +echo -e "Container successfully stopped : $CONT_NAME" + diff --git a/src/remoteinstaller/__init__.py b/src/remoteinstaller/__init__.py new file mode 100644 index 0000000..287b513 --- /dev/null +++ b/src/remoteinstaller/__init__.py @@ -0,0 +1,15 @@ +# Copyright 2019 Nokia + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +__import__('pkg_resources').declare_namespace(__name__) diff --git a/src/remoteinstaller/installer/__init__.py b/src/remoteinstaller/installer/__init__.py new file mode 100644 index 0000000..41eaa50 --- /dev/null +++ b/src/remoteinstaller/installer/__init__.py @@ -0,0 +1,13 @@ +# Copyright 2019 Nokia + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. diff --git a/src/remoteinstaller/installer/bmc_management/__init__.py b/src/remoteinstaller/installer/bmc_management/__init__.py new file mode 100644 index 0000000..41eaa50 --- /dev/null +++ b/src/remoteinstaller/installer/bmc_management/__init__.py @@ -0,0 +1,13 @@ +# Copyright 2019 Nokia + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. diff --git a/src/remoteinstaller/installer/bmc_management/bmctools.py b/src/remoteinstaller/installer/bmc_management/bmctools.py new file mode 100644 index 0000000..dfdeaea --- /dev/null +++ b/src/remoteinstaller/installer/bmc_management/bmctools.py @@ -0,0 +1,356 @@ +# Copyright 2019 Nokia + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import subprocess +import time +import logging +import ipaddress +import pexpect + +class BMCException(Exception): + pass + +class BMC(object): + def __init__(self, host, user, passwd, log_path=None): + self._host = host + self._user = user + self._passwd = passwd + if log_path: + self._log_path = log_path + else: + self._log_path = 'console.log' + self._sol = None + self._host_name = None + + def set_host_name(self, host_name): + if not self._host_name: + self._host_name = host_name + + def get_host_name(self): + if self._host_name: + return self._host_name + + return '' + + def get_host(self): + return self._host + + def get_user(self): + return self._user + + def get_passwd(self): + return self._passwd + + def reset(self): + logging.info('Reset BMC of %s: %s', self.get_host_name(), self.get_host()) + + self._run_ipmitool_command('bmc reset cold') + + success = self._wait_for_bmc_reset(180) + if not success: + raise BMCException('BMC reset failed, BMC did not come up') + + def _set_boot_from_virtual_media(self): + raise NotImplementedError + + def _detach_virtual_media(self): + raise NotImplementedError + + def _get_bmc_nfs_service_status(self): + raise NotImplementedError + + def _wait_for_bmc_responding(self, timeout, expected_to_respond=True): + if expected_to_respond: + logging.debug('Wait for BMC to start responding') + else: + logging.debug('Wait for BMC to stop responding') + + start_time = int(time.time()*1000) + + response = (not expected_to_respond) + while response != expected_to_respond: + rc, _ = self._run_ipmitool_command('bmc info', can_fail=True) + response = (rc == 0) + + if response == expected_to_respond: + break + + time_now = int(time.time()*1000) + if time_now-start_time > timeout*1000: + logging.debug('Wait timed out') + break + + logging.debug('Still waiting for BMC') + time.sleep(10) + + return response == expected_to_respond + + def _wait_for_bmc_webpage(self, timeout): + host = ipaddress.ip_address(unicode(self._host)) + if host.version == 6: + host = "[%s]" %host + + command = 'curl -g --insecure -o /dev/null https://{}/index.html'.format(host) + + start_time = int(time.time()*1000) + rc = 1 + while rc != 0: + p = subprocess.Popen(command.split(), stdout=subprocess.PIPE, stderr=subprocess.STDOUT) + _, _ = p.communicate() + rc = p.returncode + + if rc == 0: + break + + time_now = int(time.time()*1000) + if time_now-start_time > timeout*1000: + logging.debug('Wait timed out') + break + + logging.debug('Still waiting for BMC webpage') + time.sleep(10) + + return rc == 0 + + def _wait_for_bmc_not_responding(self, timeout): + return self._wait_for_bmc_responding(timeout, False) + + def _wait_for_bmc_reset(self, timeout): + logging.debug('Wait for BMC to reset') + + success = True + if not self._wait_for_bmc_not_responding(timeout): + success = False + msg = 'BMC did not go down as expected' + logging.warning(msg) + else: + logging.debug('As expected, BMC is not responding') + + if not self._wait_for_bmc_responding(timeout): + success = False + msg = 'BMC did not come up as expected' + logging.warning(msg) + else: + logging.debug('As expected, BMC is responding') + + if not self._wait_for_bmc_webpage(timeout): + success = False + msg = 'BMC webpage did not start to respond' + logging.warning(msg) + else: + logging.debug('As expected, BMC webpage is responding') + + return success + + def setup_boot_options_for_virtual_media(self): + logging.debug('Setup boot options') + + self._disable_boot_flag_timeout() + self._set_boot_from_virtual_media() + + def power(self, power_command): + logging.debug('Run host power command (%s) %s', self._host, power_command) + + return self._run_ipmitool_command('power {}'.format(power_command)).strip() + + def wait_for_bootup(self): + logging.debug('Wait for prompt after booting from hd') + + try: + self._expect_flag_in_console('localhost login:', timeout=1200) + except BMCException as ex: + self._send_to_console('\n') + self._expect_flag_in_console('localhost login:', timeout=30) + + def setup_sol(self): + logging.debug('Setup SOL for %s', self._host) + + self._run_ipmitool_command('sol set non-volatile-bit-rate 115.2') + self._run_ipmitool_command('sol set volatile-bit-rate 115.2') + + def boot_from_virtual_media(self): + logging.info('Boot from virtual media') + + self._trigger_boot() + self._wait_for_virtual_media_detach_phase() + + self._detach_virtual_media() + self._set_boot_from_hd_no_boot() + + logging.info('Boot should continue from disk now') + + def close(self): + if self._sol: + self._sol.terminate() + self._sol = None + + @staticmethod + def _convert_to_hex(ascii_string, padding=False, length=0): + hex_value = ''.join('0x{} '.format(c.encode('hex')) for c in ascii_string).strip() + if padding and (len(ascii_string) < length): + hex_value += ''.join(' 0x00' for _ in range(len(ascii_string), length)) + + return hex_value + + @staticmethod + def _convert_to_ascii(hex_string): + return ''.join('{}'.format(c.decode('hex')) for c in hex_string) + + def _execute_ipmitool_command(self, ipmi_command): + command = 'ipmitool -I lanplus -H {} -U {} -P {} {}'.format(self._host, self._user, self._passwd, ipmi_command) + + p = subprocess.Popen(command.split(), stdout=subprocess.PIPE, stderr=subprocess.STDOUT) + out, _ = p.communicate() + rc = p.returncode + + return (rc, out) + + def _run_ipmitool_command(self, ipmi_command, can_fail=False, retries=5): + logging.debug('Run ipmitool command: %s', ipmi_command) + + if can_fail: + return self._execute_ipmitool_command(ipmi_command) + else: + while retries > 0: + rc, out = self._execute_ipmitool_command(ipmi_command) + if not rc: + break + + if retries > 0: + logging.debug('Retry command') + time.sleep(5) + + retries -= 1 + + if rc: + logging.warning('ipmitool failed: %s', out) + raise BMCException('ipmitool call failed with rc: {}'.format(rc)) + + return out + + def _run_ipmitool_raw_command(self, ipmi_raw_command): + logging.debug('Run ipmitool raw command') + + out = self._run_ipmitool_command('raw {}'.format(ipmi_raw_command)) + + out_bytes = out.replace('\n', '').strip().split(' ') + return out_bytes + + def _disable_boot_flag_timeout(self): + logging.debug('Disable boot flag timeout (%s)', self._host) + + status_code = self._run_ipmitool_raw_command('0x00 0x08 0x03 0x1f') + if status_code[0] != '': + raise BMCException('Could not disable boot flag timeout (rc={})'.format(status_code[0])) + + def _open_console(self): + logging.debug('Open SOL console (log in %s)', self._log_path) + + expect_session = pexpect.spawn('ipmitool -I lanplus -H {} -U {} -P {} sol deactivate'.format(self._host, self._user, self._passwd)) + expect_session.expect(pexpect.EOF) + + logfile = open(self._log_path, 'ab') + + expect_session = pexpect.spawn('ipmitool -I lanplus -H {} -U {} -P {} sol activate'.format(self._host, self._user, self._passwd), timeout=None, logfile=logfile) + + return expect_session + + def _send_to_console(self, chars): + logging.debug('Sending %s to console', chars.replace('\n', '\\n')) + + if not self._sol: + self._sol = self._open_console() + + self._sol.send(chars) + + def _expect_flag_in_console(self, flags, timeout=600): + logging.debug('Expect a flag in console output within %s seconds ("%s")', timeout, flags) + + time_begin = time.time() + + remaining_time = timeout + + while remaining_time > 0: + if not self._sol: + try: + self._sol = self._open_console() + except pexpect.TIMEOUT as e: + logging.debug(e) + raise BMCException('Could not open console: {}'.format(str(e))) + + try: + self._sol.expect(flags, timeout=remaining_time) + logging.debug('Flag found in log') + return + except pexpect.TIMEOUT as e: + logging.debug(e) + raise BMCException('Expected message in console did not occur in time ({})'.format(flags)) + except pexpect.EOF as e: + logging.warning('Got EOF from console') + if 'SOL session closed by BMC' in self._sol.before: + logging.debug('Found: "SOL session closed by BMC" in console') + elapsed_time = time.time()-time_begin + remaining_time = timeout-elapsed_time + if remaining_time > 0: + logging.info('Retry to expect a flag in console, %s seconds remaining', remaining_time) + self.close() + + def _wait_for_bios_settings_done(self): + logging.debug('Wait until BIOS settings are updated') + + self._expect_flag_in_console('Booting...', timeout=300) + + def _set_boot_from_hd_no_boot(self): + logging.debug('Set boot from hd (%s), no boot', self._host) + + self._run_ipmitool_command('chassis bootdev disk options=persistent') + + def _wait_for_virtual_media_detach_phase(self): + logging.debug('Wait until virtual media can be detached') + + self._expect_flag_in_console(['Copying cloud guest image', + 'Installing OS to HDD', + 'Extending partition and filesystem size'], + timeout=1200) + + def _wait_for_bmc_nfs_service(self, timeout, expected_status): + logging.debug('Wait for BMC NFS service') + + start_time = int(time.time()*1000) + + status = '' + while status != expected_status: + status = self._get_bmc_nfs_service_status() + + if status == expected_status or status == 'nfserror': + logging.debug('Breaking from wait loop. status = %s', status) + break + + time_now = int(time.time()*1000) + if time_now-start_time > timeout*1000: + logging.debug('Wait timed out') + break + time.sleep(10) + + return status == expected_status + + def _trigger_boot(self): + logging.debug('Trigger boot') + + power_state = self.power('status') + logging.debug('State is: %s', power_state) + if power_state == 'Chassis Power is off': + self.power('on') + else: + self.power('reset') diff --git a/src/remoteinstaller/installer/bmc_management/hw17.py b/src/remoteinstaller/installer/bmc_management/hw17.py new file mode 100644 index 0000000..a7b5bec --- /dev/null +++ b/src/remoteinstaller/installer/bmc_management/hw17.py @@ -0,0 +1,103 @@ +# Copyright 2019 Nokia + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from .bmctools import BMC +import logging +import time + +class BMCException(Exception): + pass + +class HW17(BMC): + def __init__(self, host, user, passwd, log_path=None): + super(HW17, self).__init__(host, user, passwd, log_path) + + def attach_virtual_cd(self, nfs_host, nfs_mount, boot_iso_filename): + for _ in range(2): + self._setup_bmc_nfs_service(nfs_host, nfs_mount, boot_iso_filename) + success = self._wait_for_bmc_nfs_service(90, 'mounted') + if success: + return True + else: + logging.debug('BMC NFS server did not start yet') + self.reset() + + raise BMCException('NFS service setup failed') + + def _detach_virtual_media(self): + logging.debug('Detach virtual media') + + comp_code = self._run_ipmitool_raw_command('0x3c 0x00') + if comp_code[0] == '80': + raise BMCException('BMC NFS service reset failed, cannot get configuration') + elif comp_code[0] == '81': + raise BMCException('BMC NFS service reset failed, cannot set configuration') + else: + BMCException('BMC NFS service reset failed (rc={})'.format(comp_code)) + + def _set_boot_from_virtual_media(self): + logging.debug('Set boot from cd (%s), and boot after that', self._host) + self._run_ipmitool_command('chassis bootdev floppy options=persistent') + + def _get_bmc_nfs_service_status(self): + logging.debug('Get BMC NFS service status') + + status_code = self._run_ipmitool_raw_command('0x3c 0x03') + if status_code[0] == '00': + status = 'mounted' + elif status_code[0] == '64': + status = 'mounting' + elif status_code[0] == 'ff': + status = 'dismounted' + elif status_code[0] == '20': + status = 'nfserror' + else: + raise BMCException('Could not get BMC NFS service status (rc={})'.format(status_code)) + + logging.debug('Returned status: %s', status) + return status + + def _set_bmc_nfs_configuration(self, nfs_host, mount_path, image_name): + logging.debug('Set BMC NFS configuration') + + nfs_host_hex = self._convert_to_hex(nfs_host) + mount_path_hex = self._convert_to_hex(mount_path) + image_name_hex = self._convert_to_hex(image_name) + + logging.debug('Set the IP address of the BMC NFS service (%s)', self._host) + comp_code = self._run_ipmitool_raw_command('0x3c 0x01 0x00 {} 0x00'.format(nfs_host_hex)) + if comp_code[0] != '': + raise BMCException('Failed to set BMC NFS service IP address (rc={})'.format(comp_code)) + + logging.debug('Set the path of the BMC NFS service (%s)', mount_path) + comp_code = self._run_ipmitool_raw_command('0x3c 0x01 0x01 {} 0x00'.format(mount_path_hex)) + if comp_code[0] != '': + raise BMCException('Failed to set BMC NFS service path (rc={})'.format(comp_code)) + + logging.debug('Set the ISO image name of the BMC NFS service (%s)', image_name) + comp_code = self._run_ipmitool_raw_command('0x3c 0x01 0x02 {} 0x00'.format(image_name_hex)) + if comp_code[0] != '': + raise BMCException('Failed to set BMC NFS service iso image name (rc={})'.format(comp_code)) + + def _setup_bmc_nfs_service(self, nfs_host, mount_path, image_name): + logging.debug('Setup BMC NFS service') + + self._detach_virtual_media() + self._set_bmc_nfs_configuration(nfs_host, mount_path, image_name) + + logging.debug('Start the BMC NFS service') + comp_code = self._run_ipmitool_raw_command('0x3c 0x02 0x01') + if comp_code[0] != '': + raise BMCException('Failed to start the BMC NFS service (rc={})'.format(comp_code)) + diff --git a/src/remoteinstaller/installer/bmc_management/oe19.py b/src/remoteinstaller/installer/bmc_management/oe19.py new file mode 100644 index 0000000..b514fd5 --- /dev/null +++ b/src/remoteinstaller/installer/bmc_management/oe19.py @@ -0,0 +1,27 @@ +# Copyright 2019 Nokia + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from .or18 import OR18 +import logging + +class BMCException(Exception): + pass + +class OE19(OR18): + def __init__(self, host, user, passwd, log_path=None): + super(OE19, self).__init__(host, user, passwd, log_path) + + def _set_boot_from_virtual_media(self): + logging.debug('Set boot from floppy (%s), and boot after that', self._host) + self._run_ipmitool_command('chassis bootdev floppy options=persistent') diff --git a/src/remoteinstaller/installer/bmc_management/or18.py b/src/remoteinstaller/installer/bmc_management/or18.py new file mode 100644 index 0000000..d41df4e --- /dev/null +++ b/src/remoteinstaller/installer/bmc_management/or18.py @@ -0,0 +1,384 @@ +# Copyright 2019 Nokia + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import logging +import time +from .bmctools import BMC + +class BMCException(Exception): + pass + +class OR18(BMC): + def __init__(self, host, user, passwd, log_path=None): + super(OR18, self).__init__(host, user, passwd, log_path) + + def _clear_ris_configuration(self): + # Clear RIS configuration + try: + logging.debug('Clear RIS configuration.') + self._run_ipmitool_raw_command('0x32 0x9f 0x01 0x0d') + except Exception as err: + logging.warning('Exception when clearing RIS NFS configuration: %s', str(err)) + return False + return True + + def _check_virtual_media_started(self): + # check virtmedia service status + try: + out = self._run_ipmitool_raw_command('0x32 0xca 0x08') + logging.debug('Virtual media service status: %s', str(out[0])) + except Exception as err: + logging.warning('Exception when checking virtual media service: %s', str(err)) + if out[0] == '01': + return True + return False + + def _start_virtual_media(self): + # Enable "Remote Media Support" in GUI (p145) + try: + logging.debug('Start virtual media service') + self._run_ipmitool_raw_command('0x32 0xcb 0x08 0x01') + except Exception as err: + logging.warning('Exception when starting virtual media service: %s', str(err)) + + def _set_setup_nfs(self, nfs_host, mount_path): + + # Set share type NFS + try: + logging.debug('Virtual media share type to NFS.') + self._run_ipmitool_raw_command('0x32 0x9f 0x01 0x05 0x00 0x6e 0x66 0x73 0x00 0x00 0x00') + except Exception as err: + logging.warning('Exception when setting virtual media service type NFS: %s', str(err)) + return False + + # NFS server IP + try: + cmd = '0x32 0x9f 0x01 0x02 0x00 %s' % (self._convert_to_hex(nfs_host, True, 63)) + logging.debug('Virtual media server "%s"', nfs_host) + self._run_ipmitool_raw_command(cmd) + except Exception as err: + logging.warning('Exception when setting virtual media server: %s', str(err)) + return False + + # Set NFS Mount Root path + try: + logging.debug('Virtual media path to "%s"', mount_path) + # set progress bit (hmm. seems to return error if it is already set.. So should check..) + time.sleep(2) + cmd = '0x32 0x9f 0x01 0x01 0x00 0x00' + self._run_ipmitool_raw_command(cmd) + time.sleep(2) + cmd = '0x32 0x9f 0x01 0x01 0x00 0x01' + self._run_ipmitool_raw_command(cmd) + time.sleep(2) + cmd = '0x32 0x9f 0x01 0x01 0x01 %s' % (self._convert_to_hex(mount_path, True, 64)) + self._run_ipmitool_raw_command(cmd) + time.sleep(2) + # clear progress bit + cmd = '0x32 0x9f 0x01 0x01 0x00 0x00' + self._run_ipmitool_raw_command(cmd) + except Exception as err: + logging.warning('Exception when setting virtual media path: %s', str(err)) + return False + return True + + def _enable_virtual_media(self): + logging.debug('Enable Virtual Media') + + # 0x32 0xcb command will cause vmedia service restart automatically after 2 seconds according doc. + # restart is delayed by 2 seconds if new command is received during 2 second delay + # In other words, do all config in batch and it will only be restarted once. + + # Speed up things if it service is already running + if self._check_virtual_media_started(): + logging.debug('Virtual media service already running.') + # Service is already started + return True + + self._start_virtual_media() + + _max_tries = 6 + _try = 1 + # Just enabling the service doe not seem to start it (in all HW) + # Resetting it after enabling helps + self._restart_virtual_media_service() + while not self._check_virtual_media_started(): + if _try > _max_tries: + logging.warning('Ensure virtual media service start failed, attempts exceeded.') + return False + time.sleep(5) + _try = _try + 1 + return True + + def _get_virtual_media_device_count(self, devicetype): + try: + _num_inst = 0 + # Get num of enabled devices + if devicetype == 'CD': + _devparam = '0x04' + logging.debug('Get virtual CD count') + elif devicetype == 'FD': + _devparam = '0x05' + logging.debug('Get virtual FD count') + elif devicetype == 'HD': + _devparam = '0x06' + logging.debug('Get virtual HD count') + else: + logging.warning('Unknown device type "%s"', devicetype) + return _num_inst + + cmd = '0x32 0xca %s' % _devparam + out = self._run_ipmitool_raw_command(cmd) + _num_inst = int(out[0], 16) + + logging.debug('Number of enabled %s devices is %d', devicetype, _num_inst) + + return _num_inst + except Exception as err: + raise BMCException('Exception when getting number of enabled %s devices. error: %s' % (devicetype, str(err))) + + def _set_virtual_media_device_count(self, devicetype, devicecount): + # Chapter 46.2 page 181 + if not 0 <= devicecount <= 4: + logging.warning('Number of devices must be in range 0 to 4') + return False + + if devicetype == 'CD': + _devparam = '0x04' + logging.debug('Setting virtual CD count to %d', devicecount) + elif devicetype == 'HD': + _devparam = '0x06' + logging.debug('Setting virtual HD count to %d', devicecount) + else: + logging.warning('_set_virtual_media_device_count: Unknown device type "%s"', devicetype) + return False + + try: + cmd = '0x32 0xcb %s 0x%s' % (_devparam, str(devicecount)) + self._run_ipmitool_raw_command(cmd) + + _conf_device_num = self._get_virtual_media_device_count(devicetype) + _tries = 4 + while _conf_device_num != devicecount and _tries > 0: + logging.debug('Virtual %s count is %d expecting %d', devicetype, _conf_device_num, devicecount) + time.sleep(5) + _conf_device_num = self._get_virtual_media_device_count(devicetype) + _tries = _tries -1 + except Exception as err: + raise BMCException('Exception when setting virtual media device count : %s' % str(err)) + return True + + def _restart_virtual_media_service(self): + try: + cmd = '0x32 0xcb 0x0a 0x01' + logging.debug('Restart virtual media service') + self._run_ipmitool_raw_command(cmd) + except Exception as err: + raise BMCException('Exception when restarting virtual media service: %s' % str(err)) + + def _restart_ris(self): + try: + logging.debug('Restart RIS') + cmd = '0x32 0x9f 0x08 0x0b' + self._run_ipmitool_raw_command(cmd) + except Exception as err: + raise BMCException('Exception when restarting RIS: %s'% str(err)) + + return True + + def _restart_ris_cd(self): + try: + logging.debug('Restart RIS CD media') + cmd = '0x32 0x9f 0x01 0x0b 0x01' + self._run_ipmitool_raw_command(cmd) + except Exception as err: + raise BMCException('Exception when restarting RIS CD media: %s' % str(err)) + + return True + + def _check_cd_dvd_enabled(self, enabled): + try: + out = self._run_ipmitool_raw_command('0x32 0xca 0x0') + logging.debug('Virtual cd_dvd status: %s', str(out[0])) + except Exception as err: + logging.warning('Exception when checking cd_dvd status: %s', str(err)) + if (out[0] == '01' and enabled) or (out[0] == '00' and not enabled): + return True + return False + + def _enable_disable_cd_dvd(self, enabled): + _max_tries = 6 + _try = 1 + logging.debug('Enable/Disable cd_dvd') + while not self._check_cd_dvd_enabled(enabled): + if _try > _max_tries: + logging.warning('Ensure cd_dvd enable/disable failed, attempts exceeded. Ignoring and trying to continue.') + return True + time.sleep(5) + _try = _try + 1 + return True + + def _toggle_virtual_device(self, enabled): + # Enable "Mount CD/DVD" in GUI (p144) should cause vmedia restart withing 2 seconds. + # Seems "Mount CD/DVD" need to be enabled (or toggled) after config. refresh/vmedia restart + # is not enough(?) + try: + logging.debug('Enable/Disable mount CD/DVD.') + time.sleep(1) + #This will fail with new firmware on OR18 + self._run_ipmitool_raw_command('0x32 0xcb 0x00 0x0%s' %(str(int(enabled)))) + return self._enable_disable_cd_dvd(enabled) + except Exception as err: + logging.warning('Exception when CD/DVD virtual media new firmware? ignoring... Error: %s', str(err)) + return True + + def _mount_virtual_device(self): + return self._toggle_virtual_device(True) + + def _demount_virtual_device(self): + return self._toggle_virtual_device(False) + + def _get_mounted_image_count(self): + count = 0 + try: + out = self._run_ipmitool_raw_command('0x32 0xd8 0x00 0x01') + count = int(out[1], 16) + logging.warning('Available image count: %d', count) + except Exception as err: + logging.warning('Exception when trying to get the image count: %s', str(err)) + return count + + def _wait_for_mount_count(self): + # Poll until we got some images from server + _max_tries = 12 + _try = 1 + while self._get_mounted_image_count() == 0: + logging.debug('Check available images count try %d/%d', _try, _max_tries) + if _try > _max_tries: + logging.warning('Available images count 0, attempts exceeded.') + return False + time.sleep(10) + _try = _try + 1 + return True + + def _set_image_name(self, image_filename): + try: + logging.debug('Setting virtual media image: %s', image_filename) + self._run_ipmitool_raw_command('0x32 0xd7 0x01 0x01 0x01 0x01 %s' % (self._convert_to_hex(image_filename, True, 64))) + except Exception as err: + logging.debug('Exception when setting virtual media image: %s', str(err)) + return False + return True + + def _get_bmc_nfs_service_status(self): + # Check NFS Service Status + try: + out = self._run_ipmitool_raw_command('0x32 0xd8 0x06 0x01 0x01 0x00') + _image_name = str(bytearray.fromhex(''.join(out))) + return 'mounted' + except Exception: + return 'nfserror' + + def _stop_remote_redirection(self): + # Get num of enabled devices + _num_inst = self._get_virtual_media_device_count('CD') + for driveindex in range(0, _num_inst): + cmd = '0x32 0xd7 0x00 0x01 0x01 0x00 %s' % hex(driveindex) + logging.debug('Stop redirection CD/DVD drive index %d', driveindex) + try: + out = self._run_ipmitool_raw_command(cmd) + logging.debug('ipmitool out = %s', (out)) + except Exception as err: + # Drive might not be mounted to start with + logging.debug('_stop_remote_redirection: Ignoring exception when stopping redirection CD/DVD drive index %d error: %s', driveindex, str(err)) + + def _set_boot_from_virtual_media(self): + logging.debug('Set boot from cd (%s), and boot after that', self._host) + self._run_ipmitool_command('chassis bootdev cdrom options=persistent') + + #logging.debug('Set boot from cd (%s), and boot after that', self._host) + #try: + # self._run_ipmitool_raw_command('0x00 0x08 0x05 0xC0 0x20 0x00 0x00 0x00') + #except Exception as err: + # logging.warning('Set Boot to CD failed: %s' % str(err)) + # raise BMCException('Set Boot to CD failed') + + def _detach_virtual_media(self): + logging.debug('Detach virtual media') + + #Enable virtual media + if not self._enable_virtual_media(): + raise BMCException("detach_virtual_cd: Failed to enable virtual media") + + # Restart Remote Image Service + if not self._restart_ris(): + raise BMCException("Failed to restart RIS") + + # Stop redirection + self._stop_remote_redirection() + + #Clear RIS configuration + if not self._clear_ris_configuration(): + raise BMCException("detach_virtual_cd: Failed to clear RIS configuration") + + #Demount virtual device + if not self._demount_virtual_device(): + raise BMCException('detach_virtual_cd: Exception when disabling CD/DVD virtual media') + + # Reduce the number of virtual devices (both HD and CD default to 4 devices each) + if not self._set_virtual_media_device_count('HD', 0): + BMCException('Failed to set virtual media device count for HD') + if not self._set_virtual_media_device_count('CD', 1): + BMCException('Failed to set virtual media device count for CD') + + def attach_virtual_cd(self, nfs_host, nfs_mount, boot_iso_filename): + # Detach first + self._detach_virtual_media() + + logging.debug('Attach virtual media') + + #Enable virtual media + if not self._enable_virtual_media(): + raise BMCException("Failed to enable virtual media") + + #Enable CD/DVD device + if not self._toggle_virtual_device(True): + raise BMCException("Failed to enable virtual device") + + #Clear RIS configuration + if not self._clear_ris_configuration(): + raise BMCException("Failed to clear RIS configuration") + + #Setup nfs + if not self._set_setup_nfs(nfs_host, nfs_mount): + raise BMCException("Failed to setup nfs") + + # Restart Remote Image CD + if not self._restart_ris_cd(): + raise BMCException("Failed to restart RIS CD") + + #Wait for device to be mounted + if not self._wait_for_mount_count(): + raise BMCException("Failed when waiting for the device to appear") + + # Set Image Name + time.sleep(2) + if not self._set_image_name(boot_iso_filename): + raise BMCException("Failed to set image name") + + success = self._wait_for_bmc_nfs_service(90, 'mounted') + if success: + return True + else: + raise BMCException('NFS service setup failed') diff --git a/src/remoteinstaller/installer/bmc_management/rm18.py b/src/remoteinstaller/installer/bmc_management/rm18.py new file mode 100644 index 0000000..e498129 --- /dev/null +++ b/src/remoteinstaller/installer/bmc_management/rm18.py @@ -0,0 +1,27 @@ +# Copyright 2019 Nokia + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from .or18 import OR18 +import logging + +class BMCException(Exception): + pass + +class RM18(OR18): + def __init__(self, host, user, passwd, log_path=None): + super(RM18, self).__init__(host, user, passwd, log_path) + + def _set_boot_from_virtual_media(self): + logging.debug('Set boot from floppy (%s), and boot after that', self._host) + self._run_ipmitool_command('chassis bootdev floppy options=persistent') diff --git a/src/remoteinstaller/installer/catfile.py b/src/remoteinstaller/installer/catfile.py new file mode 100644 index 0000000..a653721 --- /dev/null +++ b/src/remoteinstaller/installer/catfile.py @@ -0,0 +1,133 @@ +# Copyright 2019 Nokia + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import sys +import argparse +import logging +import pexpect + +class CatFileException(Exception): + pass + +class CatFile(object): + def __init__(self, bmc_host, bmc_user, bmc_password, login_user, login_password): + self._host = bmc_host + self._user = bmc_user + self._password = bmc_password + self._sol = None + + self._login_user = login_user + self._login_password = login_password + + def _open_console(self, log): + logging.info('Open SOL console') + + logging.debug('deactivate sol') + expect_session = pexpect.spawn('ipmitool -I lanplus -H {} -U {} -P {} sol deactivate'.format(self._host, self._user, self._password)) + expect_session.expect(pexpect.EOF) + + logging.debug('activate sol, output will go to %s', log) + self._sol = pexpect.spawn('ipmitool -I lanplus -H {} -U {} -P {} sol activate'.format(self._host, self._user, self._password), timeout=None) + logfile = open(log, 'wb') + self._sol.logfile_read = logfile + + def _close_console(self): + logging.info('Close SOL console') + + if self._sol: + logging.debug('Logout from host') + self._sol.sendline('logout\r\n') + self._sol.sendline() + self._sol.expect('login:', timeout=10) + self._sol.terminate() + + logging.debug('deactivate sol') + session = pexpect.spawn('ipmitool -I lanplus -H {} -U {} -P {} sol deactivate'.format(self._host, self._user, self._password)) + session.expect(pexpect.EOF) + + def _expect_cmd_prompt(self): + self._sol.expect('# ', timeout=10) + + def _login(self): + logging.info('Login to host') + + try: + self._sol.sendline() + + self._expect_cmd_prompt() + logging.debug('Command prompt found') + + return + except pexpect.exceptions.TIMEOUT as e: + pass + + try: + self._sol.sendline() + + self._sol.expect('login:', timeout=10) + logging.debug('Login prompt found') + + self._sol.sendline(self._login_user) + + self._sol.expect('Password:', timeout=10) + logging.debug('Password prompt found') + + self._sol.sendline(self._login_password) + + self._sol.sendline() + self._expect_cmd_prompt() + logging.debug('Command prompt found') + except pexpect.exceptions.TIMEOUT as e: + logging.debug(e) + raise + + def _cat_log(self, path, timeout=120): + logging.debug('Catting %s', path) + + self._sol.sendline('cat {}; echo CONSOLE_CAT_DONE\r\n'.format(path)) + self._sol.expect('CONSOLE_CAT_DONE', timeout=timeout) + logging.debug('Catting done') + + self._expect_cmd_prompt() + + def cat(self, path, log_file, timeout=None): + try: + self._open_console(log_file) + self._login() + self._cat_log(path, timeout) + self._close_console() + except Exception as ex: + logging.warn('Cat file failed: %s', str(ex)) + raise CatFileException(str(ex)) + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument('-H', '--bmc_host', required=True, help='BMC host') + parser.add_argument('-U', '--bmc_user', required=True, help='BMC user') + parser.add_argument('-P', '--bmc_password', required=True, help='BMC user password') + parser.add_argument('-u', '--user', required=True, help='Login user') + parser.add_argument('-p', '--password', required=True, help='Login user password') + parser.add_argument('-f', '--file', required=True, help='File path to cat') + parser.add_argument('-o', '--output_file', required=True, help='Output file name of the log') + parser.add_argument('-t', '--timeout', required=False, help='Timeout for catting the file') + + args = parser.parse_args() + + logging.basicConfig(level=logging.DEBUG) + + cat_file = CatFile(args.bmc_host, args.bmc_user, args.bmc_password, args.user, args.password) + cat_file.cat(args.file, args.output_file, args.password) + +if __name__ == "__main__": + sys.exit(main()) diff --git a/src/remoteinstaller/installer/install.py b/src/remoteinstaller/installer/install.py new file mode 100644 index 0000000..1d60066 --- /dev/null +++ b/src/remoteinstaller/installer/install.py @@ -0,0 +1,389 @@ +# Copyright 2019 Nokia + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import logging +import sys +import argparse +import subprocess +import os +import importlib +import time + +from yaml import load +from netaddr import IPNetwork +from netaddr import IPAddress + +import hw_detector.hw_detect_lib as hw_detect +from hw_detector.hw_exception import HWException +from remoteinstaller.installer.bmc_management.bmctools import BMCException +from remoteinstaller.installer.catfile import CatFile +from remoteinstaller.installer.catfile import CatFileException + +class InstallException(Exception): + pass + +class Installer(object): + SSH_OPTS = '-o StrictHostKeyChecking=no \ + -o UserKnownHostsFile=/dev/null \ + -o ServerAliveInterval=60' + + def __init__(self, args): + self._yaml_path = args.yaml + self._boot_iso_path = args.boot_iso + self._iso_url = args.iso + self._logdir = args.logdir + self._callback_url = args.callback_url + self._client_key = args.client_key + self._client_cert = args.client_cert + self._ca_cert = args.ca_cert + self._own_ip = args.host_ip + self._tag = args.tag + self._http_port = args.http_port + + # TODO + self._disable_bmc_initial_reset = True + self._disable_other_bmc_reset = True + + self._uc = self._read_user_config(self._yaml_path) + self._vip = None + self._first_controller_ip = None + self._first_controller_bmc = None + + @staticmethod + def _read_user_config(config_file_path): + logging.debug('Read user config from %s', config_file_path) + + try: + with open(config_file_path, 'r') as f: + y = load(f) + + return y + except Exception as ex: + raise InstallException(str(ex)) + + @staticmethod + def _execute_shell(command, desc=''): + logging.debug('Execute %s with command: %s', desc, command) + + p = subprocess.Popen(command.split(), stdout=subprocess.PIPE, stderr=subprocess.STDOUT) + out, _ = p.communicate() + if p.returncode: + logging.warning('Failed to %s: %s (rc=%s)', desc, out, p.returncode) + raise InstallException('Failed to {}'.format(desc)) + + return (p.returncode, out) + + def _attach_iso_as_virtual_media(self, file_list): + logging.info('Attach ISO as virtual media') + + nfs_mount = os.path.dirname(self._boot_iso_path) + boot_iso_filename = os.path.basename(self._boot_iso_path) + patched_iso_filename = '{}/{}_{}'.format(nfs_mount, self._tag, boot_iso_filename) + + self._patch_iso(patched_iso_filename, file_list) + + self._first_controller_bmc.attach_virtual_cd(self._own_ip, nfs_mount, os.path.basename(patched_iso_filename)) + + def _create_cloud_config(self, first_controller): + logging.info('Create network config file') + + domain = self._uc['hosts'][first_controller].get('network_domain') + extnet = self._uc['networking']['infra_external']['network_domains'][domain] + + vlan = extnet.get('vlan') + first_ip = extnet['ip_range_start'] + gateway = extnet['gateway'] + dns = self._uc['networking']['dns'][0] + cidr = extnet['cidr'] + prefix = IPNetwork(cidr).prefixlen + + self._vip = str(IPAddress(first_ip)) + + pre_allocated_ips = self._uc['hosts'][first_controller].get('pre_allocated_ips', None) + if pre_allocated_ips: + pre_allocated_infra_external_ip = pre_allocated_ips.get('infra_external', None) + self._first_controller_ip = str(IPAddress(pre_allocated_infra_external_ip)) + + if not self._first_controller_ip: + self._first_controller_ip = str(IPAddress(first_ip)+1) + + controller_network_profile = self._uc['hosts'][first_controller]['network_profiles'][0] + mappings = self._uc['network_profiles'][controller_network_profile]['interface_net_mapping'] + for interface in mappings: + if 'infra_external' in mappings[interface]: + infra_external_interface = interface + break + + if infra_external_interface.startswith('bond'): + bonds = self._uc['network_profiles'][controller_network_profile]['bonding_interfaces'] + device = bonds[infra_external_interface][0] + else: + device = infra_external_interface + + # TODO + # ROOTFS_DISK + + logging.debug('VLAN=%s', vlan) + logging.debug('DEV=%s', device) + logging.debug('IP=%s/%s', self._first_controller_ip, prefix) + logging.debug('DGW=%s', gateway) + logging.debug('NAMESERVER=%s', dns) + logging.debug('ISO_URL="%s"', self._iso_url) + + network_config_filename = '{}/network_config'.format(os.getcwd()) + with open(network_config_filename, 'w') as f: + if vlan: + f.write('VLAN={}\n'.format(vlan)) + f.write('DEV={}\n'.format(device)) + f.write('IP={}/{}\n'.format(self._first_controller_ip, prefix)) + f.write('DGW={}\n'.format(gateway)) + f.write('NAMESERVER={}\n'.format(dns)) + f.write('\n') + f.write('ISO_URL="{}"'.format(self._iso_url)) + + logging.debug('CALLBACK_URL="%s"', self._callback_url) + + callback_url_filename = '{}/callback_url'.format(os.getcwd()) + with open(callback_url_filename, 'w') as f: + f.write(self._callback_url) + + if self._client_cert: + return [self._yaml_path, + network_config_filename, + callback_url_filename, + self._client_key, + self._client_cert, + self._ca_cert] + + return [self._yaml_path, + network_config_filename, + callback_url_filename] + + def _patch_iso(self, iso_target, file_list): + logging.info('Patch boot ISO') + logging.debug('Original ISO: %s', self._boot_iso_path) + logging.debug('Target ISO: %s', iso_target) + + file_list_str = ' '.join(file_list) + logging.debug('Files to add: %s', file_list_str) + + self._execute_shell('/usr/bin/patchiso.sh {} {} {}'.format(self._boot_iso_path, + iso_target, + file_list_str), 'patch ISO') + + def _put_file(self, ip, user, passwd, file_name, to_file=''): + self._execute_shell('sshpass -p {} scp {} {} {}@{}:{}'.format(passwd, + Installer.SSH_OPTS, + file_name, + user, + ip, + to_file)) + + def _get_file(self, log_dir, ip, user, passwd, file_name, recursive=False): + if recursive: + self._execute_shell('sshpass -p {} scp {} -r {}@{}:{} {}'.format(passwd, + Installer.SSH_OPTS, + user, + ip, + file_name, + log_dir)) + else: + self._execute_shell('sshpass -p {} scp {} {}@{}:{} {}'.format(passwd, + Installer.SSH_OPTS, + user, + ip, + file_name, + log_dir)) + + def _get_node_logs(self, log_dir, ip, user, passwd): + self._get_file(log_dir, ip, user, passwd, '/srv/deployment/log/cm.log') + self._get_file(log_dir, ip, user, passwd, '/srv/deployment/log/bootstrap.log') + self._get_file(log_dir, ip, user, passwd, '/var/log/ironic', recursive=True) + + def _get_journal_logs(self, log_dir, ip, user, passwd): + self._put_file(ip, user, passwd, '/opt/remoteinstaller/get_journals.sh') + self._put_file(ip, user, passwd, '/opt/remoteinstaller/print_hosts.py') + + self._execute_shell('sh ./get_journals.sh') + + self._get_file(log_dir, ip, user, passwd, '/tmp/node_journals.tgz') + + def _get_logs_from_console(self, log_dir, bmc, admin_user, admin_passwd): + bmc_host = bmc.get_host() + bmc_user = bmc.get_user() + bmc_passwd = bmc.get_passwd() + + log_file = '{}/cat_bootstrap.log'.format(log_dir) + try: + cat_file = CatFile(bmc_host, bmc_user, bmc_passwd, admin_user, admin_passwd) + cat_file.cat('/srv/deployment/log/bootstrap.log', log_file) + except CatFileException as ex: + logging.info('Could not cat file from console: %s', str(ex)) + + cat_file = CatFile(bmc_host, bmc_user, bmc_passwd, 'root', 'root') + cat_file.cat('/srv/deployment/log/bootstrap.log', log_file) + + def get_logs(self, log_dir, admin_passwd): + admin_user = self._uc['users']['admin_user_name'] + + ssh_command = 'nc -w1 {} 22 /dev/null'.format(self._first_controller_ip) + ssh_responds = self._execute_shell(ssh_command) + + if ssh_responds: + self._get_node_logs(log_dir, self._first_controller_ip, admin_user, admin_passwd) + + self._get_journal_logs(log_dir, self._first_controller_ip, admin_user, admin_passwd) + else: + self._get_logs_from_console(log_dir, + self._first_controller_bmc, + admin_user, + admin_passwd) + + def install(self): + try: + logging.info('Start install') + + if os.path.dirname(self._boot_iso_path) == '': + self._boot_iso_path = '{}/{}'.format(os.getcwd(), self._boot_iso_path) + + if self._logdir: + if not os.path.exists(self._logdir): + os.makedirs(self._logdir) + else: + self._logdir = '.' + + other_bmcs = [] + first_controller = None + for hw in sorted(self._uc['hosts']): + logging.info('HW node name is %s', hw) + + if not first_controller: + if 'controller' in self._uc['hosts'][hw]['service_profiles'] or \ + 'caas_master' in self._uc['hosts'][hw]['service_profiles']: + first_controller = hw + logging.info('HW is first controller') + + host = self._uc['hosts'][hw]['hwmgmt']['address'] + user = self._uc['hosts'][hw]['hwmgmt']['user'] + passwd = self._uc['hosts'][hw]['hwmgmt']['password'] + + bmc_log_path = '{}/{}.log'.format(self._logdir, hw) + + try: + hw_data = hw_detect.get_hw_data(host, user, passwd, False) + except HWException as e: + error = "Harware not detected for {}: {}".format(hw, str(e)) + logging.error(error) + raise BMCException(error) + + logging.info("Hardware belongs to %s product family", (hw_data['product_family'])) + if 'Unknown' in hw_data['product_family']: + error = "Hardware not detected for %s" % hw + logging.error(error) + raise BMCException(error) + + bmc_mod_name = 'remoteinstaller.installer.bmc_management.{}'.format(hw_data['product_family'].lower()) + bmc_mod = importlib.import_module(bmc_mod_name) + bmc_class = getattr(bmc_mod, hw_data['product_family']) + bmc = bmc_class(host, user, passwd, bmc_log_path) + bmc.set_host_name(hw) + + bmc.setup_sol() + + if hw != first_controller: + other_bmcs.append(bmc) + bmc.power('off') + else: + self._first_controller_bmc = bmc + + logging.debug('First controller: %s', first_controller) + + if not self._disable_bmc_initial_reset: + self._first_controller_bmc.reset() + time_after_reset = int(time.time()) + + if not self._disable_other_bmc_reset: + for bmc in other_bmcs: + bmc.reset() + + if not self._disable_bmc_initial_reset: + # Make sure we sleep at least 6min after the first controller BMC reset + sleep_time = 6*60-int(time.time())-time_after_reset + if sleep_time > 0: + logging.debug('Waiting for first controller BMC to stabilize \ + (%s sec) after reset', sleep_time) + time.sleep(sleep_time) + + config_file_names = self._create_cloud_config(first_controller) + + self._first_controller_bmc.setup_boot_options_for_virtual_media() + + self._attach_iso_as_virtual_media(config_file_names) + + self._first_controller_bmc.boot_from_virtual_media() + + self._first_controller_bmc.wait_for_bootup() + + self._first_controller_bmc.close() + + access_info = {'vip': self._vip, + 'installer_node_ip': self._first_controller_ip, + 'admin_user': self._uc['users']['admin_user_name']} + + return access_info + except BMCException as ex: + logging.error('Installation failed: %s', str(ex)) + raise InstallException(str(ex)) + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument('-y', '--yaml', required=True, + help='User config yaml file path') + parser.add_argument('-b', '--boot-iso', required=True, + help='Path to boot ISO image in NFS mount') + parser.add_argument('-i', '--iso', required=True, + help='URL to ISO image') + parser.add_argument('-d', '--debug', action='store_true', required=False, + help='Debug level for logging') + parser.add_argument('-l', '--logdir', required=False, + help='Directory path for log files') + parser.add_argument('-c', '--callback-url', required=True, + help='Callback URL for progress reporting') + parser.add_argument('-K', '--client-key', required=True, + help='Client key file path') + parser.add_argument('-C', '--client-cert', required=True, + help='Client cert file path') + parser.add_argument('-A', '--ca-cert', required=True, + help='CA cert file path') + parser.add_argument('-H', '--host-ip', required=True, + help='IP for hosting HTTPD and NFS') + parser.add_argument('-T', '--http-port', required=False, + help='Port for HTTPD') + + parsed_args = parser.parse_args() + + if parsed_args.debug: + log_level = logging.DEBUG + else: + log_level = logging.INFO + + logging.basicConfig(stream=sys.stdout, level=log_level) + + logging.debug('args: %s', parsed_args) + installer = Installer(parsed_args) + + installer.install() + +if __name__ == "__main__": + sys.exit(main()) diff --git a/src/remoteinstaller/server/__init__.py b/src/remoteinstaller/server/__init__.py new file mode 100644 index 0000000..41eaa50 --- /dev/null +++ b/src/remoteinstaller/server/__init__.py @@ -0,0 +1,13 @@ +# Copyright 2019 Nokia + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. diff --git a/src/remoteinstaller/server/server.py b/src/remoteinstaller/server/server.py new file mode 100644 index 0000000..b925dd4 --- /dev/null +++ b/src/remoteinstaller/server/server.py @@ -0,0 +1,589 @@ +# Copyright 2019 Nokia + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import sys +import argparse +import logging +import os +from threading import Thread +import time +import json +import urllib +import urlparse +import uuid as uuid_module +import ssl + +from wsgiref.simple_server import make_server +import routes + +from remoteinstaller.installer.install import Installer +from remoteinstaller.installer.install import InstallException + + +class LoggingSSLSocket(ssl.SSLSocket): + def accept(self, *args, **kwargs): + try: + result = super(LoggingSSLSocket, self).accept(*args, **kwargs) + except Exception as ex: + logging.warn('SSLSocket.accept raised exception: %s', str(ex)) + raise + return result + + +class InstallationWorker(Thread): + def __init__(self, server, uuid, admin_passwd, logdir, args=None): + super(InstallationWorker, self).__init__(name=uuid) + self._server = server + self._uuid = uuid + self._admin_passwd = admin_passwd + self._logdir = logdir + self._args = args + + def run(self): + access_info = None + if self._args: + try: + installer = Installer(self._args) + #access_info = installer.install() + + logging.info('Installation triggered for %s', self._uuid) + except InstallException as ex: + logging.warn('Installation triggering failed for %s: %s', self._uuid, str(ex)) + self._server.set_state(self._uuid, 'failed', str(ex), 0) + return + + installation_finished = False + while not installation_finished: + state = self._server.get_state(self._uuid) + if not state['status'] == 'ongoing': + installation_finished = True + else: + time.sleep(10) + + logging.info('Installation finished for %s: %s', self._uuid, state) + if access_info: + logging.info('Login details for installation %s: %s', self._uuid, str(access_info)) + + logging.info('Getting logs for installation %s...', uuid) + #installer.get_logs(self._logdir, self._admin_passwd) + logging.info('Logs retrieved for %s', uuid) + +class Server(object): + DEFAULT_PATH = '/opt/remoteinstaller' + USER_CONFIG_PATH = 'user-configs' + ISO_PATH = 'images' + CERTIFICATE_PATH = 'certificates' + INSTALLATIONS_PATH = 'installations' + #CLOUD_ISO_PATH = '{}/rec.iso'.format(ISO_PATH) + BOOT_ISO_PATH = '{}/boot.iso'.format(ISO_PATH) + + def __init__(self, host, port, cert=None, key=None, client_cert=None, client_key=None, ca_cert=None, path=None, http_port=None): + self._host = host + self._port = port + self._http_port = http_port + + self._path = path + if not self._path: + self._path = Server.DEFAULT_PATH + + self._cert = '{}/{}/{}'.format(self._path, Server.CERTIFICATE_PATH, cert) + self._key = '{}/{}/{}'.format(self._path, Server.CERTIFICATE_PATH, key) + self._client_cert = '{}/{}/{}'.format(self._path, Server.CERTIFICATE_PATH, client_cert) + self._client_key = '{}/{}/{}'.format(self._path, Server.CERTIFICATE_PATH, client_key) + self._ca_cert = '{}/{}/{}'.format(self._path, Server.CERTIFICATE_PATH, ca_cert) + + self._ongoing_installations = {} + self._load_states() + + def get_server_keys(self): + return {'cert': self._cert, 'key': self._key, 'ca_cert': self._ca_cert} + + def _read_admin_passwd(self, cloud_name): + with open('{}/{}/{}/admin_passwd'.format(self._path, + Server.USER_CONFIG_PATH, + cloud_name)) as pwf: + admin_passwd = pwf.readline() + + return admin_passwd + + def _load_states(self): + uuid_list = os.listdir('{}/{}'.format(self._path, Server.INSTALLATIONS_PATH)) + for uuid in uuid_list: + state_file_name = '{}/{}/{}.state'.format(self._path, Server.INSTALLATIONS_PATH, uuid) + if os.path.exists(state_file_name): + with open(state_file_name) as sf: + state_json = sf.readline() + self._ongoing_installations[uuid] = json.loads(state_json) + + if self._ongoing_installations[uuid]['status'] == 'ongoing': + logdir = '{}/{}/{}'.format(self._path, Server.INSTALLATIONS_PATH, uuid) + cloud_name = self._ongoing_installations[uuid]['cloud_name'] + admin_passwd = self._read_admin_passwd(cloud_name) + worker = InstallationWorker(self, uuid, admin_passwd, logdir) + worker.start() + + def _set_state(self, uuid, status, description, percentage, cloud_name=None): + self._ongoing_installations[uuid] = {} + self._ongoing_installations[uuid]['status'] = status + self._ongoing_installations[uuid]['description'] = description + self._ongoing_installations[uuid]['percentage'] = percentage + if cloud_name: + self._ongoing_installations[uuid]['cloud_name'] = cloud_name + + state_file = '{}/{}/{}.state'.format(self._path, Server.INSTALLATIONS_PATH, uuid) + with open(state_file, 'w') as sf: + sf.write(json.dumps(self._ongoing_installations[uuid])) + + def set_state(self, uuid, status, description, percentage): + logging.info('uuid=%s, status=%s, description=%s, percentage=%s', + uuid, status, description, percentage) + + if not uuid in self._ongoing_installations: + raise ServerError('Installation id {} not found'.format(uuid)) + + if not status in ['ongoing', 'failed', 'completed']: + raise ServerError('Invalid state: {}'.format(status)) + + self._set_state(uuid, status, description, percentage) + + def get_state(self, uuid): + logging.info('uuid=%s', uuid) + + if not uuid in self._ongoing_installations: + raise ServerError('Installation id {} not found'.format(uuid)) + + return {'status': self._ongoing_installations[uuid]['status'], + 'description': self._ongoing_installations[uuid]['description'], + 'percentage': self._ongoing_installations[uuid]['percentage']} + + def start_installation(self, cloud_name, iso): + logging.info('start_installation(%s, %s)', cloud_name, iso) + + uuid = str(uuid_module.uuid4()) + + args = argparse.Namespace() + + args.yaml = '{}/{}/{}/user_config.yml'.format(self._path, + Server.USER_CONFIG_PATH, + cloud_name) + if not os.path.isfile(args.yaml): + raise ServerError('YAML file {} not found'.format(args.yaml)) + + iso_path = '{}/{}/{}'.format(self._path, Server.ISO_PATH, iso) + if not os.path.isfile(iso_path): + raise ServerError('ISO file {} not found'.format(iso_path)) + + http_port_part = '' + if self._http_port: + http_port_part = ':{}'.format(self._http_port) + + args.iso = 'https://{}{}/{}/{}'.format(self._host, http_port_part, Server.ISO_PATH, iso) + + args.logdir = '{}/{}/{}'.format(self._path, Server.INSTALLATIONS_PATH, uuid) + + os.makedirs(args.logdir) + + args.boot_iso = '{}/{}'.format(self._path, Server.BOOT_ISO_PATH) + + args.tag = uuid + args.callback_url = 'http://{}:{}/v1/installations/{}/state'.format(self._host, + self._port, + uuid) + + args.client_cert = self._client_cert + args.client_key = self._client_key + args.ca_cert = self._ca_cert + args.host_ip = self._host + + self._set_state(uuid, 'ongoing', '', 0, cloud_name) + + admin_passwd = self._read_admin_passwd(cloud_name) + worker = InstallationWorker(self, uuid, admin_passwd, args.logdir, args) + worker.start() + + return uuid + + +class ServerError(Exception): + pass + + +class HTTPErrors(object): + # response for a successful GET, PUT, PATCH, DELETE, + # can also be used for POST that does not result in creation. + HTTP_OK = 200 + # response to a POST which results in creation. + HTTP_CREATED = 201 + # response to a successfull request that won't be returning any body like a DELETE request + HTTP_NO_CONTENT = 204 + # used when http caching headers are in play + HTTP_NOT_MODIFIED = 304 + # the request is malformed such as if the body does not parse + HTTP_BAD_REQUEST = 400 + # when no or invalid authentication details are provided. + # also useful to trigger an auth popup API is used from a browser + HTTP_UNAUTHORIZED_OPERATION = 401 + # when authentication succeeded but authenticated user doesn't have access to the resource + HTTP_FORBIDDEN = 403 + # when a non-existent resource is requested + HTTP_NOT_FOUND = 404 + # when an http method is being requested that isn't allowed for the authenticated user + HTTP_METHOD_NOT_ALLOWED = 405 + # indicates the resource at this point is no longer available + HTTP_GONE = 410 + # if incorrect content type was provided as part of the request + HTTP_UNSUPPORTED_MEDIA_TYPE = 415 + # used for validation errors + HTTP_UNPROCESSABLE_ENTITY = 422 + # when request is rejected due to rate limiting + HTTP_TOO_MANY_REQUESTS = 429 + # Other errrors + HTTP_INTERNAL_ERROR = 500 + + @staticmethod + def get_ok_status(): + return '%d OK' % HTTPErrors.HTTP_OK + + @staticmethod + def get_object_created_successfully_status(): + return '%d Created' % HTTPErrors.HTTP_CREATED + + @staticmethod + def get_request_not_ok_status(): + return '%d Bad request' % HTTPErrors.HTTP_BAD_REQUEST + + @staticmethod + def get_resource_not_found_status(): + return '%d Not found' % HTTPErrors.HTTP_NOT_FOUND + + @staticmethod + def get_unsupported_content_type_status(): + return '%d Unsupported content type' % HTTPErrors.HTTP_UNSUPPORTED_MEDIA_TYPE + + @staticmethod + def get_validation_error_status(): + return '%d Validation error' % HTTPErrors.HTTP_UNPROCESSABLE_ENTITY + + @staticmethod + def get_internal_error_status(): + return '%d Internal error' % HTTPErrors.HTTP_INTERNAL_ERROR + + +class HTTPRPC(object): + def __init__(self): + self.req_body = '' + self.req_filter = '' + self.req_params = {} + self.req_method = '' + self.req_content_type = '' + self.req_content_size = 0 + self.req_path = '' + + self.rep_body = '' + self.rep_status = '' + + def __str__(self): + return str.format('REQ: body:{body} filter:{filter} ' + 'params:{params} method:{method} path:{path} ' + 'content_type:{content_type} content_size:{content_size} ' + 'REP: body:{rep_body} status:{status}', + body=self.req_body, filter=self.req_filter, + params=str(self.req_params), method=self.req_method, path=self.req_path, + content_type=self.req_content_type, content_size=self.req_content_size, + rep_body=self.rep_body, status=self.rep_status) + +class WSGIHandler(object): + def __init__(self, server): + logging.debug('WSGIHandler constructor called') + + self.server = server + + self.mapper = routes.Mapper() + self.mapper.connect(None, '/apis', action='get_apis') + self.mapper.connect(None, '/{api}/installations', action='handle_installations') + self.mapper.connect(None, '/{api}/installations/{uuid}/state', action='handle_state') + + def handle_installations(self, rpc): + if rpc.req_method == 'POST': + self._start_installation(rpc) + else: + rpc.rep_status = HTTPErrors.get_request_not_ok_status() + rpc.rep_status += ', only POST are possible to this resource' + + def handle_state(self, rpc): + if rpc.req_method == 'GET': + self._get_state(rpc) + elif rpc.req_method == 'POST': + self._set_state(rpc) + else: + rpc.rep_status = HTTPErrors.get_request_not_ok_status() + rpc.rep_status += ', only GET/POST are possible to this resource' + + def _start_installation(self, rpc): + """ + Request: POST http:///v1/installations + { + 'cloud-name': , + 'iso': , + } + Response: http status set correctly + { + 'uuid': + } + """ + + logging.debug('_start_installation called') + try: + if not rpc.req_body: + rpc.rep_status = HTTPErrors.get_request_not_ok_status() + else: + request = json.loads(rpc.req_body) + cloud_name = request['cloud-name'] + iso = request['iso'] + + uuid = self.server.start_installation(cloud_name, iso) + + rpc.rep_status = HTTPErrors.get_ok_status() + reply = {'uuid': uuid} + rpc.rep_body = json.dumps(reply) + except KeyError as ex: + rpc.rep_status = HTTPErrors.get_request_not_ok_status() + raise ServerError('Missing request parameter: {}'.format(str(ex))) + except Exception as exp: # pylint: disable=broad-except + rpc.rep_status = HTTPErrors.get_internal_error_status() + rpc.rep_status += ',' + rpc.rep_status += str(exp) + + def _get_state(self, rpc): + """ + Request: GET http:///v1/installations//state + { + } + Response: http status set correctly + { + 'status': , + 'description': , + 'percentage': + } + """ + + logging.debug('_get_state called') + try: + if not rpc.req_body: + rpc.rep_status = HTTPErrors.get_request_not_ok_status() + else: + uuid = rpc.req_params['uuid'] + + reply = self.server.get_state(uuid) + + rpc.rep_status = HTTPErrors.get_ok_status() + rpc.rep_body = json.dumps(reply) + except KeyError as ex: + rpc.rep_status = HTTPErrors.get_request_not_ok_status() + raise ServerError('Missing request parameter: {}'.format(str(ex))) + except Exception as exp: # pylint: disable=broad-except + rpc.rep_status = HTTPErrors.get_internal_error_status() + rpc.rep_status += ',' + rpc.rep_status += str(exp) + + def _set_state(self, rpc): + """ + Request: POST http:///v1/installations//state + { + 'status': , + 'description': , + 'percentage': + } + Response: http status set correctly + { + } + """ + + logging.debug('set_state called') + try: + if not rpc.req_body: + rpc.rep_status = HTTPErrors.get_request_not_ok_status() + else: + request = json.loads(rpc.req_body) + uuid = rpc.req_params['uuid'] + status = request['status'] + description = request['description'] + percentage = request['percentage'] + + self.server.set_state(uuid, status, description, percentage) + + rpc.rep_status = HTTPErrors.get_ok_status() + reply = {} + rpc.rep_body = json.dumps(reply) + except ServerError: + raise + except KeyError as ex: + rpc.rep_status = HTTPErrors.get_request_not_ok_status() + raise ServerError('Missing request parameter: {}'.format(str(ex))) + except Exception as exp: # pylint: disable=broad-except + rpc.rep_status = HTTPErrors.get_internal_error_status() + rpc.rep_status += ',' + rpc.rep_status += str(exp) + + def _read_header(self, rpc, environ): + rpc.req_method = environ['REQUEST_METHOD'] + rpc.req_path = environ['PATH_INFO'] + try: + rpc.req_filter = urlparse.parse_qs(urllib.unquote(environ['QUERY_STRING'])) + except KeyError: + rpc.req_filter = {} + rpc.req_content_type = environ['CONTENT_TYPE'] + try: + rpc.req_content_size = int(environ['CONTENT_LENGTH']) + except KeyError: + rpc.req_content_size = 0 + + def _get_action(self, rpc): + # get the action to be done + action = '' + match_result = self.mapper.match(rpc.req_path) + if not match_result: + rpc.rep_status = HTTPErrors.get_resource_not_found_status() + raise ServerError('URL does not match') + + resultdict = {} + if isinstance(match_result, dict): + resultdict = match_result + else: + resultdict = match_result[0] + + try: + action = resultdict['action'] + for key, value in resultdict.iteritems(): + if key != 'action': + rpc.req_params[key] = value + except KeyError: + rpc.rep_status = HTTPErrors.get_internal_error_status() + raise ServerError('No action found') + + return action + + def _read_body(self, rpc, environ): + # get the body if available + if rpc.req_content_size: + if rpc.req_content_type == 'application/json': + rpc.req_body = environ['wsgi.input'].read(rpc.req_content_size) + else: + rpc.rep_status = HTTPErrors.get_unsupported_content_type_status() + raise ServerError('Content type is not json') + + def __call__(self, environ, start_response): + logging.debug('Handling request started, environ=%s', str(environ)) + + # For request and resonse data + rpc = HTTPRPC() + rpc.rep_status = HTTPErrors.get_ok_status() + + try: + self._read_header(rpc, environ) + + action = self._get_action(rpc) + + self._read_body(rpc, environ) + + logging.info('Calling %s with rpc=%s', action, str(rpc)) + actionfunc = getattr(self, action) + actionfunc(rpc) + except ServerError as ex: + rpc.rep_status = HTTPErrors.get_request_not_ok_status() + rpc.rep_status += ',' + rpc.rep_status += str(ex) + except AttributeError: + rpc.rep_status = HTTPErrors.get_internal_error_status() + rpc.rep_status += ',' + rpc.rep_status += 'Missing action function' + except Exception as exp: # pylint: disable=broad-except + rpc.rep_status = HTTPErrors.get_internal_error_status() + rpc.rep_status += ',' + rpc.rep_status += str(exp) + + logging.info('Replying with rpc=%s', str(rpc)) + response_headers = [('Content-type', 'application/json')] + start_response(rpc.rep_status, response_headers) + return [rpc.rep_body] + +def wrap_socket(sock, keyfile=None, certfile=None, + server_side=False, cert_reqs=ssl.CERT_NONE, + ssl_version=ssl.PROTOCOL_SSLv23, ca_certs=None, + do_handshake_on_connect=True, + suppress_ragged_eofs=True, + ciphers=None): + + return LoggingSSLSocket(sock=sock, keyfile=keyfile, certfile=certfile, + server_side=server_side, cert_reqs=cert_reqs, + ssl_version=ssl_version, ca_certs=ca_certs, + do_handshake_on_connect=do_handshake_on_connect, + suppress_ragged_eofs=suppress_ragged_eofs, + ciphers=ciphers) + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument('-H', '--host', required=True, help='binding ip of the server') + parser.add_argument('-P', '--listen', required=True, help='binding port of the server') + parser.add_argument('-S', '--server', required=False, help='externally visible ip of the server') + parser.add_argument('-B', '--port', required=False, help='externally visible port of the server') + parser.add_argument('-C', '--cert', required=False, help='server cert file name') + parser.add_argument('-K', '--key', required=False, help='server private key file name') + parser.add_argument('-c', '--client-cert', required=False, help='client cert file name') + parser.add_argument('-k', '--client-key', required=False, help='client key file name') + parser.add_argument('-A', '--ca-cert', required=False, help='CA cert file name') + parser.add_argument('-p', '--path', required=False, help='path for remote installer files') + parser.add_argument('-T', '--http-port', required=False, help='port for HTTPD') + parser.add_argument('-d', '--debug', required=False, help='Debug level for logging', + action='store_true') + + args = parser.parse_args() + + if args.debug: + log_level = logging.DEBUG + else: + log_level = logging.INFO + + format = '%(asctime)s %(threadName)s:%(levelname)s %(message)s' + logging.basicConfig(stream=sys.stdout, level=log_level, format=format) + + logging.debug('args: %s', args) + + host = args.server + if not host: + host = args.host + + port = args.port + if not port: + port = args.listen + + server = Server(host, port, args.cert, args.key, args.client_cert, args.client_key, args.ca_cert, args.path, args.http_port) + + wsgihandler = WSGIHandler(server) + + wsgi_server = make_server(args.host, int(args.listen), wsgihandler) + + if args.cert: + server_keys = server.get_server_keys() + wsgi_server.socket = wrap_socket(wsgi_server.socket, + certfile=server_keys['cert'], + keyfile=server_keys['key'], + server_side=True, + ca_certs=server_keys['ca_cert'], + cert_reqs=ssl.CERT_REQUIRED) + + wsgi_server.serve_forever() + +if __name__ == "__main__": + sys.exit(main()) diff --git a/src/scripts/patchiso.sh b/src/scripts/patchiso.sh new file mode 100755 index 0000000..2b4a280 --- /dev/null +++ b/src/scripts/patchiso.sh @@ -0,0 +1,48 @@ +#!/bin/bash + +# Copyright 2019 Nokia + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +abort() +{ + echo "abort: $*" + exit 1 +} + +syntax() +{ + cat < index.txt +echo '01' > serial.txt +echo -n > index-ri.txt +echo '01' > serial-ri.txt + +# Sign server certificate with CA private key +echo -e "y\ny\n" | openssl ca -config openssl-ca-sign.cnf -policy signing_policy -extensions signing_req -out servercert.pem -infiles servercert.csr +# Sign client certificate with server private key +echo -e "y\ny\n" | openssl ca -config openssl-ca-sign.cnf -policy signing_policy -extensions signing_req -out clientcert.pem -infiles clientcert.csr +echo -e "y\ny\n" | openssl ca -config openssl-ca-sign.cnf -policy signing_policy -extensions signing_req -out badboycert.pem -infiles badboycert.csr + +# openssl x509 -in cacert.pem -text -noout diff --git a/test/certificates/openssl-badboy.cnf b/test/certificates/openssl-badboy.cnf new file mode 100644 index 0000000..7cc21ae --- /dev/null +++ b/test/certificates/openssl-badboy.cnf @@ -0,0 +1,32 @@ +HOME = . +RANDFILE = $ENV::HOME/.rnd + +#################################################################### +[ req ] +prompt = no +default_bits = 2048 +default_keyfile = badboykey.pem +distinguished_name = badboy_distinguished_name +req_extensions = badboy_req_extensions +string_mask = utf8only + +#################################################################### +[ badboy_distinguished_name ] +countryName = SE +organizationName = bad +commonName = boy +emailAddress = test@badboy.com + +#################################################################### +[ badboy_req_extensions ] + +subjectKeyIdentifier = hash +basicConstraints = CA:FALSE +keyUsage = digitalSignature, keyEncipherment +subjectAltName = @alternate_names +nsComment = "OpenSSL Generated Certificate" + +#################################################################### +[ alternate_names ] + +DNS.1 = bad-boy.net diff --git a/test/certificates/openssl-ca-sign.cnf b/test/certificates/openssl-ca-sign.cnf new file mode 100644 index 0000000..93ddb95 --- /dev/null +++ b/test/certificates/openssl-ca-sign.cnf @@ -0,0 +1,71 @@ +HOME = . +RANDFILE = $ENV::HOME/.rnd + +#################################################################### +[ ca ] +default_ca = CA_default # The default ca section + +[ CA_default ] + +default_days = 1000 # How long to certify for +default_crl_days = 30 # How long before next CRL +default_md = sha256 # Use public key default MD +preserve = no # Keep passed DN ordering + +x509_extensions = ca_extensions # The extensions to add to the cert + +email_in_dn = no # Don't concat the email in the DN +copy_extensions = copy # Required to copy SANs from CSR to cert +base_dir = . +certificate = $base_dir/cacert.pem # The CA certifcate +private_key = $base_dir/cakey.pem # The CA private key +new_certs_dir = $base_dir # Location for new certs after signing +database = $base_dir/index.txt # Database index file +serial = $base_dir/serial.txt # The current serial number + +unique_subject = no # Set to 'no' to allow creation of + # several certificates with same subject. + +#################################################################### +[ req ] +prompt = no +default_bits = 4096 +default_keyfile = cakey.pem +distinguished_name = ca_distinguished_name +x509_extensions = ca_extensions +string_mask = utf8only + +#################################################################### +[ ca_distinguished_name ] +countryName = FI +organizationName = Nokia OY +# commonName = Nokia +# commonName_default = Test Server +# emailAddress = test@server.com +stateOrProvinceName = Uusimaa +localityName = Espoo + +#################################################################### +[ ca_extensions ] + +subjectKeyIdentifier = hash +authorityKeyIdentifier = keyid:always, issuer +basicConstraints = critical, CA:true +keyUsage = keyCertSign, cRLSign + +#################################################################### +[ signing_policy ] +countryName = optional +stateOrProvinceName = optional +localityName = optional +organizationName = optional +organizationalUnitName = optional +commonName = supplied +emailAddress = optional + +#################################################################### +[ signing_req ] +subjectKeyIdentifier = hash +authorityKeyIdentifier = keyid,issuer +basicConstraints = CA:FALSE +keyUsage = digitalSignature, keyEncipherment diff --git a/test/certificates/openssl-ca.cnf b/test/certificates/openssl-ca.cnf new file mode 100644 index 0000000..c79ddf2 --- /dev/null +++ b/test/certificates/openssl-ca.cnf @@ -0,0 +1,46 @@ +HOME = . +RANDFILE = $ENV::HOME/.rnd + +#################################################################### +[ ca ] +default_ca = CA_default # The default ca section + +[ CA_default ] + +dir = /root/ca +default_days = 1000 # How long to certify for +default_crl_days = 30 # How long before next CRL +default_md = sha256 # Use public key default MD +preserve = no # Keep passed DN ordering + +x509_extensions = ca_extensions # The extensions to add to the cert + +email_in_dn = no # Don't concat the email in the DN +copy_extensions = copy # Required to copy SANs from CSR to cert + +#################################################################### +[ req ] +prompt = no +default_bits = 4096 +default_keyfile = cakey.pem +distinguished_name = ca_distinguished_name +x509_extensions = ca_extensions +string_mask = utf8only + +#################################################################### +[ ca_distinguished_name ] +countryName = FI +organizationName = Nokia OY +# commonName = Nokia +# commonName_default = Test Server +# emailAddress = test@server.com +stateOrProvinceName = Uusimaa +localityName = Espoo + +#################################################################### +[ ca_extensions ] + +subjectKeyIdentifier = hash +authorityKeyIdentifier = keyid:always, issuer +basicConstraints = critical, CA:true +keyUsage = keyCertSign, cRLSign diff --git a/test/certificates/openssl-client-sign.cnf b/test/certificates/openssl-client-sign.cnf new file mode 100644 index 0000000..1ba1a72 --- /dev/null +++ b/test/certificates/openssl-client-sign.cnf @@ -0,0 +1,78 @@ +HOME = . +RANDFILE = $ENV::HOME/.rnd + +#################################################################### +[ req ] +prompt = no +default_bits = 2048 +default_keyfile = clientkey.pem +distinguished_name = client_distinguished_name +req_extensions = client_req_extensions +string_mask = utf8only + +#################################################################### +[ client_distinguished_name ] +countryName = DE +organizationName = Customer X +commonName = Customer +emailAddress = test@client.com + +#################################################################### +[ client_req_extensions ] + +subjectKeyIdentifier = hash +basicConstraints = CA:FALSE +keyUsage = digitalSignature, keyEncipherment +subjectAltName = @alternate_names +nsComment = "OpenSSL Generated Certificate" + +#################################################################### +[ ca ] +default_ca = CA_default # The default ca section + +[ CA_default ] + +default_days = 1000 # How long to certify for +default_crl_days = 30 # How long before next CRL +default_md = sha256 # Use public key default MD +preserve = no # Keep passed DN ordering + +x509_extensions = ca_extensions # The extensions to add to the cert + +email_in_dn = no # Don't concat the email in the DN +copy_extensions = copy # Required to copy SANs from CSR to cert +base_dir = . +certificate = $base_dir/clientcert.pem # The CA certifcate +private_key = $base_dir/clientkey.pem # The CA private key +new_certs_dir = $base_dir # Location for new certs after signing +database = $base_dir/index-ri.txt # Database index file +serial = $base_dir/serial-ri.txt # The current serial number + +unique_subject = no # Set to 'no' to allow creation of + # several certificates with same subject. + +#################################################################### +[ signing_policy ] +countryName = optional +stateOrProvinceName = optional +localityName = optional +organizationName = optional +organizationalUnitName = optional +commonName = supplied +emailAddress = optional + +#################################################################### +[ signing_req ] +subjectKeyIdentifier = hash +authorityKeyIdentifier = keyid,issuer +# authorityKeyIdentifier = issuer +basicConstraints = CA:FALSE +keyUsage = digitalSignature, keyEncipherment + +#################################################################### +[ alternate_names ] + +DNS.1 = ramuller.zoo.dynamic.nsn-net.net +DNS.2 = www.client.com +DNS.3 = mail.client.com +DNS.4 = ftp.client.com diff --git a/test/certificates/openssl-client.cnf b/test/certificates/openssl-client.cnf new file mode 100644 index 0000000..21f9c05 --- /dev/null +++ b/test/certificates/openssl-client.cnf @@ -0,0 +1,35 @@ +HOME = . +RANDFILE = $ENV::HOME/.rnd + +#################################################################### +[ req ] +prompt = no +default_bits = 2048 +default_keyfile = clientkey.pem +distinguished_name = client_distinguished_name +req_extensions = client_req_extensions +string_mask = utf8only + +#################################################################### +[ client_distinguished_name ] +countryName = DE +organizationName = Customer X +commonName = Customer +emailAddress = test@client.com + +#################################################################### +[ client_req_extensions ] + +subjectKeyIdentifier = hash +basicConstraints = CA:FALSE +keyUsage = digitalSignature, keyEncipherment +subjectAltName = @alternate_names +nsComment = "OpenSSL Generated Certificate" + +#################################################################### +[ alternate_names ] + +DNS.1 = ramuller.zoo.dynamic.nsn-net.net +DNS.2 = www.client.com +DNS.3 = mail.client.com +DNS.4 = ftp.client.com diff --git a/test/certificates/openssl-server-sign.cnf b/test/certificates/openssl-server-sign.cnf new file mode 100644 index 0000000..065871e --- /dev/null +++ b/test/certificates/openssl-server-sign.cnf @@ -0,0 +1,76 @@ +HOME = . +RANDFILE = $ENV::HOME/.rnd + +#################################################################### +[ req ] +prompt = no +default_bits = 2048 +default_keyfile = serverkey.pem +distinguished_name = server_distinguished_name +req_extensions = server_req_extensions +string_mask = utf8only + +[ ca ] +default_ca = CA_default # The default ca section + +[ CA_default ] + +default_days = 1000 # How long to certify for +default_crl_days = 30 # How long before next CRL +default_md = sha256 # Use public key default MD +preserve = no # Keep passed DN ordering + +x509_extensions = ca_extensions # The extensions to add to the cert + +email_in_dn = no # Don't concat the email in the DN +copy_extensions = copy # Required to copy SANs from CSR to cert +base_dir = . +certificate = $base_dir/servercert.pem # The CA certifcate +private_key = $base_dir/serverkey.pem # The CA private key +new_certs_dir = $base_dir # Location for new certs after signing +database = $base_dir/index-ri.txt # Database index file +serial = $base_dir/serial-ri.txt # The current serial number + +unique_subject = no # Set to 'no' to allow creation of + # several certificates with same subject. +#################################################################### +[ signing_policy ] +countryName = optional +stateOrProvinceName = optional +localityName = optional +organizationName = optional +organizationalUnitName = optional +commonName = supplied +emailAddress = optional + +#################################################################### +[ signing_req ] +subjectKeyIdentifier = hash +authorityKeyIdentifier = keyid,issuer +# authorityKeyIdentifier = issuer +basicConstraints = CA:FALSE +keyUsage = digitalSignature, keyEncipherment + +#################################################################### +[ server_distinguished_name ] +countryName = FI +organizationName = Nokia NET +commonName = Test Server +# emailAddress = test@server.com +stateOrProvinceName = Uusimaa +localityName = Espoo + +#################################################################### +[ server_req_extensions ] + +subjectKeyIdentifier = hash +basicConstraints = CA:FALSE +keyUsage = digitalSignature, keyEncipherment +subjectAltName = @alternate_names +nsComment = "OpenSSL Generated Certificate" + +#################################################################### +[ alternate_names ] + +DNS.1 = server.com + diff --git a/test/certificates/openssl-server.cnf b/test/certificates/openssl-server.cnf new file mode 100644 index 0000000..a4fb8db --- /dev/null +++ b/test/certificates/openssl-server.cnf @@ -0,0 +1,35 @@ +HOME = . +RANDFILE = $ENV::HOME/.rnd + +#################################################################### +[ req ] +prompt = no +default_bits = 2048 +default_keyfile = serverkey.pem +distinguished_name = server_distinguished_name +req_extensions = server_req_extensions +string_mask = utf8only + +#################################################################### +[ server_distinguished_name ] +countryName = FI +organizationName = Nokia NET +commonName = Test Server +# emailAddress = test@server.com +stateOrProvinceName = Uusimaa +localityName = Espoo + +#################################################################### +[ server_req_extensions ] + +subjectKeyIdentifier = hash +basicConstraints = CA:FALSE +keyUsage = digitalSignature, keyEncipherment +subjectAltName = @alternate_names +nsComment = "OpenSSL Generated Certificate" + +#################################################################### +[ alternate_names ] + +DNS.1 = server.com + diff --git a/test/certificates/openssl.cnf b/test/certificates/openssl.cnf new file mode 100644 index 0000000..6c761d7 --- /dev/null +++ b/test/certificates/openssl.cnf @@ -0,0 +1,133 @@ +# OpenSSL root CA configuration file. +# Copy to `/root/ca/openssl.cnf`. + +[ ca ] +# `man ca` +default_ca = CA_default + +[ CA_default ] +# Directory and file locations. +dir = /root/ca +certs = $dir/certs +crl_dir = $dir/crl +new_certs_dir = $dir/newcerts +database = $dir/index.txt +serial = $dir/serial +RANDFILE = $dir/private/.rand + +# The root key and root certificate. +private_key = $dir/private/ca.key.pem +certificate = $dir/certs/ca.cert.pem + +# For certificate revocation lists. +crlnumber = $dir/crlnumber +crl = $dir/crl/ca.crl.pem +crl_extensions = crl_ext +default_crl_days = 30 + +# SHA-1 is deprecated, so use SHA-2 instead. +default_md = sha256 + +name_opt = ca_default +cert_opt = ca_default +default_days = 375 +preserve = no +policy = policy_strict + +[ policy_strict ] +# The root CA should only sign intermediate certificates that match. +# See the POLICY FORMAT section of `man ca`. +countryName = match +stateOrProvinceName = match +organizationName = match +organizationalUnitName = optional +commonName = supplied +emailAddress = optional + +[ policy_loose ] +# Allow the intermediate CA to sign a more diverse range of certificates. +# See the POLICY FORMAT section of the `ca` man page. +countryName = optional +stateOrProvinceName = optional +localityName = optional +organizationName = optional +organizationalUnitName = optional +commonName = supplied +emailAddress = optional + +[ req ] +# Options for the `req` tool (`man req`). +default_bits = 2048 +distinguished_name = req_distinguished_name +string_mask = utf8only + +# SHA-1 is deprecated, so use SHA-2 instead. +default_md = sha256 + +# Extension to add when the -x509 option is used. +x509_extensions = v3_ca + +[ req_distinguished_name ] +# See . +countryName = FI +stateOrProvinceName = Uusimaa +localityName = Espoo +0.organizationName = Nokia +organizationalUnitName = NET +commonName = Nokia NET +# emailAddress = Email Address +emailAddress = + +# Optionally, specify some defaults. +# countryName_default = GB +# stateOrProvinceName_default = England +# localityName_default = +# 0.organizationName_default = Alice Ltd +# organizationalUnitName_default = +# emailAddress_default = + +[ v3_ca ] +# Extensions for a typical CA (`man x509v3_config`). +subjectKeyIdentifier = hash +authorityKeyIdentifier = keyid:always,issuer +basicConstraints = critical, CA:true +keyUsage = critical, digitalSignature, cRLSign, keyCertSign + +[ v3_intermediate_ca ] +# Extensions for a typical intermediate CA (`man x509v3_config`). +subjectKeyIdentifier = hash +authorityKeyIdentifier = keyid:always,issuer +basicConstraints = critical, CA:true, pathlen:0 +keyUsage = critical, digitalSignature, cRLSign, keyCertSign + +[ usr_cert ] +# Extensions for client certificates (`man x509v3_config`). +basicConstraints = CA:FALSE +nsCertType = client, email +nsComment = "OpenSSL Generated Client Certificate" +subjectKeyIdentifier = hash +authorityKeyIdentifier = keyid,issuer +keyUsage = critical, nonRepudiation, digitalSignature, keyEncipherment +extendedKeyUsage = clientAuth, emailProtection + +[ server_cert ] +# Extensions for server certificates (`man x509v3_config`). +basicConstraints = CA:FALSE +nsCertType = server +nsComment = "OpenSSL Generated Server Certificate" +subjectKeyIdentifier = hash +authorityKeyIdentifier = keyid,issuer:always +keyUsage = critical, digitalSignature, keyEncipherment +extendedKeyUsage = serverAuth + +[ crl_ext ] +# Extension for CRLs (`man x509v3_config`). +authorityKeyIdentifier=keyid:always + +[ ocsp ] +# Extension for OCSP signing certificates (`man ocsp`). +basicConstraints = CA:FALSE +subjectKeyIdentifier = hash +authorityKeyIdentifier = keyid,issuer +keyUsage = critical, digitalSignature +extendedKeyUsage = critical, OCSPSigning