--- /dev/null
+.coveragepy27-pytest
+.idea/
+.pytest-cache/
+.tox/
+.coverage
+htmlcov/
+__pycache__/
+*.egg-info/
+*.egg
+*.pyc
+*.pyo
--- /dev/null
+
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
+
+ APPENDIX: How to apply the Apache License to your work.
+
+ To apply the Apache License to your work, attach the following
+ boilerplate notice, with the fields enclosed by brackets "[]"
+ replaced with your own identifying information. (Don't include
+ the brackets!) The text should be enclosed in the appropriate
+ comment syntax for the file format. We also recommend that a
+ file or class name and description of purpose be included on the
+ same "printed page" as the copyright notice for easier
+ identification within third-party archives.
+
+ Copyright [yyyy] [name of copyright owner]
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from setuptools import setup, find_packages
+setup(
+ name='cmdatahandlers',
+ version='1.0',
+ license='Apache-2.0',
+ long_description='README',
+ author='Baha Mesleh',
+ author_email='baha.mesleh@nokia.com',
+ namespace_packages=['cmdatahandlers'],
+ packages=find_packages('src'),
+ include_package_data=True,
+ package_dir={'': 'src'},
+ description='Configuration Data Handlers',
+ zip_safe=False,
+)
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+__import__('pkg_resources').declare_namespace(__name__)
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from cmdatahandlers.api import configerror
+
+class Config(object):
+ def __init__(self, confman):
+ self.confman = confman
+ self.config = confman.get_config()
+ self.DOMAIN = None
+ self.ROOT = None
+ self.MASK = '*******'
+
+ def init(self):
+ raise configerror.ConfigError('Not implemented')
+
+ def validate_root(self):
+ if self.ROOT not in self.config:
+ raise configerror.ConfigError('No %s configuration found' % self.DOMAIN)
+
+ def get_domain(self):
+ return self.DOMAIN
+
+ def validate(self):
+ raise configerror.ConfigError('Not implemented')
+
+ def mask_sensitive_data(self):
+ pass
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import sys
+
+class ConfigError(Exception):
+ def __init__(self, description):
+ self.description = description
+
+ def get_description(self):
+ return self.description
+
+ def __str__(self):
+ return '%s' % self.description
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import json
+import os
+import sys
+import inspect
+import imp
+import types
+import argparse
+
+from cmdatahandlers.api import configerror
+
+class ConfigManager(object):
+ #This needs to be updated when new domain are introduced, a getter function
+ #should be added to the domain config handler
+ """
+ def __init__(self, schema, configjson):
+ #validate according to schema
+ try:
+ self.configjson = configjson
+ builder = pjo.ObjectBuilder(schema)
+ ns = builder.build_classes()
+ self.config = ns.Config(**configjson)
+ self._load_config_handlers()
+ except Exception as exp:
+ raise configerror.ConfigError(str(exp))
+ """
+
+ def __init__(self, configjson):
+ #validate according to schema
+ self.configmap = {}
+ try:
+ self.configjson = configjson
+ self._load_config_handlers()
+ except Exception as exp:
+ raise configerror.ConfigError(str(exp))
+
+ def get_config(self):
+ return self.configjson
+
+ def get_cloud_name(self):
+ if 'cloud.name' not in self.configjson:
+ raise configerror.ConfigError('Cloud name not defined')
+
+ return self.configjson['cloud.name']
+
+ def get_cloud_description(self):
+ if 'cloud.description' not in self.configjson:
+ raise configerror.ConfigError('Cloud description not defined')
+ return self.configjson['cloud.description']
+
+ def get_cloud_installation_date(self):
+ if 'cloud.installation_date' not in self.configjson:
+ raise configerror.ConfigError('Cloud installation date is not defined')
+ return self.configjson['cloud.installation_date']
+
+ def get_cloud_installation_phase(self):
+ if 'cloud.installation_phase' not in self.configjson:
+ raise configerror.ConfigError('Cloud installation phase is not defined')
+ return self.configjson['cloud.installation_phase']
+
+ def _load_config_handlers(self):
+ myfolder = os.path.realpath(os.path.abspath(os.path.split(inspect.getfile(inspect.currentframe()))[0]))
+ dirn = os.path.dirname(myfolder)
+ basen = os.path.basename(myfolder)
+
+ for d in os.listdir(dirn):
+ if d == basen:
+ continue
+ if not os.path.isdir(dirn + '/' + d):
+ continue
+
+ configmodule = dirn + '/' + d + '/config.py'
+
+
+ if not os.path.isfile(configmodule):
+ continue
+
+ mod = imp.load_source(d, configmodule)
+
+ config = mod.Config(self)
+
+ domain = config.get_domain()
+
+ self.configmap[domain] = config
+
+ domhandlerfunc = 'get_' + domain + '_config_handler'
+
+ setattr(self, domhandlerfunc, types.MethodType(self._get_domain_config_handler, domain))
+
+
+ #finalize initialization after objects are created
+ #this is needed to handle inter-handler dependencies
+ for domain, handler in self.configmap.iteritems():
+ handler.init()
+
+
+
+ def _get_domain_config_handler(self, domain):
+ if domain not in self.configmap:
+ raise configerror.ConfigError('Invalid domain')
+
+ return self.configmap[domain]
+
+
+ def mask_sensitive_data(self):
+ for handler in self.configmap.values():
+ try:
+ handler.validate_root()
+ except configerror.ConfigError:
+ continue
+
+ handler.mask_sensitive_data()
+
+if __name__ == '__main__':
+ parser = argparse.ArgumentParser(description='Config Manager', prog=sys.argv[0])
+
+ parser.add_argument('--domain',
+ required=True,
+ dest='domain',
+ metavar='DOMAIN',
+ help='The configuration domain',
+ type=str,
+ action='store')
+
+ parser.add_argument('--json',
+ required=True,
+ dest='jsonfile',
+ metavar='JSONFILE',
+ help='The json file containing the configuration',
+ type=str,
+ action='store')
+
+ parser.add_argument('--api',
+ required=True,
+ dest='api',
+ metavar='API',
+ help='The API to call in the domain',
+ type=str,
+ action='store')
+
+ parser.add_argument('--dump',
+ required=False,
+ dest='dump',
+ help='Dump the configuration',
+ action='store_true')
+
+ args, unknownargs = parser.parse_known_args(sys.argv[1:])
+ print("args = %r" % args)
+ print("unknownargs = %r" % unknownargs)
+ f = open(args.jsonfile)
+ data = json.load(f)
+ f.close()
+ manager = ConfigManager(data)
+ #domain handler func
+ funcname = 'get_'+args.domain+'_config_handler'
+ objfunc = getattr(manager, funcname)
+ obj = objfunc()
+ print('Got handler for %s' % obj.get_domain())
+
+ domainfunc = getattr(obj, args.api)
+ result = None
+ if unknownargs:
+ result = domainfunc(*unknownargs)
+ else:
+ result = domainfunc()
+ print("result is %r" % result)
+
+ if args.dump:
+ import pprint
+ pp = pprint.PrettyPrinter(indent=4)
+ pp.pprint(data)
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import socket
+import netaddr
+import subprocess
+import json
+
+from cmdatahandlers.api import configerror
+
+def validate_ipv4_address(address):
+ try:
+ socket.inet_pton(socket.AF_INET, address)
+ except AttributeError:
+ try:
+ socket.inet_aton(address)
+ except socket.error:
+ raise configerror.ConfigError('Invalid ip %s' % address)
+ if address.count('.') != 3:
+ raise configerror.ConfigError('Invalid ip %s' % address)
+ except socket.error:
+ raise configerror.ConfigError('Invalid ip %s' % address)
+
+
+def validate_list_items_unique(l):
+ if len(l) != len(set(l)):
+ raise configerror.ConfigError('List is not unique')
+
+
+def validate_cidr(cidr):
+ try:
+ tok = cidr.split('/')
+ if len(tok) != 2:
+ raise configerror.ConfigError('Invalid cidr address %s' % cidr)
+ validate_ipv4_address(tok[0])
+ except Exception as exp:
+ raise configerror.ConfigError(str(exp))
+
+def validate_ip_in_network(ip, cidr):
+ try:
+ if netaddr.IPAddress(ip) not in netaddr.IPNetwork(cidr):
+ raise configerror.ConfigError('%s does not belong to network %s' % (ip, cidr))
+ except Exception as exp:
+ raise configerror.ConfigError(str(exp))
+
+def validate_keys_in_dictionary(keys, dictionary):
+ for key in keys:
+ if key not in dictionary:
+ raise configerror.ConfigError('%s is not found' % key)
+
+def validate_vlan(vlan):
+ if vlan < 0 or vlan > 4096:
+ raise configerror.ConfigError('Vlan %d is not valid' % vlan)
+
+def get_own_hwmgmt_ip():
+ try:
+ #try both ipv4 and ipv6 addresses
+ ips=[]
+ output=subprocess.check_output(['sudo', 'ipmitool', 'lan', 'print'])
+ lines = output.split('\n')
+ for line in lines:
+ if 'IP Address' in line and 'IP Address Source' not in line:
+ data = line.split(':')
+ if len(data) != 2:
+ raise configerror.ConfigError('Invalid hwmgmt ip configured')
+ ip=data[1]
+ import re
+ ip=re.sub('[\s+]', '', ip)
+ ips.append(ip)
+
+ output_lan6=subprocess.check_output(['sudo', 'ipmitool', 'lan6', 'print'],stderr=subprocess.STDOUT)
+ lines = output_lan6.split('\n')
+ num_static_addr = 0
+ num_dynamic_addr = 0
+ #get max number of ipv6 static address
+ for line in lines:
+ if 'Static address max' in line:
+ data = line.split(':')
+ static_address = data[1]
+ import re
+ num_static_addr = int(re.sub('[\s+]', '', static_address))
+ if 'Dynamic address max' in line:
+ data = line.split(':')
+ dynamic_address = data[1]
+ import re
+ num_dynamic_addr = int(re.sub('[\s+]', '', dynamic_address))
+
+ for x in range(num_static_addr):
+ address = 'IPv6 Static Address %s' %x
+ for idx,val in enumerate(lines):
+ if address in val:
+ if 'Address' in lines[idx+2]:
+ data=lines[idx+2].split(':',1)
+ ip=data[1]
+ import re
+ ip=re.sub('[\s+]', '', ip)
+ ip=ip.split('/',1)[0]
+ ips.append(ip.strip())
+
+ for x in range(num_dynamic_addr):
+ address = 'IPv6 Dynamic Address %s' %x
+ for idx,val in enumerate(lines):
+ if address in val:
+ if 'Address' in lines[idx+2]:
+ data=lines[idx+2].split(':',1)
+ ip=data[1]
+ import re
+ ip=re.sub('[\s+]', '', ip)
+ ip=ip.split('/',1)[0]
+ ips.append(ip.strip())
+ return ips
+
+ except Exception as exp:
+ raise configerror.ConfigError(str(exp))
+
+def is_virtualized():
+ f=open('/proc/cpuinfo')
+ lines = f.readlines()
+ f.close()
+ for line in lines:
+ if line.startswith('flags') and 'hypervisor' in line:
+ return True
+ return False
+
+def flatten_config_data(jsondata):
+ result = {}
+ for key, value in jsondata.iteritems():
+ try:
+ result[key] = json.dumps(value)
+ except Exception as exp:
+ result[key] = value
+ return result
+
+
+def unflatten_config_data(props):
+ propsjson = {}
+ for name, value in props.iteritems():
+ try:
+ propsjson[name] = json.loads(value)
+ except Exception as exp:
+ propsjson[name] = value
+ return propsjson
+
+def add_lists(l1, l2):
+ result = l1
+ for v2 in l2:
+ if v2 not in result:
+ result.append(v2)
+ return result
--- /dev/null
+#!/usr/bin/python
+
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import logging
+import re
+
+from netaddr import IPAddress
+from netaddr import IPNetwork
+from netaddr import IPRange
+from netaddr import AddrFormatError
+
+from cmdatahandlers.api import configerror
+
+
+class ValidationError(configerror.ConfigError):
+ def __init__(self, description):
+ configerror.ConfigError.__init__(self, ' Validation error: %s' % description)
+
+
+
+class ValidationUtils:
+
+ def validate_ip_address(self, addr):
+ try:
+ IPAddress(addr)
+ except AddrFormatError as exc:
+ raise ValidationError("Invalid ip address: {0}".format(exc))
+
+
+ def validate_subnet_address(self, cidr):
+ try:
+ net = IPNetwork(cidr)
+ except AddrFormatError as exc:
+ raise ValidationError('Invalid subnet address: {0}'.format(exc))
+ #it seems ipnetwork compress ipv6 cidr.
+ #skip this for ipv6
+ if net.version == 4:
+ if cidr != str(net.cidr):
+ raise ValidationError('Given CIDR %s is not equal to network CIDR %s'
+ % (cidr, str(net.cidr)))
+
+
+ def validate_ip_range(self, start_ip, end_ip):
+ try:
+ IPRange(start_ip, end_ip)
+ except AddrFormatError as exc:
+ raise ValidationError('Invalid ip range: {0}'.format(exc))
+
+
+ def validate_ip_in_subnet(self, ip_addr, cidr):
+ self.validate_ip_address(ip_addr)
+ ip = IPAddress(ip_addr)
+
+ self.validate_subnet_address(cidr)
+ subnet = IPNetwork(cidr)
+ if not ip in subnet or ip == subnet.ip or ip == subnet.broadcast:
+ raise ValidationError('IP %s is not a valid address in subnet %s' % (ip_addr, cidr))
+
+
+ def validate_ip_in_range(self, addr, start, end):
+ self.validate_ip_address(addr)
+ self.validate_ip_range(start, end)
+ if IPAddress(addr) not in IPRange(start, end):
+ raise ValidationError('IP %s is not in range %s - %s' % (addr, start, end))
+
+
+ def validate_ip_not_in_range(self, addr, start, end):
+ self.validate_ip_address(addr)
+ self.validate_ip_range(start, end)
+ if IPAddress(addr) in IPRange(start, end):
+ raise ValidationError('IP %s is in reserved range %s - %s' % (addr, start, end))
+
+
+ def validate_username(self, user):
+ if not re.match('^[a-zA-Z][\da-zA-Z-_]+[\da-zA-Z]$', user):
+ raise ValidationError('Invalid user name %s' % user)
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from cmdatahandlers.api import config
+from cmdatahandlers.api import utils
+from cmdatahandlers.api import configerror
+from serviceprofiles import profiles
+import yaml
+import jinja2
+
+CAAS_CONFIG_FILE_PATH = "/etc/cmframework/config/"
+CAAS_CONFIG_FILE = "caas.yaml"
+
+
+class Config(config.Config):
+ valid_redundancy_models = ['non-redundant', 'active-cold-standby']
+
+ def __init__(self, confman):
+ super(Config, self).__init__(confman)
+ self.ROOT = 'cloud.caas'
+ self.DOMAIN = 'caas'
+
+ def init(self):
+ pass
+
+ def validate(self):
+ print("validate")
+
+ def flavour_set(self):
+ hostsconf = self.confman.get_hosts_config_handler()
+ caas_masters = []
+ for host in hostsconf.get_hosts():
+ if 'caas_master' in hostsconf.get_service_profiles(host):
+ caas_masters.append(host)
+
+ if len(caas_masters) > 1:
+ return "multi"
+ else:
+ return "single"
+
+ def set_dynamic_config(self):
+ if utils.is_virtualized():
+ self.config[self.ROOT]['vnf_embedded_deployment'] = self.get_vnf_flag()
+ user_conf = self.confman.get_users_config_handler()
+ self.config[self.ROOT]['helm_home'] = "/home/" + user_conf.get_admin_user() + "/.helm"
+ self.config[self.ROOT]['flavour'] = self.flavour_set()
+
+ def set_static_config(self):
+ try:
+ template = jinja2.Environment(
+ loader=jinja2.FileSystemLoader(
+ CAAS_CONFIG_FILE_PATH)).get_template(CAAS_CONFIG_FILE)
+ with open(CAAS_CONFIG_FILE_PATH + CAAS_CONFIG_FILE) as config_file:
+ data = yaml.load(config_file)
+ outputText = template.render(data)
+ config_data = yaml.load(outputText)
+ for key in config_data:
+ self.config[self.ROOT][key] = config_data[key]
+ except jinja2.exceptions.TemplateNotFound:
+ return
+ except Exception:
+ raise configerror.ConfigError("Unexpected issue occured!")
+
+ def add_defaults(self):
+ if not self.config.get('cloud.caas', ''):
+ return
+ self.set_dynamic_config()
+ self.set_static_config()
+
+ def is_vnf_embedded_deployment(self):
+ return (self.get_caas_only() and self.get_vnf_flag())
+
+ def get_vnf_flag(self):
+ return bool(self.config.get(self.ROOT, {}).get('vnf_embedded_deployment',
+ False))
+
+ def get_caas_only(self):
+ return self.is_caas_deployment() and not self.is_openstack_deployment()
+
+ def is_openstack_deployment(self):
+ return bool(self.get_controller_hosts())
+
+ def is_caas_deployment(self):
+ return bool(self.get_caas_master_hosts())
+
+ def is_hybrid_deployment(self):
+ return self.is_caas_deployment() and self.is_openstack_deployment()
+
+ def get_caas_master_hosts(self):
+ service_profiles_lib = profiles.Profiles()
+ return self._get_hosts_for_service_profile(service_profiles_lib.get_caasmaster_service_profile())
+
+ def _get_hosts_for_service_profile(self, profile):
+ hostsconf = self.confman.get_hosts_config_handler()
+ return hostsconf.get_service_profile_hosts(profile)
+
+ def get_controller_hosts(self):
+ service_profiles_lib = profiles.Profiles()
+ return self._get_hosts_for_service_profile(service_profiles_lib.get_controller_service_profile())
+
+ def get_apiserver_in_hosts(self):
+ return self.config.get(self.ROOT, {}).get('apiserver_in_hosts', '')
+
+ def get_registry_url(self):
+ return self.config.get(self.ROOT, {}).get('registry_url', '')
+
+ def get_update_registry_url(self):
+ return self.config.get(self.ROOT, {}).get('update_registry_url', '')
+
+ def get_swift_url(self):
+ return self.config.get(self.ROOT, {}).get('swift_url', '')
+
+ def get_swift_update_url(self):
+ return self.config.get(self.ROOT, {}).get('swift_update_url', '')
+
+ def get_ldap_master_url(self):
+ return self.config.get(self.ROOT, {}).get('ldap_master_url', '')
+
+ def get_ldap_slave_url(self):
+ return self.config.get(self.ROOT, {}).get('ldap_slave_url', '')
+
+ def get_chart_repo_url(self):
+ return self.config.get(self.ROOT, {}).get('chart_repo_url', '')
+
+ def get_tiller_url(self):
+ return self.config.get(self.ROOT, {}).get('tiller_url', '')
+
+ def get_apiserver_svc_ip(self):
+ return self.config.get(self.ROOT, {}).get('apiserver_svc_ip', '')
+
+ def get_caas_parameter(self, parameter):
+ return self.config.get(self.ROOT, {}).get(parameter, '')
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from cmdatahandlers.api import config
+
+
+class Config(config.Config):
+ def __init__(self, confman):
+ super(Config, self).__init__(confman)
+ self.ROOT = 'cloud.host_os'
+ self.DOMAIN = 'host_os'
+
+ def init(self):
+ pass
+
+ def validate(self):
+ pass
+
+ def mask_sensitive_data(self):
+ self.config[self.ROOT]['grub2_password'] = self.MASK
+
+
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import re
+
+from cmdatahandlers.api import configerror
+from cmdatahandlers.api import config
+from cmdatahandlers.api import utils
+from serviceprofiles import profiles
+
+
+class Config(config.Config):
+ def __init__(self, confman):
+ super(Config, self).__init__(confman)
+ self.ROOT = 'cloud.hosts'
+ self.DOMAIN = 'hosts'
+ try:
+ self.update_service_profiles()
+ except Exception:
+ pass
+
+ def init(self):
+ pass
+
+ def validate(self):
+ hosts = []
+ try:
+ hosts = self.get_hosts()
+ except configerror.ConfigError:
+ pass
+
+ if hosts:
+ utils.validate_list_items_unique(hosts)
+
+ for host in hosts:
+ self._validate_host(host)
+
+ def mask_sensitive_data(self):
+ for hostname in self.config[self.ROOT].keys():
+ self.config[self.ROOT][hostname]['hwmgmt']['password'] = self.MASK
+ self.config[self.ROOT][hostname]['hwmgmt']['snmpv2_trap_community_string'] = self.MASK
+ self.config[self.ROOT][hostname]['hwmgmt']['snmpv3_authpass'] = self.MASK
+ self.config[self.ROOT][hostname]['hwmgmt']['snmpv3_privpass'] = self.MASK
+
+ def _validate_host(self, hostname):
+ self._validate_hwmgmt(hostname)
+ self._validate_service_profiles(hostname)
+ self._validate_network_profiles(hostname)
+ self._validate_performance_profiles(hostname)
+ self._validate_storage_profiles(hostname)
+
+ def _validate_hwmgmt(self, hostname):
+ ip = self.get_hwmgmt_ip(hostname)
+ utils.validate_ipv4_address(ip)
+ self.get_hwmgmt_user(hostname)
+ self.get_hwmgmt_password(hostname)
+ netconf = self.confman.get_networking_config_handler()
+
+ hwmgmtnet = None
+ try:
+ hwmgmtnet = netconf.get_hwmgmt_network_name()
+ except configerror.ConfigError:
+ pass
+
+ if hwmgmtnet:
+ domain = self.get_host_network_domain(hostname)
+ cidr = netconf.get_network_cidr(hwmgmtnet, domain)
+ utils.validate_ip_in_network(ip, cidr)
+
+ def _validate_service_profiles(self, hostname):
+ node_profiles = self.get_service_profiles(hostname)
+ utils.validate_list_items_unique(node_profiles)
+ service_profiles_lib = profiles.Profiles()
+ serviceprofiles = service_profiles_lib.get_service_profiles()
+ for profile in node_profiles:
+ if profile not in serviceprofiles:
+ raise configerror.ConfigError('Invalid service profile %s specified for host %s' % (profile, hostname))
+
+ def _validate_network_profiles(self, hostname):
+ node_profiles = self.get_network_profiles(hostname)
+ utils.validate_list_items_unique(profiles)
+ netprofconf = self.confman.get_network_profiles_config_handler()
+ netprofiles = netprofconf.get_network_profiles()
+ for profile in node_profiles:
+ if profile not in netprofiles:
+ raise configerror.ConfigError('Invalid network profile %s specified for host %s' % (profile, hostname))
+
+ def _validate_performance_profiles(self, hostname):
+ node_performance_profiles = []
+ try:
+ node_performance_profiles = self.get_performance_profiles(hostname)
+ except configerror.ConfigError:
+ pass
+
+ if node_performance_profiles:
+ utils.validate_list_items_unique(node_performance_profiles)
+ perfprofconf = self.confman.get_performance_profiles_config_handler()
+ perfprofiles = perfprofconf.get_performance_profiles()
+ for profile in node_performance_profiles:
+ if profile not in perfprofiles:
+ raise configerror.ConfigError('Invalid performance profile %s specified for host %s' % (profile, hostname))
+
+ def _validate_storage_profiles(self, hostname):
+ node_storage_profiles = []
+ try:
+ node_storage_profiles = self.get_storage_profiles(hostname)
+ except configerror.ConfigError:
+ pass
+
+ if node_storage_profiles:
+ utils.validate_list_items_unique(node_storage_profiles)
+ storageprofconf = self.confman.get_storage_profiles_config_handler()
+ storageprofiles = storageprofconf.get_storage_profiles()
+ for profile in node_storage_profiles:
+ if profile not in storageprofiles:
+ raise configerror.ConfigError('Invalid storage profile %s specific for %s' % (profile, hostname))
+
+ def get_hosts(self):
+ """ get the list of hosts in the cloud
+
+ Return:
+
+ A sorted list of host names
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self.validate_root()
+
+ return sorted(self.config[self.ROOT].keys())
+
+ def get_labels(self, hostname):
+ noderole_label = "node-role.kubernetes.io/{}".format(self.get_noderole(hostname))
+ mandatory_labels = \
+ {"nodetype": self.get_nodetype(hostname),
+ "nodeindex": self.get_nodeindex(hostname),
+ "nodename": self.get_nodename(hostname),
+ noderole_label: ""}
+ labels = self.config[self.ROOT][hostname].get('labels', {}).copy()
+ labels.update(mandatory_labels)
+
+ if self.is_sriov_enabled(hostname):
+ labels.update({"sriov": "enabled"})
+
+ black_list = ['name']
+ return {name: attributes
+ for name, attributes in labels.iteritems()
+ if name not in black_list}
+
+ def get_nodetype(self, hostname):
+ service_profiles_lib = profiles.Profiles()
+ service_profiles = self.get_service_profiles(hostname)
+
+ if service_profiles_lib.get_caasmaster_service_profile() in service_profiles:
+ return service_profiles_lib.get_caasmaster_service_profile()
+ if service_profiles_lib.get_caasworker_service_profile() in service_profiles:
+ return service_profiles_lib.get_caasworker_service_profile()
+
+ return service_profiles[0]
+
+ def get_nodeindex(self, hostname):
+ return re.search(r'[-_](\d+)$', hostname).group(1)
+
+ def get_nodename(self, hostname):
+ return "{}{}".format(self.get_nodetype(hostname), self.get_nodeindex(hostname))
+
+ def get_noderole(self, hostname):
+ service_profiles_lib = profiles.Profiles()
+ service_profiles = self.get_service_profiles(hostname)
+
+ if service_profiles_lib.get_caasmaster_service_profile() in service_profiles:
+ return "master"
+ return "worker"
+
+ def is_sriov_enabled(self, hostname):
+ netprofs = self.get_network_profiles(hostname)
+ netprofconf = self.confman.get_network_profiles_config_handler()
+ for netprof in netprofs:
+ if 'sriov_provider_networks' in self.config[netprofconf.ROOT][netprof]:
+ return True
+ return False
+
+ def get_enabled_hosts(self):
+ """ get the list of enabled hosts in the cloud
+
+ Return:
+
+ A list of host names
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self.validate_root()
+ hosts = self.get_hosts()
+ ret = []
+ for host in hosts:
+ if self.is_host_enabled(host):
+ ret.append(host)
+ return ret
+
+ def get_hwmgmt_ip(self, hostname):
+ """get the hwmgmt ip address
+
+ Arguments:
+
+ hostname: The name of the node
+
+ Return:
+
+ The BMC ip address as a string
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self._validate_hostname(hostname)
+
+ if 'hwmgmt' not in self.config[self.ROOT][hostname] or 'address' not in self.config[self.ROOT][hostname]['hwmgmt']:
+ raise configerror.ConfigError('No hwmgmt info defined for host')
+
+ return self.config[self.ROOT][hostname]['hwmgmt']['address']
+
+ def get_hwmgmt_user(self, hostname):
+ """get the hwmgmt user
+
+ Arguments:
+
+ hostname: The name of the node
+
+ Return:
+
+ The BMC user name.
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self._validate_hostname(hostname)
+
+ if 'hwmgmt' not in self.config[self.ROOT][hostname] or 'user' not in self.config[self.ROOT][hostname]['hwmgmt']:
+ raise configerror.ConfigError('No hwmgmt info defined for host')
+
+ return self.config[self.ROOT][hostname]['hwmgmt']['user']
+
+ def get_hwmgmt_password(self, hostname):
+ """get the hwmgmt password
+
+ Arguments:
+
+ hostname: The name of the node
+
+ Return:
+
+ The BMC password
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self._validate_hostname(hostname)
+
+ if 'hwmgmt' not in self.config[self.ROOT][hostname] or 'password' not in self.config[self.ROOT][hostname]['hwmgmt']:
+ raise configerror.ConfigError('No hwmgmt info defined for host')
+
+ return self.config[self.ROOT][hostname]['hwmgmt']['password']
+
+ def get_service_profiles(self, hostname):
+ """get the node service profiles
+
+ Arguments:
+
+ hostname: The name of the node
+
+ Return:
+
+ A list containing service profile names
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self._validate_hostname(hostname)
+
+ if 'service_profiles' not in self.config[self.ROOT][hostname]:
+ raise configerror.ConfigError('No service profiles found')
+
+ return self.config[self.ROOT][hostname]['service_profiles']
+
+ def get_performance_profiles(self, hostname):
+ """ get the performance profiles
+
+ Arguments:
+
+ hostname: The name of the node
+
+ Return:
+
+ A list containing the perfromance profile names.
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self._validate_hostname(hostname)
+
+ if 'performance_profiles' not in self.config[self.ROOT][hostname]:
+ raise configerror.ConfigError('No performance profiles found')
+
+ return self.config[self.ROOT][hostname]['performance_profiles']
+
+ def get_network_profiles(self, hostname):
+ """get the node network profiles
+
+ Arguments:
+
+ hostname: The name of the node
+
+ Return:
+
+ A list containing network profile names
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self._validate_hostname(hostname)
+
+ if 'network_profiles' not in self.config[self.ROOT][hostname]:
+ raise configerror.ConfigError('No network profiles found')
+
+ return self.config[self.ROOT][hostname]['network_profiles']
+
+ def get_storage_profiles(self, hostname):
+ """get the node storage profiles
+
+ Arguments:
+
+ hostname: The name of the node
+
+ Return:
+
+ A list containing storage profile names
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self._validate_hostname(hostname)
+
+ if 'storage_profiles' not in self.config[self.ROOT][hostname]:
+ raise configerror.ConfigError('No storage profiles found')
+
+ return self.config[self.ROOT][hostname]['storage_profiles']
+
+ def _validate_hostname(self, hostname):
+ if not self.is_valid_host(hostname):
+ raise configerror.ConfigError('Invalid hostname given %s' % hostname)
+
+ def is_valid_host(self, hostname):
+ """check if a host is valid
+
+ Arguments:
+
+ hostname: The name of the node
+
+ Return:
+
+ True or False
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self.validate_root()
+ if hostname in self.config[self.ROOT]:
+ return True
+ return False
+
+ def get_service_profile_hosts(self, profile):
+ """ get hosts having some service profile
+
+ Argument:
+
+ service profile name
+
+ Return:
+
+ A list of host names
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ hosts = self.get_hosts()
+ result = []
+ for host in hosts:
+ node_profiles = self.get_service_profiles(host)
+ if profile in node_profiles:
+ result.append(host)
+
+ return result
+
+ def get_network_profile_hosts(self, profile):
+ """ get hosts having some network profile
+
+ Argument:
+
+ network profile name
+
+ Return:
+
+ A list of host names
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ hosts = self.get_hosts()
+ result = []
+ for host in hosts:
+ node_network_profiles = self.get_network_profiles(host)
+ if profile in node_network_profiles:
+ result.append(host)
+ if not result:
+ raise configerror.ConfigError('No hosts found for profile %s' % profile)
+
+ return result
+
+ def get_performance_profile_hosts(self, profile):
+ """ get hosts having some performance profile
+
+ Argument:
+
+ performance profile name
+
+ Return:
+
+ A list of host names
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ hosts = self.get_hosts()
+ result = []
+ for host in hosts:
+ node_performance_profiles = self.get_performance_profiles(host)
+ if profile in node_performance_profiles:
+ result.append(host)
+ if not result:
+ raise configerror.ConfigError('No hosts found for profile %s' % profile)
+
+ return result
+
+ def get_storage_profile_hosts(self, profile):
+ """ get hosts having some storage profile
+
+ Argument:
+
+ storage profile name
+
+ Return:
+
+ A list of host names
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ hosts = self.get_hosts()
+ result = []
+ for host in hosts:
+ try:
+ node_storage_profiles = self.get_storage_profiles(host)
+ if profile in node_storage_profiles:
+ result.append(host)
+ except configerror.ConfigError:
+ pass
+
+ if not result:
+ raise configerror.ConfigError('No hosts found for profile %s' % profile)
+
+ return result
+
+ def get_host_network_interface(self, host, network):
+ """ get the host interface used for some network
+
+ Argument:
+
+ the host name
+
+ the network name
+
+ Return:
+
+ The interface name
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ node_network_profiles = self.get_network_profiles(host)
+ netprofconf = self.confman.get_network_profiles_config_handler()
+ for profile in node_network_profiles:
+ interfaces = netprofconf.get_profile_network_mapped_interfaces(profile)
+ for interface in interfaces:
+ networks = netprofconf.get_profile_interface_mapped_networks(profile, interface)
+ if network in networks:
+ return interface
+
+ raise configerror.ConfigError('No interfaces found for network %s in host %s' % (network, host))
+
+ def get_host_network_ip_holding_interface(self, host, network):
+ """ get the host ip holding interface some network
+
+ Argument:
+
+ the host name
+
+ the network name
+
+ Return:
+
+ The interface name
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ networkingconf = self.confman.get_networking_config_handler()
+ vlan = None
+ try:
+ domain = self.get_host_network_domain(host)
+ vlan = networkingconf.get_network_vlan_id(network, domain)
+ except configerror.ConfigError as exp:
+ pass
+
+ if vlan:
+ return 'vlan'+str(vlan)
+
+ return self.get_host_network_interface(host, network)
+
+ def get_host_networks(self, hostname):
+ """ get the host networks
+
+ Argument:
+
+ The host name
+
+ Return:
+
+ A list of network names
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ node_network_profiles = self.get_network_profiles(hostname)
+ netprofconf = self.confman.get_network_profiles_config_handler()
+ result = []
+ for profile in node_network_profiles:
+ interfaces = netprofconf.get_profile_network_mapped_interfaces(profile)
+ for interface in interfaces:
+ networks = netprofconf.get_profile_interface_mapped_networks(profile, interface)
+ for network in networks:
+ if network not in result:
+ result.append(network)
+ if not result:
+ raise configerror.ConfigError('No networks found for host %s' % hostname)
+
+ return result
+
+ def get_host_having_hwmgmt_address(self, hwmgmtips):
+ """ get the node name matching an ipmi address
+
+ Argument:
+
+ The ipmi address
+
+ Return:
+
+ The node name
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ import ipaddress
+ hosts = self.get_hosts()
+ for host in hosts:
+ ip = self.get_hwmgmt_ip(host)
+ for hwmgtip in hwmgmtips:
+ addr=ipaddress.ip_address(unicode(hwmgtip))
+ if addr.version == 6:
+ hwmgtip=addr.compressed
+ ip=ipaddress.ip_address(unicode(ip))
+ ip=ip.compressed
+ if ip == hwmgtip:
+ return host
+ raise configerror.ConfigError('No hosts are matching the provided hw address %s' % hwmgmtips)
+
+ def set_installation_host(self, name):
+ """ set the installation node
+
+ Argument:
+
+ The installation node name
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self._validate_hostname(name)
+
+ self.config[self.ROOT][name]['installation_host'] = True
+
+ def is_installation_host(self, name):
+ """ get if the node is an installation node
+
+ Argument:
+
+ The node name
+
+ Return:
+
+ True if installation node
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self._validate_hostname(name)
+
+ if 'installation_host' in self.config[self.ROOT][name]:
+ return self.config[self.ROOT][name]['installation_host']
+
+ return False
+
+ def get_installation_host(self):
+ """ get the name of the node used for installation
+
+ Return:
+
+ The node name
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ hosts = self.get_hosts()
+ for host in hosts:
+ if self.is_installation_host(host):
+ return host
+
+ raise configerror.ConfigError('No installation host found')
+
+ def disable_host(self, host):
+ """ disable the hosts visible via configuration.
+ This can be used in bootstrapping phase.
+
+ Argument:
+
+ host to disable
+
+ Raise:
+
+ ConfigError in-case if provided host is not valid
+ """
+ self._validate_hostname(host)
+
+ self.config[self.ROOT][host]['disabled'] = True
+
+ def enable_host(self, host):
+ """ enable the hosts visible via configuration.
+ This can be used in bootstrapping phase.
+
+ Argument:
+
+ host to enable
+
+ Raise:
+
+ ConfigError in-case if provided host is not valid
+ """
+ self._validate_hostname(host)
+
+ self.config[self.ROOT][host]['disabled'] = False
+
+ def is_host_enabled(self, host):
+ """ is the host enabled
+
+ Argument:
+
+ the host to be checked
+
+ Raise:
+
+ ConfigError in-case if provided host is not valid
+ """
+ self._validate_hostname(host)
+
+ if 'disabled' in self.config[self.ROOT][host]:
+ return not self.config[self.ROOT][host]['disabled']
+
+ return True
+
+ def get_mgmt_mac(self, host):
+ self._validate_hostname(host)
+
+ if 'mgmt_mac' in self.config[self.ROOT][host]:
+ return self.config[self.ROOT][host]['mgmt_mac']
+ return []
+
+ def get_host_network_domain(self, host):
+ self._validate_hostname(host)
+ if 'network_domain' not in self.config[self.ROOT][host]:
+ raise configerror.ConfigError('Missing network domain for host %s' % host)
+ return self.config[self.ROOT][host]['network_domain']
+
+ def get_controllers_network_domain(self):
+ controllers = self.get_service_profile_hosts('controller')
+ domains = set()
+ for controller in controllers:
+ domains.add(self.get_host_network_domain(controller))
+
+ if len(domains) != 1:
+ raise configerror.ConfigError('Controllers in different networking domains not supported')
+ return domains.pop()
+
+ def get_managements_network_domain(self):
+ managements = self.get_service_profile_hosts('management')
+ domains = set()
+ for management in managements:
+ domains.add(self.get_host_network_domain(management))
+ if len(domains) != 1:
+ raise configerror.ConfigError('Management in different networking domains not supported')
+ return domains.pop()
+
+ def update_service_profiles(self):
+ profs = profiles.Profiles()
+ hosts = self.get_hosts()
+ for host in hosts:
+ new_profiles = []
+ current_profiles = self.config[self.ROOT][host]['service_profiles']
+ new_profiles = current_profiles
+ for profile in current_profiles:
+ included_profiles = profs.get_included_profiles(profile)
+ new_profiles = utils.add_lists(new_profiles, included_profiles)
+ self.config[self.ROOT][host]['service_profiles'] = new_profiles
+
+ def get_pre_allocated_ips(self, host, network):
+ ips_field = "pre_allocated_ips"
+ self._validate_hostname(host)
+ if (ips_field not in self.config[self.ROOT][host]
+ or network not in self.config[self.ROOT][host][ips_field]):
+ return None
+ return self.config[self.ROOT][host][ips_field][network]
+
+ def allocate_port(self, host, base, name):
+ used_ports = []
+ hosts = self.get_hosts()
+ for node in hosts:
+ if name in self.config[self.ROOT][node]:
+ used_ports.append(self.config[self.ROOT][node][name])
+
+ free_port = 0
+
+ for port in range(base, base+1000):
+ if port not in used_ports:
+ free_port = port
+ break
+
+ if free_port == 0:
+ raise configerror.ConfigError('No free ports available')
+
+ self.config[self.ROOT][host][name] = free_port
+
+ def add_vbmc_port(self, host):
+ base_vbmc_port = 61600
+ name = 'vbmc_port'
+ self._validate_hostname(host)
+ if name not in self.config[self.ROOT][host]:
+ self.allocate_port(host, base_vbmc_port, name)
+
+ def add_ipmi_terminal_port(self, host):
+ base_console_port = 61401
+ name = 'ipmi_terminal_port'
+ self._validate_hostname(host)
+ if name not in self.config[self.ROOT][host]:
+ self.allocate_port(host, base_console_port, name)
+
+ def get_ceph_osd_disks(self, host):
+ self._validate_hostname(host)
+ caas_disks = self.config[self.ROOT][host].get('caas_disks', [])
+ osd_disks = filter(lambda disk: disk.get('osd_disk', False), caas_disks)
+ return map(lambda disk: _get_path_for_virtio_id(disk), osd_disks)
+
+
+def _get_path_for_virtio_id(disk):
+ disk_id = disk.get('id', '')
+ if disk_id:
+ return "/dev/disk/by-id/virtio-{}".format(disk_id[:20])
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from cmdatahandlers.api import configerror
+from cmdatahandlers.api import config
+from cmdatahandlers.api import utils
+
+class Config(config.Config):
+ def __init__(self, confman):
+ super(Config, self).__init__(confman)
+ self.ROOT='cloud.localstorage'
+ self.DOMAIN='localstorage'
+
+ def init(self):
+ pass
+
+ def validate(self):
+ self.validate_root()
+
+ def add_localstorage(self, localstorage_dict):
+ self.config[self.ROOT] = localstorage_dict
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from cmdatahandlers.api import configerror
+from cmdatahandlers.api import config
+from cmdatahandlers.api import utils
+
+class Config(config.Config):
+ def __init__(self, confman):
+ super(Config, self).__init__(confman)
+ self.ROOT = 'cloud.network_profiles'
+ self.DOMAIN = 'network_profiles'
+
+ def init(self):
+ pass
+
+ def validate(self):
+ self.validate_root()
+ self._validate_network_profiles()
+
+ def _validate_network_profiles(self):
+ profiles = self.get_network_profiles()
+ utils.validate_list_items_unique(profiles)
+ for profile in profiles:
+ self._validate_network_profile(profile)
+
+ def _validate_network_profile(self, profile):
+ bondinginterfaces = None
+ try:
+ bondinginterfaces = self.get_profile_bonding_interfaces(profile)
+ except configerror.ConfigError:
+ pass
+
+ if bondinginterfaces:
+ utils.validate_list_items_unique(bondinginterfaces)
+
+ for bond in bondinginterfaces:
+ bondedinterfaces = self.get_profile_bonded_interfaces(profile, bond)
+ utils.validate_list_items_unique(bondedinterfaces)
+ if len(bondedinterfaces) < 2:
+ raise configerror.ConfigError('Number of bonded interfaces should be at least 2 in %s' % bond)
+
+ mappedinterfaces = self.get_profile_network_mapped_interfaces(profile)
+
+ utils.validate_list_items_unique(mappedinterfaces)
+
+ netconf = self.confman.get_networking_config_handler()
+ validnetworks = netconf.get_networks()
+ for interface in mappedinterfaces:
+ networks = self.get_profile_interface_mapped_networks(profile, interface)
+ utils.validate_list_items_unique(networks)
+ for network in networks:
+ if network not in validnetworks:
+ raise configerror.ConfigError('Network %s is not valid' % network)
+
+ def is_valid_profile(self, profile):
+ """
+ Check if given profile exists
+
+ Arguments:
+ The profile name
+
+ Returns:
+ True if given profile exists
+
+ Raises:
+ ConfigError in-case of an error
+ """
+ self.validate_root()
+ profiles = self.get_network_profiles()
+ if profile not in profiles:
+ raise configerror.ConfigError('Invalid profile name %s' % profile)
+
+ def get_network_profiles(self):
+ """
+ Get the network profiles list
+
+ Returns:
+ A list of network profile(s) names
+
+ Raises:
+ ConfigError in-case of an error
+ """
+ self.validate_root()
+ return self.config[self.ROOT].keys()
+
+ def get_profile_linux_bonding_options(self, profile):
+ """
+ Get the linux bonding options of a profile
+
+ Arguments:
+ The profile name
+
+ Returns:
+ The linux bonding options
+
+ Raises:
+ ConfigError in-case of an error
+ """
+ self.is_valid_profile(profile)
+
+ if 'linux_bonding_options' not in self.config[self.ROOT][profile]:
+ raise configerror.ConfigError('profile %s has no linux bonding options' % profile)
+
+ return self.config[self.ROOT][profile]['linux_bonding_options']
+
+ def get_profile_bonding_interfaces(self, profile):
+ """
+ Get the bonding interfaces in a profile
+
+ Arguments:
+ The profile name
+
+ Returns:
+ A list of bonding interfaces names
+
+ Raises:
+ ConfigError in-case of an error
+ """
+ self.is_valid_profile(profile)
+
+ if 'bonding_interfaces' not in self.config[self.ROOT][profile]:
+ raise configerror.ConfigError('Profile %s has no bonding interfaces' % profile)
+
+ return self.config[self.ROOT][profile]['bonding_interfaces'].keys()
+
+ def get_profile_bonded_interfaces(self, profile, bond):
+ """
+ Get the bonded interfaces in bond interface
+
+ Arguments:
+ The name of the profile
+ The name of the bond interface
+
+ Returns:
+ A list of bonded interfaces names
+
+ Raises:
+ ConfigError in-case of an error
+ """
+ self.validate_root()
+ bondinterfaces = self.get_profile_bonding_interfaces(profile)
+ if bond not in bondinterfaces:
+ raise configerror.ConfigError('Invalid bond interface name %s in profile %s' % (bond, profile))
+
+ return self.config[self.ROOT][profile]['bonding_interfaces'][bond]
+
+ def get_profile_network_mapped_interfaces(self, profile):
+ """
+ Get the network mapped interfaces
+
+ Arguments:
+ The profile name
+
+ Returns:
+ A list of network mapped interfaces
+
+ Raises:
+ ConfigError in-case of an error
+ """
+ self.is_valid_profile(profile)
+
+ if 'interface_net_mapping' not in self.config[self.ROOT][profile]:
+ raise configerror.ConfigError('Profile %s has now interface to network mapping' % profile)
+
+ return self.config[self.ROOT][profile]['interface_net_mapping'].keys()
+
+ def get_profile_interface_mapped_networks(self, profile, interface):
+ """
+ Get the networks mapped to a specific interface
+
+ Arguments:
+ The profile name
+ The interface name
+
+ Returns:
+ A list of network names
+
+ Raises:
+ ConfigError in-case of an error
+ """
+ self.is_valid_profile(profile)
+ mappedinterfaces = self.get_profile_network_mapped_interfaces(profile)
+ if interface not in mappedinterfaces:
+ raise configerror.ConfigError('Interface %s is not valid for profile %s' % (interface, profile))
+
+ return self.config[self.ROOT][profile]['interface_net_mapping'][interface]
+
+ def get_profile_provider_network_interfaces(self, profile):
+ """
+ Get the list of provider network interfaces
+
+ Arguments:
+ The profile name
+
+ Returns:
+ A sorted list of network interface names
+
+ Raises:
+ ConfigError in-case of an error
+ """
+ self.is_valid_profile(profile)
+ if 'provider_network_interfaces' not in self.config[self.ROOT][profile]:
+ raise configerror.ConfigError('Profile %s has no provider network interfaces' % profile)
+
+ return sorted(self.config[self.ROOT][profile]['provider_network_interfaces'].keys())
+
+ def _get_profile_provider_network_interface_dict(self, profile, interface):
+ self.is_valid_profile(profile)
+ interfaces = self.get_profile_provider_network_interfaces(profile)
+ if interface not in interfaces:
+ raise configerror.ConfigError('Profile %s has no provider interface with name %s' % (profile, interface))
+
+ return self.config[self.ROOT][profile]['provider_network_interfaces'][interface]
+
+ def get_profile_provider_network_interface_type(self, profile, interface):
+ """
+ Get the type of a provider network interface
+
+ Arguments:
+ The profile name
+ The interface name
+
+ Returns:
+ The type of the network interface
+
+ Raises:
+ ConfigError in-case of an error
+ """
+ iface_dict = self._get_profile_provider_network_interface_dict(profile, interface)
+ if 'type' not in iface_dict:
+ raise configerror.ConfigError('Provider network interface %s in profile %s does not have a type!' % (interface, profile))
+
+ return iface_dict['type']
+
+ def get_profile_provider_interface_networks(self, profile, interface):
+ """
+ Get provider networks for the interface
+
+ Arguments:
+ The profile name
+ The interface name
+
+ Returns:
+ A list of provider network names
+
+ Raises:
+ ConfigError in-case of an error
+ """
+ iface_dict = self._get_profile_provider_network_interface_dict(profile, interface)
+ if 'provider_networks' not in iface_dict:
+ raise configerror.ConfigError('Profile %s has no provider networks for interface %s' % (profile, interface))
+
+ return iface_dict['provider_networks']
+
+ def get_profile_sriov_provider_networks(self, profile):
+ """
+ Get SR-IOV provider networks
+
+ Arguments:
+ The profile name
+
+ Returns:
+ A list of SR-IOV provider network names
+
+ Raises:
+ ConfigError in-case of an error
+ """
+ self.is_valid_profile(profile)
+ if 'sriov_provider_networks' not in self.config[self.ROOT][profile]:
+ raise configerror.ConfigError('Profile %s has no SR-IOV provider networks' % profile)
+
+ return self.config[self.ROOT][profile]['sriov_provider_networks'].keys()
+
+ def get_profile_sriov_network_interfaces(self, profile, network):
+ """
+ Get SR-IOV provider network interfaces
+
+ Arguments:
+ The profile name
+ The SR-IOV network name
+
+ Returns:
+ A list of SR-IOV provider network interface names
+
+ Raises:
+ ConfigError in-case of an error
+ """
+ if network not in self.get_profile_sriov_provider_networks(profile):
+ raise configerror.ConfigError('Profile %s has no SR-IOV provider network %s' % (profile, network))
+
+ if 'interfaces' not in self.config[self.ROOT][profile]['sriov_provider_networks'][network]:
+ raise configerror.ConfigError('Profile %s has no SR-IOV provider network interfaces for the network %s' % (profile, network))
+
+ return self.config[self.ROOT][profile]['sriov_provider_networks'][network]['interfaces']
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import re
+
+from cmdatahandlers.api import configerror
+from cmdatahandlers.api import config
+from serviceprofiles import profiles
+from netaddr import IPNetwork, IPSet, IPRange
+
+
+VALID_NETWORKS = \
+ ['infra_external', 'infra_storage_cluster', 'infra_hw_management', 'infra_internal', 'cloud_tenant', 'infra_access']
+
+NETWORK_DOMAINS = 'network_domains'
+
+class Config(config.Config):
+ def __init__(self, confman):
+ super(Config, self).__init__(confman)
+ self.ROOT = 'cloud.networking'
+ self.DOMAIN = 'networking'
+ self.freepool = {}
+ self.external_vip = None
+
+ def init(self):
+ if self.ROOT not in self.config:
+ return
+ try:
+ # a mapping between network and free IPSet
+ self.freepool = {}
+ for network in self.config[self.ROOT].keys():
+ if network in VALID_NETWORKS:
+ if NETWORK_DOMAINS not in self.config[self.ROOT][network]:
+ raise configerror.ConfigError('No network domains for network %s' % network)
+
+ self.freepool[network] = {}
+ for domain in self.config[self.ROOT][network][NETWORK_DOMAINS].keys():
+ self.freepool[network][domain] = self._get_free_set(network, domain)
+
+ except configerror.ConfigError:
+ raise
+
+ except Exception as exp:
+ raise configerror.ConfigError(str(exp))
+
+ def _validate_network(self, network, domain=None):
+ networks = self.get_networks()
+ if network not in networks:
+ raise configerror.ConfigError('Invalid network name %s' % network)
+
+ if NETWORK_DOMAINS not in self.config[self.ROOT][network]:
+ raise configerror.ConfigError('No network domains for network %s' % network)
+
+ if domain and domain not in self.config[self.ROOT][network][NETWORK_DOMAINS]:
+ raise configerror.ConfigError('Invalid network domain name %s' % domain)
+
+ def _get_free_set(self, network, domain):
+ ip_range_start = self.get_network_ip_range_start(network, domain)
+ ip_range_end = self.get_network_ip_range_end(network, domain)
+ select_range = IPRange(ip_range_start, ip_range_end)
+ netset = IPSet(select_range)
+ if (network == self.get_infra_external_network_name() and
+ domain == self._get_vip_domain()):
+ iterator = netset.__iter__()
+ self.external_vip = str(iterator.next())
+ netset.remove(self.external_vip)
+
+ # check for the IP(s) taken by the nodes
+ try:
+ hostsconfig = self.confman.get_hosts_config_handler()
+ hosts = hostsconfig.get_hosts()
+ for host in hosts:
+ try:
+ hostip = self.get_host_ip(host, network)
+ netset.remove(hostip)
+ except configerror.ConfigError:
+ pass
+ except configerror.ConfigError:
+ pass
+
+ service_profiles_lib = profiles.Profiles()
+
+ # check for the IP(s) taken as VIPs
+ if network == self.get_infra_internal_network_name() and domain == self._get_vip_domain():
+ vips = self.get_net_vips(network)
+ for _, vip in vips.iteritems():
+ try:
+ netset.remove(vip)
+ except configerror.ConfigError:
+ pass
+
+ return netset
+
+ def get_dns(self):
+ """ get the list of dns servers
+
+ Return:
+
+ A list of dns servers
+
+ Raise:
+
+ ConfigError is raised in-case of an error
+ """
+ self.validate_root()
+ if 'dns' not in self.config[self.ROOT]:
+ raise configerror.ConfigError('dns not found!')
+
+ return self.config[self.ROOT]['dns']
+
+ def get_mtu(self):
+ """ get the mtu value
+
+ Return:
+
+ A number representing the mtu size
+
+ Raise:
+
+ ConfigError is raised in-case of an error
+ """
+ self.validate_root()
+ if 'mtu' not in self.config['cloud.networking']:
+ raise configerror.ConfigError('mtu not found!')
+ return self.config[self.ROOT]['mtu']
+
+ def get_networks(self):
+ """ get the list of network names
+
+ Return:
+
+ A list of network names
+
+ Raise:
+
+ ConfigError is raised in-case of an error
+ """
+ self.validate_root()
+ networks = []
+ for entry in self.config[self.ROOT]:
+ if entry in VALID_NETWORKS:
+ networks.append(entry)
+ return networks
+
+ def allocate_ip(self, network, domain):
+ """ get a new free ip in some network
+
+ Arguments:
+
+ Network name
+
+ Network domain
+
+ Return:
+
+ The free ip address
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self._validate_network(network, domain)
+
+ try:
+ iterator = self.freepool[network][domain].__iter__()
+ ip = str(iterator.next())
+ self.freepool[network][domain].remove(ip)
+ return ip
+ except Exception:
+ raise configerror.ConfigError('Failed to allocate ip for network %s in %s' % (network, domain))
+
+ def allocate_static_ip(self, ip, network, domain=None):
+ """ allocate the given ip in some network
+
+ Arguments:
+
+ Ip address
+
+ Network name
+
+ Network domain
+
+ Return:
+
+ The allocated ip address
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self._validate_network(network, domain)
+
+ try:
+ self.freepool[network][domain].remove(ip)
+ return ip
+ except Exception:
+ raise configerror.ConfigError('Failed to allocate %s for network %s in %s' % (ip, network, domain))
+
+ def allocate_vip(self, network):
+ return self.allocate_ip(network, self._get_vip_domain())
+
+ def get_network_domains(self, network):
+ self._validate_network(network)
+ return self.config[self.ROOT][network][NETWORK_DOMAINS].keys()
+
+ def get_network_cidr(self, network, domain):
+ """ get the network cidr
+
+ Arguments:
+
+ Network name
+
+ Network domain
+
+ Return:
+
+ The cidr address
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self._validate_network(network, domain)
+
+ if 'cidr' not in self.config[self.ROOT][network][NETWORK_DOMAINS][domain]:
+ raise configerror.ConfigError('No CIDR for network %s in %s' % (network, domain))
+
+ return self.config[self.ROOT][network][NETWORK_DOMAINS][domain]['cidr']
+
+ def get_vip_network_cidr(self, network):
+ return self.get_network_cidr(network, self._get_vip_domain())
+
+ def get_network_mask(self, network, domain):
+ """ get the network mask
+
+ Arguments:
+
+ Network name
+
+ Network domain
+
+ Return:
+
+ A number representing the mask
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ cidr = self.get_network_cidr(network, domain)
+ try:
+ mask = cidr.split('/')[1]
+ return int(mask)
+ except Exception as exp:
+ raise configerror.ConfigError('Invalid network mask in %s: %s' % (cidr, str(exp)))
+
+ def get_vip_network_mask(self, network):
+ return self.get_network_mask(network, self._get_vip_domain())
+
+ def get_network_gateway(self, network, domain):
+ """ get the network gateway
+
+ Arguments:
+
+ Network name
+
+ Network domain
+
+ Return:
+
+ The gateway address
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self._validate_network(network, domain)
+
+ if 'gateway' not in self.config[self.ROOT][network][NETWORK_DOMAINS][domain]:
+ raise configerror.ConfigError('No gateway configured for network %s in %s' % (network, domain))
+
+ return self.config[self.ROOT][network][NETWORK_DOMAINS][domain]['gateway']
+
+ def get_network_routes(self, network, domain):
+ self._validate_network(network, domain)
+
+ if 'routes' not in self.config[self.ROOT][network][NETWORK_DOMAINS][domain]:
+ raise configerror.ConfigError('No routes configured for network %s in %s' % (network, domain))
+
+ return self.config[self.ROOT][network][NETWORK_DOMAINS][domain]['routes']
+
+ def get_network_ip_range_start(self, network, domain):
+ """ get the network allocation range start
+
+ Arguments:
+
+ Network name
+
+ Network domain
+
+ Return:
+
+ The starting allocation range
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ net = IPNetwork(self.get_network_cidr(network, domain))
+
+ if 'ip_range_start' in self.config[self.ROOT][network][NETWORK_DOMAINS][domain]:
+ return self.config[self.ROOT][network][NETWORK_DOMAINS][domain]['ip_range_start']
+ else:
+ return str(net[1])
+
+ def get_network_ip_range_end(self, network, domain):
+ """ get the network allocation range end
+
+ Arguments:
+
+ Network name
+
+ Network domain
+
+ Return:
+
+ The end of the allocation range
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ net = IPNetwork(self.get_network_cidr(network, domain))
+
+ if 'ip_range_end' in self.config[self.ROOT][network][NETWORK_DOMAINS][domain]:
+ return self.config[self.ROOT][network][NETWORK_DOMAINS][domain]['ip_range_end']
+ else:
+ return str(net[-2])
+
+ def get_network_vlan_id(self, network, domain):
+ """ get the network vlan id
+
+ Arguments:
+
+ Network name
+
+ Network domain
+
+ Return:
+
+ The vlan id
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self._validate_network(network, domain)
+
+ if 'vlan' not in self.config[self.ROOT][network][NETWORK_DOMAINS][domain]:
+ raise configerror.ConfigError('No vlan specified for %s in %s' % (network, domain))
+
+ return self.config[self.ROOT][network][NETWORK_DOMAINS][domain]['vlan']
+
+ def get_vip_network_vlan_id(self, network):
+ return self.get_network_vlan_id(network, self._get_vip_domain())
+
+ def get_network_mtu(self, network):
+ """ get the network mtu
+
+ Argument:
+
+ Network name
+
+ Return:
+
+ The mtu of the network
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self._validate_network(network)
+
+ if 'mtu' not in self.config[self.ROOT][network]:
+ raise configerror.ConfigError('No mtu specified for %s' % network)
+
+ return self.config[self.ROOT][network]['mtu']
+
+ def get_host_ip(self, host, network):
+ """ get the host ip allocated from a specific network
+
+ Argument:
+
+ hostname: The name of the host
+ networkname: The name of the network
+
+ Return:
+
+ The ip address assigned for the host
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self._validate_network(network)
+
+ hostnetconfigkey = host + '.' + self.DOMAIN
+ if hostnetconfigkey not in self.config:
+ raise configerror.ConfigError('No network configuration available for %s' % host)
+
+ if network not in self.config[hostnetconfigkey]:
+ raise configerror.ConfigError('No network configuration available for %s' % host)
+
+ if 'ip' not in self.config[hostnetconfigkey][network]:
+ raise configerror.ConfigError('No IP assigned for %s in network %s' % (host, network))
+
+ return self.config[hostnetconfigkey][network]['ip']
+
+ def _get_vip_domain(self):
+ return self.confman.get_hosts_config_handler().get_managements_network_domain()
+
+ @staticmethod
+ def get_infra_external_network_name():
+ """ get the network name for the external network
+
+ Return:
+
+ The external network name
+
+ Raise:
+
+ ConfigError in-case the network is not configured
+ """
+ return 'infra_external'
+
+ @staticmethod
+ def get_infra_storage_cluster_network_name():
+ """ get the infra storage cluster network name
+
+ Return:
+
+ The infra stroage cluster network name
+
+ Raise:
+
+ ConfigError in-case the network is not configured
+ """
+ return 'infra_storage_cluster'
+
+ @staticmethod
+ def get_hwmgmt_network_name():
+ """ get the hwmgmt network name
+
+ Return:
+
+ The hwmgmt network name
+
+ Raise:
+
+ ConfigError in-case the network is not defined
+ """
+ return 'infra_hw_management'
+
+ @staticmethod
+ def get_infra_internal_network_name():
+ """ get the infra management network name
+
+ Return:
+
+ The infra management network name
+
+ Raise:
+
+ ConfigError in-case the network is not defined
+ """
+ return 'infra_internal'
+
+ def get_cloud_tenant_network_name(self):
+ """ get the network name for the cloud tenant network
+
+ Return:
+
+ The cloud tenant network name
+
+ Raise:
+
+ ConfigError in-case the network is not configured
+ """
+ return 'cloud_tenant'
+
+ def get_infra_access_network_name(self):
+ """ get the network name for the infra access network
+
+ Return:
+
+ The infra access network name
+
+ Raise:
+
+ ConfigError in-case the network is not configured
+ """
+ return 'infra_access'
+
+ def add_host_networks(self, host):
+ """ add host network data
+
+ Argument:
+
+ Host name
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ hostsconf = self.confman.get_hosts_config_handler()
+ networks = hostsconf.get_host_networks(host)
+ domain = hostsconf.get_host_network_domain(host)
+ for network in networks:
+ try:
+ ip = self.get_host_ip(host, network)
+ continue
+ except configerror.ConfigError:
+ pass
+
+ static_ip = hostsconf.get_pre_allocated_ips(host, network)
+ if static_ip:
+ ip = self.allocate_static_ip(static_ip, network, domain)
+ else:
+ ip = self.allocate_ip(network, domain)
+ interface = hostsconf.get_host_network_ip_holding_interface(host, network)
+ netmask = self.get_network_mask(network, domain)
+ networkdata = {'ip': ip, 'interface': interface, 'mask': netmask}
+
+ try:
+ vlan = self.get_network_vlan_id(network, domain)
+ networkdata['vlan'] = vlan
+ except configerror.ConfigError:
+ pass
+
+ try:
+ gw = self.get_network_gateway(network, domain)
+ networkdata['gateway'] = gw
+ except configerror.ConfigError:
+ pass
+
+ try:
+ routes = self.get_network_routes(network, domain)
+ networkdata['routes'] = routes
+ except configerror.ConfigError:
+ pass
+
+ key = host + '.' + self.DOMAIN
+ if key not in self.config:
+ self.config[key] = {}
+ if network not in self.config[key]:
+ self.config[key][network] = {}
+
+ self.config[key][network] = networkdata
+
+ def delete_host_networks(self, host):
+ """ delete host network data
+
+ Argument:
+
+ Host name
+ """
+ key = '{}.{}'.format(host, self.DOMAIN)
+ if key in self.config:
+ del self.config[key]
+
+ def get_networking_hosts(self):
+ """ get hosts with networking data
+
+ Return:
+
+ List of host names with existing networking data
+ """
+ hosts = []
+ match = r'^[^.]*\.networking$'
+ for key in self.config.keys():
+ if key != self.ROOT and re.match(match, key):
+ hosts.append(key.split('.')[0])
+ return hosts
+
+ def get_host_interface(self, host, network):
+ """ get the host interface allocated from a specific network
+
+ Argument:
+
+ hostname: The name of the host
+ networkname: The name of the network
+
+ Return:
+
+ The interface for the host
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self._validate_network(network)
+
+ hostnetconfigkey = host + '.' + self.DOMAIN
+ if hostnetconfigkey not in self.config:
+ raise configerror.ConfigError('No network configuration available for %s' % host)
+
+ if network not in self.config[hostnetconfigkey]:
+ raise configerror.ConfigError('No network configuration available for %s' % host)
+
+ if 'interface' not in self.config[hostnetconfigkey][network]:
+ raise configerror.ConfigError(
+ 'No interface assigned for %s in network %s' % (host, network))
+
+ return self.config[hostnetconfigkey][network]['interface']
+
+ def get_host_mask(self, host, network):
+ """ get the network mask for the host
+
+ Argument:
+
+ hostname: The name of the host
+ networkname: The name of the network
+
+ Return:
+
+ The network mask
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self._validate_network(network)
+
+ hostnetconfigkey = host + '.' + self.DOMAIN
+ if hostnetconfigkey not in self.config:
+ raise configerror.ConfigError('No network configuration available for %s' % host)
+
+ if network not in self.config[hostnetconfigkey]:
+ raise configerror.ConfigError('No network configuration available for %s' % host)
+
+ if 'mask' not in self.config[hostnetconfigkey][network]:
+ raise configerror.ConfigError('No mask assigned for %s in network %s' % (host, network))
+
+ return self.config[hostnetconfigkey][network]['mask']
+
+
+ def get_external_vip(self):
+ """ get the external vip ip, this is always the first ip in the range
+ """
+ return self.external_vip
+
+
+ def get_provider_networks(self):
+ """
+ Get provider network names
+
+ Returns:
+ A list of provider network names
+
+ Raises:
+ ConfigError in-case of an error
+ """
+ if 'provider_networks' not in self.config[self.ROOT]:
+ raise configerror.ConfigError('No provider networks configured')
+
+ return self.config[self.ROOT]['provider_networks'].keys()
+
+ def is_shared_provider_network(self, network):
+ """
+ Is shared provider network
+
+ Arguments:
+ Provider network name
+
+ Returns:
+ True if given provider network is shared, False otherwise
+
+ Raises:
+ ConfigError in-case of an error
+ """
+ networks = self.get_provider_networks()
+ if network not in networks:
+ raise configerror.ConfigError('Missing configuration for provider network %s' % network)
+
+ return (self.config[self.ROOT]['provider_networks'][network].get('shared') is True)
+
+ def get_provider_network_vlan_ranges(self, network):
+ """
+ Get vlan ranges for the given provider network
+
+ Arguments:
+ Provider network name
+
+ Returns:
+ Vlan ranges for the provider network
+
+ Raises:
+ ConfigError in-case of an error
+ """
+ networks = self.get_provider_networks()
+ if network not in networks:
+ raise configerror.ConfigError('Missing configuration for provider network %s' % network)
+
+ if 'vlan_ranges' not in self.config[self.ROOT]['provider_networks'][network]:
+ raise configerror.ConfigError(
+ 'Missing vlan ranges configuration for provider network %s' % network)
+
+ return self.config[self.ROOT]['provider_networks'][network]['vlan_ranges']
+
+
+ def get_provider_network_mtu(self, network):
+ """
+ Get mtu for the given provider network
+
+ Arguments:
+ Provider network name
+
+ Returns:
+ mtu for the provider network
+
+ Raises:
+ ConfigError in-case of an error
+ """
+ networks = self.get_provider_networks()
+ if network not in networks:
+ raise configerror.ConfigError('Missing configuration for provider network %s' % network)
+
+ if 'mtu' not in self.config[self.ROOT]['provider_networks'][network]:
+ raise configerror.ConfigError(
+ 'Missing mtu configuration for provider network %s' % network)
+
+ return self.config[self.ROOT]['provider_networks'][network]['mtu']
+
+ def is_l3_ha_enabled(self):
+ """ is L3 HA enabled
+
+ Return:
+
+ True if L3 HA is enabled, False otherwise
+ """
+ return True if 'l3_ha' in self.config[self.ROOT] else False
+
+ def _get_l3_ha_config(self):
+ if 'l3_ha' not in self.config[self.ROOT]:
+ raise configerror.ConfigError('Missing L3 HA configuration')
+
+ return self.config[self.ROOT]['l3_ha']
+
+ def get_l3_ha_provider_network(self):
+ """ get L3 HA provider network
+
+ Return:
+
+ L3 HA provider network name
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ conf = self._get_l3_ha_config()
+ if 'provider_network' not in conf:
+ raise configerror.ConfigError('Missing L3 HA provider network configuration')
+
+ return conf['provider_network']
+
+ def get_l3_ha_cidr(self):
+ """ get L3 HA CIDR
+
+ Return:
+
+ L3 HA CIDR
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ conf = self._get_l3_ha_config()
+ if 'cidr' not in conf:
+ raise configerror.ConfigError('Missing L3 HA CIDR configuration')
+
+ return conf['cidr']
+
+ def add_ovs_config_defaults(self, host):
+ """ Add Openvswitch default config """
+
+ ovs_defaults = { 'tx-flush-interval': 0, 'rxq-rebalance': 0 }
+
+ key = self.ROOT
+ if key not in self.config:
+ self.config[key] = {}
+ if 'ovs_config' not in self.config[key]:
+ self.config[key]['ovs_config'] = {}
+ if host not in self.config[key]['ovs_config']:
+ self.config[key]['ovs_config'][host] = {}
+
+ self.config[key]['ovs_config'][host] = ovs_defaults
+
+ def del_ovs_config(self, host):
+ """ Delete Openvswitch config """
+ if host in self.config[self.ROOT]['ovs_config']:
+ self.config[self.ROOT]['ovs_config'].pop(host, None)
+
+ def get_ovs_config(self, host):
+ return self.config[self.ROOT]['ovs_config'].get(host, None)
+
+ def _validate_ovs_config_args(self, host, args):
+ ovs_conf = self.config[self.ROOT]['ovs_config']
+
+ if args.get('tx_flush_interval') is not None:
+ if int(args['tx_flush_interval']) >= 0 and int(args['tx_flush_interval']) <= 1000000:
+ ovs_conf[host]['tx-flush-interval'] = int(args['tx_flush_interval'])
+ else:
+ raise ValueError("tx-flush-interval value must be 0..1000000")
+
+ if args.get('rxq_rebalance_interval') is not None:
+ if int(args['rxq_rebalance_interval']) >= 0 and int(args['rxq_rebalance_interval']) <= 1000000:
+ ovs_conf[host]['rxq-rebalance'] = int(args['rxq_rebalance_interval'])
+ else:
+ raise ValueError("rxq_rebalance_interval value must be 0..1000000")
+
+ def update_ovs_config(self, host, args):
+ if self.config[self.ROOT]['ovs_config'].get(host, None) is None:
+ return None
+ self._validate_ovs_config_args(host, args)
+ return self.config[self.ROOT]
+
+ def get_ovs_config_hosts(self):
+ return [host for host in self.config[self.ROOT]['ovs_config']]
+
+ def add_vip(self, network, name, ip):
+ if 'vips' not in self.config[self.ROOT]:
+ self.config[self.ROOT]['vips'] = {}
+
+ if network not in self.config[self.ROOT]['vips']:
+ self.config[self.ROOT]['vips'][network] = {}
+
+ self.config[self.ROOT]['vips'][network][name] = ip
+
+ def add_external_vip(self):
+ external_vip = self.get_external_vip()
+ self.add_vip('infra_external', 'external_vip', external_vip)
+
+ def add_internal_vip(self):
+ internal_vip = self.allocate_vip('infra_internal')
+ self.add_vip('infra_internal', 'internal_vip', internal_vip)
+
+ def get_internal_vip(self):
+ try:
+ return self.config[self.ROOT]['vips']['infra_internal']['internal_vip']
+ except KeyError as exp:
+ raise configerror.ConfigError('Internal vip not found')
+
+ def get_vips(self):
+ if 'vips' not in self.config[self.ROOT]:
+ return {}
+
+ return self.config[self.ROOT]['vips']
+
+ def get_net_vips(self, net):
+ if 'vips' not in self.config[self.ROOT]:
+ return {}
+
+ if net not in self.config[self.ROOT]['vips']:
+ return {}
+
+ return self.config[self.ROOT]['vips'][net]
+
+ return self.config[self.ROOT]['vips']
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from cmdatahandlers.api import configerror
+from cmdatahandlers.api import config
+
+class Config(config.Config):
+ def __init__(self, confman):
+ super(Config, self).__init__(confman)
+ self.ROOT = 'cloud.openstack'
+ self.DOMAIN = 'openstack'
+
+ def init(self):
+ pass
+
+ def _getopenstack(self):
+ hostsconf = self.confman.get_hosts_config_handler()
+ return hostsconf.get_service_profile_hosts('controller')
+
+ def validate(self):
+ if self._getopenstack():
+ self.validate_root()
+ self._validate_storage_backend()
+ self.get_admin_password()
+
+ def _validate_storage_backend(self):
+ backend = self.get_storage_backend()
+ storageconf = self.confman.get_storage_config_handler()
+ backends = storageconf.get_storage_backends()
+ if backend not in backends:
+ raise configerror.ConfigError('Invalid storage backend %s' % backend)
+
+ def get_admin_password(self):
+ """ get the admin password
+
+ Return:
+
+ The admin password
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ if not self._getopenstack():
+ return ''
+
+ self.validate_root()
+
+ if 'admin_password' not in self.config[self.ROOT]:
+ raise configerror.ConfigError('No admin_password specified')
+
+ return self.config[self.ROOT]['admin_password']
+
+ def get_storage_backend(self):
+ """ get the openstack storage backend
+
+ Return:
+
+ The openstack storage backend name
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ if not self._getopenstack():
+ storageconf = self.confman.get_storage_config_handler()
+ enabled_backends = storageconf.get_enabled_backends()
+ if enabled_backends:
+ return enabled_backends[0]
+ return ''
+
+ self.validate_root()
+
+ if 'storage_backend' not in self.config[self.ROOT]:
+ raise configerror.ConfigError('No storage backend configured')
+
+ return self.config[self.ROOT]['storage_backend']
+
+ def get_instance_default_backend(self):
+ """ get the openstack instance backend
+
+ Return:
+
+ The openstack instance backend name
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ if not self._getopenstack():
+ return "default"
+
+ self.validate_root()
+
+ if 'instance_default_backend' not in self.config[self.ROOT]:
+ return "default"
+
+ return self.config[self.ROOT]['instance_default_backend']
+
+ def mask_sensitive_data(self):
+ self.config[self.ROOT]['admin_password'] = self.MASK
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from cmdatahandlers.api import configerror
+from cmdatahandlers.api import config
+from cmdatahandlers.api import utils
+
+class Config(config.Config):
+
+ def __init__(self, confman):
+ super(Config, self).__init__(confman)
+ self.ROOT = 'cloud.osoverrides'
+ self.DOMAIN = 'osoverrides'
+
+ def init(self):
+ pass
+
+ def validate(self):
+ if self.ROOT not in self.config:
+ return
+ overrides = self.config[self.ROOT].get('osoverrides', {})
+ if not isinstance(overrides, dict):
+ raise configerror.ConfigError("OS overrides (osoverrides) must be defined as dictionary")
+
+
+ def get_config_overrides(self):
+ return self.config.get(self.ROOT, {}).get('osoverrides', {})
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from cmdatahandlers.api import configerror
+from cmdatahandlers.api import config
+from cmdatahandlers.api import utils
+
+
+class Config(config.Config):
+
+ DEFAULT_HUGEPAGESZ = 'default_hugepagesz'
+ HUGEPAGESZ = 'hugepagesz'
+ HUGEPAGES = 'hugepages'
+ PLATFORM_CPUS = 'platform_cpus'
+ OVS_DPDK_CPUS = 'ovs_dpdk_cpus'
+ PROFILE_OPTIONS = {DEFAULT_HUGEPAGESZ: 'get_profile_default_hugepage_size',
+ HUGEPAGESZ: 'get_profile_hugepage_size',
+ HUGEPAGES: 'get_profile_hugepage_count',
+ PLATFORM_CPUS: 'get_platform_cpus',
+ OVS_DPDK_CPUS: 'get_ovs_dpdk_cpus'}
+
+ ERR_INVALID_PROFILE = 'Invalid profile name {}'
+ ERR_MISSING_PROFILE_KEY = 'Profile {} does not have %s'
+ ERR_MISSING_DEFAULT_HUGEPAGESZ = ERR_MISSING_PROFILE_KEY % DEFAULT_HUGEPAGESZ
+ ERR_MISSING_HUGEPAGESZ = ERR_MISSING_PROFILE_KEY % HUGEPAGESZ
+ ERR_MISSING_HUGEPAGES = ERR_MISSING_PROFILE_KEY % HUGEPAGES
+ ERR_MISSING_PLATFORM_CPUS = ERR_MISSING_PROFILE_KEY % PLATFORM_CPUS
+ ERR_MISSING_OVS_DPDK_CPUS = ERR_MISSING_PROFILE_KEY % OVS_DPDK_CPUS
+
+ @staticmethod
+ def raise_error(context, err_type):
+ raise configerror.ConfigError(err_type.format(context))
+
+ def __init__(self, confman):
+ super(Config, self).__init__(confman)
+ self.ROOT = 'cloud.performance_profiles'
+ self.DOMAIN = 'performance_profiles'
+
+ def init(self):
+ pass
+
+ def validate(self):
+ self.validate_root()
+ self._validate_performance_profiles()
+
+ def _validate_performance_profiles(self):
+ profiles = self.get_performance_profiles()
+ utils.validate_list_items_unique(profiles)
+ for profile in profiles:
+ self._validate_performance_profile(profile)
+
+ def _validate_performance_profile(self, profile):
+ self.get_profile_default_hugepage_size(profile)
+ self.get_profile_hugepage_size(profile)
+ self.get_profile_hugepage_count(profile)
+ self.get_platform_cpus(profile)
+ self.get_ovs_dpdk_cpus(profile)
+
+ def is_valid_profile(self, profile):
+ profiles = self.get_performance_profiles()
+ if profile not in profiles:
+ self.raise_error(profile, self.ERR_INVALID_PROFILE)
+
+ def get_performance_profiles(self):
+ """ get the performance profiles list
+
+ Return:
+
+ A list of performance profile(s) names
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self.validate_root()
+ return self.config[self.ROOT].keys()
+
+ # pylint: disable=invalid-name
+ def get_profile_default_hugepage_size(self, profile):
+ """ get the default hugepage size
+
+ Argument:
+
+ profile name
+
+ Return:
+
+ The default hugepage size
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self.is_valid_profile(profile)
+
+ if self.DEFAULT_HUGEPAGESZ not in self.config[self.ROOT][profile]:
+ self.raise_error(profile, self.ERR_MISSING_DEFAULT_HUGEPAGESZ)
+
+ return self.config[self.ROOT][profile][self.DEFAULT_HUGEPAGESZ]
+
+ def get_profile_hugepage_size(self, profile):
+ """ get the hugepage size
+
+ Argument:
+
+ profile name
+
+ Return:
+
+ The hugepage size
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self.is_valid_profile(profile)
+
+ if self.HUGEPAGESZ not in self.config[self.ROOT][profile]:
+ self.raise_error(profile, self.ERR_MISSING_HUGEPAGESZ)
+
+ return self.config[self.ROOT][profile][self.HUGEPAGESZ]
+
+ def get_profile_hugepage_count(self, profile):
+ """ get the hugepage count
+
+ Argument:
+
+ profile name
+
+ Return:
+
+ The hugepage count
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self.is_valid_profile(profile)
+
+ if self.HUGEPAGES not in self.config[self.ROOT][profile]:
+ self.raise_error(profile, self.ERR_MISSING_HUGEPAGES)
+
+ return self.config[self.ROOT][profile][self.HUGEPAGES]
+
+ def get_platform_cpus(self, profile):
+ """ get the Platforma CPUs (isolate CPUs from the general scheduler).
+
+ Argument:
+
+ profile name
+
+ Return:
+
+ The platform CPUs dictionary
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self.is_valid_profile(profile)
+
+ if self.PLATFORM_CPUS not in self.config[self.ROOT][profile]:
+ self.raise_error(profile, self.ERR_MISSING_PLATFORM_CPUS)
+
+ return self.config[self.ROOT][profile][self.PLATFORM_CPUS]
+
+ def get_ovs_dpdk_cpus(self, profile):
+ """ get the ovs-dpdk cpu(s)
+
+ Argument:
+
+ profile name
+
+ Return:
+
+ The ovs-dpdk dedicated cpu(s) string
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self.is_valid_profile(profile)
+
+ if self.OVS_DPDK_CPUS not in self.config[self.ROOT][profile]:
+ self.raise_error(profile, self.ERR_MISSING_OVS_DPDK_CPUS)
+
+ return self.config[self.ROOT][profile][self.OVS_DPDK_CPUS]
+
+ def dump(self):
+ """ Dump all performaceprofiles data. """
+
+ self.validate_root()
+ return self.config[self.ROOT]
+
+ def _fill_option_value(self, profile_data, profile, option, value):
+ if value is None:
+ try:
+ value = getattr(self, self.PROFILE_OPTIONS[option])(profile)
+ except configerror.ConfigError:
+ return
+ profile_data.update({option:value})
+
+ # pylint: disable=too-many-arguments
+ def update(self, name, platform_cpus=None, ovs_dpdk_cpus=None, hugepages=None,
+ default_hugepagesz=None, hugepagesz=None):
+ """ Update performance profile, overwriting existing profile.
+
+ Parameters
+ ----------
+ name : str
+ profile name.
+ platform_cpus : dict, optional
+ Platform CPUs.
+ The syntax is: {'numa0': <int>, 'numa1': <int>, ..., 'numaN': <int>}
+ ovs_dpdk_cpus : dict, optional
+ OVS-DPDK dedicated cores.
+ The syntax is the same as platform_cpus.
+ hugepages : int, optional
+ The number of allocated persistent huge pages.
+ default_hugepagesz : str, optional
+ Default huge page size (the default value is '1G').
+ Valid values are '2M' and '1G'
+ hugepagesz : str, optional
+ Huge page size (the default value is '1G').
+ Valid values are '2M' and '1G'
+
+ """
+ data = {}
+ self._fill_option_value(data, name, 'platform_cpus', platform_cpus)
+ self._fill_option_value(data, name, 'ovs_dpdk_cpus', ovs_dpdk_cpus)
+ self._fill_option_value(data, name, 'hugepages', hugepages)
+ self._fill_option_value(data, name, 'default_hugepagesz', default_hugepagesz)
+ self._fill_option_value(data, name, 'hugepagesz', hugepagesz)
+ self.config[self.ROOT].update({name:data})
+
+ def delete(self, name):
+ """ Remove profile.
+
+ Parametes
+ ---------
+ name : str
+ profile name.
+
+ Raises
+ ------
+ Raises ConfigError if profile does not exist.
+ """
+ try:
+ self.config[self.ROOT].pop(name)
+ except KeyError:
+ self.raise_error(name, self.ERR_INVALID_PROFILE)
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from cmdatahandlers.api import configerror
+from cmdatahandlers.api import config
+
+
+class Config(config.Config):
+ def __init__(self, confman):
+ super(Config, self).__init__(confman)
+ self.ROOT = 'cloud.storage'
+ self.DOMAIN = 'storage'
+
+ def init(self):
+ pass
+
+ def validate(self):
+ self.validate_root()
+ self._validate_storage_backends()
+
+ def _validate_storage_backends(self):
+ backends = self.get_storage_backends()
+ for backend in backends:
+ if backend == 'ceph':
+ self._validate_ceph_backend()
+ elif backend == 'external_ceph':
+ self._validate_external_ceph_backend()
+ elif backend == 'lvm':
+ self._validate_lvm_backend()
+ else:
+ raise configerror.ConfigError('Invalid backend %s specified' % backend)
+
+ def _validate_ceph_backend(self):
+ self.is_ceph_enabled()
+ osdpoolsize = self.get_ceph_osd_pool_size()
+ if osdpoolsize <= 1:
+ raise configerror.ConfigError('Invalid osd pool size configured %d' % osdpoolsize)
+
+ def _validate_external_ceph_backend(self):
+ self.is_external_ceph_enabled()
+ self.get_ext_ceph_fsid()
+ self.get_ext_ceph_mon_hosts()
+ self.get_ext_ceph_ceph_s3_endpoint()
+ self.get_ext_ceph_ceph_s3_keystone_user()
+ self.get_ext_ceph_ceph_s3_keystone_adminpw()
+ self.get_ext_ceph_ceph_user()
+ self.get_ext_ceph_ceph_user_key()
+ self.get_ext_ceph_cinder_pool()
+ self.get_ext_ceph_glance_pool()
+ self.get_ext_ceph_nova_pool()
+ self.get_ext_ceph_platform_pool()
+ self.get_ext_ceph_cidr()
+
+ def _validate_lvm_backend(self):
+ self.is_lvm_enabled()
+
+ def _set_hostlist(self, key, value):
+ self.validate_root()
+ if len(value) <= 0:
+ raise configerror.ConfigError('The host list for {} is empty'.format(key))
+ self.config[self.ROOT][key] = value
+
+ def get_storage_backends(self):
+ """ get the list of backends
+
+ Return:
+
+ A list of backend names
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self.validate_root()
+
+ if 'backends' not in self.config[self.ROOT]:
+ raise configerror.ConfigError('No backends configured')
+
+ return self.config[self.ROOT]['backends'].keys()
+
+ def get_ceph_osd_pool_size(self):
+ """ get the ceph osd pool size
+
+ Return:
+
+ The ceph osd pool size
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self.validate_root()
+ backends = self.get_storage_backends()
+
+ if 'ceph' not in backends:
+ raise configerror.ConfigError('No ceph backend configured')
+
+ if 'osd_pool_default_size' not in self.config[self.ROOT]['backends']['ceph']:
+ raise configerror.ConfigError('No ceph osd configuration found')
+
+ return self.config[self.ROOT]['backends']['ceph']['osd_pool_default_size']
+
+ def is_lvm_enabled(self):
+ """ Is lvm enabled or not.
+
+ Return:
+ True if lvm is enabled otherwise False.
+ """
+ return self.is_backend_enabled('lvm')
+
+ def is_ceph_enabled(self):
+ """ Is ceph enabled or not.
+
+ Return:
+ True if ceph is enabled otherwise False.
+ """
+ return self.is_backend_enabled('ceph')
+
+ def is_external_ceph_enabled(self):
+ """ Is external ceph enabled or not.
+
+ Return:
+ True if external ceph is enabled otherwise False.
+ """
+ return self.is_backend_enabled('external_ceph')
+
+ def is_backend_enabled(self, backend):
+ """ Is the given backend enabled.
+
+ Argument:
+ The storage backend.
+
+ Return:
+ True if the backend is enabled otherwise False.
+
+ Raise:
+ -
+ """
+ self.validate_root()
+ backends = self.get_storage_backends()
+
+ if backend not in backends:
+ return False
+
+ if 'enabled' not in self.config[self.ROOT]['backends'][backend]:
+ raise configerror.ConfigError(
+ 'The enabled parameter not configured for {}'.format(backend))
+
+ return self.config[self.ROOT]['backends'][backend]['enabled']
+
+ def get_enabled_backends(self):
+ """ Gets enabled storage backends.
+
+ Argument:
+ The storage backend.
+
+ Return:
+ List of enabled backends.
+ """
+ return [backend for backend, values in self.config[self.ROOT]['backends'].iteritems()
+ if values.get('enabled', False)]
+
+ def set_mons(self, hosts):
+ """ Set the ceph monitors
+
+ Argument:
+
+ A list of ceph monitor hosts.
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self._set_hostlist('mons', hosts)
+
+ def set_ceph_mons(self, hosts):
+ """ Set the ceph monitors for openstack-ansible.
+
+ Argument:
+
+ A list of ceph monitor hosts.
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self._set_hostlist('ceph_mons', hosts)
+
+ def set_osds(self, hosts):
+ """ Set the ceph osds
+
+ Argument:
+
+ A list of ceph osd hosts.
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self._set_hostlist('osds', hosts)
+
+ def _get_external_ceph_attribute(self, attribute):
+ return self._get_backends_attribute('external_ceph', attribute)
+
+ def _is_valid_backend(self, backend):
+ self.validate_root()
+ if backend not in self.config[self.ROOT]['backends']:
+ raise configerror.ConfigError(
+ 'The cloud.storage.backends does not have the '
+ 'backend {} configured'.format(backend))
+
+ def _is_valid_backends_attribute(self, backend, attribute):
+ self.validate_root()
+ if attribute not in self.config[self.ROOT]['backends'][backend]:
+ raise configerror.ConfigError(
+ 'The cloud.storage.backends.{} does not have '
+ 'the attribute {} configured'.format(backend, attribute))
+
+ def _get_backends_attribute(self, backend, attribute):
+ self.validate_root()
+ self._is_valid_backend(backend)
+ self._is_valid_backends_attribute(backend, attribute)
+ return self.config[self.ROOT]['backends'][backend][attribute]
+
+ def get_ext_ceph_fsid(self):
+ """ Get the file system id of the external ceph cluster.
+
+ Return:
+ The fsid of the external ceph cluster.
+
+ Raise:
+ ConfigError in-case of an error
+ """
+ return self._get_external_ceph_attribute('fsid')
+
+ def get_ext_ceph_mon_hosts(self):
+ """ Get the monitor hosts of the external ceph system.
+
+ Return:
+ A list of monitor hosts.
+
+ Raise:
+ ConfigError in-case of an error
+ """
+ return self._get_external_ceph_attribute('mon_hosts')
+
+ def get_ext_ceph_ceph_s3_endpoint(self):
+ """ Get the ceph s3 endpoint. The endpoint consists of a
+ service name and port.
+
+ Return:
+ A ceph s3 endpoint in format: <service>:<port>
+
+ Raise:
+ ConfigError in-case of an error
+ """
+ return self._get_external_ceph_attribute('ceph_s3_endpoint')
+
+ def get_ext_ceph_ceph_s3_keystone_user(self):
+ """ Get the ceph s3 endpoint user.
+
+ Return:
+ The external ceph s3 keystone user.
+
+ Raise:
+ ConfigError in-case of an error.
+ """
+ return self._get_external_ceph_attribute('ceph_s3_keystone_user')
+
+ def get_ext_ceph_ceph_s3_keystone_adminpw(self):
+ """ Get the ceph s3 endpoint adminpw.
+
+ Return:
+ The external ceph s3 keystone adminpw.
+
+ Raise:
+ ConfigError in-case of an error.
+ """
+ return self._get_external_ceph_attribute('ceph_s3_keystone_adminpw')
+
+ def get_ext_ceph_ceph_user(self):
+ """ Get the ceph user of the external ceph server.
+
+ Return:
+ The external ceph user.
+
+ Raise:
+ ConfigError in-case of an error
+ """
+ return self._get_external_ceph_attribute('ceph_user')
+
+ def get_ext_ceph_ceph_user_key(self):
+ """ Get the external ceph server ceph user key.
+
+ Return:
+ The external ceph user key.
+
+ Raise:
+ ConfigError in-case of an error
+ """
+ return self._get_external_ceph_attribute('ceph_user_key')
+
+ def get_ext_ceph_cinder_pool(self):
+ """ Get the external ceph server cinder pool name.
+
+ Return:
+ External ceph server cinder pool name.
+
+ Raise:
+ ConfigError in-case of an error
+ """
+ return self._get_external_ceph_attribute('cinder_pool')
+
+ def get_ext_ceph_glance_pool(self):
+ """ Get the external ceph server glance pool name.
+
+ Return:
+ External ceph server glance pool name.
+
+ Raise:
+ ConfigError in-case of an error
+ """
+ return self._get_external_ceph_attribute('glance_pool')
+
+ def get_ext_ceph_nova_pool(self):
+ """ Get the external ceph server nova pool name.
+
+ Return:
+ External ceph server nova pool name.
+
+ Raise:
+ ConfigError in-case of an error
+ """
+ return self._get_external_ceph_attribute('nova_pool')
+
+ def get_ext_ceph_platform_pool(self):
+ """ Get the external ceph server platform pool name.
+
+ Return:
+ External ceph server platform pool name.
+
+ Raise:
+ ConfigError in-case of an error
+ """
+ return self._get_external_ceph_attribute('platform_pool')
+
+ def get_ext_ceph_cidr(self):
+ """ Get the external ceph cidr.
+
+ Return:
+ External ceph cidr.
+
+ Raise:
+ ConfigError in-case of an error
+ """
+ return self._get_external_ceph_attribute('cidr')
+
+ def set_ceph_enabled(self):
+ """ Set the ceph enabled
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+
+ if 'backends' not in self.config[self.ROOT]:
+ raise configerror.ConfigError('No backends configured')
+
+ if 'external_ceph' in self.config[self.ROOT]['backends']:
+ return
+
+ if self.is_lvm_enabled():
+ return
+
+ try:
+ self.is_ceph_enabled()
+ except configerror.ConfigError:
+ self.config[self.ROOT]['backends']['ceph'].update({"enabled": True})
+
+ def mask_sensitive_data(self):
+ if 'external_ceph' in self.get_storage_backends():
+ self.config[self.ROOT]['backends']['external_ceph']['ceph_user_key'] = self.MASK
+ self.config[self.ROOT]['backends']['external_ceph']['ceph_s3_keystone_adminpw'] = self.MASK
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from cmdatahandlers.api import configerror
+from cmdatahandlers.api import config
+from cmdatahandlers.api import utils
+
+
+class Config(config.Config):
+ def __init__(self, confman):
+ super(Config, self).__init__(confman)
+ self.ROOT = 'cloud.storage_profiles'
+ self.DOMAIN = 'storage_profiles'
+
+ def init(self):
+ pass
+
+ def validate(self):
+ self.validate_root()
+ self._validate_storage_profiles()
+
+ def _validate_storage_profiles(self):
+ profiles = self.get_storage_profiles()
+ utils.validate_list_items_unique(profiles)
+ for profile in profiles:
+ self._validate_storage_profile(profile)
+
+ def _validate_storage_profile(self, profile):
+ backend = self.get_profile_backend(profile)
+ storageconf = self.confman.get_storage_config_handler()
+ backends = storageconf.get_storage_backends()
+ if backend not in backends:
+ raise configerror.ConfigError(
+ 'Invalid backend %s provided in profile %s' % (backend, profile))
+ if backend == 'ceph':
+ self.get_profile_nr_of_ceph_osd_disks(profile)
+ elif backend == 'lvm':
+ self.get_profile_lvm_cinder_storage_partitions(profile)
+ self.get_profile_lvm_instance_storage_partitions(profile)
+ self.get_profile_lvm_instance_cow_lv_storage_percentage(profile)
+ self.get_profile_instance_storage_percentage(profile)
+
+ def is_valid_profile(self, profile):
+ self.validate_root()
+ profiles = self.get_storage_profiles()
+ if profile not in profiles:
+ raise configerror.ConfigError('Invalid profile name %s' % profile)
+
+ def get_storage_profiles(self):
+ """ get the storage profiles list
+
+ Return:
+
+ A list of storage profile(s) names
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ return self.config[self.ROOT].keys()
+
+ def get_profile_ceph_osd_disks(self, profile):
+ """ get the ceph osd disks
+
+ Argument:
+
+ profile name
+
+ Return:
+
+ The ceph osd disks
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ return self._get_attribute(profile, 'ceph_osd_disks')
+
+ def get_profile_ceph_osd_journal_disk(self, profile):
+ """ get the ceph osd journal disk
+
+ Argument:
+
+ profile name
+
+ Return:
+
+ The ceph osd journal disk
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ return self._get_attribute(profile, 'ceph_osd_journal_disk')
+
+ def get_profile_ceph_openstack_pg_proportion(self, profile):
+ openstack_pg_ratio, caas_pg_ratio = self._get_ceph_pg_share_ratios(profile)
+ return openstack_pg_ratio / (openstack_pg_ratio + caas_pg_ratio)
+
+ def _get_ceph_pg_share_ratios(self, profile):
+ pg_share_ratios = self.get_profile_ceph_openstack_caas_pg_ratio(profile).split(':')
+ return map(lambda r: float(r), pg_share_ratios)
+
+ def get_profile_ceph_caas_pg_proportion(self, profile):
+ openstack_pg_ratio, caas_pg_ratio = self._get_ceph_pg_share_ratios(profile)
+ return caas_pg_ratio / (openstack_pg_ratio + caas_pg_ratio)
+
+ def get_profile_ceph_openstack_caas_pg_ratio(self, profile):
+ """ get the ceph openstack-caas pg share ratio
+
+ Argument:
+
+ profile name
+
+ Return:
+
+ The ceph osd share ratio
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ return self._get_optional_attribute(
+ profile, 'ceph_pg_openstack_caas_share_ratio', "1:0")
+
+ def get_profile_nr_of_ceph_osd_disks(self, profile):
+ """ get the number of ceph osd disks
+
+ Argument:
+
+ profile name
+
+ Return:
+
+ The number of ceph osd disks
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self.validate_root()
+ self.is_valid_profile(profile)
+
+ if 'nr_of_ceph_osd_disks' not in self.config[self.ROOT][profile]:
+ ceph_osd_disks = self._get_attribute(profile, 'ceph_osd_disks')
+ return len(ceph_osd_disks)
+ return self.config[self.ROOT][profile]['nr_of_ceph_osd_disks']
+
+ def get_profile_lvm_cinder_storage_partitions(self, profile):
+ """ get the lvm_cinder_storage_partitions
+
+ Argument:
+
+ profile name
+
+ Return:
+
+ The lvm_cinder_storage_partitions
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ return self._get_attribute(profile, 'lvm_cinder_storage_partitions')
+
+ def get_profile_backend(self, profile):
+ """ get the storage profile backend
+
+ Argument:
+
+ profile name
+
+ Return:
+
+ The profile backend
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ return self._get_attribute(profile, 'backend')
+
+ def get_profile_lvm_instance_storage_partitions(self, profile):
+ """ get the lvm_instance_storage_partitions
+
+ Argument:
+
+ profile name
+
+ Return:
+
+ The lvm_instance_storage_partitions
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ return self._get_attribute(profile, 'lvm_instance_storage_partitions')
+
+ def get_profile_lvm_instance_cow_lv_storage_percentage(self, profile):
+ """ get the lvm_instance_cow_lv_storage_percentage
+
+ Argument:
+
+ profile name
+
+ Return:
+
+ The lvm_instance_cow_lv_storage_percentage
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ return self._get_attribute(profile, 'lvm_instance_cow_lv_storage_percentage')
+
+ def get_profile_instance_storage_percentage(self, profile):
+ """ get the instance_storage_percentage
+
+ Argument:
+
+ profile name
+
+ Return:
+
+ The instance_storage_percentage
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ return self._get_attribute(profile, 'instance_storage_percentage')
+
+ def get_profile_bare_lvm_mount_options(self, profile):
+ """ get the mount_options
+
+ Argument:
+
+ profile name
+
+ Return:
+
+ The mount_options
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ return self._get_optional_attribute(profile, 'mount_options')
+
+ def get_profile_bare_lvm_mount_dir(self, profile):
+ """ get the mount_dir
+
+ Argument:
+
+ profile name
+
+ Return:
+
+ The mount_dir
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ return self._get_attribute(profile, 'mount_dir')
+
+ def get_profile_bare_lvm_lv_name(self, profile):
+ """ get the lv_name
+
+ Argument:
+
+ profile name
+
+ Return:
+
+ The lv_name
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ return self._get_attribute(profile, 'lv_name')
+
+ def _get_attribute(self, profile, attribute):
+ """ get arbirary storage profile attribute
+
+ Arguments:
+
+ - profile name
+ - attribute name
+
+ Return:
+
+ The attribute
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self.validate_root()
+ self.is_valid_profile(profile)
+ if attribute not in self.config[self.ROOT][profile]:
+ raise configerror.ConfigError(
+ 'Profile %s does not have %s configured' % (attribute, profile))
+ return self.config[self.ROOT][profile][attribute]
+
+ def _get_optional_attribute(self, profile, attribute, default_value=""):
+ """ get arbirary optional storage profile attribute
+
+ Arguments:
+
+ - profile name
+ - attribute name
+
+ Return:
+
+ The attribute
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self.validate_root()
+ self.is_valid_profile(profile)
+ if attribute not in self.config[self.ROOT][profile]:
+ return default_value
+ return self.config[self.ROOT][profile][attribute]
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from cmdatahandlers.api import configerror
+from cmdatahandlers.api import config
+from cmdatahandlers.api import utils
+
+class Config(config.Config):
+ def __init__(self, confman):
+ super(Config, self).__init__(confman)
+ self.ROOT='cloud.time'
+ self.DOMAIN='time'
+
+ def init(self):
+ pass
+
+ def validate(self):
+ self.validate_root()
+
+ if 'zone' not in self.config[self.ROOT]:
+ raise configerror.ConfigError('No zone configuration found')
+
+ if 'ntp_servers' not in self.config[self.ROOT]:
+ raise configerror.ConfigError('No ntp servers found')
+
+ if 'auth_type' not in self.config[self.ROOT]:
+ self.config[self.ROOT]['auth_type'] = 'crypto'
+
+ if 'serverkeys_path' not in self.config[self.ROOT]:
+ self.config[self.ROOT]['serverkeys_path'] = ''
+
+ self._validate_time_zone()
+ self._validate_ntp_servers()
+
+ def _validate_time_zone(self):
+ import pytz
+ try:
+ zone = self.get_zone()
+ pytz.timezone(zone)
+ except:
+ raise configerror.ConfigError('The timezone %s is not valid' % zone)
+
+ def _validate_ntp_servers(self):
+ servers = self.get_ntp_servers()
+ utils.validate_list_items_unique(servers)
+
+ for server in servers:
+ utils.validate_ipv4_address(server)
+
+
+ """ get the time zone
+
+ Return:
+
+ A string representing time zone
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ def get_zone(self):
+ self.validate_root()
+ return self.config[self.ROOT]['zone']
+
+ """ get the ntp servers
+
+ Return:
+
+ A list of ntp server addresses.
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ def get_ntp_servers(self):
+ self.validate_root()
+ return self.config[self.ROOT]['ntp_servers']
+
+ def get_auth_type(self):
+ self.validate_root()
+ return self.config[self.ROOT]['auth_type']
+
+ def get_time_config(self):
+ self.validate_root()
+ return self.config[self.ROOT]
+
+ def set_config(self, **args):
+ if args['auth_type'] is not None:
+ self.config[self.ROOT]['auth_type'] = args['auth_type']
+ if args['servers'] is not None:
+ self.config[self.ROOT]['ntp_servers'] = args['servers']
+ if args['keyfile_url'] is not None:
+ self.config[self.ROOT]['serverkeys_path'] = args['keyfile_url']
+ if args['zone'] is not None:
+ self.config[self.ROOT]['zone'] = args['zone']
+ self.validate()
+ return self.config[self.ROOT]
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from cmdatahandlers.api import configerror
+from cmdatahandlers.api import config
+
+class Config(config.Config):
+ def __init__(self, confman):
+ super(Config, self).__init__(confman)
+ self.ROOT = 'cloud.users'
+ self.DOMAIN = 'users'
+
+ def init(self):
+ pass
+
+ def validate(self):
+ self.validate_root()
+
+ def get_users(self):
+ """ get the users list
+
+ Return:
+
+ A list of users
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ self.validate_root()
+ return []
+
+ def get_user_password(self, user):
+ """ get the password for a user
+
+ Return:
+
+ A string representing the password
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ raise configerror.ConfigError('Invalid user %s' % user)
+
+ def get_admin_user_password(self):
+ """ get the admin user password
+
+ Return:
+
+ A string representing the admin user password
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ return self.config[self.ROOT]['admin_user_password']
+
+ def get_admin_user(self):
+ """ get the admin user
+
+ Return:
+
+ A string representing the admin user
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ return self.config[self.ROOT]['admin_user_name']
+
+ def get_initial_user_name(self):
+ """ get the initial user name
+
+ Return:
+
+ A string representing the initial user name
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ return self.config[self.ROOT]['initial_user_name']
+
+ def get_initial_user_password(self):
+ """ get the initial user password
+
+ Return:
+
+ A string representing the initial user password
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ return self.config[self.ROOT]['initial_user_password']
+
+ def mask_sensitive_data(self):
+ self.config[self.ROOT]['admin_user_password'] = self.MASK
+ self.config[self.ROOT]['initial_user_password'] = self.MASK
+
+ def get_admin_password(self):
+ """ get the admin password
+
+ Return:
+
+ The admin password
+
+ Raise:
+
+ ConfigError in-case of an error
+ """
+ return self.config[self.ROOT]['admin_password']
+
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+name:base
+description:The basic profile containing services to be run in all nodes
--- /dev/null
+name:caas_master
+description:CAAS master node profile
+inherits:management
--- /dev/null
+name:caas_worker
+description:CAAS worker node profile
+inherits:base
--- /dev/null
+name:compute
+description:Openstack compute profile
+inherits:base
--- /dev/null
+name:controller
+description:Openstack controller profile
+inherits:management
--- /dev/null
+name:management
+description:The base profile for management nodes
+inherits:base
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import os
+import re
+import inspect
+
+class Profile(object):
+ def __init__(self):
+ self.name = None
+ self.description = None
+ self.inherits = []
+ self.included_profiles = []
+ def __str__(self):
+ return 'name:{}\ndescription:{}\ninherits:{}\nincluded_profiles:{}\n'.format(self.name, self.description, self.inherits, self.included_profiles)
+
+class Profiles(object):
+ def __init__(self, location=os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))):
+ self.location = location
+ self.profiles = {}
+ try:
+ self._load_profiles()
+ except:
+ pass
+
+ def _load_profiles(self):
+ files = self._get_profiles_files()
+ for f in files:
+ self._profile_from_file(f)
+
+ #update the included profiles
+ for name, profile in self.profiles.iteritems():
+ included_profiles = []
+ self._update_included_profiles(profile, included_profiles)
+ profile.included_profiles = included_profiles
+
+
+ def _update_included_profiles(self, profile, included_profiles):
+ included_profiles.append(profile.name)
+ for b in profile.inherits:
+ self._update_included_profiles(self.profiles[b], included_profiles)
+
+ def _get_profiles_files(self):
+ files = os.listdir(self.location)
+ pattern = re.compile('.*[.]profile$')
+ result = []
+ for f in files:
+ fullpath = self.location + '/' + f
+ if os.path.isfile(fullpath) and pattern.match(f):
+ result.append(fullpath)
+ return result
+
+
+
+ def _profile_from_file(self, filename):
+ profile = Profile()
+ with open(filename) as f:
+ lines=f.read().splitlines()
+ for l in lines:
+ data = l.split(':')
+ if len(data) != 2:
+ raise Exception('Invalid line %s in file %s' % (l, filename))
+ elif data[0] == 'name':
+ profile.name = data[1]
+ elif data[0] == 'description':
+ profile.description = data[1]
+ elif data[0] == 'inherits':
+ profile.inherits = data[1].split(',')
+ else:
+ raise Exception('Invalid line %s in file %s' % (l, filename))
+ self.profiles[profile.name] = profile
+
+ def get_included_profiles(self, name):
+ return self.profiles[name].included_profiles
+
+ def get_profiles(self):
+ return self.profiles
+
+ def get_children_profiles(self, name):
+ ret = []
+ for pfname, profile in self.profiles.iteritems():
+ if name in profile.inherits:
+ ret.append(pfname)
+ return ret
+
+
+if __name__ == '__main__':
+ import sys
+ import traceback
+ import argparse
+
+ parser = argparse.ArgumentParser(description='Test service profiles',
+ prog=sys.argv[0])
+
+
+ parser.add_argument('--get-included-profiles',
+ dest='get_included_profiles',
+ help='Get the profiles included in some profile name',
+ action='store_true')
+
+ parser.add_argument('--get-all-profiles',
+ help='Get the profiles list',
+ dest='get_all_profiles',
+ action='store_true')
+
+ parser.add_argument('--get-children-profiles',
+ dest='get_children_profiles',
+ help='Get the children of a profile',
+ action='store_true')
+
+ parser.add_argument('--name',
+ metavar='NAME',
+ dest='name',
+ help='The name of the profile',
+ type=str,
+ action='store')
+ try:
+ args = parser.parse_args(sys.argv[1:])
+ profiles = Profiles()
+ if args.get_included_profiles or args.get_children_profiles:
+ if not args.name:
+ raise Exception('Missing profile name')
+
+ if args.get_included_profiles:
+ included_profiles = profiles.get_included_profiles(args.name)
+ print('Included profiles')
+ for p in included_profiles:
+ print(p)
+ if args.get_children_profiles:
+ children_profiles = profiles.get_children_profiles(args.name)
+ print('Children profiles')
+ for p in children_profiles:
+ print(p)
+ elif args.get_all_profiles:
+ all = profiles.get_profiles()
+ for name, p in all.iteritems():
+ print(p)
+ except Exception as exp:
+ print('Failed with error %s' % exp)
+ traceback.print_exc()
+ sys.exit(1)
--- /dev/null
+name:storage
+description:The storage nodes profile
+inherits:base
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from unittest import TestCase
+from cmdatahandlers.api import configmanager
+import cmdatahandlers.performance_profiles.config
+from cmdatahandlers.api.configerror import ConfigError
+
+
+class PerformanceProfilesConfigTest(TestCase):
+
+
+ profile = 'dpdk_profile'
+ profile_data = {profile: {'platform_cpus': {'numa0': 1, 'numa1': 1},
+ 'ovs_dpdk_cpus': {'numa0': 2, 'numa1': 2},
+ 'default_hugepagesz': '1M',
+ 'hugepagesz': '1G',
+ 'hugepages': 192
+ }
+ }
+ hosts_data = {'controller-1':{'service_profiles': ['controller']}}
+
+ config = {'cloud.performance_profiles': profile_data,
+ 'cloud.hosts': hosts_data }
+
+ fail_profile ='dpdk_fail_profile'
+ config_fail = {'cloud.performance_profiles': {fail_profile: {} },
+ 'cloud.hosts': hosts_data }
+
+ def setUp(self):
+ confman = configmanager.ConfigManager(self.config)
+ self.pp_handler = confman.get_performance_profiles_config_handler()
+
+ confman_fail = configmanager.ConfigManager(self.config_fail)
+ self.pp_handler_fail = confman_fail.get_performance_profiles_config_handler()
+
+ def test_validate(self):
+ self.pp_handler.validate()
+
+ def test_is_valid_profile_raises_error(self):
+ with self.assertRaisesRegexp(ConfigError, "Invalid profile name foo_profile"):
+ self.pp_handler.is_valid_profile('foo_profile')
+
+ def test_get_performance_profiles(self):
+ profiles_data = self.pp_handler.get_performance_profiles()
+ expected_data = [self.profile]
+ self.assertEqual(profiles_data, expected_data)
+
+ def test_get_profile_default_hugepage_size(self):
+ default_hugepagesz_data = self.pp_handler.get_profile_default_hugepage_size(self.profile)
+ expected_data = '1M'
+ self.assertEqual(default_hugepagesz_data, expected_data)
+
+ def test_get_profile_default_hugepage_size_raises_error(self):
+ error_text = "Profile {} does not have default_hugepagesz".format(self.fail_profile)
+ with self.assertRaisesRegexp(ConfigError, error_text):
+ self.pp_handler_fail.get_profile_default_hugepage_size(self.fail_profile)
+
+ def test_get_profile_hugepage_size(self):
+ hugepagesz_data = self.pp_handler.get_profile_hugepage_size(self.profile)
+ expected_data = '1G'
+ self.assertEqual(hugepagesz_data, expected_data)
+
+ def test_get_profile_hugepage_size_raises_error(self):
+ error_text = "Profile {} does not have hugepagesz".format(self.fail_profile)
+ with self.assertRaisesRegexp(ConfigError, error_text):
+ self.pp_handler_fail.get_profile_hugepage_size(self.fail_profile)
+
+ def test_get_profile_hugepage_count(self):
+ hugepages_data = self.pp_handler.get_profile_hugepage_count(self.profile)
+ expected_data = 192
+ self.assertEqual(hugepages_data, expected_data)
+
+ def test_get_profile_hugepage_count_raises_error(self):
+ error_text = "Profile {} does not have hugepages".format(self.fail_profile)
+ with self.assertRaisesRegexp(ConfigError, error_text):
+ self.pp_handler_fail.get_profile_hugepage_count(self.fail_profile)
+
+ def test_get_platform_cpus(self):
+ platform_cpus_data = self.pp_handler.get_platform_cpus(self.profile)
+ expected_data = {'numa0': 1, 'numa1': 1}
+ self.assertEqual(platform_cpus_data, expected_data)
+
+ def test_get_platform_cpus_raises(self):
+ error_text = "Profile {} does not have platform_cpus".format(self.fail_profile)
+ with self.assertRaisesRegexp(ConfigError, error_text):
+ self.pp_handler_fail.get_platform_cpus(self.fail_profile)
+
+ def test_get_ovs_dpdk_cpus(self):
+ ovs_dpdk_cpus_data = self.pp_handler.get_ovs_dpdk_cpus(self.profile)
+ expected_data = {'numa0': 2, 'numa1': 2}
+ self.assertEqual(ovs_dpdk_cpus_data, expected_data)
+
+ def test_get_ovs_dpdk_cpus_raises_error(self):
+ error_text = "Profile {} does not have ovs_dpdk_cpus".format(self.fail_profile)
+ with self.assertRaisesRegexp(ConfigError, error_text):
+ self.pp_handler_fail.get_ovs_dpdk_cpus(self.fail_profile)
+
+ def test__fill_option_value(self):
+ data = {}
+ self.pp_handler._fill_option_value(data, self.profile, 'platform_cpus', None)
+ self.assertEqual(data, {'platform_cpus': {'numa0': 1, 'numa1': 1}})
+ self.pp_handler._fill_option_value(data, self.profile, 'platform_cpus', {'numa0': 5})
+ self.assertEqual(data, {'platform_cpus': {'numa0': 5}})
+
+ def test_update(self):
+ cman = configmanager.ConfigManager({'cloud.performance_profiles': {}, 'cloud.hosts': self.hosts_data})
+ h = cman.get_performance_profiles_config_handler()
+ profile = 'niksnaks'
+ platform_cpus = {'numa0': 1, 'numa1': 1}
+ ovs_dpdk_cpus = {'numa0': 2, 'numa1': 2}
+ hugepages = 8
+ h.update(profile, platform_cpus, ovs_dpdk_cpus, hugepages)
+ self.assertEqual(h.get_performance_profiles(), [profile])
+ self.assertEqual(h.get_platform_cpus(profile), platform_cpus)
+ self.assertEqual(h.get_ovs_dpdk_cpus(profile), ovs_dpdk_cpus)
+ self.assertEqual(h.get_profile_hugepage_count(profile), hugepages)
+ with self.assertRaisesRegexp(ConfigError, 'Profile niksnaks does not have hugepagesz'):
+ h.get_profile_hugepage_size(profile)
+ with self.assertRaisesRegexp(ConfigError, 'Profile niksnaks does not have default_hugepagesz'):
+ h.get_profile_default_hugepage_size(profile)
+
+ def test_delete(self):
+ cman = configmanager.ConfigManager({'cloud.performance_profiles': {}, 'cloud.hosts': self.hosts_data})
+ h = cman.get_performance_profiles_config_handler()
+ profile = 'nice_profile'
+ h.update(profile, '1-8', '4', 2)
+ h.delete(profile)
+ error_text = 'Invalid profile name {}'.format(profile)
+ self.assertEqual(h.get_performance_profiles(), [])
+ with self.assertRaisesRegexp(ConfigError, error_text):
+ h.delete(profile)
+
+ def test_dump(self):
+ data = self.pp_handler.dump()
+ self.assertEqual(data, self.profile_data)
+
+
+if __name__ == '__main__':
+ unittest.main()
--- /dev/null
+[tox]
+envlist = py27-pytest,pylint
+
+[testenv]
+basepython = python2.7
+changedir = tests
+
+setenv =
+ PYTHONPATH = {toxinidir}/src
+ COVERAGE_FILE = .coverage{envname}
+
+passenv = COVERAGE_FILE
+
+commands = /bin/cp -R {toxinidir}/tests/mocked_dependencies/cmdatahandlers {toxworkdir}/py27-pytest/lib/python2.7/site-packages/
+ pytest -vv \
+ --basetemp={envtmpdir} \
+ --pep8 \
+ --cov cmdatahandlers.performance_profiles \
+ --cov-config .coveragerc \
+ --cov-branch \
+ --cov-report term-missing \
+ --cov-report html:htmlcov \
+ {posargs:.}
+
+deps=
+ pip==10.0.1
+ pytest
+ mock
+ pytest-cov
+ pytest-flakes
+ pytest-pep8
+ netaddr
+ pyyaml
+ jinja2
+ # more-itertools above version 5.0.0 down not support Python 2.7
+ more-itertools==5.0.0
+
+[pytest]
+cache_dir = .pytest-cache
+pep8maxlinelength = 100
+pep8ignore = src/setup.py ALL
+ src/cmdatahandlers/api/* ALL
+ src/cmdatahandlers/has/* ALL
+ src/cmdatahandlers/hosts/* ALL
+ src/cmdatahandlers/localstorage/* ALL
+ src/cmdatahandlers/network_profiles/* ALL
+ src/cmdatahandlers/networking/* ALL
+ src/cmdatahandlers/openstack/* ALL
+ src/cmdatahandlers/storage_profiles/* ALL
+ src/cmdatahandlers/time/* ALL
+ src/cmdatahandlers/users/* ALL
+ tests/* ALL
+
+[testenv:pylint]
+commands = /bin/cp -R {toxinidir}/tests/mocked_dependencies/cmdatahandlers {toxworkdir}/pylint/lib/python2.7/site-packages/
+ -pylint --rcfile={toxinidir}/pylintrc {posargs:cmdatahandlers.performance_profiles}
+
+deps=
+ pip==10.0.1
+ pylint==1.9.2
+ mock
--- /dev/null
+.coveragepy27-pytest
+.idea/
+.pytest-cache/
+.tox/
+.coverage
+htmlcov/
+test/__pycache__/
+src/cmframework.egg-info/
+*.pyc
+*.pyo
--- /dev/null
+[MASTER]
+
+# Specify a configuration file.
+#rcfile=
+
+# Python code to execute, usually for sys.path manipulation such as
+# pygtk.require().
+#init-hook=
+
+# Add files or directories to the blacklist. They should be base names, not
+# paths.
+ignore=CVS
+
+# Add files or directories matching the regex patterns to the blacklist. The
+# regex matches against base names, not paths.
+ignore-patterns=
+
+# Pickle collected data for later comparisons.
+persistent=yes
+
+# List of plugins (as comma separated values of python modules names) to load,
+# usually to register additional checkers.
+load-plugins=
+
+# Use multiple processes to speed up Pylint.
+jobs=1
+
+# Allow loading of arbitrary C extensions. Extensions are imported into the
+# active Python interpreter and may run arbitrary code.
+unsafe-load-any-extension=no
+
+# A comma-separated list of package or module names from where C extensions may
+# be loaded. Extensions are loading into the active Python interpreter and may
+# run arbitrary code
+extension-pkg-whitelist=
+
+# Allow optimization of some AST trees. This will activate a peephole AST
+# optimizer, which will apply various small optimizations. For instance, it can
+# be used to obtain the result of joining multiple strings with the addition
+# operator. Joining a lot of strings can lead to a maximum recursion error in
+# Pylint and this flag can prevent that. It has one side effect, the resulting
+# AST will be different than the one from reality. This option is deprecated
+# and it will be removed in Pylint 2.0.
+optimize-ast=no
+
+
+[MESSAGES CONTROL]
+
+# Only show warnings with the listed confidence levels. Leave empty to show
+# all. Valid levels: HIGH, INFERENCE, INFERENCE_FAILURE, UNDEFINED
+confidence=
+
+# Enable the message, report, category or checker with the given id(s). You can
+# either give multiple identifier separated by comma (,) or put this option
+# multiple time (only on the command line, not in the configuration file where
+# it should appear only once). See also the "--disable" option for examples.
+#enable=
+
+# Disable the message, report, category or checker with the given id(s). You
+# can either give multiple identifiers separated by comma (,) or put this
+# option multiple times (only on the command line, not in the configuration
+# file where it should appear only once).You can also use "--disable=all" to
+# disable everything first and then reenable specific checks. For example, if
+# you want to run only the similarities checker, you can use "--disable=all
+# --enable=similarities". If you want to run only the classes checker, but have
+# no Warning level messages displayed, use"--disable=all --enable=classes
+# --disable=W"
+disable=import-star-module-level,old-octal-literal,oct-method,print-statement,unpacking-in-except,parameter-unpacking,backtick,old-raise-syntax,old-ne-operator,long-suffix,dict-view-method,dict-iter-method,metaclass-assignment,next-method-called,raising-string,indexing-exception,raw_input-builtin,long-builtin,file-builtin,execfile-builtin,coerce-builtin,cmp-builtin,buffer-builtin,basestring-builtin,apply-builtin,filter-builtin-not-iterating,using-cmp-argument,useless-suppression,range-builtin-not-iterating,suppressed-message,no-absolute-import,old-division,cmp-method,reload-builtin,zip-builtin-not-iterating,intern-builtin,unichr-builtin,reduce-builtin,standarderror-builtin,unicode-builtin,xrange-builtin,coerce-method,delslice-method,getslice-method,setslice-method,input-builtin,round-builtin,hex-method,nonzero-method,map-builtin-not-iterating,too-many-function-args, missing-docstring,too-few-public-methods,invalid-name,too-many-public-methods,too-many-locals,E402
+
+
+[REPORTS]
+
+# Set the output format. Available formats are text, parseable, colorized, msvs
+# (visual studio) and html. You can also give a reporter class, eg
+# mypackage.mymodule.MyReporterClass.
+output-format=text
+
+# Put messages in a separate file for each module / package specified on the
+# command line instead of printing them on stdout. Reports (if any) will be
+# written in a file name "pylint_global.[txt|html]". This option is deprecated
+# and it will be removed in Pylint 2.0.
+files-output=no
+
+# Tells whether to display a full report or only the messages
+reports=yes
+
+# Python expression which should return a note less than 10 (10 is the highest
+# note). You have access to the variables errors warning, statement which
+# respectively contain the number of errors / warnings messages and the total
+# number of statements analyzed. This is used by the global evaluation report
+# (RP0004).
+evaluation=10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10)
+
+# Template used to display messages. This is a python new-style format string
+# used to format the message information. See doc for all details
+#msg-template=
+
+
+[BASIC]
+
+# Good variable names which should always be accepted, separated by a comma
+good-names=i,j,k,ex,Run,_
+
+# Bad variable names which should always be refused, separated by a comma
+bad-names=foo,bar,baz,toto,tutu,tata
+
+# Colon-delimited sets of names that determine each other's naming style when
+# the name regexes allow several styles.
+name-group=
+
+# Include a hint for the correct naming format with invalid-name
+include-naming-hint=no
+
+# List of decorators that produce properties, such as abc.abstractproperty. Add
+# to this list to register other decorators that produce valid properties.
+property-classes=abc.abstractproperty
+
+# Regular expression matching correct function names
+function-rgx=[a-z_][a-z0-9_]{2,30}$
+
+# Naming hint for function names
+function-name-hint=[a-z_][a-z0-9_]{2,30}$
+
+# Regular expression matching correct variable names
+variable-rgx=[a-z_][a-z0-9_]{2,30}$
+
+# Naming hint for variable names
+variable-name-hint=[a-z_][a-z0-9_]{2,30}$
+
+# Regular expression matching correct constant names
+const-rgx=(([A-Z_][A-Z0-9_]*)|(__.*__))$
+
+# Naming hint for constant names
+const-name-hint=(([A-Z_][A-Z0-9_]*)|(__.*__))$
+
+# Regular expression matching correct attribute names
+attr-rgx=[a-z_][a-z0-9_]{2,30}$
+
+# Naming hint for attribute names
+attr-name-hint=[a-z_][a-z0-9_]{2,30}$
+
+# Regular expression matching correct argument names
+argument-rgx=[a-z_][a-z0-9_]{2,30}$
+
+# Naming hint for argument names
+argument-name-hint=[a-z_][a-z0-9_]{2,30}$
+
+# Regular expression matching correct class attribute names
+class-attribute-rgx=([A-Za-z_][A-Za-z0-9_]{2,30}|(__.*__))$
+
+# Naming hint for class attribute names
+class-attribute-name-hint=([A-Za-z_][A-Za-z0-9_]{2,30}|(__.*__))$
+
+# Regular expression matching correct inline iteration names
+inlinevar-rgx=[A-Za-z_][A-Za-z0-9_]*$
+
+# Naming hint for inline iteration names
+inlinevar-name-hint=[A-Za-z_][A-Za-z0-9_]*$
+
+# Regular expression matching correct class names
+class-rgx=[A-Z_][a-zA-Z0-9]+$
+
+# Naming hint for class names
+class-name-hint=[A-Z_][a-zA-Z0-9]+$
+
+# Regular expression matching correct module names
+module-rgx=(([a-z_][a-z0-9_]*)|([A-Z][a-zA-Z0-9]+))$
+
+# Naming hint for module names
+module-name-hint=(([a-z_][a-z0-9_]*)|([A-Z][a-zA-Z0-9]+))$
+
+# Regular expression matching correct method names
+method-rgx=[a-z_][a-z0-9_]{2,30}$
+
+# Naming hint for method names
+method-name-hint=[a-z_][a-z0-9_]{2,30}$
+
+# Regular expression which should only match function or class names that do
+# not require a docstring.
+no-docstring-rgx=^_
+
+# Minimum line length for functions/classes that require docstrings, shorter
+# ones are exempt.
+docstring-min-length=-1
+
+
+[ELIF]
+
+# Maximum number of nested blocks for function / method body
+max-nested-blocks=5
+
+
+[FORMAT]
+
+# Maximum number of characters on a single line.
+max-line-length=100
+
+# Regexp for a line that is allowed to be longer than the limit.
+ignore-long-lines=^\s*(# )?<?https?://\S+>?$
+
+# Allow the body of an if to be on the same line as the test if there is no
+# else.
+single-line-if-stmt=no
+
+# List of optional constructs for which whitespace checking is disabled. `dict-
+# separator` is used to allow tabulation in dicts, etc.: {1 : 1,\n222: 2}.
+# `trailing-comma` allows a space between comma and closing bracket: (a, ).
+# `empty-line` allows space-only lines.
+no-space-check=trailing-comma,dict-separator
+
+# Maximum number of lines in a module
+max-module-lines=1000
+
+# String used as indentation unit. This is usually " " (4 spaces) or "\t" (1
+# tab).
+indent-string=' '
+
+# Number of spaces of indent required inside a hanging or continued line.
+indent-after-paren=4
+
+# Expected format of line ending, e.g. empty (any line ending), LF or CRLF.
+expected-line-ending-format=
+
+
+[LOGGING]
+
+# Logging modules to check that the string format arguments are in logging
+# function parameter format
+logging-modules=logging
+
+
+[MISCELLANEOUS]
+
+# List of note tags to take in consideration, separated by a comma.
+notes=FIXME,XXX,TODO
+
+
+[SIMILARITIES]
+
+# Minimum lines number of a similarity.
+min-similarity-lines=4
+
+# Ignore comments when computing similarities.
+ignore-comments=yes
+
+# Ignore docstrings when computing similarities.
+ignore-docstrings=yes
+
+# Ignore imports when computing similarities.
+ignore-imports=no
+
+
+[SPELLING]
+
+# Spelling dictionary name. Available dictionaries: none. To make it working
+# install python-enchant package.
+spelling-dict=
+
+# List of comma separated words that should not be checked.
+spelling-ignore-words=
+
+# A path to a file that contains private dictionary; one word per line.
+spelling-private-dict-file=
+
+# Tells whether to store unknown words to indicated private dictionary in
+# --spelling-private-dict-file option instead of raising a message.
+spelling-store-unknown-words=no
+
+
+[TYPECHECK]
+
+# Tells whether missing members accessed in mixin class should be ignored. A
+# mixin class is detected if its name ends with "mixin" (case insensitive).
+ignore-mixin-members=yes
+
+# List of module names for which member attributes should not be checked
+# (useful for modules/projects where namespaces are manipulated during runtime
+# and thus existing member attributes cannot be deduced by static analysis. It
+# supports qualified module names, as well as Unix pattern matching.
+ignored-modules=
+
+# List of class names for which member attributes should not be checked (useful
+# for classes with dynamically set attributes). This supports the use of
+# qualified names.
+ignored-classes=optparse.Values,thread._local,_thread._local
+
+# List of members which are set dynamically and missed by pylint inference
+# system, and so shouldn't trigger E1101 when accessed. Python regular
+# expressions are accepted.
+generated-members=
+
+# List of decorators that produce context managers, such as
+# contextlib.contextmanager. Add to this list to register other decorators that
+# produce valid context managers.
+contextmanager-decorators=contextlib.contextmanager
+
+
+[VARIABLES]
+
+# Tells whether we should check for unused import in __init__ files.
+init-import=no
+
+# A regular expression matching the name of dummy variables (i.e. expectedly
+# not used).
+dummy-variables-rgx=(_+[a-zA-Z0-9]*?$)|dummy
+
+# List of additional names supposed to be defined in builtins. Remember that
+# you should avoid to define new builtins when possible.
+additional-builtins=
+
+# List of strings which can identify a callback function by name. A callback
+# name must start or end with one of those strings.
+callbacks=cb_,_cb
+
+# List of qualified module names which can have objects that can redefine
+# builtins.
+redefining-builtins-modules=six.moves,future.builtins
+
+
+[CLASSES]
+
+# List of method names used to declare (i.e. assign) instance attributes.
+defining-attr-methods=__init__,__new__,setUp
+
+# List of valid names for the first argument in a class method.
+valid-classmethod-first-arg=cls
+
+# List of valid names for the first argument in a metaclass class method.
+valid-metaclass-classmethod-first-arg=mcs
+
+# List of member names, which should be excluded from the protected access
+# warning.
+exclude-protected=_asdict,_fields,_replace,_source,_make
+
+
+[DESIGN]
+
+# Maximum number of arguments for function / method
+max-args=5
+
+# Argument names that match this expression will be ignored. Default to name
+# with leading underscore
+ignored-argument-names=_.*
+
+# Maximum number of locals for function / method body
+max-locals=15
+
+# Maximum number of return / yield for function / method body
+max-returns=6
+
+# Maximum number of branch for function / method body
+max-branches=12
+
+# Maximum number of statements in function / method body
+max-statements=50
+
+# Maximum number of parents for a class (see R0901).
+max-parents=7
+
+# Maximum number of attributes for a class (see R0902).
+max-attributes=7
+
+# Minimum number of public methods for a class (see R0903).
+min-public-methods=2
+
+# Maximum number of public methods for a class (see R0904).
+max-public-methods=20
+
+# Maximum number of boolean expressions in a if statement
+max-bool-expr=5
+
+
+[IMPORTS]
+
+# Deprecated modules which should not be used, separated by a comma
+deprecated-modules=regsub,TERMIOS,Bastion,rexec
+
+# Create a graph of every (i.e. internal and external) dependencies in the
+# given file (report RP0402 must not be disabled)
+import-graph=
+
+# Create a graph of external dependencies in the given file (report RP0402 must
+# not be disabled)
+ext-import-graph=
+
+# Create a graph of internal dependencies in the given file (report RP0402 must
+# not be disabled)
+int-import-graph=
+
+# Force import order to recognize a module as part of the standard
+# compatibility libraries.
+known-standard-library=
+
+# Force import order to recognize a module as part of a third party library.
+known-third-party=enchant
+
+# Analyse import fallback blocks. This can be used to support both Python 2 and
+# 3 compatible code, which means that the block might have code that exists
+# only in one or another interpreter, leading to false positives when analysed.
+analyse-fallback-blocks=no
+
+
+[EXCEPTIONS]
+
+# Exceptions that will emit a warning when being caught. Defaults to
+# "Exception"
+overgeneral-exceptions=Exception
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# json list of configuration item names whose value should be masked
+# when logging.
+
+[Passwords]
+names = [
+ "password",
+ "admin_password",
+ "admin_user_password",
+ "initial_user_password"
+ ]
--- /dev/null
+#!/bin/bash
+
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+COMMAND=$(basename "${0}")
+DO_REBOOT_IF_NEEDED=true
+
+#
+# Source log variables and functions
+#
+source $(dirname "${BASH_SOURCE[0]}")/log.sh
+
+function source_common()
+{
+ local SCRIPT_PATH
+ SCRIPT_PATH=$(dirname "${BASH_SOURCE[0]}")
+ local COMMON_SH_FILE=${SCRIPT_PATH}/common.sh
+ # shellcheck disable=SC1091
+ # shellcheck source=.
+ source "${COMMON_SH_FILE}" && return 0
+ log_error "Failed to source ${COMMON_SH_FILE}"
+ return 1
+}
+
+export CONFIG_PHASE="bootstrapping"
+
+function main()
+{
+ log_info "Bootstrapping started"
+
+ declare -a FUNCTIONS=(source_common start_db start_cm start_installation)
+ local func
+ for func in "${FUNCTIONS[@]}"
+ do
+ if ! ${func}
+ then
+ cleanup
+ return 1
+ fi
+ done
+
+ local rc
+ wait_installation_complete
+ rc=$?
+
+ if [ $rc -eq 0 ]; then
+ export CONFIG_PHASE="postconfig"
+ log_info "Generate inventory file to prepare for extra playbooks run"
+ run_cmd "$CMCLI ansible-inventory > $INVENTORY_FILE"
+
+ admin="x"
+ for d in $(ls -d /home/*); do
+ if [ -f "$d/openrc" ]; then
+ admin=$(basename "$d")
+ break
+ fi
+ done
+
+ #take a copy of the initial configuration data
+ mkdir /root/.initconfig
+ cp /var/lib/redis/dump.rdb /root/.initconfig/
+
+ su - "$admin" -c "/usr/local/bin/openstack-ansible -b -u $admin /opt/openstack-ansible/playbooks/finalize-playbook.yml" &>> $BOOTSTRAP_LOG
+ rc=$?
+
+ if [ $rc -eq 0 ]; then
+ execute_post_install
+ rc=$?
+ fi
+
+ fi
+
+ cleanup
+
+ log_info "starting redis again"
+ systemctl start redis
+
+ if [ $rc -eq 0 ]; then
+ if has_kernel_parameters_changed;
+ then
+ # The status of the installation will be logged by one of the following services after the host is rebooted.
+ #
+ # 1) finalize-bootstrap.service: When the performance porfile is enabled on the controller-1 and
+ # the network type is "ovs"
+ # 2) enable-dpdk.service: When the performance profile is enabled on the controller-1 and the
+ # network type is "ovs-dpdk"
+ #
+ if [ ${DO_REBOOT_IF_NEEDED} == true ]; then
+ reboot_host
+ else
+ log_info "Rebooting of host is skipped as requested."
+ fi
+ else
+ log_installation_success
+ fi
+ else
+ log_installation_failure
+ fi
+
+ return $rc
+}
+
+function show_help()
+{
+ echo "Usage:"
+ echo "# ${COMMAND} <full path to user-config-yaml|restore-config-yaml>"
+ echo "Or to skip the controller-1 reboot in the case kernel boot parameters are changed"
+ echo "# ${COMMAND} <full path to user-config-yaml|restore-config-yaml> (--install | --restore) --no-reboot"
+}
+
+#
+# Assume that the first argument is the configuration file to maintain backwards compatibility
+# so handle it separately.
+#
+
+if [ $# -lt 1 ]; then
+ show_help
+ exit 1
+else
+ CONFIG_FILE=$1
+ shift
+fi
+
+if ! [ -f "${CONFIG_FILE}" ]; then
+ log_error "Failed to open file:${CONFIG_FILE}"
+ show_help
+ exit 1
+fi
+
+
+#
+# And then the remaing arguments in any order
+#
+
+IS_INSTALL_ARG_SPECIFIED=false
+for arg in "$@"
+do
+ case ${arg} in
+ --no-reboot)
+ DO_REBOOT_IF_NEEDED=false
+ shift
+ ;;
+ --install)
+ IS_INSTALL_ARG_SPECIFIED=true
+ shift
+ ;;
+ --help)
+ show_help
+ exit 0
+ ;;
+ *)
+ log_error "Unknown option: ${arg}"
+ show_help
+ exit 1
+ ;;
+ esac
+done
+
+log_info "====================================================================="
+log_info "Boot strapping the environment with $CONFIG_FILE"
+log_info "====================================================================="
+
+main
--- /dev/null
+#! /bin/bash
+
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+exec /usr/local/bin/cmagent --ip config-manager --port 61100 --log-level debug --log-dest syslog --verbose
--- /dev/null
+#! /bin/bash
+
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+while [ 1 ]; do
+ instances=$(ps -elf | grep python | grep config-manager | grep -v grep | wc -l)
+ if [ $instances -eq 0 ]; then
+ echo "cmserver ready to start"
+ break
+ fi
+ echo "cmserver not ready to start yet"
+ sleep 1
+done
+
+exec /usr/local/bin/cmserver --file /etc/cmframework/config.ini
--- /dev/null
+#!/bin/bash
+
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+#
+# Collection of variables and functions for the bootstrap.sh
+
+set -o nounset
+
+#
+# Variables
+#
+
+CM_IP=127.0.0.1
+CM_PORT=61100
+CMCLI="/usr/local/bin/cmcli --ip $CM_IP --port $CM_PORT --client-lib cmframework.lib.cmclientimpl.CMClientImpl"
+CM_ACTIVATORS=/opt/cmframework/activators
+CM_PID=
+CM_VALIDATORS=/opt/cmframework/validators
+CM_INVENTORY_HANDLERS=/opt/cmframework/inventoryhandlers
+DB_IP=127.0.0.1
+DB_CHECK_CMD="/bin/redis-cli -h $DB_IP --scan --pattern '*'"
+DB_PORT=6379
+DB_STARTUP_CMD="/bin/redis-server ./redis.conf"
+INVENTORY_FILE=/opt/cmframework/inventory.data
+STATE_FILE=/etc/installation_state
+USER_CONFIG_HANDLERS=/opt/cmframework/userconfighandlers
+
+#
+# Source log variables and functions
+#
+source $(dirname "${BASH_SOURCE[0]}")/log.sh
+
+#
+# Functions
+#
+
+function run_cmd()
+{
+ local result
+ local ret
+ log_info "Running $*"
+ result=$(eval "$*" 2>&1)
+ ret=$?
+ if [ $ret -ne 0 ]; then
+ log_error "Failed with error $result"
+ else
+ log_info "Command succeeded: $result"
+ fi
+
+ return $ret
+}
+
+function stop_process()
+{
+ local pid=$1
+ log_info "Stopping process $pid"
+ if ! [ -z $pid ]; then
+ if [ -d /proc/$pid ]; then
+ log_info "Shutting down process $pid gracefully"
+ run_cmd "pkill -TERM -g $pid"
+ log_info "Waiting for process $pid to exit"
+ for ((i=0; i<10; i++)); do
+ if ! [ -d /proc/$pid ]; then
+ log_info "Process $pid exited"
+ break
+ fi
+ log_info "Process $pid is still running"
+ sleep 2
+ done
+
+ if [ -d /proc/$pid ]; then
+ log_error "Process $pid is still running, forcefully shutting it down"
+ run_cmd "pkill -KILL -g $pid"
+ fi
+ fi
+ fi
+}
+
+function cleanup()
+{
+ log_info "Cleaning up"
+ systemctl stop redis
+ stop_process $CM_PID
+ rm -f $INVENTORY_FILE
+}
+
+function start_db()
+{
+ log_info "Starting redis db using $DB_STARTUP_CMD"
+ systemctl start redis
+ log_info "Wait till DB is serving"
+ local dbok
+ dbok=0
+ for ((i=0; i<10; i++)); do
+ run_cmd "$DB_CHECK_CMD"
+ dbok=$?
+ if [ $dbok -eq 0 ]; then
+ break
+ fi
+ log_info "DB still not running"
+ sleep 2
+ done
+
+ return $dbok
+}
+
+function start_cm()
+{
+ rm -f "${STATE_FILE}"
+ log_info "Starting CM server"
+ setsid /usr/local/bin/cmserver --ip $CM_IP --port $CM_PORT \
+ --backend-api cmframework.redisbackend.cmredisdb.CMRedisDB \
+ --backend-uri redis://:@$DB_IP:$DB_PORT \
+ --log-level debug \
+ --validators $CM_VALIDATORS \
+ --activators $CM_ACTIVATORS \
+ --disable-remote-activation \
+ --log-dest console \
+ --log-level debug \
+ --inventory-handlers $CM_INVENTORY_HANDLERS \
+ --inventory-data $INVENTORY_FILE \
+ --activationstate-handler-api cmframework.utils.cmdsshandler.CMDSSHandler \
+ --activationstate-handler-uri /run/.dss-server \
+ --alarmhandler-api cmframework.lib.cmalarmhandler_dummy.AlarmHandler_Dummy \
+ --install-phase \
+ --snapshot-handler-api cmframework.utils.cmdsshandler.CMDSSHandler \
+ --snapshot-handler-uri /run/.dss-server \
+ --verbose 2> "$CM_LOG" 1>&2 &
+ export CM_PID=$!
+ log_info "cmserver pid is $CM_PID"
+ if ! [ -d /proc/$CM_PID ]; then
+ log_error "CM server is not running!"
+ log_info "Check redis.log and $BOOTSTRAP_LOG for details"
+ return 1
+ fi
+
+ log_info "Wait till cmserver is ready to serve"
+ local out
+ while true; do
+ out=$($CMCLI get-properties --matching-filter '.*' 2>&1)
+ if [ $? -eq 0 ]; then
+ break
+ fi
+ echo $out | grep "Not found" 2> /dev/null 1>&2
+ if [ $? -eq 0 ]; then
+ break
+ fi
+ log_info "cmserver not ready yet, got error $out"
+ sleep 1
+ done
+ return 0
+}
+
+function handle_user_config()
+{
+ log_info "Handling user configuration from file $CONFIG_FILE"
+ run_cmd "$CMCLI bootstrap --config $CONFIG_FILE --plugin_path $USER_CONFIG_HANDLERS"
+ return $?
+}
+
+function start_installation()
+{
+ log_info "Start installation"
+ handle_user_config
+ return $?
+}
+
+function wait_installation_complete()
+{
+ log_info "Waiting for installation to complete"
+ while true; do
+ if [ -f $STATE_FILE ]; then
+ log_info "installation completed"
+ break
+ fi
+ sleep 5
+ done
+ local result
+ result=$(cat $STATE_FILE)
+
+ if [ "$result" == "success" ]; then
+ log_info "exiting with success :)"
+ return 0
+ else
+ log_error "exiting with failure :("
+ return 1
+ fi
+}
+
+function execute_post_install()
+{
+ log_info "Start post installation"
+ return 0
+}
+
+function parameter_exists_in_list()
+{
+ local EXPECTED_PARAM=$1
+ shift
+ local PARAM_LIST=$*
+ local PARAM
+ for PARAM in ${PARAM_LIST}
+ do
+ if [ "${PARAM}" == "${EXPECTED_PARAM}" ]
+ then
+ return 0
+ fi
+ done
+ return 1
+}
+
+function get_next_boot_args()
+(
+ GRUB_CMDLINE_LINUX=""
+ GRUB_CMDLINE_LINUX_DEFAULT=""
+
+ if source /etc/default/grub > /dev/null
+ then
+ echo "${GRUB_CMDLINE_LINUX} ${GRUB_CMDLINE_LINUX_DEFAULT}"
+ fi
+)
+
+function get_current_boot_args()
+{
+ local CURRENT_BOOT_ARGS
+ local CMDLINE_PATTERN='BOOT_IMAGE='
+ CURRENT_BOOT_ARGS=$(grep ${CMDLINE_PATTERN} /proc/cmdline)
+ CURRENT_BOOT_ARGS=${CURRENT_BOOT_ARGS#${CMDLINE_PATTERN}}
+ CURRENT_BOOT_ARGS=${CURRENT_BOOT_ARGS#\"}
+ CURRENT_BOOT_ARGS=${CURRENT_BOOT_ARGS%\"}
+ echo "${CURRENT_BOOT_ARGS}"
+}
+
+function has_kernel_parameters_changed()
+{
+ local CURRENT_BOOT_ARGS
+ CURRENT_BOOT_ARGS=$(get_current_boot_args)
+ local NEXT_BOOT_ARG
+ for NEXT_BOOT_ARG in $(get_next_boot_args)
+ do
+ if ! parameter_exists_in_list "${NEXT_BOOT_ARG}" "${CURRENT_BOOT_ARGS}"
+ then
+ log_info "kernel parameter <${NEXT_BOOT_ARG}> does not exist in [${CURRENT_BOOT_ARGS}]"
+ return 0
+ fi
+ done
+ return 1
+}
+
+function reboot_host()
+{
+ local DELAY=5
+ log_info "Reboot the host in ${DELAY} seconds to apply the kernel parameter changes"
+ sleep ${DELAY}
+ systemctl reboot
+}
--- /dev/null
+ _____ _ _ _ _ _ ______ _ _ _ __
+|_ _| | | | | | | | (_) | ___| (_) | | | _ / /
+ | | _ __ ___| |_ __ _| | | __ _| |_ _ ___ _ __ | |_ __ _ _| | ___ __| | (_) | |
+ | || '_ \/ __| __/ _` | | |/ _` | __| |/ _ \| '_ \ | _/ _` | | |/ _ \/ _` | | |
+ _| || | | \__ \ || (_| | | | (_| | |_| | (_) | | | | | || (_| | | | __/ (_| | _ | |
+ \___/_| |_|___/\__\__,_|_|_|\__,_|\__|_|\___/|_| |_| \_| \__,_|_|_|\___|\__,_| (_) | |
+ \_\
+
--- /dev/null
+ _____ _ _ _ _ _ _____ _ _ __
+|_ _| | | | | | | | (_) / ___| | | | | _ \ \
+ | | _ __ ___| |_ __ _| | | __ _| |_ _ ___ _ __ \ `--. _ _ ___ ___ ___ ___ __| | ___ __| | (_) | |
+ | || '_ \/ __| __/ _` | | |/ _` | __| |/ _ \| '_ \ `--. \ | | |/ __/ __/ _ \/ _ \/ _` |/ _ \/ _` | | |
+ _| || | | \__ \ || (_| | | | (_| | |_| | (_) | | | | /\__/ / |_| | (_| (_| __/ __/ (_| | __/ (_| | _ | |
+ \___/_| |_|___/\__\__,_|_|_|\__,_|\__|_|\___/|_| |_| \____/ \__,_|\___\___\___|\___|\__,_|\___|\__,_| (_) | |
+ /_/
+
--- /dev/null
+#!/bin/bash
+
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+if [ -f /opt/cmframework/inventory.data ]; then
+ cat /opt/cmframework/inventory.data
+else
+ exec /usr/local/bin/cmcli ansible-inventory
+fi
--- /dev/null
+#!/bin/bash
+
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# Collection of logging variables and functions for the bootstrap.sh
+
+set -o nounset
+
+#
+# Variables
+#
+
+LOG_BASE_DIR="${LOG_BASE_DIR:-/srv/deployment/log}"
+BOOTSTRAP_LOG="${BOOTSTRAP_LOG:-${LOG_BASE_DIR}/bootstrap.log}"
+CM_LOG=${LOG_BASE_DIR}/cm.log
+
+#
+# Create the log dir
+#
+mkdir -p ${LOG_BASE_DIR}
+
+#
+# Functions
+#
+
+function log()
+{
+ local priority=$1
+ shift
+ local message=$1
+
+ local caller_function=""
+ if [ -z ${FUNCNAME[2]+x} ]; then
+ caller_function="${FUNCNAME[1]}"
+ else
+ caller_function="${FUNCNAME[2]}"
+ fi
+
+ echo "$(date) ($priority) ${caller_function} ${message}"
+ echo "$(date) ($priority) ${caller_function} ${message}" >> $BOOTSTRAP_LOG
+}
+
+function log_info()
+{
+ log info "$@"
+}
+
+function log_error()
+{
+ log error "$@"
+}
+
+function log_installation_success()
+{
+ log_info "Installation complete, Installation Succeeded :)"
+}
+
+function log_installation_failure()
+{
+ log_error "Installation complete, Installation Failed :("
+}
--- /dev/null
+# Redis configuration file example.
+#
+# Note that in order to read the configuration file, Redis must be
+# started with the file path as first argument:
+#
+# ./redis-server /path/to/redis.conf
+
+# Note on units: when memory size is needed, it is possible to specify
+# it in the usual form of 1k 5GB 4M and so forth:
+#
+# 1k => 1000 bytes
+# 1kb => 1024 bytes
+# 1m => 1000000 bytes
+# 1mb => 1024*1024 bytes
+# 1g => 1000000000 bytes
+# 1gb => 1024*1024*1024 bytes
+#
+# units are case insensitive so 1GB 1Gb 1gB are all the same.
+
+################################## INCLUDES ###################################
+
+# Include one or more other config files here. This is useful if you
+# have a standard template that goes to all Redis servers but also need
+# to customize a few per-server settings. Include files can include
+# other files, so use this wisely.
+#
+# Notice option "include" won't be rewritten by command "CONFIG REWRITE"
+# from admin or Redis Sentinel. Since Redis always uses the last processed
+# line as value of a configuration directive, you'd better put includes
+# at the beginning of this file to avoid overwriting config change at runtime.
+#
+# If instead you are interested in using includes to override configuration
+# options, it is better to use include as the last line.
+#
+# include /path/to/local.conf
+# include /path/to/other.conf
+
+################################## NETWORK #####################################
+
+# By default, if no "bind" configuration directive is specified, Redis listens
+# for connections from all the network interfaces available on the server.
+# It is possible to listen to just one or multiple selected interfaces using
+# the "bind" configuration directive, followed by one or more IP addresses.
+#
+# Examples:
+#
+# bind 192.168.1.100 10.0.0.1
+# bind 127.0.0.1 ::1
+#
+# ~~~ WARNING ~~~ If the computer running Redis is directly exposed to the
+# internet, binding to all the interfaces is dangerous and will expose the
+# instance to everybody on the internet. So by default we uncomment the
+# following bind directive, that will force Redis to listen only into
+# the IPv4 lookback interface address (this means Redis will be able to
+# accept connections only from clients running into the same computer it
+# is running).
+#
+# IF YOU ARE SURE YOU WANT YOUR INSTANCE TO LISTEN TO ALL THE INTERFACES
+# JUST COMMENT THE FOLLOWING LINE.
+# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+bind 127.0.0.1
+
+# Protected mode is a layer of security protection, in order to avoid that
+# Redis instances left open on the internet are accessed and exploited.
+#
+# When protected mode is on and if:
+#
+# 1) The server is not binding explicitly to a set of addresses using the
+# "bind" directive.
+# 2) No password is configured.
+#
+# The server only accepts connections from clients connecting from the
+# IPv4 and IPv6 loopback addresses 127.0.0.1 and ::1, and from Unix domain
+# sockets.
+#
+# By default protected mode is enabled. You should disable it only if
+# you are sure you want clients from other hosts to connect to Redis
+# even if no authentication is configured, nor a specific set of interfaces
+# are explicitly listed using the "bind" directive.
+protected-mode yes
+
+# Accept connections on the specified port, default is 6379 (IANA #815344).
+# If port 0 is specified Redis will not listen on a TCP socket.
+port 6379
+
+# TCP listen() backlog.
+#
+# In high requests-per-second environments you need an high backlog in order
+# to avoid slow clients connections issues. Note that the Linux kernel
+# will silently truncate it to the value of /proc/sys/net/core/somaxconn so
+# make sure to raise both the value of somaxconn and tcp_max_syn_backlog
+# in order to get the desired effect.
+tcp-backlog 511
+
+# Unix socket.
+#
+# Specify the path for the Unix socket that will be used to listen for
+# incoming connections. There is no default, so Redis will not listen
+# on a unix socket when not specified.
+#
+unixsocket /run/redis/redis.sock
+unixsocketperm 700
+
+# Close the connection after a client is idle for N seconds (0 to disable)
+timeout 0
+
+# TCP keepalive.
+#
+# If non-zero, use SO_KEEPALIVE to send TCP ACKs to clients in absence
+# of communication. This is useful for two reasons:
+#
+# 1) Detect dead peers.
+# 2) Take the connection alive from the point of view of network
+# equipment in the middle.
+#
+# On Linux, the specified value (in seconds) is the period used to send ACKs.
+# Note that to close the connection the double of the time is needed.
+# On other kernels the period depends on the kernel configuration.
+#
+# A reasonable value for this option is 300 seconds, which is the new
+# Redis default starting with Redis 3.2.1.
+tcp-keepalive 300
+
+################################# GENERAL #####################################
+
+# By default Redis does not run as a daemon. Use 'yes' if you need it.
+# Note that Redis will write a pid file in /var/run/redis.pid when daemonized.
+daemonize no
+
+# If you run Redis from upstart or systemd, Redis can interact with your
+# supervision tree. Options:
+# supervised no - no supervision interaction
+# supervised upstart - signal upstart by putting Redis into SIGSTOP mode
+# supervised systemd - signal systemd by writing READY=1 to $NOTIFY_SOCKET
+# supervised auto - detect upstart or systemd method based on
+# UPSTART_JOB or NOTIFY_SOCKET environment variables
+# Note: these supervision methods only signal "process is ready."
+# They do not enable continuous liveness pings back to your supervisor.
+supervised no
+
+# If a pid file is specified, Redis writes it where specified at startup
+# and removes it at exit.
+#
+# When the server runs non daemonized, no pid file is created if none is
+# specified in the configuration. When the server is daemonized, the pid file
+# is used even if not specified, defaulting to "/var/run/redis.pid".
+#
+# Creating a pid file is best effort: if Redis is not able to create it
+# nothing bad happens, the server will start and run normally.
+pidfile /var/run/redis/redis.pid
+
+# Specify the server verbosity level.
+# This can be one of:
+# debug (a lot of information, useful for development/testing)
+# verbose (many rarely useful info, but not a mess like the debug level)
+# notice (moderately verbose, what you want in production probably)
+# warning (only very important / critical messages are logged)
+loglevel notice
+
+# Specify the log file name. Also the empty string can be used to force
+# Redis to log on the standard output. Note that if you use standard
+# output for logging but daemonize, logs will be sent to /dev/null
+logfile /srv/deployment/log/redis.log
+
+# To enable logging to the system logger, just set 'syslog-enabled' to yes,
+# and optionally update the other syslog parameters to suit your needs.
+# syslog-enabled no
+
+# Specify the syslog identity.
+# syslog-ident redis
+
+# Specify the syslog facility. Must be USER or between LOCAL0-LOCAL7.
+# syslog-facility local0
+
+# Set the number of databases. The default database is DB 0, you can select
+# a different one on a per-connection basis using SELECT <dbid> where
+# dbid is a number between 0 and 'databases'-1
+databases 16
+
+################################ SNAPSHOTTING ################################
+#
+# Save the DB on disk:
+#
+# save <seconds> <changes>
+#
+# Will save the DB if both the given number of seconds and the given
+# number of write operations against the DB occurred.
+#
+# In the example below the behaviour will be to save:
+# after 900 sec (15 min) if at least 1 key changed
+# after 300 sec (5 min) if at least 10 keys changed
+# after 60 sec if at least 10000 keys changed
+#
+# Note: you can disable saving completely by commenting out all "save" lines.
+#
+# It is also possible to remove all the previously configured save
+# points by adding a save directive with a single empty string argument
+# like in the following example:
+#
+# save ""
+
+save 900 1
+save 300 10
+save 60 10000
+
+# By default Redis will stop accepting writes if RDB snapshots are enabled
+# (at least one save point) and the latest background save failed.
+# This will make the user aware (in a hard way) that data is not persisting
+# on disk properly, otherwise chances are that no one will notice and some
+# disaster will happen.
+#
+# If the background saving process will start working again Redis will
+# automatically allow writes again.
+#
+# However if you have setup your proper monitoring of the Redis server
+# and persistence, you may want to disable this feature so that Redis will
+# continue to work as usual even if there are problems with disk,
+# permissions, and so forth.
+stop-writes-on-bgsave-error yes
+
+# Compress string objects using LZF when dump .rdb databases?
+# For default that's set to 'yes' as it's almost always a win.
+# If you want to save some CPU in the saving child set it to 'no' but
+# the dataset will likely be bigger if you have compressible values or keys.
+rdbcompression yes
+
+# Since version 5 of RDB a CRC64 checksum is placed at the end of the file.
+# This makes the format more resistant to corruption but there is a performance
+# hit to pay (around 10%) when saving and loading RDB files, so you can disable it
+# for maximum performances.
+#
+# RDB files created with checksum disabled have a checksum of zero that will
+# tell the loading code to skip the check.
+rdbchecksum yes
+
+# The filename where to dump the DB
+dbfilename dump.rdb
+
+# The working directory.
+#
+# The DB will be written inside this directory, with the filename specified
+# above using the 'dbfilename' configuration directive.
+#
+# The Append Only File will also be created inside this directory.
+#
+# Note that you must specify a directory here, not a file name.
+dir /var/lib/redis
+
+################################# REPLICATION #################################
+
+# Master-Slave replication. Use slaveof to make a Redis instance a copy of
+# another Redis server. A few things to understand ASAP about Redis replication.
+#
+# 1) Redis replication is asynchronous, but you can configure a master to
+# stop accepting writes if it appears to be not connected with at least
+# a given number of slaves.
+# 2) Redis slaves are able to perform a partial resynchronization with the
+# master if the replication link is lost for a relatively small amount of
+# time. You may want to configure the replication backlog size (see the next
+# sections of this file) with a sensible value depending on your needs.
+# 3) Replication is automatic and does not need user intervention. After a
+# network partition slaves automatically try to reconnect to masters
+# and resynchronize with them.
+#
+# slaveof <masterip> <masterport>
+
+# If the master is password protected (using the "requirepass" configuration
+# directive below) it is possible to tell the slave to authenticate before
+# starting the replication synchronization process, otherwise the master will
+# refuse the slave request.
+#
+# masterauth <master-password>
+
+# When a slave loses its connection with the master, or when the replication
+# is still in progress, the slave can act in two different ways:
+#
+# 1) if slave-serve-stale-data is set to 'yes' (the default) the slave will
+# still reply to client requests, possibly with out of date data, or the
+# data set may just be empty if this is the first synchronization.
+#
+# 2) if slave-serve-stale-data is set to 'no' the slave will reply with
+# an error "SYNC with master in progress" to all the kind of commands
+# but to INFO and SLAVEOF.
+#
+slave-serve-stale-data yes
+
+# You can configure a slave instance to accept writes or not. Writing against
+# a slave instance may be useful to store some ephemeral data (because data
+# written on a slave will be easily deleted after resync with the master) but
+# may also cause problems if clients are writing to it because of a
+# misconfiguration.
+#
+# Since Redis 2.6 by default slaves are read-only.
+#
+# Note: read only slaves are not designed to be exposed to untrusted clients
+# on the internet. It's just a protection layer against misuse of the instance.
+# Still a read only slave exports by default all the administrative commands
+# such as CONFIG, DEBUG, and so forth. To a limited extent you can improve
+# security of read only slaves using 'rename-command' to shadow all the
+# administrative / dangerous commands.
+slave-read-only yes
+
+# Replication SYNC strategy: disk or socket.
+#
+# -------------------------------------------------------
+# WARNING: DISKLESS REPLICATION IS EXPERIMENTAL CURRENTLY
+# -------------------------------------------------------
+#
+# New slaves and reconnecting slaves that are not able to continue the replication
+# process just receiving differences, need to do what is called a "full
+# synchronization". An RDB file is transmitted from the master to the slaves.
+# The transmission can happen in two different ways:
+#
+# 1) Disk-backed: The Redis master creates a new process that writes the RDB
+# file on disk. Later the file is transferred by the parent
+# process to the slaves incrementally.
+# 2) Diskless: The Redis master creates a new process that directly writes the
+# RDB file to slave sockets, without touching the disk at all.
+#
+# With disk-backed replication, while the RDB file is generated, more slaves
+# can be queued and served with the RDB file as soon as the current child producing
+# the RDB file finishes its work. With diskless replication instead once
+# the transfer starts, new slaves arriving will be queued and a new transfer
+# will start when the current one terminates.
+#
+# When diskless replication is used, the master waits a configurable amount of
+# time (in seconds) before starting the transfer in the hope that multiple slaves
+# will arrive and the transfer can be parallelized.
+#
+# With slow disks and fast (large bandwidth) networks, diskless replication
+# works better.
+repl-diskless-sync no
+
+# When diskless replication is enabled, it is possible to configure the delay
+# the server waits in order to spawn the child that transfers the RDB via socket
+# to the slaves.
+#
+# This is important since once the transfer starts, it is not possible to serve
+# new slaves arriving, that will be queued for the next RDB transfer, so the server
+# waits a delay in order to let more slaves arrive.
+#
+# The delay is specified in seconds, and by default is 5 seconds. To disable
+# it entirely just set it to 0 seconds and the transfer will start ASAP.
+repl-diskless-sync-delay 5
+
+# Slaves send PINGs to server in a predefined interval. It's possible to change
+# this interval with the repl_ping_slave_period option. The default value is 10
+# seconds.
+#
+# repl-ping-slave-period 10
+
+# The following option sets the replication timeout for:
+#
+# 1) Bulk transfer I/O during SYNC, from the point of view of slave.
+# 2) Master timeout from the point of view of slaves (data, pings).
+# 3) Slave timeout from the point of view of masters (REPLCONF ACK pings).
+#
+# It is important to make sure that this value is greater than the value
+# specified for repl-ping-slave-period otherwise a timeout will be detected
+# every time there is low traffic between the master and the slave.
+#
+# repl-timeout 60
+
+# Disable TCP_NODELAY on the slave socket after SYNC?
+#
+# If you select "yes" Redis will use a smaller number of TCP packets and
+# less bandwidth to send data to slaves. But this can add a delay for
+# the data to appear on the slave side, up to 40 milliseconds with
+# Linux kernels using a default configuration.
+#
+# If you select "no" the delay for data to appear on the slave side will
+# be reduced but more bandwidth will be used for replication.
+#
+# By default we optimize for low latency, but in very high traffic conditions
+# or when the master and slaves are many hops away, turning this to "yes" may
+# be a good idea.
+repl-disable-tcp-nodelay no
+
+# Set the replication backlog size. The backlog is a buffer that accumulates
+# slave data when slaves are disconnected for some time, so that when a slave
+# wants to reconnect again, often a full resync is not needed, but a partial
+# resync is enough, just passing the portion of data the slave missed while
+# disconnected.
+#
+# The bigger the replication backlog, the longer the time the slave can be
+# disconnected and later be able to perform a partial resynchronization.
+#
+# The backlog is only allocated once there is at least a slave connected.
+#
+# repl-backlog-size 1mb
+
+# After a master has no longer connected slaves for some time, the backlog
+# will be freed. The following option configures the amount of seconds that
+# need to elapse, starting from the time the last slave disconnected, for
+# the backlog buffer to be freed.
+#
+# A value of 0 means to never release the backlog.
+#
+# repl-backlog-ttl 3600
+
+# The slave priority is an integer number published by Redis in the INFO output.
+# It is used by Redis Sentinel in order to select a slave to promote into a
+# master if the master is no longer working correctly.
+#
+# A slave with a low priority number is considered better for promotion, so
+# for instance if there are three slaves with priority 10, 100, 25 Sentinel will
+# pick the one with priority 10, that is the lowest.
+#
+# However a special priority of 0 marks the slave as not able to perform the
+# role of master, so a slave with priority of 0 will never be selected by
+# Redis Sentinel for promotion.
+#
+# By default the priority is 100.
+slave-priority 100
+
+# It is possible for a master to stop accepting writes if there are less than
+# N slaves connected, having a lag less or equal than M seconds.
+#
+# The N slaves need to be in "online" state.
+#
+# The lag in seconds, that must be <= the specified value, is calculated from
+# the last ping received from the slave, that is usually sent every second.
+#
+# This option does not GUARANTEE that N replicas will accept the write, but
+# will limit the window of exposure for lost writes in case not enough slaves
+# are available, to the specified number of seconds.
+#
+# For example to require at least 3 slaves with a lag <= 10 seconds use:
+#
+# min-slaves-to-write 3
+# min-slaves-max-lag 10
+#
+# Setting one or the other to 0 disables the feature.
+#
+# By default min-slaves-to-write is set to 0 (feature disabled) and
+# min-slaves-max-lag is set to 10.
+
+# A Redis master is able to list the address and port of the attached
+# slaves in different ways. For example the "INFO replication" section
+# offers this information, which is used, among other tools, by
+# Redis Sentinel in order to discover slave instances.
+# Another place where this info is available is in the output of the
+# "ROLE" command of a masteer.
+#
+# The listed IP and address normally reported by a slave is obtained
+# in the following way:
+#
+# IP: The address is auto detected by checking the peer address
+# of the socket used by the slave to connect with the master.
+#
+# Port: The port is communicated by the slave during the replication
+# handshake, and is normally the port that the slave is using to
+# list for connections.
+#
+# However when port forwarding or Network Address Translation (NAT) is
+# used, the slave may be actually reachable via different IP and port
+# pairs. The following two options can be used by a slave in order to
+# report to its master a specific set of IP and port, so that both INFO
+# and ROLE will report those values.
+#
+# There is no need to use both the options if you need to override just
+# the port or the IP address.
+#
+# slave-announce-ip 5.5.5.5
+# slave-announce-port 1234
+
+################################## SECURITY ###################################
+
+# Require clients to issue AUTH <PASSWORD> before processing any other
+# commands. This might be useful in environments in which you do not trust
+# others with access to the host running redis-server.
+#
+# This should stay commented out for backward compatibility and because most
+# people do not need auth (e.g. they run their own servers).
+#
+# Warning: since Redis is pretty fast an outside user can try up to
+# 150k passwords per second against a good box. This means that you should
+# use a very strong password otherwise it will be very easy to break.
+#
+# requirepass foobared
+
+# Command renaming.
+#
+# It is possible to change the name of dangerous commands in a shared
+# environment. For instance the CONFIG command may be renamed into something
+# hard to guess so that it will still be available for internal-use tools
+# but not available for general clients.
+#
+# Example:
+#
+# rename-command CONFIG b840fc02d524045429941cc15f59e41cb7be6c52
+#
+# It is also possible to completely kill a command by renaming it into
+# an empty string:
+#
+# rename-command CONFIG ""
+#
+# Please note that changing the name of commands that are logged into the
+# AOF file or transmitted to slaves may cause problems.
+
+################################### LIMITS ####################################
+
+# Set the max number of connected clients at the same time. By default
+# this limit is set to 10000 clients, however if the Redis server is not
+# able to configure the process file limit to allow for the specified limit
+# the max number of allowed clients is set to the current file limit
+# minus 32 (as Redis reserves a few file descriptors for internal uses).
+#
+# Once the limit is reached Redis will close all the new connections sending
+# an error 'max number of clients reached'.
+#
+# maxclients 10000
+
+# Don't use more memory than the specified amount of bytes.
+# When the memory limit is reached Redis will try to remove keys
+# according to the eviction policy selected (see maxmemory-policy).
+#
+# If Redis can't remove keys according to the policy, or if the policy is
+# set to 'noeviction', Redis will start to reply with errors to commands
+# that would use more memory, like SET, LPUSH, and so on, and will continue
+# to reply to read-only commands like GET.
+#
+# This option is usually useful when using Redis as an LRU cache, or to set
+# a hard memory limit for an instance (using the 'noeviction' policy).
+#
+# WARNING: If you have slaves attached to an instance with maxmemory on,
+# the size of the output buffers needed to feed the slaves are subtracted
+# from the used memory count, so that network problems / resyncs will
+# not trigger a loop where keys are evicted, and in turn the output
+# buffer of slaves is full with DELs of keys evicted triggering the deletion
+# of more keys, and so forth until the database is completely emptied.
+#
+# In short... if you have slaves attached it is suggested that you set a lower
+# limit for maxmemory so that there is some free RAM on the system for slave
+# output buffers (but this is not needed if the policy is 'noeviction').
+#
+# maxmemory <bytes>
+
+# MAXMEMORY POLICY: how Redis will select what to remove when maxmemory
+# is reached. You can select among five behaviors:
+#
+# volatile-lru -> remove the key with an expire set using an LRU algorithm
+# allkeys-lru -> remove any key according to the LRU algorithm
+# volatile-random -> remove a random key with an expire set
+# allkeys-random -> remove a random key, any key
+# volatile-ttl -> remove the key with the nearest expire time (minor TTL)
+# noeviction -> don't expire at all, just return an error on write operations
+#
+# Note: with any of the above policies, Redis will return an error on write
+# operations, when there are no suitable keys for eviction.
+#
+# At the date of writing these commands are: set setnx setex append
+# incr decr rpush lpush rpushx lpushx linsert lset rpoplpush sadd
+# sinter sinterstore sunion sunionstore sdiff sdiffstore zadd zincrby
+# zunionstore zinterstore hset hsetnx hmset hincrby incrby decrby
+# getset mset msetnx exec sort
+#
+# The default is:
+#
+# maxmemory-policy noeviction
+
+# LRU and minimal TTL algorithms are not precise algorithms but approximated
+# algorithms (in order to save memory), so you can tune it for speed or
+# accuracy. For default Redis will check five keys and pick the one that was
+# used less recently, you can change the sample size using the following
+# configuration directive.
+#
+# The default of 5 produces good enough results. 10 Approximates very closely
+# true LRU but costs a bit more CPU. 3 is very fast but not very accurate.
+#
+# maxmemory-samples 5
+
+############################## APPEND ONLY MODE ###############################
+
+# By default Redis asynchronously dumps the dataset on disk. This mode is
+# good enough in many applications, but an issue with the Redis process or
+# a power outage may result into a few minutes of writes lost (depending on
+# the configured save points).
+#
+# The Append Only File is an alternative persistence mode that provides
+# much better durability. For instance using the default data fsync policy
+# (see later in the config file) Redis can lose just one second of writes in a
+# dramatic event like a server power outage, or a single write if something
+# wrong with the Redis process itself happens, but the operating system is
+# still running correctly.
+#
+# AOF and RDB persistence can be enabled at the same time without problems.
+# If the AOF is enabled on startup Redis will load the AOF, that is the file
+# with the better durability guarantees.
+#
+# Please check http://redis.io/topics/persistence for more information.
+
+appendonly no
+
+# The name of the append only file (default: "appendonly.aof")
+
+appendfilename "appendonly.aof"
+
+# The fsync() call tells the Operating System to actually write data on disk
+# instead of waiting for more data in the output buffer. Some OS will really flush
+# data on disk, some other OS will just try to do it ASAP.
+#
+# Redis supports three different modes:
+#
+# no: don't fsync, just let the OS flush the data when it wants. Faster.
+# always: fsync after every write to the append only log. Slow, Safest.
+# everysec: fsync only one time every second. Compromise.
+#
+# The default is "everysec", as that's usually the right compromise between
+# speed and data safety. It's up to you to understand if you can relax this to
+# "no" that will let the operating system flush the output buffer when
+# it wants, for better performances (but if you can live with the idea of
+# some data loss consider the default persistence mode that's snapshotting),
+# or on the contrary, use "always" that's very slow but a bit safer than
+# everysec.
+#
+# More details please check the following article:
+# http://antirez.com/post/redis-persistence-demystified.html
+#
+# If unsure, use "everysec".
+
+# appendfsync always
+appendfsync everysec
+# appendfsync no
+
+# When the AOF fsync policy is set to always or everysec, and a background
+# saving process (a background save or AOF log background rewriting) is
+# performing a lot of I/O against the disk, in some Linux configurations
+# Redis may block too long on the fsync() call. Note that there is no fix for
+# this currently, as even performing fsync in a different thread will block
+# our synchronous write(2) call.
+#
+# In order to mitigate this problem it's possible to use the following option
+# that will prevent fsync() from being called in the main process while a
+# BGSAVE or BGREWRITEAOF is in progress.
+#
+# This means that while another child is saving, the durability of Redis is
+# the same as "appendfsync none". In practical terms, this means that it is
+# possible to lose up to 30 seconds of log in the worst scenario (with the
+# default Linux settings).
+#
+# If you have latency problems turn this to "yes". Otherwise leave it as
+# "no" that is the safest pick from the point of view of durability.
+
+no-appendfsync-on-rewrite no
+
+# Automatic rewrite of the append only file.
+# Redis is able to automatically rewrite the log file implicitly calling
+# BGREWRITEAOF when the AOF log size grows by the specified percentage.
+#
+# This is how it works: Redis remembers the size of the AOF file after the
+# latest rewrite (if no rewrite has happened since the restart, the size of
+# the AOF at startup is used).
+#
+# This base size is compared to the current size. If the current size is
+# bigger than the specified percentage, the rewrite is triggered. Also
+# you need to specify a minimal size for the AOF file to be rewritten, this
+# is useful to avoid rewriting the AOF file even if the percentage increase
+# is reached but it is still pretty small.
+#
+# Specify a percentage of zero in order to disable the automatic AOF
+# rewrite feature.
+
+auto-aof-rewrite-percentage 100
+auto-aof-rewrite-min-size 64mb
+
+# An AOF file may be found to be truncated at the end during the Redis
+# startup process, when the AOF data gets loaded back into memory.
+# This may happen when the system where Redis is running
+# crashes, especially when an ext4 filesystem is mounted without the
+# data=ordered option (however this can't happen when Redis itself
+# crashes or aborts but the operating system still works correctly).
+#
+# Redis can either exit with an error when this happens, or load as much
+# data as possible (the default now) and start if the AOF file is found
+# to be truncated at the end. The following option controls this behavior.
+#
+# If aof-load-truncated is set to yes, a truncated AOF file is loaded and
+# the Redis server starts emitting a log to inform the user of the event.
+# Otherwise if the option is set to no, the server aborts with an error
+# and refuses to start. When the option is set to no, the user requires
+# to fix the AOF file using the "redis-check-aof" utility before to restart
+# the server.
+#
+# Note that if the AOF file will be found to be corrupted in the middle
+# the server will still exit with an error. This option only applies when
+# Redis will try to read more data from the AOF file but not enough bytes
+# will be found.
+aof-load-truncated yes
+
+################################ LUA SCRIPTING ###############################
+
+# Max execution time of a Lua script in milliseconds.
+#
+# If the maximum execution time is reached Redis will log that a script is
+# still in execution after the maximum allowed time and will start to
+# reply to queries with an error.
+#
+# When a long running script exceeds the maximum execution time only the
+# SCRIPT KILL and SHUTDOWN NOSAVE commands are available. The first can be
+# used to stop a script that did not yet called write commands. The second
+# is the only way to shut down the server in the case a write command was
+# already issued by the script but the user doesn't want to wait for the natural
+# termination of the script.
+#
+# Set it to 0 or a negative value for unlimited execution without warnings.
+lua-time-limit 5000
+
+################################ REDIS CLUSTER ###############################
+#
+# ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+# WARNING EXPERIMENTAL: Redis Cluster is considered to be stable code, however
+# in order to mark it as "mature" we need to wait for a non trivial percentage
+# of users to deploy it in production.
+# ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+#
+# Normal Redis instances can't be part of a Redis Cluster; only nodes that are
+# started as cluster nodes can. In order to start a Redis instance as a
+# cluster node enable the cluster support uncommenting the following:
+#
+# cluster-enabled yes
+
+# Every cluster node has a cluster configuration file. This file is not
+# intended to be edited by hand. It is created and updated by Redis nodes.
+# Every Redis Cluster node requires a different cluster configuration file.
+# Make sure that instances running in the same system do not have
+# overlapping cluster configuration file names.
+#
+# cluster-config-file nodes-6379.conf
+
+# Cluster node timeout is the amount of milliseconds a node must be unreachable
+# for it to be considered in failure state.
+# Most other internal time limits are multiple of the node timeout.
+#
+# cluster-node-timeout 15000
+
+# A slave of a failing master will avoid to start a failover if its data
+# looks too old.
+#
+# There is no simple way for a slave to actually have a exact measure of
+# its "data age", so the following two checks are performed:
+#
+# 1) If there are multiple slaves able to failover, they exchange messages
+# in order to try to give an advantage to the slave with the best
+# replication offset (more data from the master processed).
+# Slaves will try to get their rank by offset, and apply to the start
+# of the failover a delay proportional to their rank.
+#
+# 2) Every single slave computes the time of the last interaction with
+# its master. This can be the last ping or command received (if the master
+# is still in the "connected" state), or the time that elapsed since the
+# disconnection with the master (if the replication link is currently down).
+# If the last interaction is too old, the slave will not try to failover
+# at all.
+#
+# The point "2" can be tuned by user. Specifically a slave will not perform
+# the failover if, since the last interaction with the master, the time
+# elapsed is greater than:
+#
+# (node-timeout * slave-validity-factor) + repl-ping-slave-period
+#
+# So for example if node-timeout is 30 seconds, and the slave-validity-factor
+# is 10, and assuming a default repl-ping-slave-period of 10 seconds, the
+# slave will not try to failover if it was not able to talk with the master
+# for longer than 310 seconds.
+#
+# A large slave-validity-factor may allow slaves with too old data to failover
+# a master, while a too small value may prevent the cluster from being able to
+# elect a slave at all.
+#
+# For maximum availability, it is possible to set the slave-validity-factor
+# to a value of 0, which means, that slaves will always try to failover the
+# master regardless of the last time they interacted with the master.
+# (However they'll always try to apply a delay proportional to their
+# offset rank).
+#
+# Zero is the only value able to guarantee that when all the partitions heal
+# the cluster will always be able to continue.
+#
+# cluster-slave-validity-factor 10
+
+# Cluster slaves are able to migrate to orphaned masters, that are masters
+# that are left without working slaves. This improves the cluster ability
+# to resist to failures as otherwise an orphaned master can't be failed over
+# in case of failure if it has no working slaves.
+#
+# Slaves migrate to orphaned masters only if there are still at least a
+# given number of other working slaves for their old master. This number
+# is the "migration barrier". A migration barrier of 1 means that a slave
+# will migrate only if there is at least 1 other working slave for its master
+# and so forth. It usually reflects the number of slaves you want for every
+# master in your cluster.
+#
+# Default is 1 (slaves migrate only if their masters remain with at least
+# one slave). To disable migration just set it to a very large value.
+# A value of 0 can be set but is useful only for debugging and dangerous
+# in production.
+#
+# cluster-migration-barrier 1
+
+# By default Redis Cluster nodes stop accepting queries if they detect there
+# is at least an hash slot uncovered (no available node is serving it).
+# This way if the cluster is partially down (for example a range of hash slots
+# are no longer covered) all the cluster becomes, eventually, unavailable.
+# It automatically returns available as soon as all the slots are covered again.
+#
+# However sometimes you want the subset of the cluster which is working,
+# to continue to accept queries for the part of the key space that is still
+# covered. In order to do so, just set the cluster-require-full-coverage
+# option to no.
+#
+# cluster-require-full-coverage yes
+
+# In order to setup your cluster make sure to read the documentation
+# available at http://redis.io web site.
+
+################################## SLOW LOG ###################################
+
+# The Redis Slow Log is a system to log queries that exceeded a specified
+# execution time. The execution time does not include the I/O operations
+# like talking with the client, sending the reply and so forth,
+# but just the time needed to actually execute the command (this is the only
+# stage of command execution where the thread is blocked and can not serve
+# other requests in the meantime).
+#
+# You can configure the slow log with two parameters: one tells Redis
+# what is the execution time, in microseconds, to exceed in order for the
+# command to get logged, and the other parameter is the length of the
+# slow log. When a new command is logged the oldest one is removed from the
+# queue of logged commands.
+
+# The following time is expressed in microseconds, so 1000000 is equivalent
+# to one second. Note that a negative number disables the slow log, while
+# a value of zero forces the logging of every command.
+slowlog-log-slower-than 10000
+
+# There is no limit to this length. Just be aware that it will consume memory.
+# You can reclaim memory used by the slow log with SLOWLOG RESET.
+slowlog-max-len 128
+
+################################ LATENCY MONITOR ##############################
+
+# The Redis latency monitoring subsystem samples different operations
+# at runtime in order to collect data related to possible sources of
+# latency of a Redis instance.
+#
+# Via the LATENCY command this information is available to the user that can
+# print graphs and obtain reports.
+#
+# The system only logs operations that were performed in a time equal or
+# greater than the amount of milliseconds specified via the
+# latency-monitor-threshold configuration directive. When its value is set
+# to zero, the latency monitor is turned off.
+#
+# By default latency monitoring is disabled since it is mostly not needed
+# if you don't have latency issues, and collecting data has a performance
+# impact, that while very small, can be measured under big load. Latency
+# monitoring can easily be enabled at runtime using the command
+# "CONFIG SET latency-monitor-threshold <milliseconds>" if needed.
+latency-monitor-threshold 0
+
+############################# EVENT NOTIFICATION ##############################
+
+# Redis can notify Pub/Sub clients about events happening in the key space.
+# This feature is documented at http://redis.io/topics/notifications
+#
+# For instance if keyspace events notification is enabled, and a client
+# performs a DEL operation on key "foo" stored in the Database 0, two
+# messages will be published via Pub/Sub:
+#
+# PUBLISH __keyspace@0__:foo del
+# PUBLISH __keyevent@0__:del foo
+#
+# It is possible to select the events that Redis will notify among a set
+# of classes. Every class is identified by a single character:
+#
+# K Keyspace events, published with __keyspace@<db>__ prefix.
+# E Keyevent events, published with __keyevent@<db>__ prefix.
+# g Generic commands (non-type specific) like DEL, EXPIRE, RENAME, ...
+# $ String commands
+# l List commands
+# s Set commands
+# h Hash commands
+# z Sorted set commands
+# x Expired events (events generated every time a key expires)
+# e Evicted events (events generated when a key is evicted for maxmemory)
+# A Alias for g$lshzxe, so that the "AKE" string means all the events.
+#
+# The "notify-keyspace-events" takes as argument a string that is composed
+# of zero or multiple characters. The empty string means that notifications
+# are disabled.
+#
+# Example: to enable list and generic events, from the point of view of the
+# event name, use:
+#
+# notify-keyspace-events Elg
+#
+# Example 2: to get the stream of the expired keys subscribing to channel
+# name __keyevent@0__:expired use:
+#
+# notify-keyspace-events Ex
+#
+# By default all notifications are disabled because most users don't need
+# this feature and the feature has some overhead. Note that if you don't
+# specify at least one of K or E, no events will be delivered.
+notify-keyspace-events ""
+
+############################### ADVANCED CONFIG ###############################
+
+# Hashes are encoded using a memory efficient data structure when they have a
+# small number of entries, and the biggest entry does not exceed a given
+# threshold. These thresholds can be configured using the following directives.
+hash-max-ziplist-entries 512
+hash-max-ziplist-value 64
+
+# Lists are also encoded in a special way to save a lot of space.
+# The number of entries allowed per internal list node can be specified
+# as a fixed maximum size or a maximum number of elements.
+# For a fixed maximum size, use -5 through -1, meaning:
+# -5: max size: 64 Kb <-- not recommended for normal workloads
+# -4: max size: 32 Kb <-- not recommended
+# -3: max size: 16 Kb <-- probably not recommended
+# -2: max size: 8 Kb <-- good
+# -1: max size: 4 Kb <-- good
+# Positive numbers mean store up to _exactly_ that number of elements
+# per list node.
+# The highest performing option is usually -2 (8 Kb size) or -1 (4 Kb size),
+# but if your use case is unique, adjust the settings as necessary.
+list-max-ziplist-size -2
+
+# Lists may also be compressed.
+# Compress depth is the number of quicklist ziplist nodes from *each* side of
+# the list to *exclude* from compression. The head and tail of the list
+# are always uncompressed for fast push/pop operations. Settings are:
+# 0: disable all list compression
+# 1: depth 1 means "don't start compressing until after 1 node into the list,
+# going from either the head or tail"
+# So: [head]->node->node->...->node->[tail]
+# [head], [tail] will always be uncompressed; inner nodes will compress.
+# 2: [head]->[next]->node->node->...->node->[prev]->[tail]
+# 2 here means: don't compress head or head->next or tail->prev or tail,
+# but compress all nodes between them.
+# 3: [head]->[next]->[next]->node->node->...->node->[prev]->[prev]->[tail]
+# etc.
+list-compress-depth 0
+
+# Sets have a special encoding in just one case: when a set is composed
+# of just strings that happen to be integers in radix 10 in the range
+# of 64 bit signed integers.
+# The following configuration setting sets the limit in the size of the
+# set in order to use this special memory saving encoding.
+set-max-intset-entries 512
+
+# Similarly to hashes and lists, sorted sets are also specially encoded in
+# order to save a lot of space. This encoding is only used when the length and
+# elements of a sorted set are below the following limits:
+zset-max-ziplist-entries 128
+zset-max-ziplist-value 64
+
+# HyperLogLog sparse representation bytes limit. The limit includes the
+# 16 bytes header. When an HyperLogLog using the sparse representation crosses
+# this limit, it is converted into the dense representation.
+#
+# A value greater than 16000 is totally useless, since at that point the
+# dense representation is more memory efficient.
+#
+# The suggested value is ~ 3000 in order to have the benefits of
+# the space efficient encoding without slowing down too much PFADD,
+# which is O(N) with the sparse encoding. The value can be raised to
+# ~ 10000 when CPU is not a concern, but space is, and the data set is
+# composed of many HyperLogLogs with cardinality in the 0 - 15000 range.
+hll-sparse-max-bytes 3000
+
+# Active rehashing uses 1 millisecond every 100 milliseconds of CPU time in
+# order to help rehashing the main Redis hash table (the one mapping top-level
+# keys to values). The hash table implementation Redis uses (see dict.c)
+# performs a lazy rehashing: the more operation you run into a hash table
+# that is rehashing, the more rehashing "steps" are performed, so if the
+# server is idle the rehashing is never complete and some more memory is used
+# by the hash table.
+#
+# The default is to use this millisecond 10 times every second in order to
+# actively rehash the main dictionaries, freeing memory when possible.
+#
+# If unsure:
+# use "activerehashing no" if you have hard latency requirements and it is
+# not a good thing in your environment that Redis can reply from time to time
+# to queries with 2 milliseconds delay.
+#
+# use "activerehashing yes" if you don't have such hard requirements but
+# want to free memory asap when possible.
+activerehashing yes
+
+# The client output buffer limits can be used to force disconnection of clients
+# that are not reading data from the server fast enough for some reason (a
+# common reason is that a Pub/Sub client can't consume messages as fast as the
+# publisher can produce them).
+#
+# The limit can be set differently for the three different classes of clients:
+#
+# normal -> normal clients including MONITOR clients
+# slave -> slave clients
+# pubsub -> clients subscribed to at least one pubsub channel or pattern
+#
+# The syntax of every client-output-buffer-limit directive is the following:
+#
+# client-output-buffer-limit <class> <hard limit> <soft limit> <soft seconds>
+#
+# A client is immediately disconnected once the hard limit is reached, or if
+# the soft limit is reached and remains reached for the specified number of
+# seconds (continuously).
+# So for instance if the hard limit is 32 megabytes and the soft limit is
+# 16 megabytes / 10 seconds, the client will get disconnected immediately
+# if the size of the output buffers reach 32 megabytes, but will also get
+# disconnected if the client reaches 16 megabytes and continuously overcomes
+# the limit for 10 seconds.
+#
+# By default normal clients are not limited because they don't receive data
+# without asking (in a push way), but just after a request, so only
+# asynchronous clients may create a scenario where data is requested faster
+# than it can read.
+#
+# Instead there is a default limit for pubsub and slave clients, since
+# subscribers and slaves receive data in a push fashion.
+#
+# Both the hard or the soft limit can be disabled by setting them to zero.
+client-output-buffer-limit normal 0 0 0
+client-output-buffer-limit slave 256mb 64mb 60
+client-output-buffer-limit pubsub 32mb 8mb 60
+
+# Redis calls an internal function to perform many background tasks, like
+# closing connections of clients in timeout, purging expired keys that are
+# never requested, and so forth.
+#
+# Not all tasks are performed with the same frequency, but Redis checks for
+# tasks to perform according to the specified "hz" value.
+#
+# By default "hz" is set to 10. Raising the value will use more CPU when
+# Redis is idle, but at the same time will make Redis more responsive when
+# there are many keys expiring at the same time, and timeouts may be
+# handled with more precision.
+#
+# The range is between 1 and 500, however a value over 100 is usually not
+# a good idea. Most users should use the default of 10 and raise this up to
+# 100 only in environments where very low latency is required.
+hz 10
+
+# When a child rewrites the AOF file, if the following option is enabled
+# the file will be fsync-ed every 32 MB of data generated. This is useful
+# in order to commit the file to the disk more incrementally and avoid
+# big latency spikes.
+aof-rewrite-incremental-fsync yes
--- /dev/null
+#! /bin/bash
+
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+DB_PORT=6379
+DB_IP=127.0.0.1
+LOG_BASE_DIR=/srv/deployment/log
+DB_LOG=${LOG_BASE_DIR}/redis.log
+
+CM_PORT=61100
+CM_IP=127.0.0.1
+CM_VALIDATORS=/opt/cmframework/validators
+CM_ACTIVATORS=/opt/cmframework/activators
+CM_LOG=${LOG_BASE_DIR}/cm.log
+USER_CONFIG_HANDLERS=/opt/cmframework/userconfighandlers
+INVENTORY_HANDLERS=/opt/cmframework/inventoryhandlers
+INVENTORY_DATA=/opt/cmframework/inventory.data
+
+
+BOOTSTRAP_LOG=${LOG_BASE_DIR}/bootstrap.log
+
+DB_STARTUP_CMD="/bin/redis-server ./redis.conf"
+DB_CHECK_CMD="/bin/redis-cli -h $DB_IP --scan --pattern '*'"
+CMCLI="/usr/local/bin/cmcli --ip $CM_IP --port $CM_PORT --client-lib cmframework.lib.cmclientimpl.CMClientImpl"
+USERCONFIG=
+
+export CONFIG_PHASE="bootstrapping"
+
+DB_PID=
+CM_PID=
+
+function log()
+{
+ priority=$1
+
+ echo "$(date) ($priority) ${FUNCNAME[2]} ${@:2}"
+ echo "$(date) ($priority) ${FUNCNAME[2]} ${@:2}" >> $BOOTSTRAP_LOG
+}
+
+function log_info()
+{
+ log info $@
+}
+
+function log_error()
+{
+ log error $@
+}
+
+
+function run_cmd()
+{
+ log_info "Running $@"
+ result=$(eval $@ 2>&1)
+ ret=$?
+ if [ $ret -ne 0 ]; then
+ log_error "Failed with error $result"
+ else
+ log_info "Command succeeded: $result"
+ fi
+
+ return $ret
+}
+
+function stop_process()
+{
+ pid=$1
+ log_info "Stopping process $pid"
+ if ! [ -z $pid ]; then
+ if [ -d /proc/$pid ]; then
+ log_info "Shutting down process $pid gracefully"
+ run_cmd "pkill -TERM -g $pid"
+ log_info "Waiting for process $pid to exit"
+ for ((i=0; i<10; i++)); do
+ if ! [ -d /proc/$pid ]; then
+ log_info "Process $pid exited"
+ break
+ fi
+ log_info "Process $pid is still running"
+ sleep 2
+ done
+
+ if [ -d /proc/$pid ]; then
+ log_error "Process $pid is still running, forcefully shutting it down"
+ run_cmd "pkill -KILL -g $pid"
+ fi
+ fi
+ fi
+}
+
+function cleanup()
+{
+ log_info "Cleaning up"
+ #stop_process $DB_PID
+ systemctl stop redis
+ stop_process $CM_PID
+}
+
+function start_db()
+{
+ log_info "Starting redis db using $DB_STARTUP_CMD"
+ #setsid $DB_STARTUP_CMD &
+ #export DB_PID=$!
+ #log_info "DB pid is $DB_PID"
+ #if ! [ -d /proc/$DB_PID ]; then
+ # log_error "DB is not running!"
+ # log_info "Check /var/log/redis.log and $BOOTSTRAP_LOG for details"
+ # return 1
+ #fi
+ systemctl start redis
+ log_info "Wait till DB is serving"
+ dbok=0
+ for ((i=0; i<10; i++)); do
+ run_cmd "$DB_CHECK_CMD"
+ dbok=$?
+ if [ $dbok -eq 0 ]; then
+ break
+ fi
+ log_info "DB still not running"
+ sleep 2
+ done
+
+ return $dbok
+}
+
+function start_cm()
+{
+ log_info "Starting CM server"
+ setsid /usr/local/bin/cmserver --ip $CM_IP --port $CM_PORT --backend-api cmframework.redisbackend.cmredisdb.CMRedisDB --backend-uri redis://:@$DB_IP:$DB_PORT --activationstate-handler-api cmframework.utils.cmstatedummyhandler.CMStateDummyHandler --activationstate-handler-uri dummy_uri --snapshot-handler-api cmframework.utils.cmstatedummyhandler.CMStateDummyHandler --alarmhandler-api cmframework.lib.cmalarmhandler_dummy.AlarmHandler_Dummy --snapshot-handler-uri dummy_uri --inventory-handlers $INVENTORY_HANDLERS --inventory-data $INVENTORY_DATA --validators $CM_VALIDATORS --activators $CM_ACTIVATORS --disable-remote-activation --log-dest console --log-level debug --verbose 2> $CM_LOG 1>&2 &
+ export CM_PID=$!
+ log_info "cmserver pid is $CM_PID"
+ if ! [ -d /proc/$CM_PID ]; then
+ log_error "CM server is not running!"
+ log_info "Check redis.log and $BOOTSTRATP_LOG for details"
+ return 1
+ fi
+ return 0
+}
+
+function handle_user_config()
+{
+ log_info "Handling user configuration from file $USER_CONFIG"
+ run_cmd "$CMCLI bootstrap --config $USER_CONFIG --plugin_path $USER_CONFIG_HANDLERS"
+ return $?
+}
+
+function main()
+{
+ # start the database
+ start_db
+ if [ $? -ne 0 ]; then
+ cleanup
+ return 1
+ fi
+
+ # start the configuration management server
+ start_cm
+ if [ $? -ne 0 ]; then
+ cleanup
+ return 1
+ fi
+
+ echo "============"
+ jobs -p
+
+ echo "Use CLI $CMCLI"
+
+ while [ 1 ]; do
+ read -n1 -r -p "Press space to exit..." key
+
+ if [ "$key" = '' ]; then
+ echo "exiting..."
+ break
+ fi
+ done
+
+ cleanup
+ return 0
+}
+
+if [ $# -ne 1 ]; then
+ echo "Usage:$0 <cmserver ip>"
+ exit 1
+fi
+
+CM_IP=$1
+CMCLI="/usr/local/bin/cmcli --ip $CM_IP --port $CM_PORT --client-lib cmframework.lib.cmclientimpl.CMClientImpl"
+mkdir -p ${LOG_BASE_DIR}
+
+main
--- /dev/null
+#! /bin/bash
+
+
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+:'
+This script can be used to start a private cmserver to test your CM userconfighandler and/or validator plugins.
+
+To test userconfighandlers and installation time validators do the following:
+- Copy your userconfighandler to a target system under /opt/cmframework/userconfighandlers/ directory.
+
+- Copy your validator to a target system under /opt/cmframework/validators directory.
+
+- Start a private cmserver
+ ./start-private-cmserver.sh <some port number> no-cm-data
+
+- The above command will print information about the temporary directory used for the private cmserver, it will also print the command to be used to access the cmserver e.g. the output can look like:
+
+=======================================================================================================================
+Use CLI /opt/bin/cmcli --ip 127.0.0.1 --port 51110
+Root DIR /tmp/tmp.HcyehIQQoQ
+=======================================================================================================================
+
+- Test your plugins by running the following command:
+ /opt/bin/cmcli --ip 127.0.0.1 --port <port> bootstrap --config /opt/userconfig/user_config.yaml --plugin_path /<cmserver root>/opt/cmframework/userconfighandlers
+
+- If the above succeeds then your userconfighandler and validator are working properly. You can view the configuration data by running the following command:
+ /opt/bin/cmcli --ip 127.0.0.1 --port <port> get-properties --matching-filter '.*'
+
+To test run-time validation do the following:
+- Copy your validator to a target system under /opt/cmframework/validators directory.
+
+- Start a private cmserver
+ ./start-private-cmserver.sh <some port number> no-cm-data
+
+- The above command will print information about the temporary directory used for the private cmserver, it will also print the command to be used to access the cmserver e.g. the output can look like:
+
+=======================================================================================================================
+Use CLI /opt/bin/cmcli --ip 127.0.0.1 --port 51110
+Root DIR /tmp/tmp.HcyehIQQoQ
+=======================================================================================================================
+
+- Test your plugins by changing the configuration by running the following command:
+ /opt/bin/cmcli --ip 127.0.0.1 --port <port> set-property --property NAME VALUE
+
+- If the above succeeds then your validator is working properly. You can view the configuration data by running the following command:
+ /opt/bin/cmcli --ip 127.0.0.1 --port <port> get-properties --matching-filter '.*'
+'
+
+CM_PORT=
+CM_IP=127.0.0.1
+NO_CM_DATA=0
+ROOT_DIR=$(mktemp -d)
+CM_VALIDATORS=$ROOT_DIR/opt/cmframework/validators
+CM_ACTIVATORS=$ROOT_DIR/opt/cmframework/activators
+CM_LOG=$ROOT_DIR/var/log/cm.log
+USER_CONFIG_HANDLERS=$ROOT_DIR/opt/cmframework/userconfighandlers
+INVENTORY_HANDLERS=$ROOT_DIR/opt/cmframework/inventoryhandlers
+INVENTORY_DATA=$ROOT_DIR/inventory.data
+
+LOG_FILE=$ROOT_DIR/var/log/cm.log
+
+DB_DIR=$ROOT_DIR/db
+DB_FILE=$DB_DIR/db
+
+CMCLI="/usr/local/bin/cmcli --ip $CM_IP --port $CM_PORT"
+ENV_FILE=$ROOT_DIR/
+
+export CONFIG_PHASE="bootstrapping"
+
+CM_PID=
+
+
+function log()
+{
+ priority=$1
+
+ echo "$(date) ($priority) ${FUNCNAME[2]} ${@:2}"
+ echo "$(date) ($priority) ${FUNCNAME[2]} ${@:2}" >> $LOG_FILE
+}
+
+function log_info()
+{
+ log info $@
+}
+
+function log_error()
+{
+ log error $@
+}
+
+
+function run_cmd()
+{
+ log_info "Running $@"
+ result=$(eval $@ 2>&1)
+ ret=$?
+ if [ $ret -ne 0 ]; then
+ log_error "Failed with error $result"
+ else
+ log_info "Command succeeded: $result"
+ fi
+
+ return $ret
+}
+
+function prepare_environment()
+{
+ mkdir -p $(dirname $CM_LOG)
+ log_info "Creating directories under $ROOT_DIR"
+ run_cmd "mkdir -p $USER_CONFIG_HANDLERS"
+ run_cmd "mkdir -p $INVENTORY_HANDLERS"
+ run_cmd "mkdir -p $CM_VALIDATORS"
+ run_cmd "mkdir -p $CM_ACTIVATORS"
+ run_cmd "mkdir -p $DB_DIR"
+
+ log_info "Copying inventory handlers and user config handlers to $ROOT_DIR"
+ cp -f /opt/cmframework/userconfighandlers/* $USER_CONFIG_HANDLERS/
+ cp -f /opt/cmframework/inventoryhandlers/* $INVENTORY_HANDLERS/
+ cp -f /opt/cmframework/validators/* $CM_VALIDATORS/
+
+ if [ $NO_CM_DATA -eq 0 ]; then
+ log_info "Generating DB file under $DB_FILE"
+ /usr/local/bin/cmcli get-properties --matching-filter '.*' > $DB_FILE
+ fi
+ return $?
+}
+
+function stop_process()
+{
+ pid=$1
+ log_info "Stopping process $pid"
+ if ! [ -z $pid ]; then
+ if [ -d /proc/$pid ]; then
+ log_info "Shutting down process $pid gracefully"
+ run_cmd "pkill -TERM -g $pid"
+ log_info "Waiting for process $pid to exit"
+ for ((i=0; i<10; i++)); do
+ if ! [ -d /proc/$pid ]; then
+ log_info "Process $pid exited"
+ break
+ fi
+ log_info "Process $pid is still running"
+ sleep 2
+ done
+
+ if [ -d /proc/$pid ]; then
+ log_error "Process $pid is still running, forcefully shutting it down"
+ run_cmd "pkill -KILL -g $pid"
+ fi
+ fi
+ fi
+}
+
+function cleanup()
+{
+ log_info "Cleaning up"
+ stop_process $CM_PID
+ rm -rf $ROOT_DIR
+}
+
+function start_cm()
+{
+ log_info "Starting CM server $CM_IP $CM_PORT $CM_LOG"
+ extra_args=""
+ if [ $NO_CM_DATA -eq 1 ]; then
+ extra_args="--install-phase"
+ fi
+ setsid /usr/local/bin/cmserver --ip $CM_IP --port $CM_PORT --backend-api cmframework.filebackend.cmfilebackend.CMFileBackend --backend-uri $DB_FILE --activationstate-handler-api cmframework.utils.cmstatedummyhandler.CMStateDummyHandler --activationstate-handler-uri dummy_uri --alarmhandler-api cmframework.lib.cmalarmhandler_dummy.AlarmHandler_Dummy --snapshot-handler-api cmframework.utils.cmstatedummyhandler.CMStateDummyHandler --snapshot-handler-uri dummy_uri --inventory-handlers $INVENTORY_HANDLERS --inventory-data $INVENTORY_DATA --validators $CM_VALIDATORS --activators $CM_ACTIVATORS --disable-remote-activation --log-dest console --log-level debug --verbose $extra_args 2> $CM_LOG 1>&2 &
+ export CM_PID=$!
+ log_info "cmserver pid is $CM_PID"
+ if ! [ -d /proc/$CM_PID ]; then
+ log_error "CM server is not running!"
+ log_info "Check $CM_LOG for details"
+ return 1
+ fi
+ return 0
+}
+
+function main()
+{
+ # prapare private environment
+ prepare_environment
+ if [ $? -ne 0 ]; then
+ cleanup
+ return 1
+ fi
+
+ # start the configuration management server
+ start_cm
+ if [ $? -ne 0 ]; then
+ cleanup
+ return 1
+ fi
+
+ jobs -p
+
+ echo ""
+ echo ""
+ echo "======================================================================================================================="
+ echo "Use CLI $CMCLI"
+ echo "Root DIR $ROOT_DIR"
+ echo "======================================================================================================================="
+ echo ""
+ echo ""
+ while [ 1 ]; do
+ read -n1 -r -p "Press 'q' space to exit..." key
+
+ if [ "$key" = 'q' ]; then
+ echo "exiting..."
+ break
+ fi
+ done
+
+ cleanup
+ return 0
+}
+
+if [ $# -lt 1 ]; then
+ echo "Usage:$0 <cmserver port> [no-cm-data]"
+ exit 1
+fi
+
+CM_PORT=$1
+shift 1
+
+for arg in "$@"; do
+ if [ "$arg" == "no-cm-data" ]; then
+ NO_CM_DATA=1
+ else
+ CONFIG_PHASE=$arg
+ fi
+done
+
+CMCLI="/usr/local/bin/cmcli --ip $CM_IP --port $CM_PORT"
+
+main
--- /dev/null
+recursive-include cmframework *.py *.sh
--- /dev/null
+This project provides an implementation for a generic configuration management
+framework. The framework provides the following:
+* Interfaces for getting, setting or deleting of configuration data.
+* Interfaces for taking a backup of configuration data.
+* Interfaces for restoring the configuration data from a previously taken
+ backup.
+* A CLI for manipulating the configuration data.
+* A plugin based interface for validating the change in configuration data.
+* A plugin based interface for activating the change in configuration data.
+
+At the highlevel, the framework surves the following purposes:
+* Provides a unified interface for manipulating the configuration data.
+* Isolate the configuration management users from changes in the used backend.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+__import__('pkg_resources').declare_namespace(__name__)
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+#! /usr/bin/python
+
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from __future__ import print_function
+import sys
+import argparse
+import socket
+import logging
+import subprocess
+
+from cmframework.apis import cmmanage
+from cmframework.apis import cmerror
+from cmframework.utils import cmlogger
+from cmframework.utils import cmalarm
+
+
+class VerboseLogger(object):
+ def __call__(self, msg):
+ print (msg)
+
+
+class CMAgent(object):
+ def __init__(self):
+ self.prog = 'cmagent'
+ self.ip = None
+ self.port = None
+ self.verbose = False
+ self.log_level = cmlogger.CMLogger.str_to_level('debug')
+ self.log_dest = cmlogger.CMLogger.str_to_dest('syslog')
+ self.api = None
+ self.verbose_logger = VerboseLogger()
+
+ @staticmethod
+ def log_level_parser(level):
+ try:
+ return cmlogger.CMLogger.str_to_level(level)
+ except cmerror.CMError as exp:
+ raise argparse.ArgumentTypeError(str(exp))
+
+ @staticmethod
+ def log_dest_parser(dest):
+ try:
+ return cmlogger.CMLogger.str_to_dest(dest)
+ except cmerror.CMError as exp:
+ raise argparse.ArgumentTypeError(str(exp))
+
+ def __call__(self, args):
+ parser = argparse.ArgumentParser(description='Configuration Management Agent',
+ prog=self.prog)
+ parser.add_argument('--ip',
+ dest='ip',
+ metavar='SERVER-IP',
+ default='config-manager',
+ type=str,
+ action='store')
+
+ parser.add_argument('--port',
+ dest='port',
+ metavar='SERVER-PORT',
+ default=61100,
+ type=int,
+ action='store')
+
+ parser.add_argument('--client-lib',
+ dest='client_lib',
+ metavar='CLIENT-LIB',
+ default='cmframework.lib.CMClientImpl',
+ type=str,
+ action='store')
+
+ parser.add_argument('--log-level',
+ dest='log_level',
+ metavar='LOG-LEVEL',
+ required=False,
+ help=('The enabled logging level, possible values are '
+ '{debug, info, warn, error}'),
+ type=CMAgent.log_level_parser,
+ default=self.log_level,
+ action='store')
+
+ parser.add_argument('--log-dest',
+ dest='log_dest',
+ metavar='LOG-DEST',
+ required=False,
+ help='The logs destination, possible values are {console, syslog}',
+ type=CMAgent.log_dest_parser,
+ default=self.log_dest,
+ action='store')
+
+ parser.add_argument('--verbose',
+ required=False,
+ default=False,
+ action='store_true')
+
+ args = parser.parse_args(args)
+
+ self.process(args)
+
+ def _init_api(self, ip, port, client_lib):
+ try:
+ serverip = socket.gethostbyname(ip)
+ except socket.gaierror:
+ # use localhost in-case we cannot resolve the provided hostname
+ serverip = '127.0.0.1'
+
+ self.api = cmmanage.CMManage(serverip, port, client_lib, self.verbose_logger)
+
+ @staticmethod
+ def _reboot_node():
+ cmd = 'systemctl reboot'
+ args = cmd.split()
+ process = subprocess.Popen(args)
+ return process.wait()
+
+ def process(self, args):
+ cmlogger.CMLogger(args.log_dest, args.verbose, args.log_level)
+
+ self._init_api(args.ip, args.port, args.client_lib)
+
+ node_name = socket.gethostname()
+
+ reboot_request_alarm = cmalarm.CMRebootRequestAlarm()
+ reboot_request_alarm.cancel_alarm_for_node(node_name)
+
+ reboot = self.api.activate_node(node_name)
+ if reboot:
+ logging.warn('Going to reboot this node')
+ res = self._reboot_node()
+ if res != 0:
+ logging.error('Reboot failed')
+
+
+def main():
+ try:
+ agent = CMAgent()
+ args = sys.argv[1:]
+ agent(args)
+ except cmerror.CMError as exp:
+ print ('Failed with error: %s' % str(exp))
+ return 1
+ # TODO: catch all exceptions?
+
+
+if __name__ == '__main__':
+ sys.exit(main())
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import os
+import subprocess
+import pwd
+import logging
+
+from cmframework.apis import cmerror
+
+
+class CMActivator(object):
+ ansible_bin = '/usr/local/bin/openstack-ansible'
+ admin_user_file = '/etc/admin_user'
+
+ def __init__(self):
+ self.plugin_client = None
+ try:
+ with open(CMActivator.admin_user_file, 'r') as f:
+ self.admin_user = f.read()
+ except IOError:
+ pass
+
+ # pylint: disable=no-self-use
+ def get_subscription_info(self):
+ """get the subscription filter
+
+ This API is used to get the re for matching the properties which the
+ activation plugin is concerned about.
+
+ Return:
+
+ A string representing the regular expression used to match the
+ properties which the activation plugin is concerned about.
+
+ Raise:
+
+ CMError can be raised in-case of a failure.
+ """
+ raise cmerror.CMError('Not implemented')
+
+ # pylint: disable=no-self-use, unused-argument
+ def activate_set(self, props):
+ """activate a configuration data addition/update
+
+ Arguments:
+
+ props: A dictionary of name-value pairs representing the changed
+ properties.
+
+ Raise:
+
+ CMError can be raised in-case of an error
+ """
+ raise cmerror.CMError('Not implemented')
+
+ # pylint: disable=no-self-use, unused-argument
+ def activate_delete(self, props):
+ """activate a configuration data deletion
+
+ Arguments:
+
+ props: A list of deleted property names.
+
+ Raise:
+
+ CMError can be raised in-case of an error
+ """
+ raise cmerror.CMError('Not implemented')
+
+ # pylint: disable=no-self-use, unused-argument
+ def activate_full(self, target):
+ """perform a full activation
+
+ Arguments:
+
+ target: None if activating all nodes
+ Node name string if activating only one node
+
+ Raise:
+
+ CMError can be raised in-case of an error
+ """
+ raise cmerror.CMError('Not implemented')
+
+ # pylint: disable=no-self-use
+ def get_plugin_client(self):
+ """get the plugin client object
+
+ This API can be used by the plugin to get the client object which the
+ plugin can use to access the configuration data. Notice that the data
+ accessed by this is what is stored in the backend. The changed data
+ is passed as argument to the different validate functions.
+
+ Return:
+
+ The plugin client object
+ """
+ return self.plugin_client
+
+ def run_playbook(self, playbook, target=None):
+ playbook_dir = os.path.dirname(playbook)
+
+ arguments = []
+ arguments.append('-b')
+ arguments.append('-u {}'.format(self.admin_user))
+ if target:
+ arguments.append('--limit {}'.format(target))
+ arguments.append(playbook)
+
+ cmd = '{} {}'.format(CMActivator.ansible_bin, ' '.join(arguments))
+ out, result = self._run_cmd_as_user(cmd, playbook_dir, self.admin_user)
+ if result != 0:
+ raise cmerror.CMError('Playbook {} failed: {}'.format(playbook, out))
+ logging.debug('Playbook out: %s', out)
+
+ def _run_cmd_as_user(self, cmd, cwd, user):
+ pw_record = pwd.getpwnam(user)
+ user_name = pw_record.pw_name
+ user_home_dir = pw_record.pw_dir
+ user_uid = pw_record.pw_uid
+ user_gid = pw_record.pw_gid
+ env = os.environ.copy()
+ env['CONFIG_PHASE'] = 'postconfig'
+ env['HOME'] = user_home_dir
+ env['LOGNAME'] = user_name
+ env['PWD'] = cwd
+ env['USER'] = user_name
+ p = subprocess.Popen(cmd.split(), preexec_fn=self._demote(user_uid, user_gid),
+ cwd=cwd, env=env, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
+ out, _ = p.communicate()
+ return (out, p.returncode)
+
+ def _demote(self, user_uid, user_gid):
+ def result():
+ os.setgid(user_gid)
+ os.setuid(user_uid)
+
+ return result
+
+
+class CMGlobalActivator(CMActivator):
+ def __init__(self):
+ super(CMGlobalActivator, self).__init__()
+
+
+class CMLocalActivator(CMActivator):
+ def __init__(self):
+ super(CMLocalActivator, self).__init__()
+ self.hostname = None
+
+ def get_hostname(self):
+ """get the node name
+
+ This API is used to get the name of the node where activation is
+ ongoing.
+
+ Return:
+
+ The node name
+ """
+ return self.hostname
+
+
+def main():
+ def print_type(activator):
+ if isinstance(activator, CMLocalActivator):
+ print 'Local activator'
+ elif isinstance(activator, CMActivator):
+ print 'Activator'
+ else:
+ print 'Unknown'
+
+ activator = CMActivator()
+ localactivator = CMLocalActivator()
+
+ x = 100
+
+ print_type(activator)
+ print_type(localactivator)
+ print_type(x)
+
+
+if __name__ == '__main__':
+ main()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from cmframework.apis import cmerror
+
+
+class CMAnsibleInventoryConfigPlugin(object):
+
+ def __init__(self, confman, inventory, ownhost):
+ self.confman = confman
+ self.inventory = inventory
+ self.ownhost = ownhost
+
+ def add_host_var(self, host, name, value):
+ """add a host specific variable
+ Arguments:
+ host: The name of the host
+ name: The name of the variable.
+ value: The value
+ """
+ if host not in self.inventory['_meta']['hostvars']:
+ self.inventory['_meta']['hostvars'][host] = {}
+ self.inventory['_meta']['hostvars'][host][name] = value
+
+ def add_global_var(self, name, value):
+ """add a global variable to the inventory
+ Arguments:
+ name: The name of the variable
+ value: The value
+ """
+ self.inventory['all']['vars'][name] = value
+
+ def add_host_group(self, name, value):
+ self.inventory[name] = value
+
+ # pylint: disable=no-self-use
+ def handle_bootstrapping(self):
+ """provide inventory extensions
+
+ Raise:
+
+ CMError is raised in-case of failure
+ """
+
+ raise cmerror.CMError('No implemented')
+
+ # pylint: disable=no-self-use
+ def handle_provisioning(self):
+ """provide inventory extensions
+
+ Raise:
+
+ CMError is raised in-case of failure
+ """
+
+ raise cmerror.CMError('No implemented')
+
+ # pylint: disable=no-self-use
+ def handle_postconfig(self):
+ """provide inventory extensions
+
+ Raise:
+
+ CMError is raised in-case of failure
+ """
+
+ raise cmerror.CMError('No implemented')
+
+ # pylint: disable=no-self-use
+ def handle_setup(self):
+ """provide inventory extensions
+
+ Raise:
+
+ CMError is raised in-case of failure
+ """
+
+ raise cmerror.CMError('No implemented')
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from cmframework.apis import cmerror
+
+
+class CMBackend(object):
+ # pylint: disable=no-self-use, unused-argument
+ def get_property(self, prop_name):
+ """get the value of some property
+
+ Arguments:
+
+ prop_name: The name of the property
+
+ Raise:
+
+ CMError is raised in-case of failure
+ """
+ raise cmerror.CMError('No implemented')
+
+ # pylint: disable=no-self-use, unused-argument
+ def get_properties(self, prop_filter):
+ """get the properties matching some filter
+
+ Arguments:
+
+ prop_filter: A string containing the re used to match the required
+ properties.
+
+ Raise:
+
+ CMError is raised in-case of failure
+ """
+ raise cmerror.CMError('Not implemented')
+
+ # pylint: disable=no-self-use, unused-argument
+ def set_property(self, prop_name, prop_value):
+ """set/update a value to some property
+
+ Arguments:
+
+ prop_name: The name of the property to be set/updated.
+
+ prop_value: The value of the property
+
+ Raise:
+
+ CMError is raised in-case of failure
+ """
+ raise cmerror.CMError('Not implemented')
+
+ # pylint: disable=no-self-use, unused-argument
+ def set_properties(self, properties):
+ """set/update a group of properties
+
+ Arguments:
+
+ props: A dictionary containing the changed properties.
+
+ Raise:
+
+ CMError is raised in-case of a failure.
+ """
+ raise cmerror.CMError('Not implemented')
+
+ # pylint: disable=no-self-use, unused-argument
+ def delete_property(self, prop_name):
+ """delete a property
+
+ This is the API used to delete a configuration property.
+
+ Arguments:
+
+ prop_name: The name of the property to be deleted.
+
+ Raise:
+
+ CMError is raised in-case of a failure.
+ """
+ raise cmerror.CMError('Not implemented')
+
+ # pylint: disable=no-self-use, unused-argument
+ def delete_properties(self, arg):
+ """delete a group of properties
+
+ Arguments:
+
+ arg: This can be either a string representing the re used when
+ matching the properties to be deleted, or it can be a list of
+ properties names to be deleted.
+
+ Rise:
+
+ CMError is raised in-case of a failure.
+ """
+ raise cmerror.CMError('Not implemented')
+
+if __name__ == '__main__':
+ pass
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+CM_CHANGE_STATE_OK = "ok"
+CM_CHANGE_STATE_NOK = "nok"
+CM_CHANGE_STATE_ONGOING = "ongoing"
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import re
+
+from functools import wraps
+from cmframework import apis as cmapis
+
+
+def handle_exceptions(func):
+ @wraps(func)
+ def wrapper(self, *arg, **kwargs):
+ try:
+ return func(self, *arg, **kwargs)
+ except cmapis.cmerror.CMError as exp:
+ raise
+ except Exception as exp:
+ raise cmapis.cmerror.CMError(str(exp))
+
+ return wrapper
+
+
+class CMClient(object):
+ """
+ Usage Example:
+ class VerboseLogger:
+ def __call__(self, msg):
+ print(msg)
+
+ logger = VerboseLogger()
+
+ client = CMClient(192.128.254.10, 51110, cmclient.CMClientImpl, logger)
+ try:
+ value = client.get_property('controller-1.ntp.servers')
+ except cmapis.cmerror.CMError as error:
+ print('Got exception %s' % str(error))
+ """
+
+ @handle_exceptions
+ def __init__(self, server_ip='config-manager', server_port=61100,
+ client_lib_impl_module='cmframework.lib.CMClientImpl', verbose_logger=None):
+ """ initialize the client interface
+
+ Arguments:
+
+ server_ip: The configuration management server ip address.
+
+ server_port: The configuration management server port number.
+
+ client_lib_impl_module: The module implementing the client library.
+
+ verbose_logger: The verbose logging callable. any callable which
+ takes a string as input argument can be used.
+
+ Raise:
+
+ CMError exception in-case of a failure.
+ """
+
+ import socket
+ try:
+ serverip = socket.gethostbyname(server_ip)
+ except Exception: # pylint: disable=broad-except
+ # use localhost in-case we cannot resolve the provided hostname
+ serverip = '127.0.0.1'
+
+ self.server_ip = serverip
+ self.server_port = server_port
+ self.client_lib = None
+ self.verbose_logger = verbose_logger
+
+ # Separate class path and module name
+ parts = client_lib_impl_module.rsplit('.', 1)
+ module_path = parts[0]
+ class_name = parts[1]
+ self.verbose_log('module_path = %s' % module_path)
+ self.verbose_log('class_name = %s' % class_name)
+ module = __import__(module_path, fromlist=[module_path])
+ classobj = getattr(module, class_name)
+ self.client_lib = classobj(self.server_ip, self.server_port, verbose_logger)
+
+ def verbose_log(self, msg):
+ if self.verbose_logger:
+ self.verbose_logger(msg)
+
+ @handle_exceptions
+ def get_property(self, prop_name, snapshot_name=None):
+ """get the value assoicated with a property.
+
+ This is the API used to read the value associated with a
+ configuration property.
+
+ Arguments:
+
+ prop_name: The property name
+ (optional) snapshot_name: The snapshot name
+
+ Raise:
+
+ CMError in-case of a failure.
+ """
+ result = self.client_lib.get_property(prop_name, snapshot_name)
+ return result
+
+ @handle_exceptions
+ def get_properties(self, prop_filter, snapshot_name=None):
+ """get a set of properties matching a filter.
+
+ This is the API used to read a group of properties matching some
+ filter.
+
+ Arguments:
+
+ prop_filter: A valid python re describing the filter used when
+ matching the returned properties.
+ (optional) snapshot_name: The snapshot name
+
+ Raise:
+
+ CMError is raised in-case of a failure.
+ """
+ self._check_filter(prop_filter)
+ result = self.client_lib.get_properties(prop_filter, snapshot_name)
+ return result
+
+ @handle_exceptions
+ def set_property(self, prop_name, prop_value):
+ """set/update the value of a property.
+
+ This is the API used to set/update the value associated with a
+ property.
+
+ Arguments:
+
+ prop_name: A string representing the property name.
+
+ prop_value: A string representing the property value.
+
+ Raise:
+
+ CMError is raised in-case of failure.
+ """
+ return self.client_lib.set_property(prop_name, prop_value)
+
+ @handle_exceptions
+ def set_properties(self, props, overwrite=False):
+ """set/update a group of properties as a whole
+
+ This API is used to set/update the values associated with a group of
+ properties as a whole, the change is either accepted as a whole or
+ rejected as a whole.
+
+ Arguments:
+
+ props: A dictionary containing the changed properties.
+
+ overwrite: Replace the existing configuration dictionary with the new one.
+
+ Raise:
+
+ CMError is raised in-case of a failure.
+ """
+ return self.client_lib.set_properties(props, overwrite)
+
+ @handle_exceptions
+ def delete_property(self, prop_name):
+ """delete a property
+
+ This is the API used to delete a configuration property.
+
+ Arguments:
+
+ prop_name: The name of the property to be deleted.
+
+ Raise:
+
+ CMError is raised in-case of a failure.
+ """
+ return self.client_lib.delete_property(prop_name)
+
+ @handle_exceptions
+ def delete_properties(self, arg):
+ """delete a group of properties as a whole
+
+ This is the API used to delete a group of properties as whole, if the
+ deletion of one of the properties is rejected then the whole delete
+ operation will fail.
+
+ Arguments:
+
+ arg: This can be either a string representing the re used when
+ matching the properties to be deleted, or it can be a list of
+ properties names to be deleted.
+
+ Raise:
+
+ CMError is raised in-case of a failure.
+ """
+ if isinstance(arg, str):
+ self._check_filter(arg)
+ return self.client_lib.delete_properties(arg)
+
+ @handle_exceptions
+ def get_changes_states(self, change_uuid):
+ """get the config changes states
+
+ This is the API used to get the changes states
+
+ Arguments:
+
+ arg: This can be either a valid change uuid or None.
+
+ Raise:
+
+ CMError is raised in-case of a failure.
+ """
+ return self.client_lib.get_changes_states(change_uuid)
+
+ @handle_exceptions
+ def wait_activation(self, change_uuid):
+ """wait for activation of config changes to finish
+
+ This is the API used to wait for config changes to finish
+
+ Arguments:
+
+ arg: A valid change uuid.
+
+ Raise:
+
+ CMError is raised in-case of a failure.
+ """
+ return self.client_lib.wait_activation(change_uuid)
+
+ # pylint: disable=no-self-use
+ def _check_filter(self, prop_filter):
+ re.compile(prop_filter)
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from __future__ import print_function
+import sys
+
+
+class CMError(Exception):
+ def __init__(self, description):
+ super(CMError, self).__init__(description)
+ self.description = description
+
+ def get_description(self):
+ return self.description
+
+ def __str__(self):
+ return '%s' % self.description
+
+
+if __name__ == '__main__':
+ try:
+ raise CMError(int(sys.argv[1]))
+ except CMError as error:
+ print('Got exception %s' % str(error))
+ except Exception as exp: # pylint: disable=broad-except
+ print('Got exception %s' % str(exp))
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import logging
+
+
+class CMHandler(object):
+ def __init__(self):
+ self._before = []
+
+ def get_subscription_info(self):
+ logging.debug('get_subscription info called')
+ return r''
+
+ def __str__(self):
+ return self.__class__.__name__
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from cmframework.apis import cmclient
+
+
+class CMManage(cmclient.CMClient):
+ """
+ Usage Example:
+ class VerboseLogger:
+ def __call__(self, msg):
+ print(msg)
+
+ logger = VerboseLogger()
+
+ client = CMManage(192.128.254.10, 51110, cmclient.CMClientImpl, logger)
+ try:
+ value = client.create_snapshot('snapshot1')
+ except cmapis.cmerror.CMError as error:
+ print('Got exception %s' % str(error))
+ """
+
+ def __init__(self, server_ip='config-manager', server_port=61100,
+ client_impl_module='cmframework.lib.CMClientImpl', verbose_logger=None):
+ """ initialize the client interface
+
+ Arguments:
+
+ server_ip: The configuration management server ip address.
+
+ server_port: The configuration management server port number.
+
+ client_lib_impl_module: The module implementing the client library.
+
+ verbose_logger: The verbose logging callable. any callable which
+ takes a string as input argument can be used.
+
+ Raise:
+
+ CMError exception in-case of a failure.
+ """
+ cmclient.CMClient.__init__(self, server_ip, server_port, client_impl_module, verbose_logger)
+
+ @cmclient.handle_exceptions
+ def create_snapshot(self, snapshot_name):
+ """initiate a create snapshot operation
+
+ This API is used to initiate a create snapshot operation for the configuration
+ data.
+
+ Arguments:
+
+ snapshot_name: The name of the snapshot
+
+ Raise:
+
+ CMError is raised in-case of a failure.
+ """
+ return self.client_lib.create_snapshot(snapshot_name)
+
+ @cmclient.handle_exceptions
+ def restore_snapshot(self, snapshot_name):
+ """initiate a snapshot restore operation
+
+ This API is used to initiate a snapshot restore operation for the
+ configuration data.
+
+ Arguments:
+
+ snapshot_name: The name of the snapshot.
+
+ Raise:
+
+ CMError is raised in-case of a failure.
+ """
+ return self.client_lib.restore_snapshot(snapshot_name)
+
+ @cmclient.handle_exceptions
+ def delete_snapshot(self, snapshot_name):
+ """initiate a snapshot delete operation
+
+ This API is used to initiate a snapshot delete operation for the
+ configuration data.
+
+ Arguments:
+
+ snapshot_name: The name of the snapshot.
+
+ Raise:
+
+ CMError is raised in-case of a failure.
+ """
+ return self.client_lib.delete_snapshot(snapshot_name)
+
+ @cmclient.handle_exceptions
+ def list_snapshots(self):
+ """initiate a list snapshots operation
+
+ This API is used to initiate a list snapshots operation for the
+ configuration data.
+
+ Arguments:
+
+ Raise:
+
+ CMError is raised in-case of a failure.
+ """
+ return self.client_lib.list_snapshots()
+
+ @cmclient.handle_exceptions
+ def activate(self, node_name):
+ """activate configuration in all or one specific node
+
+ This API is used to initiate full activation for all or one specific node
+
+ Arguments:
+
+ node_name: a string containing the node name where
+ the configuration is to be activated. If not specified, then activation
+ is done for all nodes.
+
+ Raise:
+
+ CMError is raised in-case of failure.
+ """
+ return self.client_lib.activate(node_name)
+
+ @cmclient.handle_exceptions
+ def activate_node(self, node_name):
+ """for cmagent to activate configuration in specified node
+
+ This API is used only by cmagent to initiate full activation for a node
+
+ Arguments:
+
+ node_name: a string containing the name of the node to be activated.
+
+ Raise:
+
+ CMError is raised in-case of failure.
+ """
+ return self.client_lib.activate_node(node_name)
+
+ @cmclient.handle_exceptions
+ def reboot_node(self, node_name):
+ """request reboot of specified node
+
+ This API is used to initiate node reboot during full activation of a node
+
+ Arguments:
+
+ node_name: a string containing the name of the node to be rebooted
+
+ Raise:
+
+ CMError is raised in-case of failure.
+ """
+ return self.client_lib.reboot_node(node_name)
+
+ @cmclient.handle_exceptions
+ def disable_automatic_activation(self):
+ """disable automatic activation
+
+ This API is used to disable automatic activation
+
+ Arguments:
+
+ Raise:
+
+ CMError is raised in-case of failure.
+ """
+ return self.client_lib.disable_automatic_activation()
+
+ @cmclient.handle_exceptions
+ def enable_automatic_activation(self):
+ """enable automatic activation
+
+ This API is used to enable automatic activation
+
+ Arguments:
+
+ Raise:
+
+ CMError is raised in-case of failure.
+ """
+ return self.client_lib.enable_automatic_activation()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from cmframework.apis import cmerror
+
+
+class CMPluginClient(object):
+ # pylint: disable=no-self-use, unused-argument
+ def get_property(self, prop_name):
+ """read the value assoicated with a property
+
+ Arguments:
+
+ prop_name: The name of the property to be read.
+
+ Raise:
+
+ CMError is raised in-case of failure
+ """
+ raise cmerror.CMError('Not implemented')
+
+ # pylint: disable=no-self-use, unused-argument
+ def get_properties(self, prop_filter):
+ """read a set of properties matching a filter.
+
+ Arguments:
+
+ prop_filter: A string containing the re used to match the read
+ properties.
+
+ Raise:
+
+ CMError is raised in-case of a failure.
+ """
+ raise cmerror.CMError('Not implemented')
+
+ # pylint: disable=no-self-use, unused-argument
+ def set_property(self, name, value):
+ raise cmerror.CMError('Not implemented')
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from cmframework.apis import cmerror
+
+
+class CMState(object):
+ # pylint: disable=no-self-use, unused-argument
+ def get(self, domain, name):
+ raise cmerror.CMError('No implemented')
+
+ # pylint: disable=no-self-use, unused-argument
+ def get_domain(self, domain):
+ raise cmerror.CMError('Not implemented')
+
+ # pylint: disable=no-self-use, unused-argument
+ def set(self, domain, name, value):
+ raise cmerror.CMError('Not implemented')
+
+ # pylint: disable=no-self-use, unused-argument
+ def get_domains(self):
+ raise cmerror.CMError('Not implemented')
+
+ # pylint: disable=no-self-use, unused-argument
+ def delete(self, domain, name):
+ raise cmerror.CMError('Not implemented')
+
+ # pylint: disable=no-self-use, unused-argument
+ def delete_domain(self, domain):
+ raise cmerror.CMError('Not implemented')
+
+if __name__ == '__main__':
+ pass
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import logging
+
+from cmframework.lib.cmupdateimpl import CMUpdateImpl
+
+
+class CMUpdate(object):
+ def __init__(self, plugins_path, server_ip='config-manager', server_port=61100,
+ client_lib_impl_module='cmframework.lib.CMClientImpl', verbose_logger=None):
+ logging.info('CMUpdate constructor, plugins_path is %s', plugins_path)
+
+ self._update_lib = CMUpdateImpl(plugins_path,
+ server_ip,
+ server_port,
+ client_lib_impl_module,
+ verbose_logger)
+
+ def update(self, confman=None):
+ self._update_lib.update(confman)
+
+ def wait_activation(self):
+ self._update_lib.wait_activation()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from cmframework.apis.cmhandler import CMHandler
+
+
+class CMUpdateHandler(CMHandler):
+ def update(self, confman):
+ raise NotImplementedError('Not implemented')
+
+ def validation_failed(self, confman):
+ pass
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from cmframework.apis import cmerror
+
+
+class CMUserConfigPlugin(object):
+
+ def __init__(self):
+ pass
+
+ # pylint: disable=no-self-use,unused-argument
+ def handle(self, confman):
+ """provide configuration to CM by interpreting user config
+
+ Arguments:
+
+ confman: The configuration manager object used to manipulate the
+ configuration data.
+
+ Raise:
+
+ CMError is raised in-case of failure
+ """
+
+ raise cmerror.CMError('No implemented')
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from cmframework.apis import cmerror
+
+
+class CMValidator(object):
+ def __init__(self):
+ self.plugin_client = None
+
+ # pylint: disable=no-self-use
+ def get_subscription_info(self):
+ """get the subscription filter
+
+ This API is used to get the re for matching the properties which the
+ validation plugin is concerned about.
+
+ Return:
+
+ A string representing the regular expression used to match the
+ properties which the validation plugin is concerned about.
+
+ Raise:
+
+ CMError can be raised in-case of a failure.
+ """
+ raise cmerror.CMError('Not implemented')
+
+ # pylint: disable=no-self-use, unused-argument
+ def validate_set(self, props):
+ """validate a configuration data addition/update
+
+ Arguments:
+
+ props: A dictionary of name-value pairs representing the changed
+ properties.
+
+ Raise:
+
+ CMError can be raised if the change is not accepted by this plugin.
+ """
+ raise cmerror.CMError('Not implemented')
+
+ # pylint: disable=no-self-use, unused-argument
+ def validate_delete(self, props):
+ """validate a configuration data delete
+
+ Arguments:
+
+ props: A list containing the names of deleted properties
+
+ Raise:
+
+ CMError can raised if the delete is not accepted by this plugin.
+ """
+ raise cmerror.CMError('Not implemented')
+
+ def get_plugin_client(self):
+ """get the plugin client object
+
+ This API can be used by the plugin to get the client object which the
+ plugin can use to access the configuration data. Notice that the data
+ accessed by this is what is stored in the backend. The changed data
+ is passed as argument to the different validate functions.
+
+ Return:
+
+ The plugin client object
+ """
+ return self.plugin_client
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+#! /usr/bin/python
+
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from __future__ import print_function
+import sys
+from cmframework.cli import cmcliprocessor
+
+
+def main():
+ try:
+ processor = cmcliprocessor.CMCLIProcessor('cmcli')
+ args = sys.argv[1:]
+ processor(args)
+ except Exception as exp: # pylint: disable=broad-except
+ print('Failed with error: %s' % str(exp))
+ return 1
+
+
+if __name__ == '__main__':
+ sys.exit(main())
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from __future__ import print_function
+import sys
+import inspect
+import socket
+import pprint
+import prettytable
+
+from cmframework.apis import cmmanage
+from cmframework.apis import cmerror
+
+
+class VerboseLogger(object):
+ def __call__(self, msg):
+ print(msg)
+
+
+class CMCLIHandler(object):
+ def __init__(self):
+ self.api = None
+ self.verbose_logger = VerboseLogger()
+
+ def _init_api(self, ip, port, client_lib, verbose):
+ logger = None
+ if verbose:
+ logger = self.verbose_logger
+
+ try:
+ serverip = socket.gethostbyname(ip)
+ except Exception: # pylint: disable=broad-except
+ # use localhost in-case we cannot resolve the provided hostname
+ serverip = '127.0.0.1'
+
+ self.api = cmmanage.CMManage(serverip, port, client_lib, logger)
+
+ def set_handler(self, subparser):
+ subparser.set_defaults(handler=self)
+
+ # pylint: disable=no-self-use, unused-argument
+ def init_subparser(self, subparsers):
+ raise cmerror.CMError('Not implemented')
+
+ # pylint: disable=no-self-use, unused-argument
+ def __call__(self, args):
+ raise cmerror.CMError('Not implemented')
+
+
+class CMCLIGetPropertyHandler(CMCLIHandler):
+ def init_subparser(self, subparsers):
+ subparser = subparsers.add_parser('get-property', help='Get a property value')
+ subparser.add_argument('--property',
+ required=True,
+ dest='name',
+ metavar='NAME',
+ action='store')
+ subparser.add_argument('--snapshot',
+ required=False,
+ dest='snapshot',
+ metavar='SNAPSHOT-NAME',
+ action='store')
+ self.set_handler(subparser)
+
+ def __call__(self, args):
+ self._init_api(args.ip, args.port, args.client_lib, args.verbose)
+ name = args.name
+ snapshot = args.snapshot
+ value = self.api.get_property(name, snapshot)
+ print(value)
+
+
+class CMCLIGetPropertiesHandler(CMCLIHandler):
+ def init_subparser(self, subparsers):
+ subparser = subparsers.add_parser('get-properties',
+ help='Get the properties matching a filter')
+ subparser.add_argument('--matching-filter',
+ required=True,
+ dest='matching_filter',
+ metavar='MATCHING-FILTER',
+ action='store')
+ subparser.add_argument('--snapshot',
+ required=False,
+ dest='snapshot',
+ metavar='SNAPSHOT-NAME',
+ action='store')
+ self.set_handler(subparser)
+
+ def __call__(self, args):
+ self._init_api(args.ip, args.port, args.client_lib, args.verbose)
+ matching_filter = args.matching_filter
+ snapshot = args.snapshot
+ props = self.api.get_properties(matching_filter, snapshot)
+ for name, value in props.iteritems():
+ print('%s=%s' % (name, value))
+
+
+class CMCLISetPropertyHandler(CMCLIHandler):
+ def init_subparser(self, subparsers):
+ subparser = subparsers.add_parser('set-property', help='Set a property')
+ subparser.add_argument('--property',
+ required=True,
+ dest='prop',
+ metavar='NAME VALUE',
+ action='store',
+ nargs=2)
+ self.set_handler(subparser)
+
+ def __call__(self, args):
+ self._init_api(args.ip, args.port, args.client_lib, args.verbose)
+ prop = args.prop
+ change_uuid = self.api.set_property(prop[0], prop[1])
+ print("change-uuid:%s" % change_uuid)
+
+
+class CMCLISetPropertiesHandler(CMCLIHandler):
+ def init_subparser(self, subparsers):
+ subparser = subparsers.add_parser('set-properties', help='Set a group of properties')
+ subparser.add_argument('--property',
+ required=True,
+ dest='props',
+ metavar='NAME VALUE',
+ action='append',
+ nargs=2)
+ self.set_handler(subparser)
+
+ def __call__(self, args):
+ self._init_api(args.ip, args.port, args.client_lib, args.verbose)
+ props = args.props
+ dic = {}
+ for prop in props:
+ name = prop[0]
+ value = prop[1]
+ dic[name] = value
+ change_uuid = self.api.set_properties(dic)
+ print("change-uuid:%s" % change_uuid)
+
+
+class CMCLIDeletePropertyHandler(CMCLIHandler):
+ def init_subparser(self, subparsers):
+ subparser = subparsers.add_parser('delete-property', help='Delete a property')
+ subparser.add_argument('--property',
+ required=True,
+ dest='name',
+ metavar='NAME',
+ action='store')
+ self.set_handler(subparser)
+
+ def __call__(self, args):
+ self._init_api(args.ip, args.port, args.client_lib, args.verbose)
+ name = args.name
+ change_uuid = self.api.delete_property(name)
+ print("change-uuid:%s" % change_uuid)
+
+
+class CMCLIDeletePropertiesWithFilterHandler(CMCLIHandler):
+ def init_subparser(self, subparsers):
+ subparser = subparsers.add_parser('delete-properties-with-filter',
+ help='Delete properties matching a filter')
+ subparser.add_argument('--matching-filter',
+ required=True,
+ dest='matching_filter',
+ metavar='MATCHING-FILTER',
+ action='store')
+ self.set_handler(subparser)
+
+ def __call__(self, args):
+ self._init_api(args.ip, args.port, args.client_lib, args.verbose)
+ matching_filter = args.matching_filter
+ change_uuid = self.api.delete_properties(matching_filter)
+ print("change-uuid:%s" % change_uuid)
+
+
+class CMCLIDeletePropertiesHandler(CMCLIHandler):
+ def init_subparser(self, subparsers):
+ subparser = subparsers.add_parser('delete-properties', help='Delete a group of properties')
+ subparser.add_argument('--property',
+ required=True,
+ dest='props',
+ metavar='PROPERY-NAME',
+ action='append')
+ self.set_handler(subparser)
+
+ def __call__(self, args):
+ self._init_api(args.ip, args.port, args.client_lib, args.verbose)
+ props = args.props
+ change_uuid = self.api.delete_properties(props)
+ print("change-uuid:%s" % change_uuid)
+
+
+class CMCLICreateSnapshotHandler(CMCLIHandler):
+ def init_subparser(self, subparsers):
+ subparser = subparsers.add_parser('create-snapshot',
+ help='Take a snapshot of the configuration data')
+ subparser.add_argument('--name',
+ required=True,
+ dest='snapshot_full_name',
+ metavar='SNAPSHOT-FULL-NAME',
+ action='store')
+ self.set_handler(subparser)
+
+ def __call__(self, args):
+ self._init_api(args.ip, args.port, args.client_lib, args.verbose)
+ snapshot = args.snapshot_full_name
+ self.api.create_snapshot(snapshot)
+
+
+class CMCLIRestoreSnapshotHandler(CMCLIHandler):
+ def init_subparser(self, subparsers):
+ subparser = subparsers.add_parser('restore-snapshot',
+ help='Restore a configuration snapshot')
+ subparser.add_argument('--name',
+ required=True,
+ dest='snapshot_full_name',
+ metavar='SNAPSHOT-FULL-NAME',
+ action='store')
+ self.set_handler(subparser)
+
+ def __call__(self, args):
+ self._init_api(args.ip, args.port, args.client_lib, args.verbose)
+ snapshot = args.snapshot_full_name
+ self.api.restore_snapshot(snapshot)
+
+
+class CMCLIDeleteSnapshotHandler(CMCLIHandler):
+ def init_subparser(self, subparsers):
+ subparser = subparsers.add_parser('delete-snapshot', help='Delete a configuration snapshot')
+ subparser.add_argument('--name',
+ required=True,
+ dest='snapshot_full_name',
+ metavar='SNAPSHOT-FULL-NAME',
+ action='store')
+ self.set_handler(subparser)
+
+ def __call__(self, args):
+ self._init_api(args.ip, args.port, args.client_lib, args.verbose)
+ snapshot = args.snapshot_full_name
+ self.api.delete_snapshot(snapshot)
+
+
+class CMCLIListSnapshotHandler(CMCLIHandler):
+ def init_subparser(self, subparsers):
+ subparser = subparsers.add_parser('list-snapshots', help='List all configuration snapshots')
+ self.set_handler(subparser)
+
+ def __call__(self, args):
+ self._init_api(args.ip, args.port, args.client_lib, args.verbose)
+ snapshots = self.api.list_snapshots()
+ sorted_snapshots = sorted(snapshots, key=lambda k: k['creation_date'])
+
+ t = prettytable.PrettyTable(['Name', 'Date', 'Release', 'Build'])
+ t.set_style(prettytable.PLAIN_COLUMNS)
+ t.align = 'l'
+ for snapshot in sorted_snapshots:
+ t.add_row([snapshot['name'],
+ snapshot['creation_date'],
+ snapshot['release'],
+ snapshot['build']])
+ print(t)
+
+
+class CMCLIActivateHandler(CMCLIHandler):
+ def init_subparser(self, subparsers):
+ subparser = subparsers.add_parser(
+ 'activate',
+ help='Activate the configuration in all or specified node')
+ subparser.add_argument('--node-name',
+ required=False,
+ dest='node_name',
+ metavar='NODE-NAME',
+ action='store')
+ self.set_handler(subparser)
+
+ def __call__(self, args):
+ self._init_api(args.ip, args.port, args.client_lib, args.verbose)
+ node_name = args.node_name
+ self.api.activate(node_name)
+
+
+class CMCLIRebootHandler(CMCLIHandler):
+ def init_subparser(self, subparsers):
+ subparser = subparsers.add_parser('reboot-request',
+ help=('Request reboot of a specified node '
+ 'during activation'))
+ subparser.add_argument('--node-name',
+ required=True,
+ dest='node_name',
+ metavar='NODE-NAME',
+ action='store')
+ self.set_handler(subparser)
+
+ def __call__(self, args):
+ self._init_api(args.ip, args.port, args.client_lib, args.verbose)
+ node_name = args.node_name
+ self.api.reboot_node(node_name)
+
+
+class CMCLIBootstrapHandler(CMCLIHandler):
+ def init_subparser(self, subparsers):
+ subparser = subparsers.add_parser('bootstrap',
+ help='Bootsrap the backend with the user config data')
+ subparser.add_argument('--config',
+ required=True,
+ dest='config',
+ metavar='INITIAL-USER-CONFIG',
+ action='store')
+ subparser.add_argument('--plugin_path',
+ required=True,
+ dest='plugin_path',
+ metavar='BOOTSTRAP-PLUGIN-PATH',
+ action='store')
+ self.set_handler(subparser)
+
+ def __call__(self, args):
+ self._init_api(args.ip, args.port, args.client_lib, args.verbose)
+ from cmframework.utils.cmuserconfig import UserConfig
+ uc = UserConfig(args.config, args.plugin_path)
+ flat_config = uc.get_flat_config()
+ self.api.set_properties(flat_config)
+
+
+class CMCLIAnsibleInventoryHandler(CMCLIHandler):
+ def init_subparser(self, subparsers):
+ subparser = subparsers.add_parser('ansible-inventory',
+ help='Prints the ansible inventory json to output')
+ subparser.add_argument('--plugin_path',
+ required=False,
+ default='/opt/cmframework/inventoryhandlers',
+ dest='plugin_path',
+ metavar='INVENTORY-HANDLERS-PLUGIN-PATH',
+ action='store')
+ self.set_handler(subparser)
+
+ def __call__(self, args):
+ self._init_api(args.ip, args.port, args.client_lib, args.verbose)
+ from cmframework.utils.cmansibleinventory import AnsibleInventory
+ import json
+
+ properties = self.api.get_properties('.*')
+
+ inventory = AnsibleInventory(properties, args.plugin_path)
+ inv = inventory.generate_inventory()
+
+ print (json.dumps(inv, indent=4, sort_keys=True))
+
+
+class CMAnsiblePlaybookHandler(CMCLIHandler):
+ def init_subparser(self, subparsers):
+ subparser = subparsers.add_parser('ansible-playbooks-generate',
+ help=('Generate the ansible playbooks for the '
+ 'bootstrapping, provisioning, '
+ 'postconfig and finalizing phases'))
+ subparser.add_argument('--bootstrapping-playbooks-path',
+ required=False,
+ default='/etc/lcm/playbooks/installation/bootstrapping',
+ dest='bootstrapping_path',
+ metavar='BOOTSTRAPPING-PLAYBOOKS-PATH',
+ action='store')
+ subparser.add_argument('--provisioning-playbooks-path',
+ required=False,
+ default='/etc/lcm/playbooks/installation/provisioning',
+ dest='provisioning_path',
+ metavar='PROVISIONING-PLAYBOOKS-PATH',
+ action='store')
+ subparser.add_argument('--postconfig-playbooks-path',
+ required=False,
+ default='/etc/lcm/playbooks/installation/postconfig',
+ dest='postconfig_path',
+ metavar='POSTCONFIG-PLAYBOOKS-PATH',
+ action='store')
+ subparser.add_argument('--finalize-playbooks-path',
+ required=False,
+ default='/etc/lcm/playbooks/installation/finalize',
+ dest='finalize_path',
+ metavar='FINALIZE-PLAYBOOKS-PATH',
+ action='store')
+ subparser.add_argument('--destination-path',
+ required=False,
+ default='/opt/openstack-ansible/playbooks/',
+ dest='destination_path',
+ metavar='DESTINATION-PATH',
+ action='store')
+ self.set_handler(subparser)
+
+ def __call__(self, args):
+ from cmframework.utils.cmansibleplaybooks import AnsiblePlaybooks
+
+ playbooks = AnsiblePlaybooks(args.destination_path, args.bootstrapping_path,
+ args.provisioning_path, args.postconfig_path,
+ args.finalize_path)
+ playbooks.generate_playbooks()
+
+
+class CMCLIDisableAutomaticActivationHandler(CMCLIHandler):
+ def init_subparser(self, subparsers):
+ subparser = subparsers.add_parser('disable-automatic-activation',
+ help='Disable automatic activation')
+ self.set_handler(subparser)
+
+ def __call__(self, args):
+ self._init_api(args.ip, args.port, args.client_lib, args.verbose)
+ self.api.disable_automatic_activation()
+
+
+class CMCLIEnableAutomaticActivationHandler(CMCLIHandler):
+ def init_subparser(self, subparsers):
+ subparser = subparsers.add_parser('enable-automatic-activation',
+ help='Enable automatic activation')
+ self.set_handler(subparser)
+
+ def __call__(self, args):
+ self._init_api(args.ip, args.port, args.client_lib, args.verbose)
+ self.api.enable_automatic_activation()
+
+
+class CMCLIGetChangesStatesHandler(CMCLIHandler):
+ def init_subparser(self, subparsers):
+ subparser = subparsers.add_parser('get-changes-states',
+ help='Get the configuration changes states')
+ subparser.add_argument('--uuid',
+ required=False,
+ dest='change_uuid',
+ metavar='CHANGE_UUID',
+ action='store')
+ self.set_handler(subparser)
+
+ def __call__(self, args):
+ self._init_api(args.ip, args.port, args.client_lib, args.verbose)
+ change_uuid = None
+ if args.change_uuid:
+ change_uuid = args.change_uuid
+
+ result = self.api.get_changes_states(change_uuid)
+ pp = pprint.PrettyPrinter(indent=4)
+ pp.pprint(result)
+
+
+def get_handlers_list():
+ handlers = []
+ for name, obj in inspect.getmembers(sys.modules[__name__]):
+ if inspect.isclass(obj):
+ if name != 'CMCLIHandler':
+ if issubclass(obj, CMCLIHandler):
+ handlers.append(obj())
+ return handlers
+
+
+def main():
+ handlers = get_handlers_list()
+ for handler in handlers:
+ print('handler is ', handler)
+
+
+if __name__ == '__main__':
+ main()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from __future__ import print_function
+import sys
+import argparse
+
+from cmframework.apis import cmerror
+from cmframework.cli import cmclihandlers
+
+
+class CMCLIProcessor(object):
+ def __init__(self, prog):
+ self.prog = prog
+
+ def __call__(self, args):
+ parser = argparse.ArgumentParser(description='Configuration Management CLI', prog=self.prog)
+ parser.add_argument('--ip',
+ dest='ip',
+ metavar='SERVER-IP',
+ default='config-manager',
+ type=str,
+ action='store')
+
+ parser.add_argument('--port',
+ dest='port',
+ metavar='SERVER-PORT',
+ default=61100,
+ type=int,
+ action='store')
+
+ parser.add_argument('--client-lib',
+ dest='client_lib',
+ metavar='CLIENT-LIB',
+ default='cmframework.lib.CMClientImpl',
+ type=str,
+ action='store')
+
+ parser.add_argument('--verbose',
+ required=False,
+ default=False,
+ action='store_true')
+
+ subparsers = parser.add_subparsers()
+ handlers = cmclihandlers.get_handlers_list()
+ for handler in handlers:
+ handler.init_subparser(subparsers)
+
+ parse_result = parser.parse_args(args)
+
+ try:
+ parse_result.handler(parse_result)
+ except cmerror.CMError:
+ raise
+ except Exception as exp:
+ raise cmerror.CMError(str(exp))
+
+
+def main():
+ processor = CMCLIProcessor('cmcli')
+ args = sys.argv[1:]
+ try:
+ processor(args)
+ except cmerror.CMError as error:
+ print('Got error %s' % str(error))
+ sys.exit(1)
+
+
+if __name__ == '__main__':
+ main()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from cmfilebackend import CMFileBackend
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from __future__ import print_function
+import logging
+import re
+import os
+import stat
+
+from cmframework.apis import cmbackend
+from cmframework.apis import cmerror
+
+
+class CMFileBackend(cmbackend.CMBackend):
+ def __init__(self, **kw):
+ self.uri = kw['uri']
+ logging.debug('CMFileBackend constructor called, uri=%s', self.uri)
+ self.data = {}
+ self.load_file()
+
+ def load_file(self):
+ logging.debug('load_file started')
+ try:
+ self.data = {}
+ with open(self.uri) as f:
+ lines = f.read().splitlines()
+ for line in lines:
+ name, var = line.partition('=')[::2]
+ logging.debug('Adding %s=%s', name, var)
+ self.data[name.strip()] = var
+ f.close()
+ except IOError:
+ logging.debug('File %s does not exist', self.uri)
+
+ def write_file(self):
+ logging.debug('write_file started')
+ try:
+ with open(self.uri, 'w') as f:
+ os.chmod(self.uri, stat.S_IRUSR | stat.S_IWUSR)
+ for key, value in self.data.iteritems():
+ logging.debug('Writing %s=%s', key, value)
+ f.write(key + '=' + value + '\n')
+ f.flush()
+ os.fsync(f.fileno())
+ except IOError as exp:
+ raise cmerror.CMError(str(exp))
+
+ def get_property(self, prop_name):
+ logging.debug('get_property called for %s', prop_name)
+ try:
+ value = self.data[prop_name]
+ return value
+ except KeyError:
+ raise cmerror.CMError('Invalid property name')
+
+ def get_properties(self, prop_filter):
+ logging.debug('get_properties called with filter %s', prop_filter)
+ returned = {}
+ pattern = re.compile(prop_filter)
+ for key, value in self.data.iteritems():
+ logging.debug('Matching %s against %s', key, prop_filter)
+ if pattern.match(key):
+ logging.debug('Adding %s', key)
+ returned[key] = value
+ return returned
+
+ def set_property(self, prop_name, prop_value):
+ logging.debug('set_property %s=%s', prop_name, prop_value)
+ props = {}
+ props[prop_name] = prop_value
+ self.set_properties(props)
+
+ def set_properties(self, properties):
+ logging.debug('set_properties called props=%s', str(properties))
+ try:
+ for key, value in properties.iteritems():
+ self.data[key] = value
+ self.write_file()
+ except cmerror.CMError:
+ self.load_file()
+ raise
+
+ def delete_property(self, prop_name):
+ logging.debug('delete_property called for %s', prop_name)
+ try:
+ del self.data[prop_name]
+ self.write_file()
+ except KeyError:
+ logging.debug('Property not found')
+ raise cmerror.CMError('Property not found')
+ except cmerror.CMError:
+ self.load_file()
+ raise
+
+ def delete_properties(self, arg):
+ logging.debug('delete_properties called with arg %s', arg)
+ try:
+ if isinstance(arg, str):
+ pattern = re.compile(arg)
+ for key in self.data.keys(): # pylint: disable=consider-iterating-dictionary
+ if pattern.match(key):
+ del self.data[key]
+ else:
+ for prop in arg:
+ for key in self.data.keys(): # pylint: disable=consider-iterating-dictionary
+ if key == prop:
+ del self.data[key]
+ self.write_file()
+ except cmerror.CMError:
+ self.load_file()
+ raise
+
+
+def main():
+ import sys
+
+ filepath = sys.argv[1]
+ try:
+ print('Initializing backend at %s' % filepath)
+ filebackend = CMFileBackend(uri=filepath)
+ print('Adding key1=value1')
+ filebackend.set_property("key1", "value1")
+ properties = {'key2': 'value2', 'key3': 'value3'}
+ print('Adding %s' % str(properties))
+ filebackend.set_properties(properties)
+ print('Getting key1')
+ value = filebackend.get_property('key1')
+ print('value of key1 is %s' % value)
+ print('Deleting key2')
+ filebackend.delete_property('key2')
+ print('Getting *')
+ properties = filebackend.get_properties('.*')
+ print('Got %s' % str(properties))
+ print('Delete all properties')
+ filebackend.delete_properties('.*')
+ except cmerror.CMError as exp:
+ print('Got exeption %s' % str(exp))
+ sys.exit(1)
+
+
+if __name__ == '__main__':
+ main()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from cmclientimpl import CMClientImpl
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import logging
+
+
+class AlarmHandler_Dummy(object):
+ def handle_alarm_work(self, work):
+ logging.debug('AlarmHandler skipping work: %s', work)
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import json
+import time
+import requests
+
+from cmframework.apis import cmerror
+from cmframework.apis import cmchangestate
+
+
+class CMClientImpl(object):
+ def __init__(self, server_ip, server_port, verbose_logger):
+ self.version = 'v1.0'
+ self.server_ip = server_ip
+ self.server_port = server_port
+ base_url = str.format('http://{ip}:{port}/cm/{api}', ip=self.server_ip,
+ port=self.server_port, api=self.version)
+ self.props_base_url = str.format('{base}/properties', base=base_url)
+ self.snapshots_base_url = str.format('{base}/snapshots', base=base_url)
+ self.activator_url = str.format('{base}/activator', base=base_url)
+ self.reboot_url = str.format('{base}/reboot', base=base_url)
+ self.changes_url = str.format('{base}/changes', base=base_url)
+ self.verbose_logger = verbose_logger
+
+ def get_property(self, prop_name, snapshot_name=None):
+ resource = str.format('{base}/{prop}', base=self.props_base_url, prop=prop_name)
+ if snapshot_name:
+ resource = str.format('{}?snapshot={snapshot}', resource, snapshot=snapshot_name)
+ result = self._get_rpc(resource)
+ try:
+ value = result['value']
+ except KeyError:
+ raise cmerror.CMError('Invalid response')
+ except TypeError:
+ raise cmerror.CMError('Invalid response')
+ except Exception as exp: # pylint: disable=broad-except
+ raise cmerror.CMError(str(exp))
+ return value
+
+ def get_properties(self, prop_filter, snapshot_name=None):
+ resource = str.format('{base}?prop-name-filter={f}', base=self.props_base_url,
+ f=prop_filter)
+ if snapshot_name:
+ resource = str.format('{}&snapshot={snapshot}', resource,
+ snapshot=snapshot_name)
+ result = self._get_rpc(resource)
+ props = {}
+ try:
+ properties = result['properties']
+ for item in properties:
+ name = item['name']
+ value = item['value']
+ props[name] = value
+ except KeyError as exp:
+ raise cmerror.CMError('Invalid response')
+ except TypeError as exp:
+ raise cmerror.CMError('Invalid response')
+ except Exception as exp:
+ raise cmerror.CMError(str(exp))
+ return props
+
+ def set_property(self, prop_name, prop_value):
+ resource = str.format('{base}/{prop}', base=self.props_base_url, prop=prop_name)
+ body = {}
+ body['value'] = prop_value
+ result = self._post_rpc(resource, body)
+ return result['change-uuid']
+
+ def set_properties(self, props, overwrite=False):
+ body = {}
+ items = []
+ for key, value in props.iteritems():
+ item = {}
+ item['name'] = key
+ item['value'] = value
+ items.append(item)
+ body['overwrite'] = overwrite
+ body['properties'] = items
+ result = self._post_rpc(self.props_base_url, body)
+ return result['change-uuid']
+
+ def delete_property(self, prop_name):
+ resource = str.format('{base}/{prop}', base=self.props_base_url, prop=prop_name)
+ result = self._delete_rpc(resource, None)
+ return result['change-uuid']
+
+ def delete_properties(self, arg):
+ result = {}
+ if isinstance(arg, str):
+ resource = str.format('{base}?prop-name-filter={f}', base=self.props_base_url, f=arg)
+ result = self._delete_rpc(resource, None)
+ else:
+ resource = str.format('{base}', base=self.props_base_url)
+ body = {}
+ body['properties'] = arg
+ result = self._delete_rpc(resource, body)
+ return result['change-uuid']
+
+ def create_snapshot(self, snapshot_name):
+ resource = str.format('{base}/{snapshot}',
+ base=self.snapshots_base_url,
+ snapshot=snapshot_name)
+ self._get_rpc(resource)
+
+ def restore_snapshot(self, snapshot_name):
+ resource = str.format('{base}/{snapshot}',
+ base=self.snapshots_base_url,
+ snapshot=snapshot_name)
+ self._post_rpc(resource, None)
+
+ def delete_snapshot(self, snapshot_name):
+ resource = str.format('{base}/{snapshot}',
+ base=self.snapshots_base_url,
+ snapshot=snapshot_name)
+ self._delete_rpc(resource, None)
+
+ def list_snapshots(self):
+ resource = str.format('{base}', base=self.snapshots_base_url)
+ result = self._get_rpc(resource)
+
+ return result['snapshots']
+
+ def activate(self, node_name):
+ if not node_name:
+ resource = str.format('{base}', base=self.activator_url)
+ else:
+ resource = str.format('{base}/{node}', base=self.activator_url, node=node_name)
+ result = self._post_rpc(resource, None)
+
+ return result['change-uuid']
+
+ def activate_node(self, node_name):
+ resource = str.format('{base}/agent/{node}', base=self.activator_url, node=node_name)
+ result = self._get_rpc(resource)
+ try:
+ return result['reboot']
+ except KeyError as exp:
+ raise cmerror.CMError('Invalid response')
+ except TypeError as exp:
+ raise cmerror.CMError('Invalid response')
+ except Exception as exp:
+ raise cmerror.CMError(str(exp))
+
+ def reboot_node(self, node_name):
+ resource = str.format('{base}?node-name={f}', base=self.reboot_url, f=node_name)
+ result = self._get_rpc(resource)
+ try:
+ return result['node-name']
+ except KeyError as exp:
+ raise cmerror.CMError('Invalid response')
+ except TypeError as exp:
+ raise cmerror.CMError('Invalid response')
+ except Exception as exp:
+ raise cmerror.CMError(str(exp))
+
+ def enable_automatic_activation(self):
+ resource = str.format('{base}/enable', base=self.activator_url)
+ self._post_rpc(resource, None)
+
+ def disable_automatic_activation(self):
+ resource = str.format('{base}/disable', base=self.activator_url)
+ self._post_rpc(resource, None)
+
+ def get_changes_states(self, change_uuid):
+ if change_uuid:
+ resource = str.format('{base}?change-uuid-filter={change_uuid}',
+ base=self.changes_url,
+ change_uuid=change_uuid)
+ else:
+ resource = str.format('{base}', base=self.changes_url)
+ result = self._get_rpc(resource)
+ return result
+
+ def wait_activation(self, change_uuid):
+ self.verbose_log('Waiting for activation (%s) to finish' % change_uuid)
+ state = None
+ failed_plugins = None
+ while True:
+ try:
+ changes = self.get_changes_states(change_uuid)
+ state = changes[change_uuid]['state']
+ failed_plugins = changes[change_uuid]['failed-plugins']
+ self.verbose_log('State of change is %s' % state)
+ if state != cmchangestate.CM_CHANGE_STATE_ONGOING:
+ break
+ time.sleep(5)
+ except Exception as exp: # pylint: disable=broad-except
+ raise cmerror.CMError(str(exp))
+
+ if state != cmchangestate.CM_CHANGE_STATE_OK:
+ raise cmerror.CMError("Activation was unsuccessful! Failed plugins: {}"
+ .format(failed_plugins))
+
+ def verbose_log(self, msg):
+ if self.verbose_logger:
+ self.verbose_logger(msg)
+
+ def _get_rpc(self, resource):
+ self.verbose_log('Sending GET %s' % resource)
+ response = requests.get(resource)
+ return self._handle_response(response)
+
+ def _post_rpc(self, resource, body):
+ self.verbose_log('Sending POST %s' % resource)
+ self.verbose_log(' BODY %s' % body)
+ if body:
+ headers = {}
+ headers['Content-type'] = 'application/json'
+ response = requests.post(resource, data=json.dumps(body), headers=headers)
+ else:
+ response = requests.post(resource)
+
+ return self._handle_response(response)
+
+ def _delete_rpc(self, resource, body):
+ self.verbose_log('Sending DELETE %s' % resource)
+ self.verbose_log(' BODY %s' % body)
+ if body:
+ headers = {}
+ headers['Content-type'] = 'application/json'
+ response = requests.delete(resource, data=json.dumps(body), headers=headers)
+ else:
+ headers = {}
+ headers['Content-type'] = 'text'
+ response = requests.delete(resource)
+ return self._handle_response(response)
+
+ def _handle_response(self, response):
+ self.verbose_log('Got STATUS %s' % response.reason)
+ self.verbose_log(' CONTENT %s' % response.content)
+ if not response.ok:
+ raise cmerror.CMError(response.reason)
+
+ try:
+ if response.content:
+ return response.json()
+ except ValueError as exp:
+ raise cmerror.CMError(str(exp))
+ return None
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import logging
+import time
+
+from cmframework.apis.cmerror import CMError
+from cmframework.apis.cmmanage import CMManage
+from cmframework.utils.cmpluginloader import CMPluginLoader
+from cmframework.utils.cmdependencysort import CMDependencySort
+
+from cmdatahandlers.api.configmanager import ConfigManager
+from cmdatahandlers.api import utils
+
+
+class CMUpdateImpl(object):
+ def __init__(self, plugins_path, server_ip='config-manager', server_port=61100,
+ client_lib_impl_module='cmframework.lib.CMClientImpl', verbose_logger=None):
+ logging.info('CMUpdateImpl constructor, plugins_path is %s', plugins_path)
+
+ self._plugins_path = plugins_path
+ self._handlers = {}
+ self._sorted_handlers = []
+
+ self._load_handlers()
+ self._load_dependencies()
+
+ self.uuid_value = None
+
+ self.api = CMManage(server_ip, server_port, client_lib_impl_module, verbose_logger)
+
+ def _load_handlers(self):
+ loader = CMPluginLoader(self._plugins_path)
+ handler_modules, _ = loader.load()
+ logging.info('Handler module(s): %r', handler_modules)
+
+ for handler_class_name, module in handler_modules.iteritems():
+ handler_class = getattr(module, handler_class_name)
+ handler = handler_class()
+ self._handlers[handler_class_name] = handler
+
+ def update(self, confman=None):
+ logging.info('Taking snapshot of the original configuration just in-case')
+ now = int(round(time.time()*1000))
+ snapshot_name = 'cmupdate-' + str(now)
+ self.api.create_snapshot(snapshot_name)
+ if not confman:
+ properties = self.api.get_properties('.*')
+ propsjson = utils.unflatten_config_data(properties)
+ confman = ConfigManager(propsjson)
+
+ for handler in self._sorted_handlers:
+ try:
+ logging.debug('Calling update for %s', handler)
+ self._handlers[handler].update(confman)
+ logging.debug('update for %s done', handler)
+ except Exception as ex: # pylint: disable=broad-except
+ logging.warning('Update handler %s failed: %s', handler, str(ex))
+ raise
+
+ properties = confman.get_config()
+ flatprops = utils.flatten_config_data(properties)
+
+ try:
+ self.uuid_value = self.api.set_properties(flatprops, True)
+ except Exception as exp: # pylint: disable=broad-except
+ for handler in self._sorted_handlers:
+ try:
+ logging.debug('Calling validation_failed for %s', handler)
+ self._handlers[handler].validation_failed(confman)
+ except Exception as ex2: # pylint: disable=broad-except
+ logging.warning('Update handler %s validation_failed raised exception: %s',
+ handler, str(ex2))
+ raise exp
+
+ def wait_activation(self):
+ if self.uuid_value:
+ self.api.wait_activation(self.uuid_value)
+
+ @staticmethod
+ def _read_dependency_file(file_name):
+ before_list = []
+ after_list = []
+
+ try:
+ with open(file_name, 'r') as deps_file:
+ while True:
+ line = deps_file.readline()
+ if not line:
+ break
+ if line.startswith('Before:'):
+ before_list = line[7:].strip().replace(' ', '').split(',')
+ if line.startswith('After:'):
+ after_list = line[6:].strip().replace(' ', '').split(',')
+ except IOError:
+ logging.debug('Dependency file %s not found.', file_name)
+
+ return (before_list, after_list)
+
+ def _load_dependencies(self):
+ before_graph = {}
+ after_graph = {}
+
+ for handler in self._handlers.values():
+ deps_filename = '{}/{}.deps'.format(self._plugins_path, handler)
+ before_list, after_list = self._read_dependency_file(deps_filename)
+
+ for before_dep in before_list:
+ if not self._handlers.get(before_dep, None):
+ raise CMError(
+ 'Unexisting handler {} referred in handler {}\'s "Before" dependencies'
+ .format(before_dep, handler))
+ for after_dep in after_list:
+ if not self._handlers.get(after_dep, None):
+ raise CMError(
+ 'Unexisting handler {} referred in handler {}\'s "After" dependencies'
+ .format(after_dep, handler))
+
+ before_graph[str(handler)] = before_list
+ after_graph[str(handler)] = after_list
+
+ sorter = CMDependencySort(after_graph, before_graph)
+ self._sorted_handlers = sorter.sort()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# vim: tabstop=4 expandtab shiftwidth=4 softtabstop=4
+from __future__ import print_function
+from urlparse import urlparse
+import re
+import time
+import functools
+import redis
+
+from cmframework.apis import cmerror
+from cmframework.apis import cmbackend
+
+
+def retry(func):
+ @functools.wraps(func)
+ def wrapper(*args, **kwargs):
+ for _ in xrange(20):
+ try:
+ return func(*args, **kwargs)
+ except Exception, e: # pylint: disable=broad-except
+ time.sleep(1)
+ retExc = e
+ raise retExc
+
+ return wrapper
+
+
+class CMRedisDB(cmbackend.CMBackend):
+
+ def __init__(self, **kw):
+ dburl = kw['uri']
+ urldata = urlparse(dburl)
+ self.vip = urldata.hostname
+ self.port = urldata.port
+ self.password = urldata.password
+ self.client = redis.StrictRedis(host=self.vip, port=self.port, password=self.password)
+
+ @retry
+ def set_property(self, prop_name, prop_value):
+ self.client.set(prop_name, prop_value)
+
+ def get_property(self, prop_name):
+ return self.client.get(prop_name)
+
+ @retry
+ def delete_property(self, prop_name):
+ self.client.delete(prop_name)
+
+ @retry
+ def set_properties(self, properties):
+ pipe = self.client.pipeline()
+ for name, value in properties.iteritems():
+ pipe.set(name, value)
+ pipe.execute()
+
+ def get_properties(self, prop_filter):
+ # seems redis does not understand regex, it understands only glob
+ # patterns, thus we need to handle the matching by ourselves :/
+ keys = self.client.keys()
+ pattern = re.compile(prop_filter)
+ props = {}
+ for key in keys:
+ if pattern.match(key):
+ value = self.client.get(key)
+ props[key] = value
+ return props
+
+ @retry
+ def delete_properties(self, arg):
+ if not isinstance(arg, list):
+ raise cmerror.CMError('Deleting with filter is not supported by the backend')
+ pipe = self.client.pipeline()
+ for prop in arg:
+ pipe.delete(prop)
+ pipe.execute()
+
+
+def main():
+ import argparse
+ import sys
+ import traceback
+
+ parser = argparse.ArgumentParser(description='Test redis db plugin', prog=sys.argv[0])
+
+ parser.add_argument('--uri',
+ required=True,
+ dest='uri',
+ metavar='URI',
+ help='The redis db uri format redis://:password@<ip>:port',
+ type=str,
+ action='store')
+
+ parser.add_argument('--api',
+ required=True,
+ dest='api',
+ metavar='API',
+ help=('The api name, can be set_property, get_property, delete_property, '
+ 'set_properties, get_properties, delete_properties'),
+ type=str,
+ action='store')
+
+ parser.add_argument('--property',
+ required=False,
+ dest='properties',
+ metavar='PROPERTY',
+ help='The property in the format name[=value]',
+ type=str,
+ action='append')
+
+ parser.add_argument('--filter',
+ required=False,
+ dest='filter',
+ metavar='FILTER',
+ help='The regular expression matching the property names',
+ type=str,
+ action='store')
+
+ args = parser.parse_args(sys.argv[1:])
+
+ try:
+ uri = args.uri
+ api = args.api
+ p = {}
+ p['uri'] = uri
+ db = CMRedisDB(**p)
+ print('Involing %s' % api)
+ func = getattr(db, api)
+ if api == 'set_property':
+ if not args.properties:
+ raise Exception('Missing --properties argument')
+ for prop in args.properties:
+ i = prop.index('=')
+ name = prop[:i]
+ value = prop[(i + 1):]
+ print("Setting %s to %s" % (name, value))
+ func(name, value)
+ elif api == 'get_property':
+ if not args.properties:
+ raise Exception('Missing --properties argument')
+ for prop in args.properties:
+ value = func(prop)
+ print('%s=%s' % (prop, value))
+ elif api == 'delete_property':
+ if not args.properties:
+ raise Exception('Missing --properties argument')
+ for prop in args.properties:
+ print('Deleting %s' % prop)
+ func(prop)
+ elif api == 'set_properties':
+ if not args.properties:
+ raise Exception('Missing --properties argument')
+ props = {}
+ for prop in args.properties:
+ i = prop.index('=')
+ name = prop[:i]
+ value = prop[(i + 1):]
+ props[name] = value
+ func(props)
+ elif api == 'get_properties':
+ if not args.filter:
+ raise Exception('Missing --filter argument')
+ print('Getting properties matching %s' % args.filter)
+ props = func(args.filter)
+ for key, value in props.iteritems():
+ print('%s=%s' % (key, value))
+ elif api == 'delete_properties':
+ if not args.properties:
+ raise Exception('Missing --properties argument')
+ func(args.properties)
+ except Exception as exp: # pylint: disable=broad-except
+ print('Failed with error %s', exp)
+ traceback.print_exc()
+ sys.exit(1)
+ sys.exit(0)
+
+
+if __name__ == '__main__':
+ main()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from cmframework.apis import cmerror
+
+
+class CMActivateHandler(object):
+ # pylint: disable=no-self-use, unused-argument
+ def activate(self, work):
+ raise cmerror.CMError('Not implemented')
+
+ def is_supported(self, activator_plugin):
+ return False
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import logging
+
+from cmframework.apis import cmactivator
+from cmframework.utils import cmactivationrmq
+from cmframework.server import cmactivatehandler
+
+
+class CMActivateRMQHandler(cmactivatehandler.CMActivateHandler):
+ def __init__(self, rmq_host, rmq_port):
+ self.rmq_host = rmq_host
+ self.rmq_port = rmq_port
+ self.rmq_publisher = cmactivationrmq.CMActivationRMQPublisher(self.rmq_host, self.rmq_port)
+
+ def activate(self, work):
+ logging.debug('CMAcivateRMQHandler activating %s', work)
+ self.rmq_publisher.send(work)
+
+ def is_supported(self, activator_plugin):
+ return isinstance(activator_plugin, cmactivator.CMLocalActivator)
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import logging
+import time
+
+from cmframework.utils import cmpluginmanager
+from cmframework.utils import cmpluginloader
+from cmframework.utils import cmactivationwork
+
+from cmframework.apis import cmactivator
+from cmframework.server import cmactivatehandler
+
+
+class CMActivateServerHandler(cmactivatehandler.CMActivateHandler,
+ cmpluginmanager.CMPluginManager,
+ cmpluginloader.CMPluginLoader.LoadingFilter):
+
+ def __init__(self, plugins_path, plugin_client, changemonitor, activationstate_handler):
+ cmpluginmanager.CMPluginManager.__init__(self, plugins_path)
+ self.load_plugin()
+ self.plugin_client = plugin_client
+ self.changemonitor = changemonitor
+ self.activationstate_handler = activationstate_handler
+
+ def is_supported(self, activator_plugin):
+ logging.debug('Checking %s', activator_plugin)
+ return isinstance(activator_plugin, cmactivator.CMGlobalActivator)
+
+ def load_plugin(self):
+ plugin_filter = self
+ pl = cmpluginloader.CMPluginLoader(self.plugins_path, plugin_filter)
+ self.pluginlist, self.filterdict = pl.load()
+ logging.info('pluginlist is %r', self.pluginlist)
+
+ def activate_set(self, indata):
+ return self._activate(indata, 'activate_set')
+
+ def activate_delete(self, indata):
+ return self._activate(indata, 'activate_delete')
+
+ def activate_full(self, target_node, startup_activation=False):
+ return self._activate(target_node, 'activate_full', startup_activation)
+
+ def activate_node(self, target_node):
+ return self._activate(target_node, 'activate_full')
+
+ def _activate(self, indata, operation, startup_activation=False):
+ logging.info('%s called with %s', operation, indata)
+ failures = {}
+ for plugin, objectname in self.pluginlist.iteritems():
+ logging.info('Running plugin %s.%s', plugin, operation)
+ func = None
+ try:
+ class_name = getattr(objectname, plugin)
+ instance = class_name()
+ instance.plugin_client = self.plugin_client
+ func = getattr(instance, operation)
+ if operation != 'activate_full':
+ filtername = self.filterdict[plugin]
+ inputdata = self.build_input(indata, filtername)
+
+ if not inputdata:
+ logging.info('Skipping plugin %s as no input data is to be processed by it',
+ plugin)
+ continue
+
+ start_time = time.time()
+ func(inputdata)
+ logging.info('Plugin %s.%s took %s seconds', plugin, operation,
+ time.time() - start_time)
+ else:
+ if startup_activation:
+ if plugin not in self.activationstate_handler.get_full_failed():
+ logging.info('Skipping plugin %s during startup as '
+ 'it has not failed in last activation', plugin)
+ continue
+
+ start_time = time.time()
+ func(indata)
+ logging.info('Plugin %s.%s took %s seconds', plugin, operation,
+ time.time() - start_time)
+ except AttributeError as exp:
+ logging.info('Plugin %s does not have %s defined', plugin, operation)
+ logging.info(str(exp))
+ failures[str(plugin)] = 'Plugin does not have {} defined'.format(operation)
+ continue
+ except Exception as exp: # pylint: disable=broad-except
+ failures[str(plugin)] = str(exp)
+ logging.error('Skipping %s, got exception %s', plugin, str(exp))
+ failures[str(plugin)] = str(exp)
+ continue
+
+ logging.info('%s done with %s', operation, indata)
+
+ return failures
+
+ def activate(self, work):
+ logging.debug('CMAcivateServerHandler activating %s', work)
+ failures = {}
+ if work.get_operation() == cmactivationwork.CMActivationWork.OPER_SET:
+ failures = self.activate_set(work.get_props())
+ elif work.get_operation() == cmactivationwork.CMActivationWork.OPER_DELETE:
+ failures = self.activate_delete(work.get_props())
+ elif work.get_operation() == cmactivationwork.CMActivationWork.OPER_FULL:
+ startup_activation = work.is_startup_activation()
+ failures = self.activate_full(work.get_target(), startup_activation)
+ elif work.get_operation() == cmactivationwork.CMActivationWork.OPER_NODE:
+ failures = self.activate_node(work.get_target())
+ else:
+ logging.error('Unsupported activation operation %s', work.get_operation())
+
+ if work.uuid_value:
+ if failures:
+ self.changemonitor.change_nok(work.uuid_value, failures)
+ else:
+ self.changemonitor.change_ok(work.uuid_value)
+
+ return failures
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from Queue import Queue
+
+from cmframework.server import cmeventletrwlock
+from cmframework.server import cmactivatorworker
+
+
+class CMActivator(object):
+ def __init__(self, worker_count):
+ self.works = Queue()
+ self.node_works = Queue()
+ self.handlers = []
+ self.workers = []
+ self.worker_count = worker_count
+ self.lock = cmeventletrwlock.CMEventletRWLock()
+
+ def add_handler(self, handler):
+ self.handlers.append(handler)
+
+ def get_parallel_work(self):
+ return self.node_works.get()
+
+ def get_work(self):
+ return self.works.get()
+
+ def add_work(self, work):
+ work.release()
+ if not work.get_target():
+ self.works.put(work)
+ else:
+ self.node_works.put(work)
+
+ def get_handlers(self):
+ return self.handlers
+
+ def start(self):
+ worker = cmactivatorworker.CMActivatorWorker(self, 0, self.lock)
+ worker.start()
+ self.workers.append(worker)
+
+ for i in range(1, self.worker_count+1):
+ worker = cmactivatorworker.CMActivatorWorker(self, i, self.lock, parallel=True)
+ worker.start()
+ self.workers.append(worker)
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import logging
+from threading import Thread
+
+
+class CMActivatorWorker(Thread):
+ def __init__(self, activator, index, lock, parallel=False):
+ super(CMActivatorWorker, self).__init__()
+
+ self.activator = activator
+ self.index = index
+ self.lock = lock
+ self.parallel = parallel
+
+ self.daemon = True
+
+ def __str__(self):
+ return 'worker-{}'.format(self.index)
+
+ def _handle_work(self, work):
+ logging.debug('%s handling work: %s', self, work)
+ failures = {}
+ for handler in self.activator.get_handlers():
+ try:
+ logging.info('%s activating using %s', self, handler.__class__.__name__)
+ handler_failures = handler.activate(work)
+ if handler_failures:
+ logging.error('%s activation failed, error count=%s',
+ self,
+ len(handler_failures))
+ failures[handler.__class__.__name__] = handler_failures
+ except Exception as exp: # pylint: disable=broad-except
+ logging.error('%s activation failed with error %s', self, str(exp))
+ failures[handler.__class__.__name__] = str(exp)
+
+ logging.debug('%s handled work: %s', self, work)
+
+ work.add_result(failures)
+
+ def run(self):
+ while True:
+ if self.parallel:
+ work = self.activator.get_parallel_work()
+ with self.lock.reader():
+ self._handle_work(work)
+ else:
+ work = self.activator.get_work()
+ with self.lock.writer():
+ self._handle_work(work)
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+import argparse
+import os
+
+from ConfigParser import SafeConfigParser
+
+from cmframework.apis import cmerror
+from cmframework.utils import cmlogger
+
+
+class CMArgsParser(object):
+ def __init__(self, prog):
+ self.prog = prog
+ self.ip = None
+ self.port = None
+ self.verbose = False
+ self.log_level = cmlogger.CMLogger.str_to_level('debug')
+ self.log_dest = cmlogger.CMLogger.str_to_dest('syslog')
+ self.disable_remote_activation = False
+ self.rmq_host = None
+ self.rmq_port = None
+ self.activators = None
+ self.validators = None
+ self.rmq_ip = None
+ self.backend_uri = None
+ self.backend_api = None
+ self.filename = None
+ self.inventory_handlers = None
+ self.inventory_data = None
+ self.install_phase = False
+ self.activationstate_handler_uri = None
+ self.activationstate_handler_api = None
+ self.activator_workers = 1
+ self.snapshot_handler_uri = None
+ self.snapshot_handler_api = None
+ self.alarmhandler_api = None
+
+ def parse(self, args):
+ argparse.ArgumentParser(description='Configuration Management Server',
+ prog=self.prog)
+
+ if '--file' in args:
+ self.parse_config(args)
+ else:
+ self.parse_cmd_line(args)
+
+ def parse_config(self, args):
+ parser = argparse.ArgumentParser(description='Configuration Management Server',
+ prog=self.prog)
+
+ parser.add_argument('--file',
+ dest='filename',
+ required=True,
+ type=str,
+ action='store')
+ try:
+ args = parser.parse_args(args)
+ self.filename = args.filename
+ except Exception as error:
+ raise cmerror.CMError(str(error))
+
+ self.parse_config_file()
+
+ def parse_config_file(self):
+ config = SafeConfigParser(
+ {'log_level': cmlogger.CMLogger.level_to_str(self.log_level),
+ 'log_dest': cmlogger.CMLogger.dest_to_str(self.log_dest),
+ 'verbose': repr(self.verbose),
+ 'disable_remote_activation': repr(self.disable_remote_activation),
+ 'activator_workers': '10',
+ 'alarmhandler_api': 'cmframework.lib.cmalarmhandler_dummy.AlarmHandler_Dummy',
+ 'snapshot_handler_api': ''})
+ try:
+ config.read(self.filename)
+ self.ip = config.get('cmserver', 'ip')
+ self.port = config.getint('cmserver', 'port')
+ self.backend_api = config.get('cmserver', 'backend_api')
+ self.backend_uri = config.get('cmserver', 'backend_uri')
+ self.verbose = config.getboolean('cmserver', 'verbose')
+ self.log_level = cmlogger.CMLogger.str_to_level(config.get('cmserver', 'log_level'))
+ self.log_dest = cmlogger.CMLogger.str_to_dest(config.get('cmserver', 'log_dest'))
+ self.validators = CMArgsParser.dir_parser(config.get('cmserver', 'validators'))
+ self.activators = CMArgsParser.dir_parser(config.get('cmserver', 'activators'))
+ self.disable_remote_activation = \
+ config.getboolean('cmserver', 'disable_remote_activation')
+ if not self.disable_remote_activation:
+ self.rmq_ip = config.get('cmserver', 'rmq_ip')
+ self.rmq_port = config.getint('cmserver', 'rmq_port')
+ self.inventory_handlers = CMArgsParser.dir_parser(config.get('cmserver',
+ 'inventory_handlers'))
+ self.inventory_data = config.get('cmserver', 'inventory_data')
+ self.install_phase = config.getboolean('cmserver', 'install_phase')
+ self.activationstate_handler_uri = config.get('cmserver', 'activationstate_handler_uri')
+ self.activationstate_handler_api = config.get('cmserver', 'activationstate_handler_api')
+ self.activator_workers = config.getint('cmserver', 'activator_workers')
+ self.snapshot_handler_uri = config.get('cmserver', 'snapshot_handler_uri')
+ self.snapshot_handler_api = config.get('cmserver', 'snapshot_handler_api')
+ self.alarmhandler_api = config.get('cmserver', 'alarmhandler_api')
+ except Exception as error:
+ raise cmerror.CMError(str(error))
+
+ def parse_cmd_line(self, args):
+ parser = argparse.ArgumentParser(description='Configuration Management Server',
+ prog=self.prog)
+
+ parser.add_argument('--ip',
+ required=True,
+ help='The IP address to listen to',
+ type=str,
+ action='store')
+
+ parser.add_argument('--port',
+ required=True,
+ help='The port number used for listening',
+ type=int,
+ action='store')
+
+ parser.add_argument('--backend-api',
+ dest='backend_api',
+ metavar='BACKEND-API',
+ required=True,
+ help='The module.class implementing the backend api',
+ type=str,
+ action='store')
+
+ parser.add_argument('--backend-uri',
+ dest='backend_uri',
+ metavar='BACKEND-URI',
+ required=True,
+ help='The uri needed by the backend api',
+ type=str, action='store')
+
+ parser.add_argument('--log-level',
+ dest='log_level',
+ metavar='LOG-LEVEL',
+ required=False,
+ help=('The enabled logging level, possible values are '
+ '{debug, info, warn, error}'),
+ type=CMArgsParser.log_level_parser,
+ default=self.log_level,
+ action='store')
+
+ parser.add_argument('--log-dest',
+ dest='log_dest',
+ metavar='LOG-DEST',
+ required=False,
+ help='The logs destination, possible values are {console, syslog}',
+ type=CMArgsParser.log_dest_parser,
+ default=self.log_dest,
+ action='store')
+
+ parser.add_argument('--verbose',
+ dest='verbose',
+ required=False,
+ default=self.verbose,
+ help='Enable verbose mode',
+ action='store_true')
+
+ parser.add_argument('--validators',
+ dest='validators',
+ metavar='VALIDATORS-PATH',
+ required=True,
+ help='The full path were validatation plugin(s) are located',
+ type=CMArgsParser.dir_parser,
+ action='store')
+
+ parser.add_argument('--activators',
+ dest='activators',
+ metavar='ACTIVATORS-PATH',
+ required=True,
+ help='The full path were activation plugin(s) are located',
+ type=CMArgsParser.dir_parser,
+ action='store')
+
+ parser.add_argument('--disable-remote-activation',
+ dest='disable_remote_activation',
+ required=False,
+ default=self.disable_remote_activation,
+ help='Enable running activators in the target nodes',
+ action='store_true')
+
+ parser.add_argument('--rmq-ip',
+ dest='rmq_ip',
+ metavar='RMQ-IP',
+ required=False,
+ help='RMQ broker IP address',
+ type=str,
+ action='store')
+
+ parser.add_argument('--rmq-port',
+ dest='rmq_port',
+ metavar='RMQ-PORT',
+ required=False,
+ help='RMQ broker port number',
+ type=int,
+ action='store')
+
+ parser.add_argument('--inventory-handlers',
+ dest='inventory_handlers',
+ metavar='INVENTORY-HANDLERS-PATH',
+ required=True,
+ help='The full path were inventory handlers are located',
+ type=CMArgsParser.dir_parser,
+ action='store')
+
+ parser.add_argument('--inventory-data',
+ dest='inventory_data',
+ metavar='INVENTORY-DATA',
+ required=True,
+ help='The full path for the inventory data file',
+ type=str,
+ action='store')
+
+ parser.add_argument('--install-phase',
+ dest='install_phase',
+ required=False,
+ default=False,
+ help='Indicate install phase startup',
+ action='store_true')
+
+ parser.add_argument('--activationstate-handler-api',
+ dest='activationstate_handler_api',
+ metavar='ACTIVATIONSTATE-HANDLER-API',
+ required=True,
+ help='The module.class implementing the activationstate handler api',
+ type=str,
+ action='store')
+
+ parser.add_argument('--activationstate-handler-uri',
+ dest='activationstate_handler_uri',
+ metavar='ACTIVATIONSTATE-HANDLER-URI',
+ required=True,
+ help='The uri needed by the activationstate handler uri',
+ type=str, action='store')
+
+ parser.add_argument('--activator-workers',
+ dest='activator_workers',
+ metavar='ACTIVATOR_WORKERS',
+ required=False,
+ default=1,
+ help='Number of activator workers',
+ type=int,
+ action='store')
+
+ parser.add_argument('--snapshot-handler-api',
+ dest='snapshot_handler_api',
+ metavar='SNAPSHOT-HANDLER-API',
+ required=True,
+ help='The module.class implementing the snapshot handler api',
+ type=str,
+ action='store')
+
+ parser.add_argument('--snapshot-handler-uri',
+ dest='snapshot_handler_uri',
+ metavar='SNAPSHOT-HANDLER-URI',
+ required=False,
+ default='',
+ help='The uri needed by the snapshot handler uri',
+ type=str, action='store')
+
+ parser.add_argument('--alarmhandler-api',
+ dest='alarmhandler_api',
+ metavar='ALARMHANDLER-API',
+ required=True,
+ help='The module.class implementing the alarmhandler api',
+ type=str,
+ action='store')
+
+ try:
+ args = parser.parse_args(args)
+ self.ip = args.ip
+ self.port = args.port
+ self.backend_api = args.backend_api
+ self.backend_uri = args.backend_uri
+ self.verbose = args.verbose
+ self.log_level = args.log_level
+ self.log_dest = args.log_dest
+ self.validators = args.validators
+ self.activators = args.activators
+ self.disable_remote_activation = args.disable_remote_activation
+ if not self.disable_remote_activation:
+ self.rmq_ip = args.rmq_ip
+ self.rmq_port = args.rmq_port
+ self.inventory_handlers = args.inventory_handlers
+ self.inventory_data = args.inventory_data
+ self.install_phase = args.install_phase
+ self.activationstate_handler_api = args.activationstate_handler_api
+ self.activationstate_handler_uri = args.activationstate_handler_uri
+ self.activator_workers = args.activator_workers
+ self.snapshot_handler_api = args.snapshot_handler_api
+ self.snapshot_handler_uri = args.snapshot_handler_uri
+ self.alarmhandler_api = args.alarmhandler_api
+ except Exception as error:
+ raise cmerror.CMError(str(error))
+
+ @staticmethod
+ def log_level_parser(level):
+ try:
+ return cmlogger.CMLogger.str_to_level(level)
+ except cmerror.CMError as exp:
+ raise argparse.ArgumentTypeError(str(exp))
+
+ @staticmethod
+ def log_dest_parser(dest):
+ try:
+ return cmlogger.CMLogger.str_to_dest(dest)
+ except cmerror.CMError as exp:
+ raise argparse.ArgumentTypeError(str(exp))
+
+ @staticmethod
+ def dir_parser(path):
+ if os.path.isdir(path):
+ return path
+ raise argparse.ArgumentTypeError('Not a directory')
+
+ def get_ip(self):
+ return self.ip
+
+ def get_port(self):
+ return self.port
+
+ def get_backend_api(self):
+ return self.backend_api
+
+ def get_backend_uri(self):
+ return self.backend_uri
+
+ def get_log_level(self):
+ return self.log_level
+
+ def get_log_dest(self):
+ return self.log_dest
+
+ def get_verbose(self):
+ return self.verbose
+
+ def get_validators(self):
+ return self.validators
+
+ def get_activators(self):
+ return self.activators
+
+ def get_disable_remote_activation(self):
+ return self.disable_remote_activation
+
+ def get_rmq_ip(self):
+ return self.rmq_ip
+
+ def get_rmq_port(self):
+ return self.rmq_port
+
+ def get_inventory_handlers(self):
+ return self.inventory_handlers
+
+ def get_inventory_data(self):
+ return self.inventory_data
+
+ def is_install_phase(self):
+ return self.install_phase
+
+ def get_activationstate_handler_api(self):
+ return self.activationstate_handler_api
+
+ def get_activationstate_handler_uri(self):
+ return self.activationstate_handler_uri
+
+ def get_activator_workers(self):
+ return self.activator_workers
+
+ def get_snapshot_handler_api(self):
+ return self.snapshot_handler_api
+
+ def get_snapshot_handler_uri(self):
+ return self.snapshot_handler_uri
+
+ def get_alarmhandler_api(self):
+ return self.alarmhandler_api
+
+
+def main():
+ cm_parser = CMArgsParser('cmserver')
+ try:
+ cm_parser.parse(sys.argv[1:])
+ print 'ip = %s' % cm_parser.get_ip()
+ print 'port = %d' % cm_parser.get_port()
+ print 'backend-api = %s' % cm_parser.get_backend_api()
+ print 'backend-uri = %s' % cm_parser.get_backend_uri()
+ print 'log-level = %s' % cmlogger.CMLogger.level_to_str(cm_parser.get_log_level())
+ print 'log-dest = %s' % cmlogger.CMLogger.dest_to_str(cm_parser.get_log_dest())
+ print 'verbose = %s' % repr(cm_parser.get_verbose())
+ print 'validators = %s' % repr(cm_parser.get_validators())
+ print 'activators = %s' % repr(cm_parser.get_activators())
+ print 'rmq-ip = %s' % repr(cm_parser.get_rmq_ip())
+ print 'rmq-port = %s' % repr(cm_parser.get_rmq_port())
+ print 'inventory-handlers = %s' % repr(cm_parser.get_inventory_handlers())
+ print 'inventory-data = %s' % repr(cm_parser.get_inventory_data())
+ print 'install-phase = %s' % repr(cm_parser.is_install_phase())
+ print 'activationstate-handler-api = %s' % repr(cm_parser.get_activationstate_handler_api())
+ print 'activationstate-handler-uri = %s' % repr(cm_parser.get_activationstate_handler_uri())
+ print 'activator-workers = %s' % repr(cm_parser.get_activator_workers())
+ print 'snapshot-handler-api = %s' % repr(cm_parser.get_snapshot_handler_api())
+ print 'snapshot-handler-uri = %s' % repr(cm_parser.get_snapshot_handler_uri())
+ print 'alarmhandler-api = %s' % cm_parser.get_alarmhandler_api()
+ except cmerror.CMError as error:
+ print 'Got error %s' % str(error)
+ sys.exit(1)
+
+
+if __name__ == '__main__':
+ main()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import logging
+import uuid
+import copy
+from cmframework.apis import cmchangestate
+from cmframework.server import cmeventletrwlock
+
+
+class CMChangeMonitorState(object):
+ def __init__(self):
+ self.state = cmchangestate.CM_CHANGE_STATE_ONGOING
+ self.failed_plugins = {}
+
+
+class CMChangeMonitor(object):
+ def __init__(self):
+ self.changes = {}
+ self.lock = cmeventletrwlock.CMEventletRWLock()
+
+ def start_change(self):
+ with self.lock.writer():
+ changestate = CMChangeMonitorState()
+ uuid_value = str(uuid.uuid4())
+ self.changes[uuid_value] = changestate
+ return uuid_value
+
+ def change_nok(self, uuid_value, failed_plugins):
+ with self.lock.writer():
+ if uuid_value in self.changes:
+ self.changes[uuid_value].state = cmchangestate.CM_CHANGE_STATE_NOK
+ self.changes[uuid_value].failed_plugins = failed_plugins
+ else:
+ logging.warning('Invalid change uuid %s', uuid_value)
+
+ def change_ok(self, uuid_value):
+ with self.lock.writer():
+ if uuid_value in self.changes:
+ self.changes[uuid_value].state = cmchangestate.CM_CHANGE_STATE_OK
+ else:
+ logging.warning('Invalid change uuid %s', uuid_value)
+
+ def get_change_state(self, uuid_value):
+ with self.lock.reader():
+ return self.changes[uuid_value]
+
+ def get_all_changes_states(self):
+ with self.lock.reader():
+ return copy.deepcopy(self.changes)
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import logging
+import copy
+import json
+
+from cmframework.apis import cmerror
+
+
+class CMCSN(object):
+ CONFIG_NAME = 'cloud.cmframework'
+
+ def __init__(self, backend_handler):
+ self.backend_handler = backend_handler
+ self.config = {'csn': {'global': 0, 'nodes': {}}}
+ self._get_config()
+ logging.info('Current csn is %d', self.get())
+
+ def _update_csn(self, node_name=None):
+ new_config = copy.deepcopy(self.config)
+
+ if not node_name:
+ new_config['csn']['global'] += 1
+ logging.info('Updating csn to %d', new_config['csn']['global'])
+ else:
+ new_config['csn']['nodes'][node_name] = new_config['csn']['global']
+ logging.info('Updating csn for node %s to %s', node_name, new_config['csn']['global'])
+
+ self.backend_handler.set_property(CMCSN.CONFIG_NAME, json.dumps(new_config))
+ self.config = new_config
+
+ def _get_config(self):
+ try:
+ self.config = json.loads(self.backend_handler.get_property(CMCSN.CONFIG_NAME))
+ except cmerror.CMError as exp:
+ logging.info('Context id not defined')
+ except Exception as exp: # pylint: disable=broad-except
+ logging.warning('Got error: %s', exp)
+
+ def increment(self):
+ self._update_csn()
+
+ def get(self):
+ return self.config['csn']['global']
+
+ def get_node_csn(self, node_name):
+ return self.config['csn']['nodes'].get(node_name, 1)
+
+ def sync_node_csn(self, node_name):
+ self._update_csn(node_name)
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from __future__ import print_function
+from eventlet import greenthread, hubs
+
+
+class CMEventletRWLock(object):
+ def __init__(self):
+ self.counter = 0
+ self._read_waiters = set()
+ self._write_waiters = set()
+
+ def reader(self):
+ return self.Reader(self)
+
+ def writer(self):
+ return self.Writer(self)
+
+ class Base(object):
+ def __init__(self, parent):
+ self.parent = parent
+
+ def _acquire(self, waiters):
+ waiters.add(greenthread.getcurrent())
+
+ try:
+ while self.parent.counter != 0:
+ hubs.get_hub().switch()
+ finally:
+ waiters.discard(greenthread.getcurrent())
+
+ def _exit(self):
+ for waiters, fn in (
+ (self.parent._read_waiters, lambda x: x >= 0),
+ (self.parent._write_waiters, lambda x: x == 0),
+ ):
+ if not waiters:
+ continue
+
+ hubs.get_hub().schedule_call_global(
+ 0, self._release, waiters, fn,
+ )
+
+ def _release(self, waiters, fn):
+ if waiters and fn(self.parent.counter):
+ waiters.pop().switch()
+
+ class Reader(Base):
+ def __enter__(self):
+ if self.parent.counter < 0:
+ self._acquire(self.parent._read_waiters)
+ self.parent.counter += 1
+
+ def __exit__(self, *args, **kwargs):
+ self.parent.counter -= 1
+ self._exit()
+
+ class Writer(Base):
+ def __enter__(self):
+ if self.parent.counter != 0:
+ self._acquire(self.parent._write_waiters)
+ self.parent.counter -= 1
+
+ def __exit__(self, *args, **kwargs):
+ self.parent.counter += 1
+ self._exit()
+
+
+def main():
+ lock = CMEventletRWLock()
+ with lock.reader():
+ print('Got reader lock')
+
+ with lock.writer():
+ print('Got write lock')
+
+
+if __name__ == '__main__':
+ main()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+class CMHTTPErrors(object):
+ # response for a successful GET, PUT, PATCH, DELETE,
+ # can also be used for POST that does not result in creation.
+ HTTP_OK = 200
+ # response to a POST which results in creation.
+ HTTP_CREATED = 201
+ # response to a successfull request that won't be returning any body like a DELETE request
+ HTTP_NO_CONTENT = 204
+ # used when http caching headers are in play
+ HTTP_NOT_MODIFIED = 304
+ # the request is malformed such as if the body does not parse
+ HTTP_BAD_REQUEST = 400
+ # when no or invalid authentication details are provided.
+ # also useful to trigger an auth popup API is used from a browser
+ HTTP_UNAUTHORIZED_OPERATION = 401
+ # when authentication succeeded but authenticated user doesn't have access to the resource
+ HTTP_FORBIDDEN = 403
+ # when a non-existent resource is requested
+ HTTP_NOT_FOUND = 404
+ # when an http method is being requested that isn't allowed for the authenticated user
+ HTTP_METHOD_NOT_ALLOWED = 405
+ # indicates the resource at this point is no longer available
+ HTTP_GONE = 410
+ # if incorrect content type was provided as part of the request
+ HTTP_UNSUPPORTED_MEDIA_TYPE = 415
+ # used for validation errors
+ HTTP_UNPROCESSABLE_ENTITY = 422
+ # when request is rejected due to rate limiting
+ HTTP_TOO_MANY_REQUESTS = 429
+ # Other errrors
+ HTTP_INTERNAL_ERROR = 500
+
+ @staticmethod
+ def get_ok_status():
+ return '%d OK' % CMHTTPErrors.HTTP_OK
+
+ @staticmethod
+ def get_object_created_successfully_status():
+ return '%d Created' % CMHTTPErrors.HTTP_CREATED
+
+ @staticmethod
+ def get_request_not_ok_status():
+ return '%d Bad request' % CMHTTPErrors.HTTP_BAD_REQUEST
+
+ @staticmethod
+ def get_resource_not_found_status():
+ return '%d Not found' % CMHTTPErrors.HTTP_NOT_FOUND
+
+ @staticmethod
+ def get_unsupported_content_type_status():
+ return '%d Unsupported content type' % CMHTTPErrors.HTTP_UNSUPPORTED_MEDIA_TYPE
+
+ @staticmethod
+ def get_validation_error_status():
+ return '%d Validation error' % CMHTTPErrors.HTTP_UNPROCESSABLE_ENTITY
+
+ @staticmethod
+ def get_internal_error_status():
+ return '%d Internal error' % CMHTTPErrors.HTTP_INTERNAL_ERROR
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+class HTTPRPC(object):
+ def __init__(self):
+ self.req_body = ''
+ self.req_filter = ''
+ self.req_params = {}
+ self.req_method = ''
+ self.rep_body = ''
+ self.rep_status = ''
+
+ def __str__(self):
+ return str.format('REQ: body:{body} filter:{filter} '
+ 'params:{params} method:{method} '
+ 'REP: body:{rep_body} status:{status}',
+ body=self.req_body, filter=self.req_filter,
+ params=str(self.req_params), method=self.req_method,
+ rep_body=self.rep_body, status=self.rep_status)
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import logging
+
+from cmframework.utils import cmactivationwork
+from cmframework.server import cmeventletrwlock
+from cmframework.server import cmcsn
+from cmframework.server import cmsnapshot
+from cmframework.utils.cmflagfile import CMFlagFile
+from cmframework.utils import cmalarm
+
+
+class CMProcessor(object):
+ SERVICE_GROUP_NAME = 'config-manager'
+
+ def __init__(self,
+ backend_handler,
+ validator,
+ activator,
+ changemonitor,
+ activationstate_handler,
+ snapshot_handler):
+ logging.debug('CMProcessor constructed')
+
+ self.backend_handler = backend_handler
+ self.lock = cmeventletrwlock.CMEventletRWLock()
+ self.csn = cmcsn.CMCSN(self.backend_handler)
+ self.validator = validator
+ self.activator = activator
+ self.reboot_requests = set()
+ self.automatic_activation_disabled = CMFlagFile('automatic_activation_disabled')
+ self.changemonitor = changemonitor
+ self.activationstate_handler = activationstate_handler
+ self.snapshot = cmsnapshot.CMSnapshot(snapshot_handler)
+
+ def reboot_request(self, node_name):
+ logging.debug('reboot_request called for %s', node_name)
+
+ self.reboot_requests.add(node_name)
+
+ def _clear_reboot_requests(self):
+ logging.debug('_clear_reboot_requests called')
+
+ self.reboot_requests.clear()
+
+ def _raise_reboot_alarms(self):
+ logging.debug('_raise_reboot_alarms called')
+
+ reboot_request_alarm = cmalarm.CMRebootRequestAlarm()
+
+ for node_name in self.reboot_requests:
+ reboot_request_alarm.raise_alarm_for_node(node_name)
+
+ def get_property(self, prop_name, snapshot_name=None):
+ logging.debug('get_property called for %s', prop_name)
+
+ with self.lock.reader():
+ if snapshot_name:
+ self.snapshot.load(snapshot_name)
+
+ return self.snapshot.get_property(prop_name)
+
+ return self.backend_handler.get_property(prop_name)
+
+ def get_properties(self, prop_filter, snapshot_name=None):
+ logging.debug('get_properties called with filter %s', prop_filter)
+
+ with self.lock.reader():
+ if snapshot_name:
+ self.snapshot.load(snapshot_name)
+
+ return self.snapshot.get_properties(prop_filter)
+
+ return self.backend_handler.get_properties(prop_filter)
+
+ def set_property(self, prop_name, prop_value):
+ logging.debug('set_property called %s=%s', prop_name, prop_value)
+
+ props = {}
+ props[prop_name] = prop_value
+ return self.set_properties(props)
+
+ def set_properties(self, props, overwrite=False):
+ logging.debug('set_properties called for %s', str(props))
+
+ with self.lock.writer():
+ self._validate_set(props)
+ if overwrite:
+ logging.debug('Deleting old configuration data as requested')
+ orig_props = self.backend_handler.get_properties('.*')
+ self.backend_handler.delete_properties(orig_props.keys())
+ self.backend_handler.set_properties(props)
+ self.csn.increment()
+
+ if not self.automatic_activation_disabled:
+ return self._activate_set(props)
+
+ return "0"
+
+ def delete_property(self, prop_name):
+ logging.debug('delete_property called for %s', prop_name)
+
+ props = []
+ props.append(prop_name)
+ return self._delete_properties(props, None)
+
+ def delete_properties(self, arg):
+ logging.debug('delete_properties called with arg %r', arg)
+
+ keys = []
+ prop_filter = None
+ if isinstance(arg, str):
+ prop_filter = arg
+ props = self.get_properties(prop_filter)
+ keys = props.keys()
+ else:
+ keys = arg
+ return self._delete_properties(keys, prop_filter)
+
+ def _delete_properties(self, props, props_filter):
+ logging.debug('_delete_properties called with props %s filter %s', props, props_filter)
+
+ with self.lock.writer():
+ self._validate_delete(props)
+ if props_filter:
+ self.backend_handler.delete_properties(props_filter)
+ else:
+ if len(props) == 1:
+ self.backend_handler.delete_property(props[0])
+ else:
+ self.backend_handler.delete_properties(props)
+ self.csn.increment()
+
+ if not self.automatic_activation_disabled:
+ return self._activate_delete(props)
+
+ return "0"
+
+ def _validate_set(self, props):
+ logging.debug('_validate_set called for %s', str(props))
+
+ self.validator.validate_set(props)
+
+ def _activate_set_no_lock(self, props):
+ logging.debug('_activate_set_no_lock called for %s', str(props))
+
+ uuid_value = self.changemonitor.start_change()
+
+ work = cmactivationwork.CMActivationWork(cmactivationwork.CMActivationWork.OPER_SET,
+ self.csn.get(), props)
+ work.uuid_value = uuid_value
+ self.activator.add_work(work)
+ return uuid_value
+
+ def _activate_set(self, props):
+ logging.debug('_activate_set called')
+
+ with self.lock.reader():
+ return self._activate_set_no_lock(props)
+
+ def _validate_delete(self, props):
+ logging.debug('_validate_delete called for %s', str(props))
+
+ self.validator.validate_delete(props)
+
+ def _activate_delete(self, props):
+ logging.debug('_activate_delete called for %s', str(props))
+
+ with self.lock.reader():
+ uuid_value = self.changemonitor.start_change()
+ work = cmactivationwork.CMActivationWork(cmactivationwork.CMActivationWork.OPER_DELETE,
+ self.csn.get(), props)
+ work.uuid_value = uuid_value
+ self.activator.add_work(work)
+ return uuid_value
+
+ def create_snapshot(self, snapshot_name):
+ logging.debug('create_snapshot called, snapshot name is %s', snapshot_name)
+
+ with self.lock.writer():
+ self.snapshot.create(snapshot_name, self.backend_handler)
+
+ def restore_snapshot(self, snapshot_name):
+ logging.debug('restore_snapshot called, snapshot name is %s', snapshot_name)
+
+ with self.lock.writer():
+ self.snapshot.load(snapshot_name)
+
+ self._validate_set(self.snapshot.get_properties())
+
+ self.snapshot.restore(self.backend_handler)
+
+ self.csn = cmcsn.CMCSN(self.backend_handler)
+
+ self._activate_set_no_lock(self.snapshot.get_properties())
+
+ def list_snapshots(self):
+ logging.debug('list_snapshots called')
+
+ snapshots = []
+ with self.lock.writer():
+ snapshots = self.snapshot.list()
+
+ return snapshots
+
+ def delete_snapshot(self, snapshot_name):
+ logging.debug('delete_snapshot called, snapshot name is %s', snapshot_name)
+
+ with self.lock.writer():
+ self.snapshot.delete(snapshot_name)
+
+ def activate(self, node_name=None, startup_activation=False):
+ logging.debug('activate called, node is %s', node_name)
+
+ activation_alarm = cmalarm.CMActivationFailedAlarm()
+ if node_name:
+ activation_alarm.cancel_alarm_for_node(node_name)
+ else:
+ activation_alarm.cancel_alarm_for_sg(CMProcessor.SERVICE_GROUP_NAME)
+
+ with self.lock.reader():
+ uuid_value = self.changemonitor.start_change()
+ if not node_name:
+ work = cmactivationwork.CMActivationWork(
+ cmactivationwork.CMActivationWork.OPER_FULL,
+ self.csn.get(), {}, None, startup_activation)
+ else:
+ work = cmactivationwork.CMActivationWork(
+ cmactivationwork.CMActivationWork.OPER_FULL,
+ self.csn.get(), {}, node_name)
+ work.uuid_value = uuid_value
+ self.activator.add_work(work)
+
+ logging.debug('activation work added, going to wait for result')
+ failures = work.get_result()
+ logging.debug('got activation result')
+
+ if self.reboot_requests:
+ self._raise_reboot_alarms()
+
+ if not node_name:
+ self.activationstate_handler.clear_full_failed()
+
+ if failures:
+ logging.warning('Activation failed: %s', failures)
+
+ failed_activators = [activator for handler in failures.keys()
+ for activator in failures[handler]]
+
+ supplementary_info = {'failed activators': failed_activators}
+
+ if node_name:
+ activation_alarm.raise_alarm_for_node(node_name, supplementary_info)
+ else:
+ self.activationstate_handler.set_full_failed(failed_activators)
+
+ activation_alarm.raise_alarm_for_sg(CMProcessor.SERVICE_GROUP_NAME,
+ supplementary_info)
+ return uuid_value
+
+ def activate_node(self, node_name):
+ logging.debug('activate_node called, node name is %s', node_name)
+
+ if self.automatic_activation_disabled:
+ return False
+
+ with self.lock.reader():
+ node_csn = self.csn.get_node_csn(node_name)
+
+ if self.csn.get() == node_csn:
+ logging.info('No change in data since last translation, last csn %d',
+ self.csn.get())
+ return False
+
+ self._clear_reboot_requests()
+ work = cmactivationwork.CMActivationWork(cmactivationwork.CMActivationWork.OPER_NODE,
+ self.csn.get(), {}, node_name)
+ self.activator.add_work(work)
+
+ activation_alarm = cmalarm.CMActivationFailedAlarm()
+ activation_alarm.cancel_alarm_for_node(node_name)
+
+ failures = work.get_result()
+ if failures:
+ logging.warning('Activation failed: %s', failures)
+
+ failed_activators = [activator for handler in failures.keys()
+ for activator in failures[handler]]
+ supplementary_info = {'failed activators': failed_activators}
+ activation_alarm.raise_alarm_for_node(node_name, supplementary_info)
+
+ else:
+ with self.lock.writer():
+ self.csn.sync_node_csn(node_name)
+
+ return node_name in self.reboot_requests
+
+ def set_automatic_activation_state(self, state):
+ logging.debug('set_automatic_activation_state called, state is %s', state)
+
+ with self.lock.writer():
+ if state:
+ self.automatic_activation_disabled.unset()
+ else:
+ self.automatic_activation_disabled.set()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import logging
+
+from cmframework.server import cmwsgicallbacks
+from cmframework.apis import cmerror
+from cmframework.server.cmhttperrors import CMHTTPErrors
+
+
+class CMRestAPI(cmwsgicallbacks.CMWSGICallbacks):
+ def __init__(self, version, status, minimum_version, processor):
+ logging.debug('CMRestAPI constructor called with '
+ '{version, status, min_version}{%s, %s, %s}',
+ version, status, minimum_version)
+ self.version = version
+ self.status = status
+ self.minimum_version = minimum_version
+ self.processor = processor
+
+ def handle_property(self, rpc):
+ logging.debug('handle_property called')
+ if rpc.req_method == 'GET':
+ self.get_property(rpc)
+ elif rpc.req_method == 'POST':
+ self.set_property(rpc)
+ elif rpc.req_method == 'DELETE':
+ self.delete_property(rpc)
+ else:
+ rpc.rep_status = CMHTTPErrors.get_request_not_ok_status()
+ rpc.rep_status += ', only GET/POST/DELETE are possible to this resource'
+
+ def handle_properties(self, rpc):
+ logging.debug('handle_properties called')
+ if rpc.req_method == 'GET':
+ self.get_properties(rpc)
+ elif rpc.req_method == 'POST':
+ self.set_properties(rpc)
+ elif rpc.req_method == 'DELETE':
+ self.delete_properties(rpc)
+ else:
+ rpc.rep_status = CMHTTPErrors.get_request_not_ok_status()
+ rpc.rep_status += ', only GET/POST/DELETE are possible to this resource'
+
+ def handle_snapshots(self, rpc):
+ logging.debug('handle_snapshots called')
+ if rpc.req_method == 'GET':
+ self.list_snapshots(rpc)
+ else:
+ rpc.rep_status = CMHTTPErrors.get_request_not_ok_status()
+ rpc.rep_status += ', only GET is possible to this resource'
+
+ def handle_snapshot(self, rpc):
+ logging.debug('handle_snapshot called')
+ if rpc.req_method == 'GET':
+ self.create_snapshot(rpc)
+ elif rpc.req_method == 'POST':
+ self.restore_snapshot(rpc)
+ elif rpc.req_method == 'DELETE':
+ self.delete_snapshot(rpc)
+ else:
+ rpc.rep_status = CMHTTPErrors.get_request_not_ok_status()
+ rpc.rep_status += ', only GET/POST/DELETE are possible to this resource'
+
+ def handle_agent_activate(self, rpc):
+ logging.debug('handle_agent_activate called')
+ if rpc.req_method == 'GET':
+ self.activate_node(rpc)
+ else:
+ rpc.rep_status = CMHTTPErrors.get_request_not_ok_status()
+ rpc.rep_status += ', only GET is possible to this resource'
+
+ def handle_activate(self, rpc):
+ logging.debug('handle_activate called')
+ if rpc.req_method == 'POST':
+ self.activate(rpc)
+ else:
+ rpc.rep_status = CMHTTPErrors.get_request_not_ok_status()
+ rpc.rep_status += ', only POST is possible to this resource'
+
+ def handle_activator_disable(self, rpc):
+ logging.debug('handle_activator_disable called')
+ if rpc.req_method == 'POST':
+ self.set_automatic_activation_state(rpc, False)
+ else:
+ rpc.rep_status = CMHTTPErrors.get_request_not_ok_status()
+ rpc.rep_status += ', only POST is possible to this resource'
+
+ def handle_activator_enable(self, rpc):
+ logging.debug('handle_activator_enable called')
+ if rpc.req_method == 'POST':
+ self.set_automatic_activation_state(rpc, True)
+ else:
+ rpc.rep_status = CMHTTPErrors.get_request_not_ok_status()
+ rpc.rep_status += ', only POST is possible to this resource'
+
+ def handle_reboot(self, rpc):
+ logging.debug('handle_reboot called')
+ if rpc.req_method == 'GET':
+ self.reboot_node(rpc)
+ else:
+ rpc.rep_status = CMHTTPErrors.get_request_not_ok_status()
+ rpc.rep_status += ', only GET is possible to this resource'
+
+ def handle_changes(self, rpc):
+ logging.debug('handle_changes called')
+ if rpc.req_method == 'GET':
+ self.get_changes_states(rpc)
+ else:
+ rpc.rep_status = CMHTTPErrors.get_request_not_ok_status()
+ rpc.rep_status += ', only GET is possible to this resource'
+
+ # pylint: disable=no-self-use
+ def get_property(self, rpc):
+ logging.error('get_property not implemented')
+ rpc.rep_status = CMHTTPErrors.get_resource_not_found_status()
+ raise cmerror.CMError('Not implemented')
+
+ def get_properties(self, rpc):
+ logging.error('get_properties not implemented')
+ rpc.rep_status = CMHTTPErrors.get_resource_not_found_status()
+ raise cmerror.CMError('Not implemented')
+
+ def set_property(self, rpc):
+ logging.error('set_property not implemented')
+ rpc.rep_status = CMHTTPErrors.get_resource_not_found_status()
+ raise cmerror.CMError('Not implemented')
+
+ def set_properties(self, rpc):
+ logging.error('set_properties not implemented')
+ rpc.rep_status = CMHTTPErrors.get_resource_not_found_status()
+ raise cmerror.CMError('Not implemented')
+
+ def delete_property(self, rpc):
+ logging.error('delete_property not implemented')
+ rpc.rep_status = CMHTTPErrors.get_resource_not_found_status()
+ raise cmerror.CMError('Not implemented')
+
+ def delete_properties(self, rpc):
+ logging.error('delete_properties not implemented')
+ rpc.rep_status = CMHTTPErrors.get_resource_not_found_status()
+ raise cmerror.CMError('Not implemented')
+
+ def create_snapshot(self, rpc):
+ logging.error('create_snapshot not implemented')
+ rpc.rep_status = CMHTTPErrors.get_resource_not_found_status()
+ raise cmerror.CMError('Not implemented')
+
+ def restore_snapshot(self, rpc):
+ logging.error('restore_snapshot not implemented')
+ rpc.rep_status = CMHTTPErrors.get_resource_not_found_status()
+ raise cmerror.CMError('Not implemented')
+
+ def delete_snapshot(self, rpc):
+ logging.error('delete_snapshot not implemented')
+ rpc.rep_status = CMHTTPErrors.get_resource_not_found_status()
+ raise cmerror.CMError('Not implemented')
+
+ def list_snapshots(self, rpc):
+ logging.error('list_snapshots not implemented')
+ rpc.rep_status = CMHTTPErrors.get_resource_not_found_status()
+ raise cmerror.CMError('Not implemented')
+
+ def activate(self, rpc):
+ logging.error('activate not implemented')
+ rpc.rep_status = CMHTTPErrors.get_resource_not_found_status()
+ raise cmerror.CMError('Not implemented')
+
+ def activate_node(self, rpc):
+ logging.error('activate_node not implemented')
+ rpc.rep_status = CMHTTPErrors.get_resource_not_found_status()
+ raise cmerror.CMError('Not implemented')
+
+ def reboot_node(self, rpc):
+ logging.error('reboot_node not implemented')
+ rpc.rep_status = CMHTTPErrors.get_resource_not_found_status()
+ raise cmerror.CMError('Not implemented')
+
+ def get_changes_states(self, rpc):
+ logging.error('get_changes_states not implemented')
+ rpc.rep_status = CMHTTPErrors.get_resource_not_found_status()
+ raise cmerror.CMError('Not implemented')
+
+ def set_automatic_activation_state(self, rpc, state):
+ logging.error('set_automatic_activation_state not implemented')
+ rpc.rep_status = CMHTTPErrors.get_resource_not_found_status()
+ raise cmerror.CMError('Not implemented')
+
+ def get_version(self):
+ return self.version
+
+ def get_status(self):
+ return self.status
+
+ def get_minimum_version(self):
+ return self.minimum_version
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import json
+import logging
+
+from cmframework.server.cmhttperrors import CMHTTPErrors
+from cmframework.server import cmrestapiv1
+
+
+class CMRestAPIFactory(object):
+ def __init__(self, processor, base_url):
+ self.apis = {}
+ api = cmrestapiv1.CMRestAPIV1(processor)
+ self.apis[api.get_version()] = api
+ self.base_url = base_url
+
+ def get_apis(self, rpc):
+ """
+ Request: GET http://<cm-vip:port>/cm/apis
+ Response:
+ {
+ "versions": [
+ {
+ "id": "<version>",
+ "href": "<http address for the api>"
+ "min-version": "<mimimum version required to support this version>",
+ "status": "<supported|deprecated|unsupported|current>"
+ },
+ ...
+ ]
+ }
+ """
+ try:
+ logging.debug('get_apis called')
+ reply = {}
+ versions = []
+ for key, value in self.apis.iteritems():
+ version = {}
+ version['id'] = value.get_version()
+ version['href'] = '%s%s/' % (self.base_url, key)
+ version['min-version'] = value.get_minimum_version()
+ version['status'] = value.get_status()
+ versions.append(version)
+ reply['versions'] = versions
+ rpc.rep_status = CMHTTPErrors.get_ok_status()
+ rpc.rep_body = json.dumps(reply)
+ logging.debug('returning, rpc=%s', str(rpc))
+ except Exception as exp: # pylint: disable=broad-except
+ logging.error('Got exception %s', str(exp))
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+
+ def get_api(self, version):
+ logging.debug('get_api called with version %s', version)
+ api = None
+ try:
+ api = self.apis[version]
+ except KeyError:
+ logging.warn('Could not find API with version %s', version)
+ return api
+
+
+if __name__ == '__main__':
+ rest_api_factory = CMRestAPIFactory(None, None)
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import logging
+import json
+
+from cmframework.apis import cmerror
+from cmframework.server import cmrestapi
+from cmframework.server.cmhttperrors import CMHTTPErrors
+
+
+class CMRestAPIV1(cmrestapi.CMRestAPI):
+ def __init__(self, processor):
+ logging.debug('CMRestAPIV1 constructor called')
+ cmrestapi.CMRestAPI.__init__(self, 'v1.0', 'current', '1.0', processor)
+
+ def get_property(self, rpc):
+ """
+ Request: GET http://<cm-vip:port>/cm/v1.0/properties/
+ <property-name>?snapshot=<snapshot name>
+ Response: {
+ "name": "<name of the property>",
+ "value": "<value of the property>",
+ }
+ """
+
+ logging.debug('get_property called')
+ try:
+ snapshot_name = rpc.req_filter.get('snapshot', None)
+ if isinstance(snapshot_name, list):
+ snapshot_name = snapshot_name[0]
+ prop_name = rpc.req_params['property']
+ value = self.processor.get_property(prop_name, snapshot_name)
+ reply = {}
+ reply['name'] = prop_name
+ reply['value'] = value
+ rpc.rep_status = CMHTTPErrors.get_ok_status()
+ rpc.rep_body = json.dumps(reply)
+ except cmerror.CMError as exp:
+ rpc.rep_status = CMHTTPErrors.get_resource_not_found_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+ except KeyError:
+ rpc.rep_status = CMHTTPErrors.get_resource_not_found_status()
+ except Exception as exp: # pylint: disable=broad-except
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+
+ def get_properties(self, rpc):
+ """
+ Request: GET http://<cm-vip:port>/cm/v1.0/properties?
+ prop-name-filter=<filter>&snapshot=<snapshot name>
+ Response: {
+ "properties": [
+ {
+ "name": "<name of the property>",
+ "value": "<value of the property>"
+ },
+ {
+ "name": "<name of the property>",
+ "value": "<value of the property>"
+ }
+ ....
+ ]
+ }
+ """
+
+ logging.debug('get_properties called')
+ try:
+ prop_name_filter = rpc.req_filter.get('prop-name-filter', '')
+ if isinstance(prop_name_filter, list):
+ prop_name_filter = prop_name_filter[0]
+ snapshot_name = rpc.req_filter.get('snapshot', None)
+ if isinstance(snapshot_name, list):
+ snapshot_name = snapshot_name[0]
+ result = self.processor.get_properties(prop_name_filter, snapshot_name)
+ if not bool(result):
+ rpc.rep_status = CMHTTPErrors.get_resource_not_found_status()
+ else:
+ reply = {}
+ items = []
+ for key, value in result.iteritems():
+ tmp = {}
+ tmp['name'] = key
+ tmp['value'] = value
+ items.append(tmp)
+ reply['properties'] = items
+ rpc.rep_status = CMHTTPErrors.get_ok_status()
+ rpc.rep_body = json.dumps(reply)
+ except cmerror.CMError as exp:
+ rpc.rep_status = CMHTTPErrors.get_resource_not_found_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+ except Exception as exp: # pylint: disable=broad-except
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+
+ def set_property(self, rpc):
+ """
+ Request: POST http://<cm-vip:port>/cm/v1.0/properties/<property-name>
+ {
+ "value": "<value of the property>"
+ }
+ Response: http status set correctly
+ {
+ "change-uuid": "<uuid>"
+ }
+ """
+
+ logging.debug('set_property called')
+ try:
+ if not rpc.req_body:
+ rpc.rep_status = CMHTTPErrors.get_request_not_ok_status()
+ else:
+ request = json.loads(rpc.req_body)
+ name = rpc.req_params['property']
+ value = request['value']
+ uuid_value = self.processor.set_property(name, value)
+ rpc.rep_status = CMHTTPErrors.get_ok_status()
+ reply = {}
+ reply['change-uuid'] = uuid_value
+ rpc.rep_body = json.dumps(reply)
+ except cmerror.CMError as exp:
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+ except KeyError:
+ rpc.rep_status = CMHTTPErrors.get_request_not_ok_status()
+ except Exception as exp: # pylint: disable=broad-except
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+
+ def set_properties(self, rpc):
+ """
+ Request: POST http://<cm-vip:port>/cm/v1.0/properties
+ {
+ "overwrite": True|False,
+ "properties": [
+ {
+ "name": "<name of the property>",
+ "value": "<value of the property>"
+ },
+ {
+ "name": "<name of the property>",
+ "value": "<value of the property>"
+ },
+ ....
+ ]
+ }
+ Response:
+ {
+ "change-uuid": "<uuid>"
+ }
+ """
+
+ logging.debug('set_properties called')
+ try:
+ if not rpc.req_body:
+ rpc.rep_status = CMHTTPErrors.get_request_not_ok_status()
+ else:
+ request = json.loads(rpc.req_body)
+ overwrite = False
+ if 'overwrite' in request:
+ overwrite = request['overwrite']
+ items = request['properties']
+ data = {}
+ for entry in items:
+ name = entry['name']
+ value = entry['value']
+ data[name] = value
+ uuid_value = self.processor.set_properties(data, overwrite)
+ rpc.rep_status = CMHTTPErrors.get_ok_status()
+ reply = {}
+ reply['change-uuid'] = uuid_value
+ rpc.rep_body = json.dumps(reply)
+ except cmerror.CMError as exp:
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+ except KeyError:
+ rpc.rep_status = CMHTTPErrors.get_request_not_ok_status()
+ except Exception as exp: # pylint: disable=broad-except
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+
+ def delete_property(self, rpc):
+ """
+ Request: DELETE http://<cm-vip:port>/cm/v1.0/properties/<property-name>
+ Response: http response with proper status
+ {
+ "change-uuid": "<uuid>"
+ }
+ """
+
+ logging.debug('delete_property called')
+ try:
+ prop = rpc.req_params['property']
+ uuid_value = self.processor.delete_property(prop)
+ rpc.rep_status = CMHTTPErrors.get_ok_status()
+ reply = {}
+ reply['change-uuid'] = uuid_value
+ rpc.rep_body = json.dumps(reply)
+ except cmerror.CMError as exp:
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+ except KeyError:
+ rpc.rep_status = CMHTTPErrors.get_request_not_ok_status()
+ except Exception as exp: # pylint: disable=broad-except
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+
+ def delete_properties(self, rpc):
+ """
+ Request: DELETE http://<cm-vip:port>/cm/v1.0/properties?prop-name-filter=<filter>
+ {
+ 'properties': [ <prop-name>,
+ <prop-name>,
+ ....
+ ]
+ }
+ Response: http response with proper status
+ {
+ "change-uuid": "<uuid>"
+ }
+ """
+
+ logging.debug('delete_properties called')
+ try:
+ if not rpc.req_filter and not rpc.req_body:
+ rpc.rep_status = CMHTTPErrors.get_request_not_ok_status()
+ else:
+ if rpc.req_filter:
+ arg = rpc.req_filter.get('prop-name-filter', '')
+ if isinstance(arg, list):
+ arg = arg[0]
+ else:
+ body = json.loads(rpc.req_body)
+ arg = body['properties']
+ uuid_value = self.processor.delete_properties(arg)
+ rpc.rep_status = CMHTTPErrors.get_ok_status()
+ reply = {}
+ reply['change-uuid'] = uuid_value
+ rpc.rep_body = json.dumps(reply)
+ except cmerror.CMError as exp:
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+ except KeyError:
+ rpc.rep_status = CMHTTPErrors.get_request_not_ok_status()
+ except Exception as exp: # pylint: disable=broad-except
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+
+ def create_snapshot(self, rpc):
+ """
+ Request: GET http://<cm-vip:port>/cm/v1.0/snapshots/<snapshot name>
+ Response: http response with proper status
+ """
+
+ logging.debug('create snapshot called')
+ try:
+ snapshot_name = rpc.req_params['snapshot']
+ self.processor.create_snapshot(snapshot_name)
+ rpc.rep_status = CMHTTPErrors.get_ok_status()
+ except cmerror.CMError as exp:
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+ except Exception as exp: # pylint: disable=broad-except
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+
+ def restore_snapshot(self, rpc):
+ """
+ Request: POST http://<cm-vip:port>/cm/v1.0/snapshots/<snapshot name>
+ Response: http response with proper status
+ """
+
+ logging.debug('restore_snapshot called')
+ try:
+ snapshot_name = rpc.req_params['snapshot']
+ self.processor.restore_snapshot(snapshot_name)
+ rpc.rep_status = CMHTTPErrors.get_ok_status()
+ except cmerror.CMError as exp:
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+ except Exception as exp:
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+
+ def delete_snapshot(self, rpc):
+ """
+ Request: DELETE http://<cm-vip:port>/cm/v1.0/snapshots/<snapshot name>
+ Response: http response with proper status
+ """
+
+ logging.debug('delete_snapshot called')
+ try:
+ snapshot_name = rpc.req_params['snapshot']
+ self.processor.delete_snapshot(snapshot_name)
+ rpc.rep_status = CMHTTPErrors.get_ok_status()
+ except cmerror.CMError as exp:
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+ except KeyError as exp:
+ rpc.rep_status = CMHTTPErrors.get_request_not_ok_status()
+ except Exception as exp:
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+
+ def list_snapshots(self, rpc):
+ """
+ Request: GET http://<cm-vip:port>/cm/v1.0/snapshots
+ Response: {
+ "snapshots": [
+ "<name of the snapshot>",
+ ....
+ ]
+ }
+ """
+
+ logging.debug('list_snapshots called')
+ try:
+ snapshots = self.processor.list_snapshots()
+ if not bool(snapshots):
+ rpc.rep_status = CMHTTPErrors.get_resource_not_found_status()
+ else:
+ reply = {}
+ reply['snapshots'] = snapshots
+ rpc.rep_status = CMHTTPErrors.get_ok_status()
+ rpc.rep_body = json.dumps(reply)
+ except cmerror.CMError as exp:
+ rpc.rep_status = CMHTTPErrors.get_resource_not_found_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+ except Exception as exp: # pylint: disable=broad-except
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+
+ def activate(self, rpc):
+ """
+ Request: POST http://<cm-vip:port>/cm/v1.0/activator/<node-name>
+ Response: http response with proper status
+ {
+ "change-uuid": "<uuid>"
+ }
+ """
+
+ logging.debug('activate called')
+ try:
+ node_name = rpc.req_params.get('node', None)
+ uuid_value = self.processor.activate(node_name)
+ rpc.rep_status = CMHTTPErrors.get_ok_status()
+ reply = {}
+ reply['change-uuid'] = uuid_value
+ rpc.rep_body = json.dumps(reply)
+ except cmerror.CMError as exp:
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+ except Exception as exp: # pylint: disable=broad-except
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+
+ def activate_node(self, rpc):
+ """
+ Request: GET http://<cm-vip:port>/cm/v1.0/activator/agent/<node name>
+ Response: {
+ "name": "<name of the node>",
+ "reboot": "<boolean value indicating whether a reboot is needed>",
+ }
+ """
+
+ logging.debug('activate_node called')
+ try:
+ node_name = rpc.req_params['node']
+ reboot_needed = self.processor.activate_node(node_name)
+ reply = {}
+ reply['name'] = node_name
+ reply['reboot'] = reboot_needed
+ rpc.rep_status = CMHTTPErrors.get_ok_status()
+ rpc.rep_body = json.dumps(reply)
+ except cmerror.CMError as exp:
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+ except Exception as exp: # pylint: disable=broad-except
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+
+ def set_automatic_activation_state(self, rpc, state):
+ """
+ Request: POST http://<cm-vip:port>/cm/v1.0/activator/disable
+ or POST http://<cm-vip:port>/cm/v1.0/activator/enable
+ Response: http response with proper status
+ """
+
+ logging.debug('set_automatic_activation_state called')
+ try:
+ self.processor.set_automatic_activation_state(state)
+ rpc.rep_status = CMHTTPErrors.get_ok_status()
+ except cmerror.CMError as exp:
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+ except Exception as exp: # pylint: disable=broad-except
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+
+ def reboot_node(self, rpc):
+ """
+ Request: GET http://<cm-vip:port>/cm/v1.0/reboot?node-name=<node name>
+ Response: {
+ "node-name": "<name of the node>",
+ }
+ """
+
+ logging.debug('reboot_node called')
+ try:
+ if not rpc.req_filter:
+ rpc.rep_status = CMHTTPErrors.get_request_not_ok_status()
+ else:
+ node_name = rpc.req_filter.get('node-name', None)
+ if isinstance(node_name, list):
+ node_name = node_name[0]
+ self.processor.reboot_request(node_name)
+ reply = {}
+ reply['node-name'] = node_name
+ rpc.rep_status = CMHTTPErrors.get_ok_status()
+ rpc.rep_body = json.dumps(reply)
+ except cmerror.CMError as exp:
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+ except Exception as exp: # pylint: disable=broad-except
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+
+ def get_changes_states(self, rpc):
+ """
+ Request: GET http://<cm-vip:port>/cm/v1.0/changes?changd-uuid-filter=<filter>
+ Response: {
+ "<change-uuid>"" : {
+ "state": "<state>",
+ "failed-plugins": { "plugin-name":"error", ... }
+ ...
+ }
+ """
+
+ logging.debug('get_changes_states called')
+ try:
+ reply = {}
+ changemonitor = self.processor.changemonitor
+ change_uuid_value = None
+ change_uuid_list = rpc.req_filter.get('change-uuid-filter', None)
+ if change_uuid_list:
+ change_uuid_value = change_uuid_list[0]
+ state = changemonitor.get_change_state(change_uuid_value)
+ reply[change_uuid_value] = {}
+ reply[change_uuid_value]["state"] = state.state
+ reply[change_uuid_value]["failed-plugins"] = state.failed_plugins
+ else:
+ changes = changemonitor.get_all_changes_states()
+ for change_uuid, state in changes.iteritems():
+ reply[change_uuid] = {}
+ reply[change_uuid]["state"] = state.state
+ reply[change_uuid]["failed-plugins"] = state.failed_plugins
+
+ rpc.rep_status = CMHTTPErrors.get_ok_status()
+ rpc.rep_body = json.dumps(reply)
+ except cmerror.CMError as exp:
+ rpc.rep_status = CMHTTPErrors.get_resource_not_found_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+ except KeyError:
+ rpc.rep_status = CMHTTPErrors.get_resource_not_found_status()
+ except Exception as exp: # pylint: disable=broad-except
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
--- /dev/null
+#! /usr/bin/python
+
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import eventlet # noqa
+from eventlet import wsgi # noqa
+eventlet.monkey_patch() # noqa
+import os
+import json
+import sys
+import logging
+import traceback
+
+import ConfigParser
+from cmframework.utils import cmbackendpluginclient
+from cmframework.utils import cmlogger
+from cmframework.utils import cmbackendhandler
+from cmframework.utils import cmactivationstatehandler
+from cmframework.utils import cmalarmhandler
+from cmframework.utils import cmsnapshothandler
+from cmframework.server import cmargs
+from cmframework.server import cmprocessor
+from cmframework.server import cmrestapifactory
+from cmframework.server import cmwsgihandler
+from cmframework.server import cmvalidator
+from cmframework.server import cmactivator
+from cmframework.server import cmactivateserverhandler
+from cmframework.server import cmchangemonitor
+from cmframework.utils.cmansibleinventory import AnsibleInventory
+
+
+def main():
+ try:
+ # parse the arguments
+ parser = cmargs.CMArgsParser('cmserver')
+ parser.parse(sys.argv[1:])
+
+ # Read configuration names which should be masked in logging
+ mask_names = _read_maskable_names()
+
+ # initialize the logger
+ _ = cmlogger.CMLogger(parser.get_log_dest(),
+ parser.get_verbose(),
+ parser.get_log_level(),
+ mask_names)
+
+ logging.info('CM server is starting up')
+
+ # initialize the change monitor object
+ changemonitor = cmchangemonitor.CMChangeMonitor()
+
+ # load backend plugin
+ logging.info('Initializing backend handler')
+ backend_args = {}
+ backend_args['uri'] = parser.get_backend_uri()
+ backend = cmbackendhandler.CMBackendHandler(parser.get_backend_api(), **backend_args)
+
+ # construct the plugin client library
+ logging.info('Initialize plugin client library')
+ plugin_client = cmbackendpluginclient.CMBackendPluginClient(parser.get_backend_api(),
+ **backend_args)
+
+ # load activation state handler
+ logging.info('Initializing activation state handler')
+ activationstatehandler_args = {}
+ activationstatehandler_args['uri'] = parser.get_activationstate_handler_uri()
+ activationstate_handler = cmactivationstatehandler.CMActivationStateHandler(
+ parser.get_activationstate_handler_api(), **activationstatehandler_args)
+
+ # initialize validator
+ logging.info('Initializing validator')
+ validator = cmvalidator.CMValidator(parser.get_validators(), plugin_client)
+
+ # initializing activation handling process
+ logging.info('Initializing activator')
+ activator = cmactivator.CMActivator(parser.get_activator_workers())
+
+ # initialize activator rmq handler
+ if not parser.get_disable_remote_activation():
+ from cmframework.server import cmactivatermqhandler
+ logging.info('Initializing activator rmq handler')
+ activatermqhandler = cmactivatermqhandler.CMActivateRMQHandler(parser.get_rmq_ip(),
+ parser.get_rmq_port())
+ activator.add_handler(activatermqhandler)
+
+ # initialize activator server handler
+ logging.info('Initializing activator server handler')
+ activateserverhandler = cmactivateserverhandler.CMActivateServerHandler(
+ parser.get_activators(), plugin_client, changemonitor, activationstate_handler)
+ activator.add_handler(activateserverhandler)
+
+ # starting activator
+ logging.info('Starting activation handling process')
+ activator.start()
+
+ # start alarm handler
+ logging.info('Starting alarm handler process')
+ alarmhandler = cmalarmhandler.AlarmHandler()
+ alarmhandler.set_library_impl(parser.get_alarmhandler_api())
+ alarmhandler.start()
+
+ # load snapshot handler
+ logging.info('Initializing snapshot handler')
+ snapshothandler_args = {}
+ snapshothandler_args['uri'] = parser.get_snapshot_handler_uri()
+ snapshot_handler = cmsnapshothandler.CMSnapshotHandler(
+ parser.get_snapshot_handler_api(), **snapshothandler_args)
+
+ # initialize processor
+ logging.info('Initializing CM processor')
+ processor = cmprocessor.CMProcessor(backend,
+ validator,
+ activator,
+ changemonitor,
+ activationstate_handler,
+ snapshot_handler)
+
+ if not parser.is_install_phase():
+ # generate inventory file
+ logging.info('Generate inventory file')
+ properties = backend.get_properties('.*')
+ inventory = AnsibleInventory(properties, parser.get_inventory_handlers())
+ inventory_data = inventory.generate_inventory()
+ with open(parser.get_inventory_data(), 'w') as inventory_file:
+ inventory_file.write(json.dumps(inventory_data, indent=4, sort_keys=True))
+
+ # publish full activate request to on-line activators
+ logging.info('Ask on-line activators to full translate')
+ processor.activate(startup_activation=True)
+
+ # remove inventory file
+ logging.info('Remove inventory file')
+ try:
+ os.remove(parser.get_inventory_data())
+ except OSError:
+ pass
+
+ # initialize rest api factory
+ logging.info('Initializing REST API factory')
+ base_url = 'http://' + parser.get_ip() + ':' + \
+ str(parser.get_port()) + '/cm/'
+ rest_api_factory = cmrestapifactory.CMRestAPIFactory(processor, base_url)
+
+ # initialize wsgi handler
+ logging.info('Initializing the WSGI handler')
+ wsgihandler = cmwsgihandler.CMWSGIHandler(rest_api_factory)
+
+ # start the http server
+ logging.info('Start listening to http requests')
+ wsgi.server(eventlet.listen((parser.get_ip(), parser.get_port())), wsgihandler)
+ except KeyboardInterrupt as exp:
+ logging.info('CM server shutting down')
+ return 0
+ except Exception as exp: # pylint: disable=broad-except
+ logging.error('Got exception %s', str(exp))
+ traceback.print_exc()
+ return 1
+
+
+def _read_maskable_names():
+ MASK_DIR = '/etc/cmframework/masks.d/'
+ all_names = []
+ filenames = [MASK_DIR + f for f in os.listdir(MASK_DIR)]
+ for filename in filenames:
+ config = ConfigParser.SafeConfigParser()
+ config.read(filename)
+ names = json.loads(config.get('Passwords', 'names'))
+ all_names.extend(names)
+ return all_names
+
+
+if __name__ == "__main__":
+ sys.exit(main())
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+class _CMSingleton(type):
+ _instances = {}
+
+ def __call__(cls, *args, **kwargs):
+ if cls not in cls._instances:
+ cls._instances[cls] = super(_CMSingleton, cls).__call__(*args, **kwargs)
+ return cls._instances[cls]
+
+
+class CMSingleton(_CMSingleton('SingletonMeta', (object,), {})):
+ pass
+
+
+def main():
+ class TestSingleton(CMSingleton):
+ def __init__(self):
+ self.counter = 0
+
+ def inc(self):
+ self.counter += 1
+
+ def dec(self):
+ self.counter -= 1
+
+ def __str__(self):
+ return 'Counter now is %d' % self.counter
+
+ instance1 = TestSingleton()
+ instance1.inc()
+ instance2 = TestSingleton()
+ instance2.inc()
+ print str(instance2)
+
+
+if __name__ == '__main__':
+ main()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import logging
+import re
+import datetime
+
+from cmframework.apis import cmerror
+
+
+class CMSnapshot(object):
+ def __init__(self, handler):
+ logging.debug('CMSnapshot constructed')
+
+ self._handler = handler
+ self._metadata = {}
+ self._data = {}
+
+ def get_property(self, prop_name):
+ if not self._metadata:
+ raise cmerror.CMError('No data: create or load first')
+
+ properties = self.get_properties(prop_name)
+
+ return properties.get(prop_name)
+
+ def get_properties(self, prop_filter='.*'):
+ if not self._data:
+ raise cmerror.CMError('No data: create or load first')
+
+ matched_properties = {}
+ pattern = re.compile(prop_filter)
+ for key, value in self._data:
+ if pattern.match(key):
+ matched_properties[key] = value
+
+ return matched_properties
+
+ def create(self, snapshot_name, source_backend, custom_metadata=None):
+ logging.debug('create_snapshot called, snapshot name is %s', snapshot_name)
+
+ if self._handler.snapshot_exists(snapshot_name):
+ raise cmerror.CMError('Snapshot already exist')
+
+ self._metadata = {}
+ self._metadata['name'] = snapshot_name
+ self._metadata['creation_date'] = datetime.datetime.now().isoformat()
+ self._metadata['custom'] = custom_metadata
+
+ self._data = source_backend.get_properties('.*')
+
+ snapshot_data = {'snapshot_properties': self._data, 'snapshot_metadata': self._metadata}
+ self._handler.set_data(snapshot_name, snapshot_data)
+
+ def load(self, snapshot_name):
+ logging.debug('load_snapshot called, snapshot name is %s', snapshot_name)
+
+ if not self._handler.snapshot_exists(snapshot_name):
+ raise cmerror.CMError('Snapshot does not exist')
+
+ snapshot_data = self._handler.get_data(snapshot_name)
+
+ self._metadata = snapshot_data.get('snapshot_metadata')
+ if not self._metadata:
+ raise cmerror.CMError('Could not load snapshot metadata for {}'.format(snapshot_name))
+
+ self._data = snapshot_data.get('snapshot_properties')
+
+ def restore(self, target_backend):
+ logging.debug('restore_snapshot called')
+
+ if not self._data:
+ raise cmerror.CMError('No data: create or load first')
+
+ current_properties = target_backend.get_properties('.*')
+ current_keys = current_properties.keys()
+
+ if len(current_keys) == 1:
+ target_backend.delete_property(current_keys[0])
+ else:
+ target_backend.delete_properties(current_keys)
+
+ target_backend.set_properties(self._data)
+
+ def list(self):
+ logging.debug('list_snapshots called')
+
+ snapshots = []
+
+ snapshot_names = self._handler.list_snapshots()
+
+ for snapshot_name in snapshot_names:
+ snapshot_data = self._handler.get_data(snapshot_name)
+ metadata = snapshot_data.get('snapshot_metadata')
+ if not metadata:
+ logging.warning('Could not load snapshot metadata for %s', snapshot_name)
+ continue
+
+ snapshots.append(metadata)
+
+ return snapshots
+
+ def delete(self, snapshot_name):
+ logging.debug('delete_snapshot called, snapshot name is %s', snapshot_name)
+
+ if not self._handler.snapshot_exists(snapshot_name):
+ raise cmerror.CMError('Snapshot does not exist')
+
+ self._handler.delete_snapshot(snapshot_name)
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# vim: tabstop=8 expandtab shiftwidth=4 softtabstop=4
+import logging
+
+from cmframework.utils.cmpluginloader import CMPluginLoader
+from cmframework.utils.cmpluginmanager import CMPluginManager
+
+
+class CMValidator(CMPluginManager):
+ def __init__(self, plugins_path, plugin_client):
+ logging.info('Validator constructor, plugins_path is %s', plugins_path)
+ CMPluginManager.__init__(self, plugins_path)
+ self.load_plugin()
+ self.plugin_client = plugin_client
+
+ def load_plugin(self):
+ pl = CMPluginLoader(self.plugins_path)
+ self.pluginlist, self.filterdict = pl.load()
+ logging.info('Plugin(s): %r', self.pluginlist)
+ logging.info('Subscription(s): %r', self.filterdict)
+
+ def validate_delete(self, indata):
+ self.validate_plugins(indata, 'validate_delete')
+
+ def validate_set(self, indata):
+ self.validate_plugins(indata, 'validate_set')
+
+ def validate_plugins(self, indata, operation):
+ # import pdb; pdb.set_trace()
+ logging.debug('validate_plugins called with data %s', indata)
+ for plugin, objectname in self.pluginlist.iteritems():
+ filtername = self.filterdict[plugin]
+ inputdata = self.build_input(indata, filtername)
+ if inputdata:
+ logging.debug('Calling validation plugin %s with %s', plugin, inputdata)
+ class_name = getattr(objectname, plugin)
+ instance = class_name()
+ instance.plugin_client = self.plugin_client
+ try:
+ func = getattr(instance, operation)
+ func(inputdata)
+ except AttributeError:
+ logging.info('Plugin %s does have function %s implemented', plugin, operation)
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import logging
+
+from cmframework.apis import cmerror
+
+
+class CMWSGICallbacks(object):
+ # pylint: disable=no-self-use, unused-argument
+ def handle_properties(self, rpc):
+ logging.error('handle_properties not implemented')
+ raise cmerror.CMError('Not implemented')
+
+ def handle_property(self, rpc):
+ logging.error('handle_property not implemented')
+ raise cmerror.CMError('Not implemented')
+
+ def handle_snapshots(self, rpc):
+ logging.error('handle_snapshots not implemented')
+ raise cmerror.CMError('Not implemented')
+
+ def handle_snapshot(self, rpc):
+ logging.error('handle_snapshot not implemented')
+ raise cmerror.CMError('Not implemented')
+
+ def handle_agent_activate(self, rpc):
+ logging.error('handle_agent_activate not implemented')
+ raise cmerror.CMError('Not implemented')
+
+ def handle_activate(self, rpc):
+ logging.error('handle_activate not implemented')
+ raise cmerror.CMError('Not implemented')
+
+ def handle_activator_disable(self, rpc):
+ logging.error('handle_activator_disable not implemented')
+ raise cmerror.CMError('Not implemented')
+
+ def handle_activator_enable(self, rpc):
+ logging.error('handle_activator_enable not implemented')
+ raise cmerror.CMError('Not implemented')
+
+ def handle_reboot(self, rpc):
+ logging.error('handle_reboot not implemented')
+ raise cmerror.CMError('Not implemented')
+
+ def handle_changes(self, rpc):
+ logging.error('handle_changes not implemented')
+ raise cmerror.CMError('Not implemented')
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import logging
+import urllib
+import urlparse
+import routes
+
+from cmframework.server.cmhttperrors import CMHTTPErrors
+from cmframework.apis import cmerror
+from cmframework.server import cmhttprpc
+
+
+class CMWSGIHandler(object):
+ def __init__(self, rest_api_factory):
+ logging.debug('CMWSGIHandler constructor called')
+ self.mapper = routes.Mapper()
+ self.mapper.connect(None, '/cm/apis', action='get_apis')
+ self.mapper.connect(None, '/cm/{api}/properties', action='handle_properties')
+ self.mapper.connect(None, '/cm/{api}/properties/{property}', action='handle_property')
+ self.mapper.connect(None, '/cm/{api}/snapshots', action='handle_snapshots')
+ self.mapper.connect(None, '/cm/{api}/snapshots/{snapshot}', action='handle_snapshot')
+ self.mapper.connect(None, '/cm/{api}/activator/enable', action='handle_activator_enable')
+ self.mapper.connect(None, '/cm/{api}/activator/disable', action='handle_activator_disable')
+ self.mapper.connect(None, '/cm/{api}/activator/agent/{node}',
+ action='handle_agent_activate')
+ self.mapper.connect(None, '/cm/{api}/activator/{node}', action='handle_activate')
+ self.mapper.connect(None, '/cm/{api}/activator', action='handle_activate')
+ self.mapper.connect(None, '/cm/{api}/reboot', action='handle_reboot')
+ self.mapper.connect(None, '/cm/{api}/changes', action='handle_changes')
+ self.rest_api_factory = rest_api_factory
+
+ def __call__(self, environ, start_response):
+ logging.debug('Handling request started, environ=%s', str(environ))
+ # for debug, print environment
+ # pprint.pprint(environ)
+
+ # For request and resonse data
+ rpc = cmhttprpc.HTTPRPC()
+ rpc.rep_status = CMHTTPErrors.get_ok_status()
+
+ # get the interesting fields
+ rpc.req_method = environ['REQUEST_METHOD']
+ path = environ['PATH_INFO']
+ try:
+ rpc.req_filter = urlparse.parse_qs(urllib.unquote(environ['QUERY_STRING']))
+ except KeyError as exp:
+ rpc.req_filter = {}
+ content_type = environ['CONTENT_TYPE']
+ try:
+ content_size = environ['CONTENT_LENGTH']
+ except KeyError:
+ content_size = None
+
+ try:
+ # get the action to be done
+ action = ''
+ actions, _ = self.mapper.routematch(path)
+ if actions and isinstance(actions, dict):
+ action = actions.get('action', '')
+ for key, value in actions.iteritems():
+ if key != 'action':
+ rpc.req_params[key] = value
+ else:
+ rpc.rep_status = CMHTTPErrors.get_resource_not_found_status()
+ raise cmerror.CMError('The requested url is not found')
+
+ # get the body if available
+ if content_size and int(content_size):
+ size = int(content_size)
+ if content_type == 'application/json':
+ totalread = 0
+ while totalread < size:
+ data = environ['wsgi.input'].read()
+ totalread += len(data)
+ rpc.req_body += data
+ else:
+ rpc.rep_status = CMHTTPErrors.get_unsupported_content_type_status()
+ raise cmerror.CMError('Only json content is supported')
+
+ # check the action
+ try:
+ logging.info('Calling %s with rpc=%s', action, str(rpc))
+ actionfunc = getattr(self, action)
+ actionfunc(rpc)
+ except AttributeError as attrerror:
+ rpc.reply_status = CMHTTPErrors.get_resource_not_found_status()
+ raise cmerror.CMError('Action %s not found, error: %s' % (action, str(attrerror)))
+
+ except cmerror.CMError as exp:
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+ except Exception as exp: # pylint: disable=broad-except
+ rpc.rep_status = CMHTTPErrors.get_internal_error_status()
+ rpc.rep_status += ','
+ rpc.rep_status += str(exp)
+ finally:
+ logging.info('Replying with rpc=%s', str(rpc))
+ response_headers = [('Content-type', 'application/json')]
+ start_response(rpc.rep_status, response_headers)
+ yield rpc.rep_body
+
+ def get_apis(self, rpc):
+ logging.debug('get_apis called')
+ if rpc.req_method != 'GET':
+ rpc.rep_status = CMHTTPErrors.get_request_not_ok_status()
+ rpc.rep_status += ', only GET operation is possible'
+ else:
+ self.rest_api_factory.get_apis(rpc)
+
+ def handle_properties(self, rpc):
+ logging.debug('handle_properties called')
+ api = self._get_api(rpc)
+ if api:
+ api.handle_properties(rpc)
+
+ def handle_property(self, rpc):
+ logging.debug('handle_property called')
+ api = self._get_api(rpc)
+ if api:
+ api.handle_property(rpc)
+
+ def handle_snapshots(self, rpc):
+ logging.debug('handle_snapshots called')
+ api = self._get_api(rpc)
+ if api:
+ api.handle_snapshots(rpc)
+
+ def handle_snapshot(self, rpc):
+ logging.debug('handle_snapshot called')
+ api = self._get_api(rpc)
+ if api:
+ api.handle_snapshot(rpc)
+
+ def handle_agent_activate(self, rpc):
+ logging.debug('handle_agent_activate called')
+ api = self._get_api(rpc)
+ if api:
+ api.handle_agent_activate(rpc)
+
+ def handle_activate(self, rpc):
+ logging.debug('handle_activate called')
+ api = self._get_api(rpc)
+ if api:
+ api.handle_activate(rpc)
+
+ def handle_activator_disable(self, rpc):
+ logging.debug('handle_activator_disable called')
+ api = self._get_api(rpc)
+ if api:
+ api.handle_activator_disable(rpc)
+
+ def handle_activator_enable(self, rpc):
+ logging.debug('handle_activator_enable called')
+ api = self._get_api(rpc)
+ if api:
+ api.handle_activator_enable(rpc)
+
+ def handle_reboot(self, rpc):
+ logging.debug('handle_reboot called')
+ api = self._get_api(rpc)
+ if api:
+ api.handle_reboot(rpc)
+
+ def handle_changes(self, rpc):
+ logging.debug('handle_changes called')
+ api = self._get_api(rpc)
+ if api:
+ api.handle_changes(rpc)
+
+ def _get_api(self, rpc):
+ logging.debug('_get_api called')
+ api = None
+ try:
+ version = rpc.req_params['api']
+ api = self.rest_api_factory.get_api(version)
+ if not api:
+ rpc.rep_status = CMHTTPErrors.get_resource_not_found_status()
+ except KeyError:
+ rpc.rep_status = CMHTTPErrors.get_request_not_ok_status()
+ return api
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from __future__ import print_function
+import logging
+import pika
+
+from cmframework.apis import cmerror
+from cmframework.utils import cmactivationwork
+
+
+class CMActivationRMQ(object):
+ EXCHANGE = 'cmframework.activator'
+
+ def __init__(self, host, port):
+ try:
+ self.host = host
+ self.port = port
+ self.connection = pika.BlockingConnection(
+ pika.ConnectionParameters(host=self.host, port=self.port))
+ self.channel = self.connection.channel()
+ self.channel.exchange_declare(exchange=CMActivationRMQ.EXCHANGE, type='direct')
+ except Exception as exp: # pylint: disable=broad-except
+ raise cmerror.CMError(str(exp))
+
+
+class CMActivationRMQPublisher(CMActivationRMQ):
+ def __init__(self, host, port):
+ CMActivationRMQ.__init__(self, host, port)
+
+ def send(self, work):
+ try:
+ data = work.serialize()
+ self.channel.basic_publish(exchange=CMActivationRMQ.EXCHANGE,
+ routing_key=work.get_target(),
+ body=data)
+ logging.debug('Sent %s to activation exchange', str(work))
+ except Exception as exp: # pylint: disable=broad-except
+ self.connection.close()
+ raise cmerror.CMError(str(exp))
+
+
+class CMActivationRMQConsumer(CMActivationRMQ):
+ class WorkConsumer(object):
+ # pylint: disable=no-self-use, unused-argument
+ def consume(self, work):
+ raise cmerror.CMError('Not implemented')
+
+ def __init__(self, host, port, consumer, node):
+ CMActivationRMQ.__init__(self, host, port)
+ self.node = node
+ result = self.channel.queue_declare(exclusive=True)
+ self.queue_name = result.method.queue
+ self.channel.queue_bind(exchange=CMActivationRMQ.EXCHANGE, queue=self.queue_name,
+ routing_key=node)
+ self.channel.queue_bind(exchange=CMActivationRMQ.EXCHANGE, queue=self.queue_name,
+ routing_key='all')
+ self.channel.basic_consume(self,
+ queue=self.queue_name,
+ no_ack=True)
+ self.consumer = consumer
+
+ def __call__(self, ch, method, properties, body):
+ logging.debug('Received %r', body)
+ work = cmactivationwork.CMActivationWork()
+ work.deserialize(body)
+ self.consumer.consume(work)
+
+ def receive(self):
+ try:
+ self.channel.start_consuming()
+ except Exception as exp: # pylint: disable=broad-except
+ self.connection.close()
+ raise cmerror.CMError(str(exp))
+
+
+def main():
+ import sys
+ import argparse
+
+ class MyConsumer(CMActivationRMQConsumer.WorkConsumer):
+ def consume(self, work):
+ print('Got work %s' % work)
+
+ parser = argparse.ArgumentParser(description='Test rabbitmq activator', prog=sys.argv[0])
+ parser.add_argument('--role', required=True, action='store')
+ parser.add_argument('--host', required=True, action='store')
+ parser.add_argument('--port', required=True, type=int, action='store')
+ parser.add_argument('--operation', required=False, type=int, action='store')
+ parser.add_argument('--csn', required=False, type=int, action='store')
+ parser.add_argument('--node', required=True, type=str, action='store')
+ parser.add_argument('--properties', required=False, nargs=2, action='append')
+ args = parser.parse_args(sys.argv[1:])
+ if args.role == 'publisher':
+ if not args.operation or not args.csn or not args.properties:
+ print('Missing options')
+ sys.exit(1)
+ publisher = CMActivationRMQPublisher(args.host, args.port)
+ props = {}
+ for prop in args.properties:
+ props[prop[0]] = prop[1]
+ work = cmactivationwork.CMActivationWork(args.operation, args.csn, props)
+ publisher.send(work)
+ elif args.role == 'consumer':
+ myconsumer = MyConsumer()
+ consumer = CMActivationRMQConsumer(args.host, args.port, myconsumer, args.node)
+ consumer.receive()
+ else:
+ print('Invalid role %s' % args.role)
+ sys.exit(1)
+
+
+if __name__ == '__main__':
+ main()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from __future__ import print_function
+import logging
+import json
+
+from cmframework.utils.cmstatehandler import CMStateHandler
+
+
+class CMActivationStateHandler(CMStateHandler):
+ def set_full_failed(self, failed_activators):
+ logging.debug('set_full_failed called with: %s', failed_activators)
+
+ self.plugin.set('cm.activation_status', 'full', json.dumps(failed_activators))
+
+ def get_full_failed(self):
+ logging.debug('get_full_failed called')
+
+ full_failed = self.plugin.get('cm.activation_status', 'full')
+ if not full_failed:
+ return []
+
+ return json.loads(full_failed)
+
+ def clear_full_failed(self):
+ logging.debug('clear_full_failed called')
+
+ self.plugin.set('cm.activation_status', 'full', json.dumps([]))
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from threading import Condition
+import json
+
+from cmframework.apis import cmerror
+
+
+class CMActivationWork(object):
+ """
+ Serialize/Deserialize the work to/from json, the structure of the json data is the following:
+ {
+ 'operation': <operation number>,
+ 'csn': <csn number>,
+ 'properties': {
+ '<name>': '<value>',
+ ....
+ }
+ }
+ """
+
+ OPER_NONE = 0
+ OPER_SET = 1
+ OPER_DELETE = 2
+ OPER_FULL = 3
+ OPER_NODE = 4
+
+ OPER_NAMES = ('NONE', 'SET', 'DELETE', 'FULL', 'NODE')
+
+ def __init__(self,
+ operation=OPER_NONE,
+ csn=0,
+ props=None,
+ target=None,
+ startup_activation=False):
+ self.operation = operation
+ self.csn = csn
+ if not props:
+ self.props = {}
+ else:
+ self.props = props
+ self.target = target
+ self.condition = Condition()
+ self.condition.acquire()
+ self.result = None
+ self.uuid_value = None
+ self.startup_activation = startup_activation
+
+ def __str__(self):
+ return '(%r %d %r %r %r)' % (self._get_operation_name(),
+ self.csn,
+ self.props,
+ self.target,
+ self.startup_activation)
+
+ def _get_operation_name(self):
+ return CMActivationWork.OPER_NAMES[self.operation]
+
+ def get_operation(self):
+ return self.operation
+
+ def get_csn(self):
+ return self.csn
+
+ def get_props(self):
+ return self.props
+
+ def get_target(self):
+ return self.target
+
+ def add_result(self, result):
+ self.condition.acquire()
+ self.result = result
+ self.condition.notify()
+ self.condition.release()
+
+ def get_result(self):
+ self.condition.acquire()
+ self.condition.wait()
+ self.condition.release()
+ return self.result
+
+ def release(self):
+ self.condition.notify()
+ self.condition.release()
+
+ def is_startup_activation(self):
+ return self.startup_activation
+
+ def serialize(self):
+ try:
+ data = {}
+ data['operation'] = self.operation
+ data['csn'] = self.csn
+ data['properties'] = self.props
+ data['result'] = self.result
+ data['startup_activation'] = self.startup_activation
+ return json.dumps(data)
+ except Exception as exp:
+ raise cmerror.CMError(str(exp))
+
+ def deserialize(self, msg):
+ try:
+ data = json.loads(msg)
+ self.operation = data['operation']
+ self.csn = data['csn']
+ self.props = data['properties']
+ self.result = data['result']
+ self.startup_activation = data['startup_activation']
+ except Exception as exp:
+ raise cmerror.CMError(str(exp))
+
+
+if __name__ == '__main__':
+ work = CMActivationWork(CMActivationWork.OPER_FULL, 10, {})
+ print 'Work is %s' % work
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import logging
+
+from cmframework.utils.cmalarmhandler import AlarmHandler
+
+
+class CMAlarm(object):
+ NODE_REBOOT_REQUEST_ALARM = '45001'
+ ACTIVATION_FAILED_ALARM = '45002'
+
+ def _get_alarm_id(self):
+ raise NotImplementedError('Not implemented alarm')
+
+ def raise_alarm_for_node(self, node_name, supplementary_info=None):
+ logging.debug('raise_alarm_for_node called with %s %s', node_name, supplementary_info)
+
+ self._raise_alarm_with_dn('NODE-{}'.format(node_name), supplementary_info)
+
+ def raise_alarm_for_sg(self, sg_name, supplementary_info=None):
+ logging.debug('raise_alarm_for_sg called with %s %s', sg_name, supplementary_info)
+
+ self._raise_alarm_with_dn('SG-{}'.format(sg_name), supplementary_info)
+
+ def _raise_alarm_with_dn(self, dn, supplementary_info=None):
+ logging.debug('raise_alarm called for %s with %s %s',
+ self._get_alarm_id(),
+ dn,
+ supplementary_info)
+
+ if not supplementary_info:
+ supplementary_info = {}
+
+ try:
+ alarm_handler = AlarmHandler()
+
+ alarm_handler.raise_alarm_with_dn(self._get_alarm_id(),
+ dn,
+ supplementary_info)
+ except Exception as ex: # pylint: disable=broad-except
+ logging.warning('Alarm raising failed: %s', str(ex))
+
+ def cancel_alarm_for_node(self, node_name, supplementary_info=None):
+ logging.debug('cancel_alarm called with %s %s', node_name, supplementary_info)
+
+ self._cancel_alarm_with_dn('NODE-{}'.format(node_name), supplementary_info)
+
+ def cancel_alarm_for_sg(self, sg_name, supplementary_info=None):
+ logging.debug('cancel_alarm called with %s %s', sg_name, supplementary_info)
+
+ self._cancel_alarm_with_dn('SG-{}'.format(sg_name), supplementary_info)
+
+ def _cancel_alarm_with_dn(self, dn, supplementary_info=None):
+ logging.debug('cancel_alarm called for %s with %s %s',
+ self._get_alarm_id(),
+ dn,
+ supplementary_info)
+
+ if not supplementary_info:
+ supplementary_info = {}
+
+ try:
+ alarm_handler = AlarmHandler()
+
+ alarm_handler.cancel_alarm_with_dn(self._get_alarm_id(),
+ dn,
+ supplementary_info)
+ except Exception as ex: # pylint: disable=broad-except
+ logging.warning('Alarm canceling failed: %s', str(ex))
+
+
+class CMRebootRequestAlarm(CMAlarm):
+ def _get_alarm_id(self):
+ return CMAlarm.NODE_REBOOT_REQUEST_ALARM
+
+
+class CMActivationFailedAlarm(CMAlarm):
+ def _get_alarm_id(self):
+ return CMAlarm.ACTIVATION_FAILED_ALARM
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import logging
+from Queue import Queue
+from threading import Thread
+
+from cmframework.server.cmsingleton import CMSingleton
+from cmframework.utils.cmalarmwork import CMAlarmWork
+from cmframework.apis import cmerror
+
+
+class AlarmHandler(Thread, CMSingleton):
+ def __init__(self):
+ super(AlarmHandler, self).__init__()
+
+ self.handler_lib = None
+ self.works = Queue()
+
+ self.daemon = True
+
+ def set_library_impl(self, handler_lib_impl_module, **kw):
+ try:
+ logging.debug('Loading alarmhandler lib from %s', handler_lib_impl_module)
+ # Separate class path and module name
+ parts = handler_lib_impl_module.rsplit('.', 1)
+ module_path = parts[0]
+ class_name = parts[1]
+ logging.debug('Importing %s from %s', class_name, module_path)
+ module = __import__(module_path, fromlist=[module_path])
+ classobj = getattr(module, class_name)
+ logging.debug('Constructing alarm handler lib with args %r', kw)
+ self.handler_lib = classobj(**kw)
+ except ImportError as exp1:
+ raise cmerror.CMError(str(exp1))
+ except Exception as exp2: # pylint: disable=broad-except
+ raise cmerror.CMError(str(exp2))
+
+ def cancel_alarm_with_dn(self,
+ alarm_id,
+ dn,
+ supplementary_info):
+ logging.debug('AlarmHandler.cancel_alarm_with_dn called')
+
+ alarmwork = CMAlarmWork(CMAlarmWork.OPER_CANCEL, alarm_id, dn, supplementary_info)
+
+ self._add_work(alarmwork)
+
+ def raise_alarm_with_dn(self,
+ alarm_id,
+ dn,
+ supplementary_info):
+ logging.debug('AlarmHandler.raise_alarm_with_dn called')
+
+ alarmwork = CMAlarmWork(CMAlarmWork.OPER_RAISE, alarm_id, dn, supplementary_info)
+
+ self._add_work(alarmwork)
+
+ def _get_work(self):
+ return self.works.get()
+
+ def _add_work(self, work):
+ logging.debug('AlarmHandler._add_work called with %s', work)
+
+ self.works.put(work)
+
+ def run(self):
+ while True:
+ work = self._get_work()
+ if self.handler_lib:
+ logging.debug('Asking handler lib to handle work: %s', work)
+ self.handler_lib.handle_alarm_work(work)
+ logging.debug('Handler lib handled work: %s', work)
+ else:
+ logging.warning('No handler lib set to handle work')
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+class CMAlarmWork(object):
+ OPER_RAISE = 1
+ OPER_CANCEL = 2
+
+ OPER_NAMES = ('NONE', 'RAISE', 'CANCEL')
+
+ def __init__(self, operation, alarm_id, dn, supplementary_info):
+ self.operation = operation
+ self.alarm_id = alarm_id
+ self.dn = dn
+ self.supplementary_info = supplementary_info
+
+ def __str__(self):
+ return '(%r %r %r %r)' % (self._get_operation_name(),
+ self.alarm_id,
+ self.dn,
+ self.supplementary_info)
+
+ def _get_operation_name(self):
+ return CMAlarmWork.OPER_NAMES[self.operation]
+
+ def get_operation(self):
+ return self.operation
+
+ def get_alarm_id(self):
+ return self.alarm_id
+
+ def get_dn(self):
+ return self.dn
+
+ def get_supplementary_info(self):
+ return self.supplementary_info
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from __future__ import print_function
+import json
+import os
+
+from cmframework.apis import cmerror
+from cmframework.utils.cmpluginloader import CMPluginLoader
+from cmdatahandlers.api import configmanager
+from cmdatahandlers.api import utils
+from serviceprofiles import profiles
+
+
+class AnsibleInventory(object):
+
+ def __init__(self, properties, plugin_path):
+ self.pluginloader = AnsibleInventoryPluginLoader(plugin_path)
+ self.props = properties
+ propsjson = {}
+ for name, value in properties.iteritems():
+ try:
+ propsjson[name] = json.loads(value)
+ except Exception: # pylint: disable=broad-except
+ continue
+ self.confman = configmanager.ConfigManager(propsjson)
+
+ # pylint: disable=no-self-use
+ def _is_setup(self):
+ if 'CONFIG_PHASE' in os.environ and os.environ['CONFIG_PHASE'] == 'setup':
+ return True
+ return False
+
+ def _is_bootstrapping(self):
+ if 'CONFIG_PHASE' in os.environ and os.environ['CONFIG_PHASE'] == 'bootstrapping':
+ return True
+ return False
+
+ def _is_provisioning(self):
+ if 'CONFIG_PHASE' in os.environ and os.environ['CONFIG_PHASE'] == 'provisioning':
+ return True
+ return False
+
+ def _is_postconfig(self):
+ if not self._is_bootstrapping() and not self._is_provisioning():
+ return True
+ return False
+
+ def _get_own_host(self):
+
+ hostsconf = self.confman.get_hosts_config_handler()
+
+ if utils.is_virtualized():
+ return hostsconf.get_installation_host()
+
+ hwmgmtip = utils.get_own_hwmgmt_ip()
+
+ return hostsconf.get_host_having_hwmgmt_address(hwmgmtip)
+
+ def set_default_route(self, hostvars, node, infra_internal_name):
+ routes = hostvars[node]['networking'][infra_internal_name].get('routes', [])
+ infra_int_ip = hostvars[node]['networking'][infra_internal_name]['ip']
+ caasconf = self.confman.get_caas_config_handler()
+ cidr_to_set = caasconf.get_caas_parameter("service_cluster_ip_cidr")
+ routes.append({"to": cidr_to_set, "via": infra_int_ip})
+ hostvars[node]['networking'][infra_internal_name]['routes'] = routes
+
+ def set_common_caas(self, hostvars, node, hostsconf):
+ hostvars[node]['nodetype'] = hostsconf.get_nodetype(node)
+ hostvars[node]['nodeindex'] = hostsconf.get_nodeindex(node)
+ hostvars[node]['nodename'] = hostsconf.get_nodename(node)
+
+ host_labels = hostsconf.get_labels(node)
+ if host_labels:
+ hostvars[node]['labels'] = host_labels
+
+ hostvars[node]['ssl_alt_name'] = {}
+ dns = [node]
+ hostvars[node]['ssl_alt_name']['dns'] = dns
+ ips = ['127.0.0.1']
+ ips.append(hostvars[node]['ansible_host'])
+ hostvars[node]['ssl_alt_name']['ip'] = ips
+
+ def set_caas_master_data(self, hostvars, node, caasconf, hostsconf):
+ dns = hostvars[node]['ssl_alt_name']['dns']
+ dns.append('kubernetes.default.svc.nokia.net')
+ dns.append(caasconf.get_apiserver_in_hosts())
+ dns.append(caasconf.get_registry_url())
+ dns.append(caasconf.get_update_registry_url())
+ dns.append(caasconf.get_swift_url())
+ dns.append(caasconf.get_swift_update_url())
+ dns.append(caasconf.get_ldap_master_url())
+ dns.append(caasconf.get_ldap_slave_url())
+ dns.append(caasconf.get_chart_repo_url())
+ dns.append(caasconf.get_caas_parameter('prometheus_url'))
+ dns.append(caasconf.get_tiller_url())
+
+ hosts = hostsconf.get_hosts()
+ for host in hosts:
+ if 'caas_master' in hostsconf.get_service_profiles(host):
+ dns.append(host)
+
+ hostvars[node]['ssl_alt_name']['dns'] = dns
+ ips = hostvars[node]['ssl_alt_name']['ip']
+ ips.append(caasconf.get_apiserver_svc_ip())
+ hostvars[node]['ssl_alt_name']['ip'] = ips
+
+ def generate_inventory(self):
+ try:
+ inventory = {}
+
+ # convert properties to inventory using the following rules:
+ # 1. cloud scoped configuration is mapped to "all" group's "vars" section
+ # 2. The host level domain configuration will be mapped to "_meta" section
+ # under "hostvars" group. Under this there will be a dictionary per host.
+ # 3. The mapping between hosts and profiles is created.
+ # This is used to allow ansible to automatically identify in which hosts
+ # the playbooks are to be run.
+ # This mapping is done as follows:
+ # - A mapping is created for each service profile.
+ # - A mapping is created for the network_profiles type.
+ # - A mapping is created for the storage_profiles type.
+ # - A mapping is created for the performance_profiles type.
+
+ # Get the host variables and all variables
+ hostvars = {}
+ allvars = {}
+
+ netconf = self.confman.get_networking_config_handler()
+ hostsconf = self.confman.get_hosts_config_handler()
+ infra_internal_name = netconf.get_infra_internal_network_name()
+ hosts = hostsconf.get_hosts()
+
+ ownhost = self._get_own_host()
+ if self._is_bootstrapping():
+ for host in hosts:
+ if host != ownhost:
+ hostsconf.disable_host(host)
+
+ hosts = hostsconf.get_enabled_hosts()
+ for name, value in self.props.iteritems():
+ try:
+ d = name.split('.')
+ if len(d) != 2:
+ continue
+ node = d[0]
+ domain = d[1]
+ if node != 'cloud':
+ if node in hosts:
+ if node not in hostvars:
+ hostvars[node] = {}
+ hostip = netconf.get_host_ip(node, infra_internal_name)
+ hostvars[node]['ansible_host'] = hostip
+
+ try:
+ hostvars[node][domain] = json.loads(value)
+ except Exception: # pylint: disable=broad-except
+ hostvars[node][domain] = value
+
+ if 'caas_master' in hostsconf.get_service_profiles(node):
+ self.set_common_caas(hostvars, node, hostsconf)
+ caasconf = self.confman.get_caas_config_handler()
+ self.set_caas_master_data(hostvars, node, caasconf, hostsconf)
+ self.set_default_route(hostvars, node, infra_internal_name)
+
+ if 'caas_worker' in hostsconf.get_service_profiles(node):
+ self.set_common_caas(hostvars, node, hostsconf)
+ self.set_default_route(hostvars, node, infra_internal_name)
+ else:
+ try:
+ allvars[domain] = json.loads(value)
+ except Exception: # pylint: disable=broad-except
+ allvars[domain] = value
+
+ except Exception: # pylint: disable=broad-except
+ pass
+
+ inventory['_meta'] = {}
+ inventory['_meta']['hostvars'] = hostvars
+ inventory['all'] = {'vars': allvars}
+
+ # add hosts to service profiles mapping
+ serviceprofiles = profiles.Profiles().get_service_profiles()
+ for profile in serviceprofiles:
+ try:
+ servicehosts = hostsconf.get_service_profile_hosts(profile)
+ tmp = []
+ for host in servicehosts:
+ if host in hosts:
+ tmp.append(host)
+ inventory[profile] = tmp
+ except Exception: # pylint: disable=broad-except
+ continue
+
+ # add mapping between profile types and hosts
+ inventory['network_profiles'] = []
+ inventory['storage_profiles'] = []
+ inventory['performance_profiles'] = []
+ for host in hosts:
+ # check for network profiles
+ try:
+ _ = hostsconf.get_network_profiles(host)
+ inventory['network_profiles'].append(host)
+ except Exception: # pylint: disable=broad-except
+ pass
+
+ # check for storage profiles
+ try:
+ _ = hostsconf.get_storage_profiles(host)
+ inventory['storage_profiles'].append(host)
+ except Exception: # pylint: disable=broad-except
+ pass
+
+ # check for perfromance profiles
+ try:
+ _ = hostsconf.get_performance_profiles(host)
+ inventory['performance_profiles'].append(host)
+ except Exception: # pylint: disable=broad-except
+ pass
+
+ self.pluginloader.load()
+ plugins = self.pluginloader.get_plugin_instances(self.confman, inventory, ownhost)
+ if self._is_setup():
+ inventory.clear()
+ for name, plugin in plugins.iteritems():
+ if self._is_bootstrapping():
+ plugin.handle_bootstrapping()
+ elif self._is_provisioning():
+ plugin.handle_provisioning()
+ elif self._is_setup():
+ plugin.handle_setup()
+ else:
+ plugin.handle_postconfig()
+
+ return inventory
+
+ except Exception as exp: # pylint: disable=broad-except
+ raise cmerror.CMError(str(exp))
+
+
+class AnsibleInventoryPluginLoader(CMPluginLoader):
+ def __init__(self, plugin_location, plugin_filter=None):
+ super(AnsibleInventoryPluginLoader, self).__init__(plugin_location, plugin_filter)
+
+ def build_filter_dict(self):
+ pass
+
+ def get_plugin_instances(self, confman, inventory, ownhost):
+ plugs = {}
+ for plugin, module in self.loaded_plugin.iteritems():
+ class_name = getattr(module, plugin)
+ instance = class_name(confman, inventory, ownhost)
+ plugs[plugin] = instance
+ return plugs
+
+
+def main():
+ import argparse
+ import sys
+ import traceback
+
+ parser = argparse.ArgumentParser(description='Test ansible inventory handler', prog=sys.argv[0])
+
+ parser.add_argument('--properties',
+ required=True,
+ dest='properties',
+ metavar='PROPERTIES',
+ help='The file containing the properties',
+ type=str,
+ action='store')
+
+ parser.add_argument('--plugins',
+ required=True,
+ dest='plugins',
+ metavar='PLUGINS',
+ help='The path to ansible inventory plugin(s)',
+ type=str,
+ action='store')
+
+ try:
+ args = parser.parse_args(sys.argv[1:])
+
+ f = open(args.properties, 'r')
+ lines = f.read().splitlines()
+ f.close()
+ properties = {}
+ for line in lines:
+ d = line.split('=')
+ if len(d) != 2:
+ continue
+ properties[d[0]] = d[1]
+ ansible = AnsibleInventory(properties, args.plugins)
+ inventory = ansible.generate_inventory()
+
+ print(json.dumps(inventory, indent=4, sort_keys=True))
+
+ except Exception as exp: # pylint: disable=broad-except
+ print(str(exp))
+ traceback.print_exc()
+ sys.exit(1)
+ sys.exit(0)
+
+
+if __name__ == '__main__':
+ main()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import os
+from cmframework.utils import cmtopologicalsort
+
+
+class AnsiblePlaybooks(object):
+ bootstrapping_playbook = 'bootstrapping-playbook.yml'
+ provisioning_playbook = 'provisioning-playbook.yml'
+ postconfig_playbook = 'postconfig-playbook.yml'
+ finalize_playbook = 'finalize-playbook.yml'
+
+ def __init__(self, dest, bootstrapping_path, provisioning_path, postconfig_path, finalize_path):
+ self.dest = dest
+ self.bootstrapping_path = bootstrapping_path
+ self.provisioning_path = provisioning_path
+ self.postconfig_path = postconfig_path
+ self.finalize_path = finalize_path
+
+ def generate_playbooks(self):
+ self._generate_playbook(self.bootstrapping_path, self.bootstrapping_playbook)
+ self._generate_playbook(self.provisioning_path, self.provisioning_playbook)
+ self._generate_playbook(self.postconfig_path, self.postconfig_playbook)
+ self._generate_playbook(self.finalize_path, self.finalize_playbook)
+
+ def _generate_playbook(self, directory, name):
+ graph = self._get_dependency_graph(directory)
+ topsort = cmtopologicalsort.TopSort(graph)
+ sortedlists = topsort.sort()
+ with open(self.dest + '/' + name, 'w') as f:
+ for entry in sortedlists:
+ for e in entry:
+ fullpath = directory + '/' + e
+ if os.path.exists(fullpath):
+ f.write('- import_playbook: ' + e + '\n')
+
+ def _get_dependency_graph(self, directory):
+ entries = os.listdir(directory)
+ graph = {}
+ for entry in entries:
+ entryfull = directory + '/' + entry
+ if os.path.isfile(entryfull) or os.path.islink(entryfull):
+ requires = self._get_required_playbooks(entryfull)
+ graph[entry] = requires
+ return graph
+
+ @staticmethod
+ def _get_required_playbooks(playbook):
+ requires = []
+ with open(playbook) as f:
+ lines = f.read().splitlines()
+ # parse the lines containing:
+ # cmframework.requires: <comma separated list of playbooks>
+ for line in lines:
+ if 'cmframework.requires:' in line:
+ data = line.split(':')
+ if len(data) != 2:
+ continue
+ tmp = data[1].replace(" ", "")
+ requires = tmp.split(',')
+ break
+ return requires
+
+
+if __name__ == '__main__':
+ playbooks = AnsiblePlaybooks('/tmp/', '/etc/lcm/playbooks/installation/bootstrapping',
+ '/etc/lcm/playbooks/installation/provisioning',
+ '/etc/lcm/playbooks/installation/postconfig',
+ '/etc/lcm/playbooks/installation/finalize')
+ playbooks.generate_playbooks()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from __future__ import print_function
+import logging
+from cmframework.apis import cmerror
+
+
+class CMBackendHandler(object):
+ def __init__(self, plugin_path, **kw):
+ try:
+ logging.debug('Loading backend plugin from %s', plugin_path)
+ # Separate class path and module name
+ parts = plugin_path.rsplit('.', 1)
+ self.plugin_path = parts[0]
+ class_name = parts[1]
+ logging.debug('Importing %s from %s', class_name, self.plugin_path)
+ module = __import__(self.plugin_path, fromlist=[self.plugin_path])
+ classobj = getattr(module, class_name)
+ logging.debug('Constructing backend handler with args %r', kw)
+ self.plugin = classobj(**kw)
+ except ImportError as exp1:
+ raise cmerror.CMError(str(exp1))
+ except Exception as exp2:
+ raise cmerror.CMError(str(exp2))
+
+ def get_property(self, prop_name):
+ logging.debug('get_property called for %s', prop_name)
+ return self.plugin.get_property(prop_name)
+
+ def get_properties(self, prop_filter):
+ logging.debug('get_properties called with filter %s', prop_filter)
+ return self.plugin.get_properties(prop_filter)
+
+ def set_property(self, prop_name, prop_value):
+ logging.debug('set_property called for setting %s=%s', prop_name, prop_value)
+ return self.plugin.set_property(prop_name, prop_value)
+
+ def set_properties(self, props):
+ logging.debug('set_properties called for properties %s', str(props))
+ return self.plugin.set_properties(props)
+
+ def delete_property(self, prop_name):
+ logging.debug('delete_property called for %s', prop_name)
+ return self.plugin.delete_property(prop_name)
+
+ def delete_properties(self, prop_filter):
+ logging.debug('delete_properties called with filter %s', prop_filter)
+ return self.plugin.delete_properties(prop_filter)
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from cmframework.apis import cmpluginclient
+from cmframework.utils import cmbackendhandler
+
+
+class CMBackendPluginClient(cmpluginclient.CMPluginClient):
+ def __init__(self, plugin_path, **args):
+ self.backend = cmbackendhandler.CMBackendHandler(plugin_path, **args)
+
+ def get_property(self, prop_name):
+ return self.backend.get_property(prop_name)
+
+ def get_properties(self, prop_filter):
+ return self.backend.get_properties(prop_filter)
+
+ def set_property(self, name, value):
+ return self.backend.set_property(name, value)
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from cmframework.apis.cmerror import CMError
+
+
+class CMDependencySort(object):
+ def __init__(self, after=None, before=None):
+ if not after:
+ after = {}
+ if not before:
+ before = {}
+
+ self._entries = set()
+ self._before = before
+
+ self._convert_after_to_before(after)
+ self._find_all_entries()
+
+ self._sorted_entries = []
+
+ def _convert_after_to_before(self, after):
+ for entry, deps in after.iteritems():
+ self._entries.add(entry)
+ for dep in deps:
+ dep_before = self._before.get(dep, None)
+ if not dep_before:
+ dep_before = []
+ self._before[dep] = dep_before
+ if entry not in dep_before:
+ dep_before.append(entry)
+
+ def _find_all_entries(self):
+ for entry, deps in self._before.iteritems():
+ self._entries.add(entry)
+ for dep in deps:
+ self._entries.add(dep)
+
+ def sort(self):
+ self._sort_entries()
+ return self._sorted_entries
+
+ def _sort_entries(self):
+ sorted_list = []
+ permanent_mark_list = []
+ for entry in self._entries:
+ if entry not in permanent_mark_list:
+ self._visit(entry, sorted_list, permanent_mark_list)
+ self._sorted_entries = sorted_list
+
+ def _visit(self, entry, sorted_list, permanent_mark_list, temporary_mark_list=None):
+ if not temporary_mark_list:
+ temporary_mark_list = []
+
+ if entry in permanent_mark_list:
+ return
+
+ if entry in temporary_mark_list:
+ raise CMError('Cycle detected in dependencies ({})'.format(entry))
+
+ temporary_mark_list.append(entry)
+ for dep in self._before.get(entry, []):
+ self._visit(dep, sorted_list, permanent_mark_list, temporary_mark_list)
+ permanent_mark_list.append(entry)
+ sorted_list.insert(0, entry)
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from __future__ import print_function
+import logging
+from urlparse import urlparse
+
+from cmframework.apis import cmerror
+from cmframework.apis import cmstate
+from dss.client import dss_client
+from dss.api import dss_error
+
+
+class CMDSSHandler(cmstate.CMState):
+ def __init__(self, **kw):
+ uridata = urlparse(kw['uri'])
+ socket = uridata.path
+ self.client = dss_client.Client(socket)
+
+ def _domain_exists(self, domain):
+ return domain in self.get_domains()
+
+ def _name_exists_in_domain(self, domain, name):
+ return name in self.get_domain(domain)
+
+ def get(self, domain, name):
+ logging.debug('get called for %s %s', domain, name)
+
+ if self._domain_exists(domain):
+ try:
+ domain_data = self.client.get_domain(domain)
+ return domain_data.get(name, None)
+ except dss_error.Error as ex:
+ raise cmerror.CMError(str(ex))
+
+ return None
+
+ def get_domain(self, domain):
+ logging.debug('get_domain called for %s', domain)
+
+ if self._domain_exists(domain):
+ try:
+ return self.client.get_domain(domain)
+ except dss_error.Error as ex:
+ raise cmerror.CMError(str(ex))
+
+ return None
+
+ def set(self, domain, name, value):
+ logging.debug('set called for setting %s %s=%s', domain, name, value)
+ try:
+ return self.client.set(domain, name, value)
+ except dss_error.Error as ex:
+ raise cmerror.CMError(str(ex))
+
+ def get_domains(self):
+ logging.debug('get_domains called')
+ try:
+ return self.client.get_domains()
+ except dss_error.Error as ex:
+ raise cmerror.CMError(str(ex))
+
+ def delete(self, domain, name):
+ logging.debug('delete called for %s %s', domain, name)
+
+ if self._domain_exists(domain):
+ if self._name_exists_in_domain(domain, name):
+ try:
+ self.client.delete(domain, name)
+ except dss_error.Error as ex:
+ raise cmerror.CMError(str(ex))
+
+ def delete_domain(self, domain):
+ logging.debug('delete_domain called for %s', domain)
+
+ if self._domain_exists(domain):
+ try:
+ self.client.delete_domain(domain)
+ except dss_error.Error as ex:
+ raise cmerror.CMError(str(ex))
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from __future__ import print_function
+import logging
+import os.path
+import os
+import stat
+
+from cmframework.apis import cmerror
+
+
+class CMFlagFile(object):
+ CM_FLAGFILE_DIR = '/mnt/config-manager'
+
+ def __init__(self, name):
+ logging.debug('CMFlagFile constructor called, name=%s', name)
+
+ self._name = '{}/{}'.format(CMFlagFile.CM_FLAGFILE_DIR, name)
+
+ def __nonzero__(self):
+ return self.is_set()
+
+ def is_set(self):
+ logging.debug('is_set called')
+
+ return os.path.exists(self._name)
+
+ def set(self):
+ logging.debug('set called')
+
+ if not self.is_set():
+ try:
+ with open(self._name, 'w') as f:
+ os.chmod(self._name, stat.S_IRUSR | stat.S_IWUSR)
+ f.write('')
+ f.flush()
+ os.fsync(f.fileno())
+ except IOError as exp:
+ raise cmerror.CMError(str(exp))
+
+ def unset(self):
+ logging.debug('unset called')
+
+ if self.is_set():
+ try:
+ os.remove(self._name)
+ except IOError as exp:
+ raise cmerror.CMError(str(exp))
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import re
+import sys
+import logging
+from logging.handlers import SysLogHandler
+
+from cmframework.apis import cmerror
+
+
+class CMMaskFormatter(object):
+ def __init__(self, orig_formatter, patterns):
+ self.orig_formatter = orig_formatter
+ self.patterns = patterns
+
+ def format(self, record):
+ msg = self.orig_formatter.format(record)
+ regex = r'([\\]*\"{}[\\]*\":\s[\\]*\")[^\"\\\s]+([\\]*\")'
+ for pattern in self.patterns:
+
+ msg = re.sub(regex.format(pattern),
+ r'\1*** password ***\2', msg)
+
+ return msg
+
+ def __getattr__(self, attr):
+ return getattr(self.orig_formatter, attr)
+
+
+class CMLogger(object):
+ levels = {'debug': logging.DEBUG,
+ 'info': logging.INFO,
+ 'warning': logging.WARNING,
+ 'error': logging.error}
+
+ DEST_CONSOLE = 1
+ DEST_SYSLOG = 2
+ dests = {'console': DEST_CONSOLE,
+ 'syslog': DEST_SYSLOG}
+
+ def __init__(self, dest, verbose, level, mask_names=None):
+ self.verbose = verbose
+ self.dest = dest
+ self.level = level
+ self.formatter = None
+ self.handler = None
+ self.mask_formatter = None
+ if mask_names is None:
+ mask_names = []
+ self.init(mask_names)
+
+ def init(self, mask_names):
+ if self.level not in CMLogger.levels.values():
+ raise cmerror.CMError('Invalid level value, possible values are %s' %
+ str(CMLogger.levels))
+
+ if self.dest not in CMLogger.dests.values():
+ raise cmerror.CMError('Invalid destination value, possible values are %s' %
+ str(CMLogger.dests))
+
+ if self.verbose:
+ if self.dest is CMLogger.DEST_CONSOLE:
+ format_str = '[%(asctime)s %(levelname)7s %(module)s(%(lineno)3s)] %(message)s'
+ else:
+ format_str = '[%(module)s(%(lineno)3s)] %(message)s'
+ else:
+ format_str = '%(message)s'
+
+ self.formatter = logging.Formatter(format_str)
+ self.mask_formatter = CMMaskFormatter(self.formatter, mask_names)
+ self.set_dest(self.dest)
+
+ logging.getLogger().setLevel(self.level)
+
+ def set_level(self, level):
+ self.level = level
+ logging.getLogger().setLevel(self.level)
+
+ def set_dest(self, dest):
+ if self.dest != dest or self.handler is None:
+ if self.handler:
+ logging.getLogger().removeHandler(self.handler)
+
+ if self.dest is CMLogger.DEST_CONSOLE:
+ self.handler = logging.StreamHandler(sys.stdout)
+ self.handler.setFormatter(self.mask_formatter)
+ elif self.dest is CMLogger.DEST_SYSLOG:
+ print '====> setting destination to syslog'
+ self.handler = SysLogHandler(address='/dev/log')
+ self.handler.setFormatter(self.mask_formatter)
+ logging.getLogger().addHandler(self.handler)
+
+ @staticmethod
+ def str_to_level(level):
+ ret = None
+ try:
+ ret = CMLogger.levels[level]
+ except KeyError:
+ raise cmerror.CMError('Invalid log level, possible values %s' %
+ str(CMLogger.levels.keys()))
+ return ret
+
+ @staticmethod
+ def str_to_dest(dest):
+ ret = None
+ try:
+ ret = CMLogger.dests[dest]
+ except KeyError:
+ raise cmerror.CMError('Invalid destination, possible values %s' %
+ str(CMLogger.dests.keys()))
+ return ret
+
+ @staticmethod
+ def level_to_str(level):
+ for key, value in CMLogger.levels.iteritems():
+ if value is level:
+ return key
+ return None
+
+ @staticmethod
+ def dest_to_str(dest):
+ for key, value in CMLogger.dests.iteritems():
+ if value is dest:
+ return key
+ return None
+
+
+def main():
+ log_dest = CMLogger.str_to_dest('console')
+ log_level = CMLogger.str_to_level('debug')
+ _ = CMLogger(log_dest, True, log_level)
+ world = 'world'
+ logging.error('hello %s!', world)
+ logging.warn('hello %s!', world)
+ logging.info('hello %s!', world)
+ logging.debug('hello %s!', world)
+
+
+if __name__ == '__main__':
+ main()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# vim: tabstop=8 expandtab shiftwidth=4 softtabstop=4
+import os
+import imp
+import sys
+import logging
+
+from cmframework.apis import cmerror
+
+
+class CMPluginLoader(object):
+ class LoadingFilter(object):
+ # pylint: disable=no-self-use, unused-argument
+ def is_supported(self, plugin):
+ return True
+
+ def __init__(self, plugin_location, plugin_filter=None):
+ self.location = plugin_location
+ sys.path.append(self.location)
+ self.pluginslist = []
+ self.loaded_plugin = {}
+ self.filterlist = {}
+ self.plugin_filter = plugin_filter
+
+ def find_plugin(self):
+ logging.debug('finding plugins in %s', self.location)
+ listofplugin = os.listdir(self.location)
+ for plugin in listofplugin:
+ if plugin.endswith('.py'):
+ logging.debug('Adding plugin %s', plugin)
+ self.pluginslist.append(plugin.replace(".py", ""))
+
+ def sort_plugin(self):
+ logging.debug('Sorting plugins')
+ self.pluginslist.sort()
+
+ def load_plugin(self):
+ for plugin in self.pluginslist:
+ logging.debug('Loading plugin %s', plugin)
+ fp, pathname, description = imp.find_module(plugin)
+ try:
+ pluginmodule = imp.load_module(plugin, fp, pathname, description)
+ add_plugin = True
+ if self.plugin_filter:
+ class_name = getattr(pluginmodule, plugin)
+ instance = class_name()
+ add_plugin = self.plugin_filter.is_supported(instance)
+ if add_plugin:
+ logging.info('Adding plugin %s to list', plugin)
+ self.loaded_plugin[plugin] = pluginmodule
+ else:
+ logging.info(
+ 'Skipping plugin %s as it does not match configured filter', plugin)
+ except Exception as exp: # pylint: disable=broad-except
+ logging.error('Failed to load plugin %s, got exp %s', plugin, str(exp))
+ raise cmerror.CMError('Loading %s plugin failed' % plugin)
+ finally:
+ if fp:
+ fp.close()
+
+ def build_filter_dict(self):
+ faulty_plugins = []
+ for plugin, objectname in self.loaded_plugin.iteritems():
+ logging.debug('Getting the subsription info from %s %s', plugin, objectname)
+ try:
+ class_name = getattr(objectname, plugin)
+ instance = class_name()
+ filtername = instance.get_subscription_info()
+ self.filterlist[plugin] = filtername
+ except Exception as exp: # pylint: disable=broad-except
+ logging.error('Getting subscription failed for %s %s, got exp %s',
+ plugin, objectname, str(exp))
+ faulty_plugins.append(plugin)
+
+ for plugin in faulty_plugins:
+ del self.loaded_plugin[plugin]
+
+ # pylint: disable=no-self-use
+ def validate_plugin(self):
+ pass
+
+ def load(self):
+ self.find_plugin()
+ self.sort_plugin()
+ self.load_plugin()
+ self.build_filter_dict()
+ return self.loaded_plugin, self.filterlist
+
+
+def main():
+ pl = CMPluginLoader("plug_in/tst")
+ a, b = pl.load()
+ print a
+ print b
+
+
+if __name__ == '__main__':
+ main()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# vim: tabstop=8 expandtab shiftwidth=4 softtabstop=4
+import re
+import logging
+
+from cmframework.apis import cmerror
+
+
+class CMPluginManager(object):
+
+ def __init__(self, plugins_path):
+ self.pluginlist = {}
+ self.filterdict = {}
+ self.plugins_path = plugins_path
+
+ # pylint: disable=no-self-use
+ def load_plugin(self):
+ raise cmerror.CMError('Not implemented')
+
+ # pylint: disable=no-self-use
+ def build_input(self, indata, filtername):
+ search_re = re.compile(filtername)
+ if isinstance(indata, dict):
+ filter_data = {}
+ for key, value in indata.iteritems():
+ logging.debug('Matching %s against %s', key, filtername)
+ if search_re.match(key):
+ filter_data[key] = value
+ else:
+ filter_data = []
+ for key in indata:
+ logging.debug('Matching %s against %s', key, filtername)
+ if search_re.match(key):
+ filter_data.append(key)
+
+ return filter_data
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from __future__ import print_function
+import logging
+import json
+
+from cmframework.utils.cmstatehandler import CMStateHandler
+
+
+class CMSnapshotHandler(CMStateHandler):
+ SNAPSHOTS_DOMAIN = 'cm.snapshots'
+
+ def get_data(self, snapshot_name):
+ logging.debug('get_data called for: %s', snapshot_name)
+
+ data_json = self.plugin.get(CMSnapshotHandler.SNAPSHOTS_DOMAIN, snapshot_name)
+
+ return json.loads(data_json)
+
+ def set_data(self, snapshot_name, data):
+ logging.debug('set_properties called for: %s with: %s', snapshot_name, data)
+
+ self.plugin.set(CMSnapshotHandler.SNAPSHOTS_DOMAIN, snapshot_name, json.dumps(data))
+
+ def snapshot_exists(self, snapshot_name):
+ logging.debug('snapshot_exists called for: %s', snapshot_name)
+
+ return self.plugin.get(CMSnapshotHandler.SNAPSHOTS_DOMAIN, snapshot_name) is not None
+
+ def list_snapshots(self):
+ logging.debug('list_snapshots called')
+
+ snapshots = self.plugin.get_domain(CMSnapshotHandler.SNAPSHOTS_DOMAIN)
+
+ return snapshots.keys()
+
+ def delete_snapshot(self, snapshot_name):
+ logging.debug('delete_snapshot called for: %s', snapshot_name)
+
+ self.plugin.delete(CMSnapshotHandler.SNAPSHOTS_DOMAIN, snapshot_name)
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+class CMStateDummyHandler(object):
+ def __init__(self, **kw):
+ pass
+
+ def get(self, domain, name):
+ return None
+
+ def get_domain(self, domain):
+ return None
+
+ def set(self, domain, name, value):
+ pass
+
+ def get_domains(self):
+ return []
+
+ def delete(self, domain, name):
+ pass
+
+ def delete_domain(self, domain):
+ pass
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from __future__ import print_function
+import logging
+from urlparse import urlparse
+from configparser import ConfigParser
+import os.path
+import os
+import stat
+
+from cmframework.apis import cmerror
+from cmframework.apis import cmstate
+
+
+class CMStateFileHandler(cmstate.CMState):
+ def __init__(self, **kw):
+ uridata = urlparse(kw['uri'])
+ self.path = uridata.path
+ self.configparser = ConfigParser()
+ self._load_config_file()
+
+ def _load_config_file(self):
+ try:
+ if os.path.isfile(self.path):
+ with open(self.path, 'r') as cf:
+ self.configparser.read_file(cf)
+ else:
+ with open(self.path, 'w') as cf:
+ pass
+ except IOError as ex:
+ raise cmerror.CMError(str(ex))
+
+ def _write_config_file(self):
+ try:
+ with open(self.path, 'w') as cf:
+ os.chmod(self.path, stat.S_IRUSR | stat.S_IWUSR)
+ self.configparser.write(cf)
+ except IOError as ex:
+ raise cmerror.CMError(str(ex))
+
+ def get(self, domain, name):
+ logging.debug('get called for %s %s', domain, name)
+
+ if self.configparser.has_section(domain):
+ if self.configparser.has_option(domain, name):
+ return self.configparser.get(domain, name)
+
+ return None
+
+ def get_domain(self, domain):
+ logging.debug('get_domain called for %s', domain)
+
+ if self.configparser.has_section(domain):
+ return self.configparser.items(domain)
+
+ return None
+
+ def set(self, domain, name, value):
+ logging.debug('set called for setting %s %s=%s', domain, name, value)
+
+ if not self.configparser.has_section(domain):
+ self.configparser.add_section(domain)
+
+ self.configparser.set(domain, name, value)
+
+ self._write_config_file()
+
+ def get_domains(self):
+ logging.debug('get_domains called')
+
+ return self.configparser.sections()
+
+ def delete(self, domain, name):
+ logging.debug('delete called for %s %s', domain, name)
+
+ if self.configparser.has_section(domain):
+ self.configparser.remove_option(domain, name)
+
+ self._write_config_file()
+
+ def delete_domain(self, domain):
+ logging.debug('delete_domain called for %s', domain)
+
+ self.configparser.remove_section(domain)
+
+ self._write_config_file()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from __future__ import print_function
+import logging
+from cmframework.apis import cmerror
+
+
+class CMStateHandler(object):
+ def __init__(self, plugin_path, **kw):
+ try:
+ logging.debug('Loading backend plugin from %s', plugin_path)
+ # Separate class path and module name
+ parts = plugin_path.rsplit('.', 1)
+ self.plugin_path = parts[0]
+ class_name = parts[1]
+ logging.debug('Importing %s from %s', class_name, self.plugin_path)
+ module = __import__(self.plugin_path, fromlist=[self.plugin_path])
+ classobj = getattr(module, class_name)
+ logging.debug('Constructing state handler with args %r', kw)
+ self.plugin = classobj(**kw)
+ except ImportError as exp1:
+ raise cmerror.CMError(str(exp1))
+ except Exception as exp2:
+ raise cmerror.CMError(str(exp2))
+
+ def get(self, domain, name):
+ logging.debug('get called for %s %s', domain, name)
+ return self.plugin.get(domain, name)
+
+ def get_domain(self, domain):
+ logging.debug('get_domain called for %s', domain)
+ return self.plugin.get_domain(domain)
+
+ def set(self, domain, name, value):
+ logging.debug('set called for setting %s %s=%s', domain, name, value)
+ return self.plugin.set(domain, name, value)
+
+ def get_domains(self):
+ logging.debug('get_domains called')
+ return self.plugin.get_domains()
+
+ def delete(self, domain, name):
+ logging.debug('delete called for %s %s', domain, name)
+ return self.plugin.delete(domain, name)
+
+ def delete_domain(self, domain):
+ logging.debug('delete_domain called for %s', domain)
+ return self.plugin.delete_domain(domain)
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from __future__ import print_function
+from cmframework.apis import cmerror
+
+
+class TopSort(object):
+ """
+ This class implements a topological sort algorithm.
+
+ It expects a dependency graph as an input and returns a sorted list as an
+ output. The returned list contains lists sorted accordingly to the dependency
+ graph:
+
+ Usage Example:
+ # The following graph indicates that 2 depends on 11, 9 depends on 11,8,10 and so on...
+ graph = {2: [11],
+ 9: [11, 8, 10],
+ 10: [11, 3],
+ 11: [7, 5],
+ 8: [7, 3]}
+
+ sort = TopSort(graph)
+ try:
+ sorted = sort.sort()
+ for entry in sorted:
+ print('%r' % entry)
+ except cmerror.CMError as exp:
+ print('Got exception %s' % str(exp))
+
+ The above example will generate the following output:
+ [3, 5, 7]
+ [8, 11]
+ [2, 10]
+ [9]
+ """
+
+ def __init__(self, graph):
+ self.graph = graph
+ self.output = {}
+ self.recursionlevel = 0
+
+ def _get_dependent_entries(self, entry):
+ result = []
+ for e, deps in self.graph.iteritems():
+ if entry in deps:
+ result.append(e)
+ return result
+
+ def _update_dependent_entries(self, entry):
+ maximumrecursion = (len(self.graph)) * (len(self.graph))
+ if (self.recursionlevel + 1) >= maximumrecursion:
+ raise cmerror.CMError('cyclic dependency detected, graph %r' % self.graph)
+ self.recursionlevel += 1
+ dependententries = self._get_dependent_entries(entry)
+ entrydepth = self.output[entry]
+ for e in dependententries:
+ if e in self.output:
+ if entrydepth >= self.output[e]:
+ self.output[e] = entrydepth + 1
+ self._update_dependent_entries(e)
+
+ def sort(self):
+ for entry, deps in self.graph.iteritems():
+ depth = 0
+ if entry in self.output:
+ depth = self.output[entry]
+
+ # calculate new depth according to dependencies
+ newdepth = depth
+ for dep in deps:
+ if dep in self.output:
+ weight = self.output[dep]
+ else:
+ weight = 0
+ self.output[dep] = 0
+ if weight >= newdepth:
+ newdepth = weight + 1
+
+ # if our depth is changed we need to update the entries depending on us
+ self.output[entry] = newdepth
+ if newdepth > depth and entry in self.output:
+ self.recursionlevel = 0
+ self._update_dependent_entries(entry)
+
+ return self._getsorted()
+
+ def _getsorted(self):
+ import operator
+ sortedoutput = sorted(self.output.items(), key=operator.itemgetter(1))
+ result = {}
+ for entry in sortedoutput:
+ if entry[1] not in result:
+ result[entry[1]] = []
+ result[entry[1]].append(entry[0])
+
+ returned = []
+ for entry, data in result.iteritems():
+ returned.append(data)
+
+ return returned
+
+
+def main():
+ graph1 = {2: [11],
+ 9: [11, 8, 10],
+ 10: [11, 3],
+ 11: [7, 5],
+ 8: [7, 3]}
+
+ graph2 = {'playbook2.yaml': [], 'playbook1.yaml': ['playbook2.yaml']}
+
+ topsort1 = TopSort(graph1)
+ topsort2 = TopSort(graph2)
+
+ try:
+ print(graph1)
+ sortedlists1 = topsort1.sort()
+ for entry in sortedlists1:
+ print('%r' % entry)
+
+ print(graph2)
+ sortedlists2 = topsort2.sort()
+ for entry in sortedlists2:
+ print('%r' % entry)
+ except cmerror.CMError as exp:
+ print(str(exp))
+
+
+if __name__ == '__main__':
+ main()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from __future__ import print_function
+import json
+import yaml
+
+from cmframework.apis import cmerror
+from cmframework.utils.cmpluginloader import CMPluginLoader
+from cmdatahandlers.api import configmanager
+
+
+class UserConfig(object):
+
+ def __init__(self, user_config_file, plugin_path):
+ self.user_config_file = user_config_file
+ self.removed_values = ['version']
+ self.pluginloader = UserConfigPluginLoader(plugin_path)
+
+ def get_flat_config(self):
+ try:
+ result = {}
+ stream = file(self.user_config_file, 'r')
+ tmp = yaml.load(stream)
+ userconfig = {}
+ for key in tmp.keys():
+ userconfig['cloud.' + key] = tmp[key]
+
+ confman = configmanager.ConfigManager(userconfig)
+ self.pluginloader.load()
+ plugins = self.pluginloader.get_plugin_instances()
+ for _, plugin in plugins.iteritems():
+ plugin.handle(confman)
+
+ # change the multi-level deictionary to one level dictionary mapping
+ # the key to a josn string representation of the value.
+ for key, value in userconfig.iteritems():
+ result[key] = json.dumps(value)
+
+ return result
+ except Exception as exp:
+ raise cmerror.CMError("Failed to load user config %(userconfig)s: %(failure)s" %
+ {'userconfig': self.user_config_file,
+ 'failure': str(exp)})
+
+
+class UserConfigPluginLoader(CMPluginLoader):
+ def __init__(self, plugin_location, plugin_filter=None):
+ super(UserConfigPluginLoader, self).__init__(plugin_location, plugin_filter)
+
+ def build_filter_dict(self):
+ pass
+
+ def get_plugin_instances(self):
+ plugs = {}
+ for plugin, module in self.loaded_plugin.iteritems():
+ class_name = getattr(module, plugin)
+ instance = class_name()
+ plugs[plugin] = instance
+ return plugs
+
+
+def main():
+ import argparse
+ import sys
+ import traceback
+
+ parser = argparse.ArgumentParser(description='Test userconfig handler', prog=sys.argv[0])
+
+ parser.add_argument('--config',
+ required=True,
+ dest='config',
+ metavar='CONFIG',
+ help='The user config yaml file',
+ type=str,
+ action='store')
+
+ parser.add_argument('--plugins',
+ required=True,
+ dest='plugins',
+ metavar='PLUGINS',
+ help='The path to userconfig plugin(s)',
+ type=str,
+ action='store')
+
+ try:
+ args = parser.parse_args(sys.argv[1:])
+ config = UserConfig(args.config, args.plugins)
+ data = config.get_flat_config()
+ for key, value in data.iteritems():
+ print('%s=%s' % (key, value))
+
+ except Exception as exp: # pylint: disable=broad-except
+ print("Got exp: %s" % exp)
+ traceback.print_exc()
+ sys.exit(1)
+ sys.exit(1)
+
+
+if __name__ == '__main__':
+ main()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from setuptools import setup, find_packages
+setup(
+ name='cmframework',
+ version='1.0',
+ license='Apache-2.0',
+ long_description='README.txt',
+ author='Baha Mesleh',
+ author_email='baha.mesleh@nokia.com',
+ namespace_packages=['cmframework'],
+ packages=find_packages(),
+ include_package_data=True,
+ package_data={'cmframework.tst': ['*.py', '*.sh']},
+ url='https://gitlab.fp.nsn-rdnet.net/mesleh/cmframework',
+ description='Configuration Management Framework',
+ entry_points={
+ 'console_scripts': [
+ 'cmserver = cmframework.server.cmserver:main',
+ 'cmcli = cmframework.cli.cmcli:main',
+ 'cmagent = cmframework.agent.cmagent:main'
+ ],
+ },
+ zip_safe=False,
+)
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+[Unit]
+Description=cmagent service
+After=network.target
+Requires=network.target
+
+[Service]
+Type=oneshot
+ExecStart=/opt/cmframework/scripts/cmagent
+
+[Install]
+WantedBy=multi-user.target
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+[Unit]
+Description=cmserver service
+After=network.target
+
+[Service]
+Type=simple
+ExecStart=/opt/cmframework/scripts/cmserver
+Restart=always
+
+#[Install]
+#WantedBy=multi-user.target
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import unittest
+import mock
+from cmframework.agent.cmagent import CMAgent
+from cmframework.apis.cmerror import CMError
+
+
+class CMAgentTest(unittest.TestCase):
+ @mock.patch('cmframework.agent.cmagent.cmmanage.CMManage')
+ @mock.patch('cmframework.agent.cmagent.cmlogger.CMLogger')
+ @mock.patch('cmframework.agent.cmagent.socket.gethostbyname')
+ @mock.patch('cmframework.agent.cmagent.CMAgent._reboot_node')
+ @mock.patch('cmframework.agent.cmagent.VerboseLogger')
+ @mock.patch('cmframework.agent.cmagent.logging')
+ def test_activate_default_args_and_reboot(self, mock_logging, mock_verboselogger,
+ mock_reboot_node, mock_socket_get_hostbyname,
+ mock_cmlogger, mock_cmmanage):
+ mock_cmmanage.return_value.activate_node = mock.MagicMock(return_value=True)
+ mock_reboot_node.return_value = 0
+ args = []
+ agent = CMAgent()
+ agent(args)
+
+ # import pdb
+ # pdb.set_trace()
+
+ mock_socket_get_hostbyname.assert_called_once_with('config-manager')
+
+ mock_cmmanage.assert_called_once_with(mock_socket_get_hostbyname.return_value, 61100,
+ 'cmframework.lib.CMClientImpl',
+ mock_verboselogger.return_value)
+ mock_cmmanage.return_value.activate_node.assert_called_once()
+ mock_reboot_node.assert_called_once()
+
+ @mock.patch('cmframework.agent.cmagent.cmmanage.CMManage')
+ @mock.patch('cmframework.agent.cmagent.cmlogger.CMLogger')
+ @mock.patch('cmframework.agent.cmagent.socket.gethostbyname')
+ @mock.patch('cmframework.agent.cmagent.CMAgent._reboot_node')
+ @mock.patch('cmframework.agent.cmagent.VerboseLogger')
+ @mock.patch('cmframework.agent.cmagent.logging')
+ def test_activate_no_reboot(self, mock_logging, mock_verboselogger, mock_reboot_node,
+ mock_socket_get_hostbyname, mock_cmlogger, mock_cmmanage):
+ mock_cmmanage.return_value.activate_node = mock.MagicMock(return_value=False)
+ args = []
+ agent = CMAgent()
+ agent(args)
+
+ mock_cmmanage.return_value.activate_node.assert_called_once()
+ mock_reboot_node.assert_not_called()
+
+ @mock.patch('cmframework.agent.cmagent.cmmanage.CMManage')
+ @mock.patch('cmframework.agent.cmagent.cmlogger.CMLogger')
+ @mock.patch('cmframework.agent.cmagent.socket.gethostbyname')
+ @mock.patch('cmframework.agent.cmagent.CMAgent._reboot_node')
+ @mock.patch('cmframework.agent.cmagent.VerboseLogger')
+ @mock.patch('cmframework.agent.cmagent.logging')
+ def test_activate_custom_args(self, mock_logging, mock_verboselogger, mock_reboot_node,
+ mock_socket_get_hostbyname, mock_cmlogger, mock_cmmanage):
+ mock_cmmanage.return_value.activate_node = mock.MagicMock(return_value=True)
+
+ args = ['--ip', 'abc.com', '--port', '1234', '--client-lib', 'abc']
+ agent = CMAgent()
+ agent(args)
+
+ mock_socket_get_hostbyname.assert_called_once_with('abc.com')
+
+ mock_cmmanage.assert_called_once_with(mock_socket_get_hostbyname.return_value, 1234, 'abc',
+ mock_verboselogger.return_value)
+ mock_cmmanage.return_value.activate_node.assert_called_once()
+
+ @mock.patch('cmframework.agent.cmagent.cmmanage.CMManage')
+ @mock.patch('cmframework.agent.cmagent.cmlogger.CMLogger')
+ @mock.patch('cmframework.agent.cmagent.socket.gethostbyname')
+ @mock.patch('cmframework.agent.cmagent.CMAgent._reboot_node')
+ @mock.patch('cmframework.agent.cmagent.VerboseLogger')
+ @mock.patch('cmframework.agent.cmagent.logging')
+ def test_activate_localhost(self, mock_logging, mock_verboselogger, mock_reboot_node,
+ mock_socket_get_hostbyname, mock_cmlogger, mock_cmmanage):
+ mock_cmmanage.return_value.activate_node = mock.MagicMock(return_value=True)
+
+ import socket
+ mock_socket_get_hostbyname.side_effect = socket.gaierror()
+
+ args = []
+ agent = CMAgent()
+ agent(args)
+
+ mock_socket_get_hostbyname.assert_called_once_with('config-manager')
+ mock_verboselogger.assert_called_once()
+
+ mock_cmmanage.assert_called_once_with('127.0.0.1', 61100,
+ 'cmframework.lib.CMClientImpl',
+ mock_verboselogger.return_value)
+ mock_cmmanage.return_value.activate_node.assert_called_once()
+
+ @mock.patch('cmframework.agent.cmagent.cmmanage.CMManage')
+ @mock.patch('cmframework.agent.cmagent.cmlogger.CMLogger')
+ @mock.patch('cmframework.agent.cmagent.socket.gethostbyname')
+ @mock.patch('cmframework.agent.cmagent.CMAgent._reboot_node')
+ @mock.patch('cmframework.agent.cmagent.VerboseLogger')
+ @mock.patch('cmframework.agent.cmagent.logging')
+ def test_activate_fails(self, mock_logging, mock_verboselogger, mock_reboot_node,
+ mock_socket_get_hostbyname, mock_cmlogger, mock_cmmanage):
+ mock_cmmanage.return_value.activate_node = mock.MagicMock()
+ mock_cmmanage.return_value.activate_node.side_effect = CMError('Test error')
+
+ args = []
+ agent = CMAgent()
+ try:
+ agent(args)
+ assert False
+ except CMError:
+ pass
+
+ # TODO assert that alarm was raised
+
+
+if __name__ == '__main__':
+ unittest.main()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import unittest
+import mock
+from mock import call
+import json
+
+from cmframework.utils.cmalarm import CMAlarm
+from cmframework.utils.cmalarm import CMRebootRequestAlarm
+from cmframework.utils.cmalarm import CMActivationFailedAlarm
+from cmframework.apis.cmerror import CMError
+
+
+class CMAlarmTest(unittest.TestCase):
+ @mock.patch('cmframework.utils.cmalarm.logging')
+ @mock.patch('cmframework.utils.cmalarm.AlarmHandler')
+ def test_abstract_alarm(self, mock_alarm_handler, mock_logging):
+ alarm = CMAlarm()
+
+ with self.assertRaises(NotImplementedError) as context:
+ alarm.raise_alarm_for_node('node-a')
+
+ @mock.patch('cmframework.utils.cmalarm.logging')
+ @mock.patch('cmframework.utils.cmalarm.AlarmHandler')
+ def test_raise_and_fail(self, mock_alarm_handler, mock_logging):
+ mock_alarm_handler.side_effect = Exception
+ alarm = CMRebootRequestAlarm()
+ alarm.raise_alarm_for_node('node-a')
+
+ mock_logging.warning.assert_called_once()
+ mock_alarm_handler.return_value.raise_alarm_with_dn.assert_not_called()
+
+ @mock.patch('cmframework.utils.cmalarm.logging')
+ @mock.patch('cmframework.utils.cmalarm.AlarmHandler')
+ def test_cancel_and_fail(self, mock_alarm_handler, mock_logging):
+ mock_alarm_handler.side_effect = Exception
+ alarm = CMRebootRequestAlarm()
+ alarm.cancel_alarm_for_node('node-a')
+
+ mock_logging.warning.assert_called_once()
+ mock_alarm_handler.return_value.cancel_alarm_with_dn.assert_not_called()
+
+ @mock.patch('cmframework.utils.cmalarm.logging')
+ @mock.patch('cmframework.utils.cmalarm.AlarmHandler')
+ def test_raise_rebootrequestalarm_for_node(self, mock_alarm_handler, mock_logging):
+ alarm = CMRebootRequestAlarm()
+ alarm.raise_alarm_for_node('node-a')
+
+ mock_alarm_handler.return_value.raise_alarm_with_dn.assert_called_once_with(
+ '45001',
+ 'NODE-node-a',
+ {})
+
+ @mock.patch('cmframework.utils.cmalarm.logging')
+ @mock.patch('cmframework.utils.cmalarm.AlarmHandler')
+ def test_raise_rebootrequestalarm_for_node_with_info(self, mock_alarm_handler, mock_logging):
+ alarm = CMRebootRequestAlarm()
+ alarm.raise_alarm_for_node('node-a', {'some': 'additional info'})
+
+ mock_alarm_handler.return_value.raise_alarm_with_dn.assert_called_once_with(
+ '45001',
+ 'NODE-node-a',
+ {'some': 'additional info'})
+
+ @mock.patch('cmframework.utils.cmalarm.logging')
+ @mock.patch('cmframework.utils.cmalarm.AlarmHandler')
+ def test_raise_rebootrequestalarm_for_sg(self, mock_alarm_handler, mock_logging):
+ alarm = CMRebootRequestAlarm()
+ alarm.raise_alarm_for_sg('config-manager')
+
+ mock_alarm_handler.return_value.raise_alarm_with_dn.assert_called_once_with(
+ '45001',
+ 'SG-config-manager',
+ {})
+
+ @mock.patch('cmframework.utils.cmalarm.logging')
+ @mock.patch('cmframework.utils.cmalarm.AlarmHandler')
+ def test_cancel_rebootrequestalarm_for_node(self, mock_alarm_handler, mock_logging):
+ alarm = CMRebootRequestAlarm()
+ alarm.cancel_alarm_for_node('node-a')
+
+ mock_alarm_handler.return_value.cancel_alarm_with_dn.assert_called_once_with(
+ '45001',
+ 'NODE-node-a',
+ {})
+
+ @mock.patch('cmframework.utils.cmalarm.logging')
+ @mock.patch('cmframework.utils.cmalarm.AlarmHandler')
+ def test_cancel_rebootrequestalarm_for_node_with_info(self, mock_alarm_handler, mock_logging):
+ alarm = CMRebootRequestAlarm()
+ alarm.cancel_alarm_for_node('node-a', {'some': 'additional info'})
+
+ mock_alarm_handler.return_value.cancel_alarm_with_dn.assert_called_once_with(
+ '45001',
+ 'NODE-node-a',
+ {'some': 'additional info'})
+
+ @mock.patch('cmframework.utils.cmalarm.logging')
+ @mock.patch('cmframework.utils.cmalarm.AlarmHandler')
+ def test_cancel_rebootrequestalarm_for_sg(self, mock_alarm_handler, mock_logging):
+ alarm = CMRebootRequestAlarm()
+ alarm.cancel_alarm_for_sg('config-manager')
+
+ mock_alarm_handler.return_value.cancel_alarm_with_dn.assert_called_once_with(
+ '45001',
+ 'SG-config-manager',
+ {})
+
+ @mock.patch('cmframework.utils.cmalarm.logging')
+ @mock.patch('cmframework.utils.cmalarm.AlarmHandler')
+ def test_raise_activationfailed_for_node(self, mock_alarm_handler, mock_logging):
+ alarm = CMActivationFailedAlarm()
+ alarm.raise_alarm_for_node('node-a')
+
+ mock_alarm_handler.return_value.raise_alarm_with_dn.assert_called_once_with(
+ '45002',
+ 'NODE-node-a',
+ {})
+
+ @mock.patch('cmframework.utils.cmalarm.logging')
+ @mock.patch('cmframework.utils.cmalarm.AlarmHandler')
+ def test_raise_activationfailed_for_sg(self, mock_alarm_handler, mock_logging):
+ alarm = CMActivationFailedAlarm()
+ alarm.raise_alarm_for_sg('config-manager')
+
+ mock_alarm_handler.return_value.raise_alarm_with_dn.assert_called_once_with(
+ '45002',
+ 'SG-config-manager',
+ {})
+
+ @mock.patch('cmframework.utils.cmalarm.logging')
+ @mock.patch('cmframework.utils.cmalarm.AlarmHandler')
+ def test_cancel_activationfailed_for_node(self, mock_alarm_handler, mock_logging):
+ alarm = CMActivationFailedAlarm()
+ alarm.cancel_alarm_for_node('node-a')
+
+ mock_alarm_handler.return_value.cancel_alarm_with_dn.assert_called_once_with(
+ '45002',
+ 'NODE-node-a',
+ {})
+
+ @mock.patch('cmframework.utils.cmalarm.logging')
+ @mock.patch('cmframework.utils.cmalarm.AlarmHandler')
+ def test_cancel_activationfailed_for_sg(self, mock_alarm_handler, mock_logging):
+ alarm = CMActivationFailedAlarm()
+ alarm.cancel_alarm_for_sg('config-manager')
+
+ mock_alarm_handler.return_value.cancel_alarm_with_dn.assert_called_once_with(
+ '45002',
+ 'SG-config-manager',
+ {})
+
+if __name__ == '__main__':
+ unittest.main()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import unittest
+import mock
+from mock import call
+import json
+
+from cmframework.server.cmcsn import CMCSN
+from cmframework.apis.cmerror import CMError
+
+
+class CMCSNTest(unittest.TestCase):
+ @mock.patch('cmframework.server.cmcsn.logging')
+ def test_initial_config(self, mock_logging):
+ mock_backend = mock.MagicMock()
+ mock_backend.get_property = mock.MagicMock()
+ mock_backend.get_property.side_effect = CMError('Test error')
+
+ csn = CMCSN(mock_backend)
+
+ mock_backend.assert_has_calls([call.get_property('cloud.cmframework')])
+ self.assertEqual(csn.get(), 0)
+
+ @mock.patch('cmframework.server.cmcsn.logging')
+ def test_bad_config(self, mock_logging):
+ mock_backend = mock.MagicMock()
+ mock_backend.get_property = mock.MagicMock()
+ mock_backend.get_property.return_value = 'bad json'
+
+ csn = CMCSN(mock_backend)
+
+ mock_backend.assert_has_calls([call.get_property('cloud.cmframework')])
+ self.assertEqual(csn.get(), 0)
+
+ @mock.patch('cmframework.server.cmcsn.logging')
+ def test_old_config(self, mock_logging):
+ mock_backend = mock.MagicMock()
+ mock_backend.get_property = mock.MagicMock()
+ mock_backend.get_property.return_value = ('{"csn": {"global": 101, "nodes": '
+ '{"node-a": 99, "node-b": 100, "node-c": 101}}}')
+
+ csn = CMCSN(mock_backend)
+
+ self.assertEqual(csn.get(), 101)
+ self.assertEqual(csn.get_node_csn('node-a'), 99)
+ self.assertEqual(csn.get_node_csn('node-b'), 100)
+ self.assertEqual(csn.get_node_csn('node-c'), 101)
+
+ @mock.patch('cmframework.server.cmcsn.logging')
+ def test_increment(self, mock_logging):
+ mock_backend = mock.MagicMock()
+ mock_backend.get_property = mock.MagicMock()
+ mock_backend.set_property = mock.MagicMock()
+ mock_backend.get_property.return_value = ('{"csn": {"global": 101, "nodes": '
+ '{"node-a": 99, "node-b": 100, "node-c": 101}}}')
+
+ csn = CMCSN(mock_backend)
+ csn.increment()
+
+ mock_backend.set_property.assert_called_once()
+ mock_backend_set_property_arg_value = json.loads(mock_backend.set_property.call_args[0][1])
+ self.assertEqual(mock_backend_set_property_arg_value['csn']['global'], 102)
+ self.assertEqual(mock_backend_set_property_arg_value['csn']['nodes']['node-a'], 99)
+ self.assertEqual(mock_backend_set_property_arg_value['csn']['nodes']['node-b'], 100)
+ self.assertEqual(mock_backend_set_property_arg_value['csn']['nodes']['node-c'], 101)
+
+ self.assertEqual(csn.get(), 102)
+
+ @mock.patch('cmframework.server.cmcsn.logging')
+ def test_sync_node(self, mock_logging):
+ mock_backend = mock.MagicMock()
+ mock_backend.get_property = mock.MagicMock()
+ mock_backend.set_property = mock.MagicMock()
+ mock_backend.get_property.return_value = ('{"csn": {"global": 101, "nodes": '
+ '{"node-a": 99, "node-b": 100, "node-c": 101}}}')
+
+ csn = CMCSN(mock_backend)
+ csn.sync_node_csn('node-a')
+
+ mock_backend.set_property.assert_called_once()
+ mock_backend_set_property_arg_value = json.loads(mock_backend.set_property.call_args[0][1])
+ self.assertEqual(mock_backend_set_property_arg_value['csn']['global'], 101)
+ self.assertEqual(mock_backend_set_property_arg_value['csn']['nodes']['node-a'], 101)
+ self.assertEqual(mock_backend_set_property_arg_value['csn']['nodes']['node-b'], 100)
+ self.assertEqual(mock_backend_set_property_arg_value['csn']['nodes']['node-c'], 101)
+
+ self.assertEqual(csn.get_node_csn('node-a'), 101)
+
+
+if __name__ == '__main__':
+ unittest.main()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import unittest
+import mock
+from mock import call
+
+from cmframework.utils.cmdependencysort import CMDependencySort as Sort
+from cmframework.apis.cmerror import CMError
+
+
+class CMDependencySortTest(unittest.TestCase):
+ def test_empty(self):
+ sort = Sort()
+ sorted_list = sort.sort()
+ assert sorted_list == []
+
+ def test_one_no_deps(self):
+ sort = Sort({'a': []})
+ sorted_list = sort.sort()
+ assert sorted_list == ['a']
+
+ sort = Sort(None, {'a': []})
+ sorted_list = sort.sort()
+ assert sorted_list == ['a']
+
+ def test_both_no_deps(self):
+ sort = Sort({'a': []}, {'a': []})
+ sorted_list = sort.sort()
+ assert sorted_list == ['a']
+
+ sort = Sort({'a': [], 'b': []}, {'a': [], 'b': []})
+ sorted_list = sort.sort()
+
+ assert len(sorted_list) == 2
+ assert 'a' in sorted_list
+ assert 'b' in sorted_list
+
+ @staticmethod
+ def _assert_orderings(after, before, sorted_list):
+ for entry, deps in after.iteritems():
+ for dep in deps:
+ assert sorted_list.index(entry) > sorted_list.index(dep)
+
+ for entry, deps in before.iteritems():
+ for dep in deps:
+ assert sorted_list.index(entry) < sorted_list.index(dep)
+
+ def test_mandbc(self):
+ after = {'a': ['m'], 'b': ['d']}
+ before = {'a': ['b', 'c', 'd', 'n'], 'b': ['c']}
+ sort = Sort(after, before)
+ sorted_list = sort.sort()
+
+ CMDependencySortTest._assert_orderings(after, before, sorted_list)
+
+ def test_simple_only_after(self):
+ after = {'a': ['b'], 'b': ['c']}
+ before = {}
+ sort = Sort(after)
+ sorted_list = sort.sort()
+
+ CMDependencySortTest._assert_orderings(after, before, sorted_list)
+
+ def test_simple_only_before(self):
+ after = {}
+ before = {'a': ['b'], 'b': ['c']}
+ sort = Sort(None, before)
+ sorted_list = sort.sort()
+
+ CMDependencySortTest._assert_orderings(after, before, sorted_list)
+
+ def test_simple(self):
+ after = {'b': ['a'], 'c': ['b']}
+ before = {'a': ['b'], 'b': ['c']}
+ sort = Sort(after, before)
+ sorted_list = sort.sort()
+
+ CMDependencySortTest._assert_orderings(after, before, sorted_list)
+
+ def test_cycle_before_after(self):
+ after = {'b': ['a']}
+ before = {'b': ['a']}
+ sort = Sort(after, before)
+
+ with self.assertRaises(CMError) as context:
+ sorted_list = sort.sort()
+
+ def test_cycle_only_before(self):
+ after = {}
+ before = {'b': ['a'], 'a': ['b']}
+ sort = Sort(after, before)
+
+ with self.assertRaises(CMError) as context:
+ sorted_list = sort.sort()
+
+ def test_cycle_only_after(self):
+ after = {'b': ['a'], 'a': ['b']}
+ before = {}
+ sort = Sort(after, before)
+
+ with self.assertRaises(CMError) as context:
+ sorted_list = sort.sort()
+
+if __name__ == '__main__':
+ unittest.main()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import unittest
+import mock
+from mock import call
+import json
+from collections import OrderedDict
+
+from cmframework.utils.cmdsshandler import CMDSSHandler
+from cmframework.apis.cmerror import CMError
+from dss.api import dss_error
+
+
+class CMDSSHandlerTest(unittest.TestCase):
+ @mock.patch('cmframework.utils.cmdsshandler.dss_client.Client')
+ @mock.patch('cmframework.utils.cmdsshandler.logging')
+ def test_init(self, mock_logging, mock_dss_client):
+ handler = CMDSSHandler(uri='test_uri')
+
+ mock_dss_client.assert_called_once_with('test_uri')
+
+ @mock.patch('cmframework.utils.cmdsshandler.dss_client.Client')
+ @mock.patch('cmframework.utils.cmdsshandler.logging')
+ def test_get_domains_exception(self, mock_logging, mock_dss_client):
+ handler = CMDSSHandler(uri='test_uri')
+
+ mock_dss_client.return_value.get_domains.side_effect = dss_error.Error('no domains')
+
+ with self.assertRaises(CMError) as context:
+ handler.get_domains()
+
+ @mock.patch('cmframework.utils.cmdsshandler.dss_client.Client')
+ @mock.patch('cmframework.utils.cmdsshandler.logging')
+ def test_get_domains(self, mock_logging, mock_dss_client):
+ handler = CMDSSHandler(uri='test_uri')
+
+ expected_result = ['a domain', 'b domain', 'c domain']
+ mock_dss_client.return_value.get_domains.return_value = expected_result
+
+ domains = handler.get_domains()
+
+ assert domains == expected_result
+
+ @mock.patch('cmframework.utils.cmdsshandler.dss_client.Client')
+ @mock.patch('cmframework.utils.cmdsshandler.logging')
+ def test_get_domain_not_existing(self, mock_logging, mock_dss_client):
+ handler = CMDSSHandler(uri='test_uri')
+
+ mock_dss_client.return_value.get_domains.return_value = ['a domain', 'b domain', 'c domain']
+
+ domain = handler.get_domain('not domain')
+
+ assert domain is None
+
+ mock_dss_client.return_value.get_domains.assert_called_once()
+ mock_dss_client.return_value.get_domain.assert_not_called()
+
+ @mock.patch('cmframework.utils.cmdsshandler.dss_client.Client')
+ @mock.patch('cmframework.utils.cmdsshandler.logging')
+ def test_get_domain_dss_fails(self, mock_logging, mock_dss_client):
+ handler = CMDSSHandler(uri='test_uri')
+
+ mock_dss_client.return_value.get_domains.return_value = ['a domain', 'b domain', 'c domain']
+ mock_dss_client.return_value.get_domain.side_effect = dss_error.Error('some error')
+
+ with self.assertRaises(CMError) as context:
+ domain = handler.get_domain('a domain')
+
+ mock_dss_client.return_value.get_domains.assert_called_once()
+ mock_dss_client.return_value.get_domain.assert_called_once_with('a domain')
+
+ @mock.patch('cmframework.utils.cmdsshandler.dss_client.Client')
+ @mock.patch('cmframework.utils.cmdsshandler.logging')
+ def test_get_domain(self, mock_logging, mock_dss_client):
+ handler = CMDSSHandler(uri='test_uri')
+
+ mock_dss_client.return_value.get_domains.return_value = ['a domain', 'b domain', 'c domain']
+
+ expected_result = OrderedDict([('name1', 'value1'), ('name2', 'value2')])
+ mock_dss_client.return_value.get_domain.return_value = expected_result
+
+ domain = handler.get_domain('a domain')
+
+ assert domain == expected_result
+
+ @mock.patch('cmframework.utils.cmdsshandler.dss_client.Client')
+ @mock.patch('cmframework.utils.cmdsshandler.logging')
+ def test_set_dss_fails(self, mock_logging, mock_dss_client):
+ handler = CMDSSHandler(uri='test_uri')
+
+ mock_dss_client.return_value.set.side_effect = dss_error.Error('some error')
+
+ with self.assertRaises(CMError) as context:
+ handler.set('a domain', 'a name', 'a value')
+
+ mock_dss_client.return_value.set.assert_called_once_with('a domain', 'a name', 'a value')
+
+ @mock.patch('cmframework.utils.cmdsshandler.dss_client.Client')
+ @mock.patch('cmframework.utils.cmdsshandler.logging')
+ def test_set(self, mock_logging, mock_dss_client):
+ handler = CMDSSHandler(uri='test_uri')
+
+ handler.set('a domain', 'a name', 'a value')
+
+ mock_dss_client.return_value.set.assert_called_once_with('a domain', 'a name', 'a value')
+
+ @mock.patch('cmframework.utils.cmdsshandler.dss_client.Client')
+ @mock.patch('cmframework.utils.cmdsshandler.logging')
+ def test_delete_dss_fails(self, mock_logging, mock_dss_client):
+ handler = CMDSSHandler(uri='test_uri')
+
+ mock_dss_client.return_value.get_domains.return_value = ['a domain', 'b domain', 'c domain']
+
+ mock_dss_client.return_value.get_domain.return_value = OrderedDict([('name', 'value')])
+
+ mock_dss_client.return_value.delete.side_effect = dss_error.Error('some error')
+
+ with self.assertRaises(CMError) as context:
+ handler.delete('a domain', 'name')
+
+ mock_dss_client.return_value.delete.assert_called_once_with('a domain', 'name')
+
+ @mock.patch('cmframework.utils.cmdsshandler.dss_client.Client')
+ @mock.patch('cmframework.utils.cmdsshandler.logging')
+ def test_delete_non_existing_name(self, mock_logging, mock_dss_client):
+ handler = CMDSSHandler(uri='test_uri')
+
+ mock_dss_client.return_value.get_domains.return_value = ['a domain', 'b domain', 'c domain']
+
+ mock_dss_client.return_value.get_domain.return_value = OrderedDict([('name', 'value')])
+
+ handler.delete('a domain', 'a name')
+
+ mock_dss_client.return_value.get_domain.assert_called_once_with('a domain')
+ mock_dss_client.return_value.delete.assert_not_called()
+
+ @mock.patch('cmframework.utils.cmdsshandler.dss_client.Client')
+ @mock.patch('cmframework.utils.cmdsshandler.logging')
+ def test_delete_non_existing_domain(self, mock_logging, mock_dss_client):
+ handler = CMDSSHandler(uri='test_uri')
+
+ mock_dss_client.return_value.get_domains.return_value = ['a domain', 'b domain', 'c domain']
+
+ handler.delete('not domain', 'no name')
+
+ mock_dss_client.return_value.get_domains.assert_called_once()
+ mock_dss_client.return_value.get_domain.assert_not_called()
+ mock_dss_client.return_value.delete.assert_not_called()
+
+ @mock.patch('cmframework.utils.cmdsshandler.dss_client.Client')
+ @mock.patch('cmframework.utils.cmdsshandler.logging')
+ def test_get_dss_fails(self, mock_logging, mock_dss_client):
+ handler = CMDSSHandler(uri='test_uri')
+
+ mock_dss_client.return_value.get_domains.return_value = ['a domain', 'b domain', 'c domain']
+
+ mock_dss_client.return_value.get_domain.side_effect = dss_error.Error('some error')
+
+ with self.assertRaises(CMError) as context:
+ handler.get('a domain', 'name')
+
+ mock_dss_client.return_value.get_domain.assert_called_once_with('a domain')
+
+ @mock.patch('cmframework.utils.cmdsshandler.dss_client.Client')
+ @mock.patch('cmframework.utils.cmdsshandler.logging')
+ def test_get_non_existing_name(self, mock_logging, mock_dss_client):
+ handler = CMDSSHandler(uri='test_uri')
+
+ mock_dss_client.return_value.get_domains.return_value = ['a domain', 'b domain', 'c domain']
+ mock_dss_client.return_value.get_domain.return_value = OrderedDict([('name', 'value')])
+
+ value = handler.get('a domain', 'a name')
+
+ assert value is None
+
+ mock_dss_client.return_value.get_domain.assert_called_once_with('a domain')
+
+ @mock.patch('cmframework.utils.cmdsshandler.dss_client.Client')
+ @mock.patch('cmframework.utils.cmdsshandler.logging')
+ def test_get_non_existing_domain(self, mock_logging, mock_dss_client):
+ handler = CMDSSHandler(uri='test_uri')
+
+ mock_dss_client.return_value.get_domains.return_value = ['a domain', 'b domain', 'c domain']
+ mock_dss_client.return_value.get_domain.return_value = OrderedDict([('name', 'value')])
+
+ value = handler.get('some domain', 'a name')
+
+ assert value is None
+
+ mock_dss_client.return_value.get_domain.assert_not_called()
+ mock_dss_client.return_value.get_domains.assert_called_once()
+
+ @mock.patch('cmframework.utils.cmdsshandler.dss_client.Client')
+ @mock.patch('cmframework.utils.cmdsshandler.logging')
+ def test_get(self, mock_logging, mock_dss_client):
+ handler = CMDSSHandler(uri='test_uri')
+
+ mock_dss_client.return_value.get_domains.return_value = ['a domain', 'b domain', 'c domain']
+ mock_dss_client.return_value.get_domain.return_value = OrderedDict([('name', 'value')])
+
+ value = handler.get('a domain', 'name')
+
+ assert value == 'value'
+
+ @mock.patch('cmframework.utils.cmdsshandler.dss_client.Client')
+ @mock.patch('cmframework.utils.cmdsshandler.logging')
+ def test_delete_domain_dss_fails(self, mock_logging, mock_dss_client):
+ handler = CMDSSHandler(uri='test_uri')
+
+ mock_dss_client.return_value.get_domains.return_value = ['a domain', 'b domain', 'c domain']
+
+ mock_dss_client.return_value.delete_domain.side_effect = dss_error.Error('some error')
+
+ with self.assertRaises(CMError) as context:
+ handler.delete_domain('a domain')
+
+ mock_dss_client.return_value.delete_domain.assert_called_once_with('a domain')
+
+ @mock.patch('cmframework.utils.cmdsshandler.dss_client.Client')
+ @mock.patch('cmframework.utils.cmdsshandler.logging')
+ def test_delete_domain_non_existent(self, mock_logging, mock_dss_client):
+ handler = CMDSSHandler(uri='test_uri')
+
+ mock_dss_client.return_value.get_domains.return_value = ['a domain', 'b domain', 'c domain']
+
+ handler.delete_domain('not domain')
+
+ mock_dss_client.return_value.get_domains.assert_called_once()
+ mock_dss_client.return_value.delete_domain.assert_not_called()
+
+ @mock.patch('cmframework.utils.cmdsshandler.dss_client.Client')
+ @mock.patch('cmframework.utils.cmdsshandler.logging')
+ def test_delete_domain(self, mock_logging, mock_dss_client):
+ handler = CMDSSHandler(uri='test_uri')
+
+ mock_dss_client.return_value.get_domains.return_value = ['a domain', 'b domain', 'c domain']
+
+ handler.delete_domain('a domain')
+
+ mock_dss_client.return_value.get_domains.assert_called_once()
+ mock_dss_client.return_value.delete_domain.assert_called_once_with('a domain')
+
+
+if __name__ == '__main__':
+ unittest.main()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import unittest
+import mock
+from mock import call
+from mock import mock_open
+import json
+
+from cmframework.utils.cmflagfile import CMFlagFile
+from cmframework.apis.cmerror import CMError
+
+
+class CMFlagFileTest(unittest.TestCase):
+ @mock.patch('cmframework.utils.cmflagfile.os')
+ @mock.patch('cmframework.utils.cmflagfile.logging')
+ def test_is_set_for_non_existing_file(self, mock_logging, mock_os):
+ mock_os.path = mock.MagicMock()
+ mock_os.path.exists = mock.MagicMock()
+ mock_os.path.exists.return_value = False
+
+ flagfile = CMFlagFile('foo')
+ self.assertFalse(flagfile)
+
+ @mock.patch('cmframework.utils.cmflagfile.os')
+ @mock.patch('cmframework.utils.cmflagfile.logging')
+ def test_is_set_for_existing_file(self, mock_logging, mock_os):
+ mock_os.path = mock.MagicMock()
+ mock_os.path.exists = mock.MagicMock()
+ mock_os.path.exists.return_value = True
+
+ flagfile = CMFlagFile('foo')
+ self.assertTrue(flagfile)
+
+ @mock.patch('cmframework.utils.cmflagfile.open', new_callable=mock_open)
+ @mock.patch('cmframework.utils.cmflagfile.os')
+ @mock.patch('cmframework.utils.cmflagfile.logging')
+ def test_set_for_nonexisting_file(self, mock_logging, mock_os, mock_file):
+ mock_os.path = mock.MagicMock()
+ mock_os.path.exists = mock.MagicMock()
+ mock_os.path.exists.return_value = False
+
+ flagfile = CMFlagFile('foo')
+ self.assertFalse(flagfile)
+
+ flagfile.set()
+
+ mock_file.assert_called_with('/mnt/config-manager/foo', 'w')
+ mock_file.return_value.write.assert_called_once()
+
+ @mock.patch('cmframework.utils.cmflagfile.open', new_callable=mock_open)
+ @mock.patch('cmframework.utils.cmflagfile.os')
+ @mock.patch('cmframework.utils.cmflagfile.logging')
+ def test_set_for_existing_file(self, mock_logging, mock_os, mock_file):
+ mock_os.path = mock.MagicMock()
+ mock_os.path.exists = mock.MagicMock()
+ mock_os.path.exists.return_value = True
+
+ flagfile = CMFlagFile('foo')
+ self.assertTrue(flagfile)
+
+ flagfile.set()
+
+ mock_file.assert_not_called()
+
+ @mock.patch('cmframework.utils.cmflagfile.open', new_callable=mock_open)
+ @mock.patch('cmframework.utils.cmflagfile.os')
+ @mock.patch('cmframework.utils.cmflagfile.logging')
+ def test_set_io_failure(self, mock_logging, mock_os, mock_file):
+ mock_os.path = mock.MagicMock()
+ mock_os.path.exists = mock.MagicMock()
+ mock_os.path.exists.return_value = False
+
+ mock_file.return_value.write.side_effect = IOError()
+
+ flagfile = CMFlagFile('foo')
+ self.assertFalse(flagfile)
+
+ with self.assertRaises(CMError) as context:
+ flagfile.set()
+
+ @mock.patch('cmframework.utils.cmflagfile.os')
+ @mock.patch('cmframework.utils.cmflagfile.logging')
+ def test_unset_for_existing_file(self, mock_logging, mock_os):
+ mock_os.path = mock.MagicMock()
+ mock_os.path.exists = mock.MagicMock()
+ mock_os.path.exists.return_value = True
+
+ flagfile = CMFlagFile('foo')
+ self.assertTrue(flagfile)
+
+ flagfile.unset()
+
+ mock_os.remove.assert_called_once_with('/mnt/config-manager/foo')
+
+ @mock.patch('cmframework.utils.cmflagfile.os')
+ @mock.patch('cmframework.utils.cmflagfile.logging')
+ def test_unset_for_nonexisting_file(self, mock_logging, mock_os):
+ mock_os.path = mock.MagicMock()
+ mock_os.path.exists = mock.MagicMock()
+ mock_os.path.exists.return_value = False
+
+ flagfile = CMFlagFile('foo')
+ self.assertFalse(flagfile)
+
+ flagfile.unset()
+
+ mock_os.remove.assert_not_called()
+
+ @mock.patch('cmframework.utils.cmflagfile.os')
+ @mock.patch('cmframework.utils.cmflagfile.logging')
+ def test_unset_io_failure(self, mock_logging, mock_os):
+ mock_os.path = mock.MagicMock()
+ mock_os.path.exists = mock.MagicMock()
+ mock_os.path.exists.return_value = True
+
+ flagfile = CMFlagFile('foo')
+ self.assertTrue(flagfile)
+
+ mock_os.remove.side_effect = IOError()
+
+ with self.assertRaises(CMError) as context:
+ flagfile.unset()
+
+if __name__ == '__main__':
+ unittest.main()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import mock
+from cmframework.utils.cmlogger import CMMaskFormatter
+
+
+class Test(object):
+
+ def test_masking_case1(self):
+ record = r'\"compute-3\": {\"hwmgmt\": {\"password\": \"secret\"'
+ formatter = mock.MagicMock()
+ formatter.format.return_value = record
+ maskformatter = CMMaskFormatter(formatter, ['password', 'admin_pass'])
+ log = maskformatter.format(record)
+ assert log == r'\"compute-3\": {\"hwmgmt\": {\"password\": \"*** password ***\"'
+
+ def test_masking_case2(self):
+ record = r'"compute-3": {"hwmgmt": {"password": "secret"'
+ formatter = mock.MagicMock()
+ formatter.format.return_value = record
+ maskformatter = CMMaskFormatter(formatter, ['password', 'admin_pass'])
+ log = maskformatter.format(record)
+ assert log == r'"compute-3": {"hwmgmt": {"password": "*** password ***"'
+
+ def test_masking_case3(self):
+ record = r'\\"compute-3\\": {\\"hwmgmt\\": {\\"password\\": \\"secret\\"'
+ formatter = mock.MagicMock()
+ formatter.format.return_value = record
+ maskformatter = CMMaskFormatter(formatter, ['password', 'admin_pass'])
+ log = maskformatter.format(record)
+ assert log == r'\\"compute-3\\": {\\"hwmgmt\\": {\\"password\\": \\"*** password ***\\"'
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import unittest
+import mock
+import json
+
+from cmframework.server.cmprocessor import CMProcessor
+from cmframework.apis.cmerror import CMError
+from cmframework.server.cmcsn import CMCSN
+from cmframework.server import cmchangemonitor
+
+
+class CMProcessorAutomaticActivationTest(unittest.TestCase):
+ @staticmethod
+ def backend_get_property(key):
+ if key == 'cloud.cmframework':
+ return '{"csn": {"global": 101, "nodes": {"node-a": 99, "node-b": 100, "node-c": 101}}}'
+ elif key == 'foo':
+ return '{"foo": "bar"}'
+
+ @mock.patch('cmframework.utils.cmflagfile.os')
+ @mock.patch('cmframework.server.cmprocessor.logging')
+ def test_set_property_automatic_activation_disabled(self, mock_logging, mock_flagfile_os):
+ mock_backend = mock.MagicMock()
+ mock_backend.get_property = CMProcessorAutomaticActivationTest.backend_get_property
+
+ mock_validator = mock.MagicMock()
+ mock_activator = mock.MagicMock()
+ mock_changemonitor = mock.MagicMock()
+ mock_activationstate_handler = mock.MagicMock()
+ mock_snapshot_handler = mock.MagicMock()
+
+ mock_flagfile_os.path = mock.MagicMock()
+ mock_flagfile_os.path.exists = mock.MagicMock()
+ mock_flagfile_os.path.exists.return_value = True
+
+ processor = CMProcessor(mock_backend, mock_validator, mock_activator,
+ mock_changemonitor, mock_activationstate_handler,
+ mock_snapshot_handler)
+
+ processor.set_property('foo', 'barbar')
+
+ mock_validator.validate_set.assert_called_once_with({'foo': 'barbar'})
+ mock_backend.set_properties.called_once_with({'foo': 'barbar'})
+ mock_activator.add_work.assert_not_called()
+
+ @mock.patch('cmframework.utils.cmflagfile.os')
+ @mock.patch('cmframework.server.cmprocessor.logging')
+ def test_delete_property_automatic_activation_disabled(self, mock_logging, mock_flagfile_os):
+ mock_backend = mock.MagicMock()
+ mock_backend.get_property = CMProcessorAutomaticActivationTest.backend_get_property
+
+ mock_validator = mock.MagicMock()
+ mock_activator = mock.MagicMock()
+ mock_changemonitor = mock.MagicMock()
+ mock_activationstate_handler = mock.MagicMock()
+ mock_snapshot_handler = mock.MagicMock()
+
+ mock_flagfile_os.path = mock.MagicMock()
+ mock_flagfile_os.path.exists = mock.MagicMock()
+ mock_flagfile_os.path.exists.return_value = True
+
+ processor = CMProcessor(mock_backend, mock_validator, mock_activator,
+ mock_changemonitor, mock_activationstate_handler,
+ mock_snapshot_handler)
+
+ processor.delete_property('foo')
+
+ mock_validator.validate_delete.assert_called_once_with(['foo'])
+ mock_backend.delete_property.assert_called_once_with('foo')
+ mock_activator.add_work.assert_not_called()
+
+ @mock.patch('cmframework.utils.cmflagfile.os')
+ @mock.patch('cmframework.server.cmprocessor.logging')
+ @mock.patch('cmframework.server.cmprocessor.cmactivationwork.CMActivationWork')
+ @mock.patch.object(CMProcessor, '_clear_reboot_requests')
+ @mock.patch.object(CMCSN, 'sync_node_csn')
+ def test_activate_node_automatic_activation_disabled(self,
+ mock_sync_node_csn,
+ mock_clear_reboot_requests,
+ mock_work,
+ mock_logging,
+ mock_flagfile_os):
+ mock_backend = mock.MagicMock()
+ mock_backend.get_property = CMProcessorAutomaticActivationTest.backend_get_property
+
+ mock_validator = mock.MagicMock()
+ mock_activator = mock.MagicMock()
+ mock_changemonitor = mock.MagicMock()
+ mock_activationstate_handler = mock.MagicMock()
+ mock_snapshot_handler = mock.MagicMock()
+
+ mock_work.return_value.get_result = mock.MagicMock()
+ mock_work.return_value.get_result.return_value = None
+
+ mock_flagfile_os.path = mock.MagicMock()
+ mock_flagfile_os.path.exists = mock.MagicMock()
+ mock_flagfile_os.path.exists.return_value = True
+
+ mock_work.OPER_NODE = mock.MagicMock()
+
+ processor = CMProcessor(mock_backend, mock_validator, mock_activator,
+ mock_changemonitor, mock_activationstate_handler,
+ mock_snapshot_handler)
+
+ self.assertEqual(processor.activate_node('node-b'), False)
+
+ mock_clear_reboot_requests.assert_not_called()
+ mock_work.assert_not_called()
+ mock_activator.add_work.assert_not_called()
+ mock_sync_node_csn.assert_not_called()
+
+if __name__ == '__main__':
+ unittest.main()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import unittest
+import mock
+from mock import call
+import json
+
+from cmframework.server.cmprocessor import CMProcessor
+from cmframework.apis.cmerror import CMError
+from cmframework.server.cmcsn import CMCSN
+
+# TODO:Add the checking for calling the changemonitorhandle
+
+
+class CMProcessorNodeActivationTest(unittest.TestCase):
+ @staticmethod
+ def backend_get_property(key):
+ if key == 'cloud.cmframework':
+ return '{"csn": {"global": 101, "nodes": {"node-a": 99, "node-b": 100, "node-c": 101}}}'
+ elif key == 'foo':
+ return '{"foo": "bar"}'
+
+ @mock.patch('cmframework.server.cmprocessor.logging')
+ @mock.patch('cmframework.server.cmprocessor.cmactivationwork.CMActivationWork')
+ def test_get_property(self, mock_work, mock_logging):
+ mock_backend = mock.MagicMock()
+ mock_backend.get_property = CMProcessorNodeActivationTest.backend_get_property
+
+ mock_validator = mock.MagicMock()
+ mock_activator = mock.MagicMock()
+ mock_changemonitor = mock.MagicMock()
+ mock_activationstate_handler = mock.MagicMock()
+ mock_snapshot_handler = mock.MagicMock()
+
+ mock_work.OPER_SET = mock.MagicMock()
+
+ processor = CMProcessor(mock_backend, mock_validator, mock_activator,
+ mock_changemonitor, mock_activationstate_handler,
+ mock_snapshot_handler)
+
+ self.assertEqual(json.loads(processor.get_property('foo'))['foo'], 'bar')
+
+ @mock.patch('cmframework.server.cmprocessor.logging')
+ @mock.patch('cmframework.server.cmprocessor.cmactivationwork.CMActivationWork')
+ def test_set_property(self, mock_work, mock_logging):
+ mock_backend = mock.MagicMock()
+ mock_backend.get_property = CMProcessorNodeActivationTest.backend_get_property
+
+ mock_validator = mock.MagicMock()
+ mock_activator = mock.MagicMock()
+ mock_changemonitor = mock.MagicMock()
+ mock_activationstate_handler = mock.MagicMock()
+ mock_snapshot_handler = mock.MagicMock()
+
+ mock_work.OPER_SET = mock.MagicMock()
+
+ processor = CMProcessor(mock_backend, mock_validator, mock_activator,
+ mock_changemonitor, mock_activationstate_handler,
+ mock_snapshot_handler)
+
+ processor.set_property('foo', 'barbar')
+
+ mock_validator.validate_set.assert_called_once_with({'foo': 'barbar'})
+ mock_backend.set_properties.called_once_with({'foo': 'barbar'})
+ mock_work.assert_called_once_with(mock_work.OPER_SET, 102, {'foo': 'barbar'})
+ mock_activator.add_work.assert_called_once_with(mock_work.return_value)
+
+ @mock.patch('cmframework.server.cmprocessor.logging')
+ @mock.patch('cmframework.server.cmprocessor.cmactivationwork.CMActivationWork')
+ @mock.patch('cmframework.server.cmprocessor.cmalarm.CMActivationFailedAlarm')
+ def test_activate_node_no_change(self, mock_alarm, mock_work, mock_logging):
+ mock_backend = mock.MagicMock()
+ mock_backend.get_property = CMProcessorNodeActivationTest.backend_get_property
+
+ mock_validator = mock.MagicMock()
+ mock_activator = mock.MagicMock()
+ mock_changemonitor = mock.MagicMock()
+ mock_activationstate_handler = mock.MagicMock()
+ mock_snapshot_handler = mock.MagicMock()
+
+ mock_work.return_value.get_result = mock.MagicMock()
+ mock_work.return_value.get_result.return_value = None
+
+ mock_work.OPER_NODE = mock.MagicMock()
+
+ processor = CMProcessor(mock_backend, mock_validator, mock_activator,
+ mock_changemonitor, mock_activationstate_handler,
+ mock_snapshot_handler)
+
+ self.assertEqual(processor.activate_node('node-c'), False)
+
+ mock_alarm.assert_not_called()
+ mock_activator.add_work.assert_not_called()
+
+ @mock.patch('cmframework.server.cmprocessor.logging')
+ @mock.patch('cmframework.server.cmprocessor.cmactivationwork.CMActivationWork')
+ @mock.patch('cmframework.server.cmprocessor.cmalarm.CMActivationFailedAlarm')
+ @mock.patch.object(CMProcessor, '_clear_reboot_requests')
+ @mock.patch.object(CMCSN, 'sync_node_csn')
+ def test_activate_node_changed_no_reboot(self, mock_sync_node_csn, mock_clear_reboot_requests,
+ mock_alarm, mock_work, mock_logging):
+ mock_backend = mock.MagicMock()
+ mock_backend.get_property = CMProcessorNodeActivationTest.backend_get_property
+
+ mock_validator = mock.MagicMock()
+ mock_activator = mock.MagicMock()
+ mock_changemonitor = mock.MagicMock()
+ mock_activationstate_handler = mock.MagicMock()
+ mock_snapshot_handler = mock.MagicMock()
+
+ mock_work.return_value.get_result = mock.MagicMock()
+ mock_work.return_value.get_result.return_value = None
+
+ mock_work.OPER_NODE = mock.MagicMock()
+
+ processor = CMProcessor(mock_backend, mock_validator, mock_activator,
+ mock_changemonitor, mock_activationstate_handler,
+ mock_snapshot_handler)
+
+ self.assertEqual(processor.activate_node('node-b'), False)
+
+ mock_clear_reboot_requests.assert_called_once()
+ mock_work.assert_called_once_with(mock_work.OPER_NODE, 101, {}, 'node-b')
+ mock_activator.add_work.assert_called_once_with(mock_work.return_value)
+ mock_alarm.return_value.cancel_alarm_for_node.assert_called_once_with('node-b')
+ mock_work.return_value.get_result.assert_called_once()
+ mock_sync_node_csn.assert_called_once()
+ mock_alarm.return_value.raise_alarm_for_node.assert_not_called()
+
+ @mock.patch('cmframework.server.cmprocessor.logging')
+ @mock.patch('cmframework.server.cmprocessor.cmactivationwork.CMActivationWork')
+ @mock.patch('cmframework.server.cmprocessor.cmalarm.CMActivationFailedAlarm')
+ @mock.patch.object(CMProcessor, '_clear_reboot_requests')
+ @mock.patch.object(CMCSN, 'sync_node_csn')
+ def test_activate_node_changed_and_reboot(self, mock_sync_node_csn, mock_clear_reboot_requests,
+ mock_alarm, mock_work, mock_logging):
+ mock_backend = mock.MagicMock()
+ mock_backend.get_property = CMProcessorNodeActivationTest.backend_get_property
+
+ mock_validator = mock.MagicMock()
+ mock_activator = mock.MagicMock()
+ mock_changemonitor = mock.MagicMock()
+ mock_activationstate_handler = mock.MagicMock()
+ mock_snapshot_handler = mock.MagicMock()
+
+ mock_work.return_value.get_result = mock.MagicMock()
+ mock_work.return_value.get_result.return_value = None
+
+ mock_work.OPER_NODE = mock.MagicMock()
+
+ processor = CMProcessor(mock_backend, mock_validator, mock_activator,
+ mock_changemonitor, mock_activationstate_handler,
+ mock_snapshot_handler)
+
+ processor.reboot_request('node-b')
+
+ self.assertEqual(processor.activate_node('node-b'), True)
+
+ mock_clear_reboot_requests.assert_called_once()
+ mock_work.assert_called_once_with(mock_work.OPER_NODE, 101, {}, 'node-b')
+ mock_activator.add_work.assert_called_once_with(mock_work.return_value)
+ mock_alarm.return_value.cancel_alarm_for_node.assert_called_once_with('node-b')
+ mock_work.return_value.get_result.assert_called_once()
+ mock_sync_node_csn.assert_called_once()
+ mock_alarm.return_value.raise_alarm_for_node.assert_not_called()
+
+ @mock.patch('cmframework.server.cmprocessor.logging')
+ @mock.patch('cmframework.server.cmprocessor.cmactivationwork.CMActivationWork')
+ @mock.patch('cmframework.server.cmprocessor.cmalarm.CMActivationFailedAlarm')
+ @mock.patch.object(CMCSN, 'sync_node_csn')
+ def test_activate_node_changed_activation_fails(self, mock_sync_node_csn,
+ mock_alarm, mock_work, mock_logging):
+ mock_backend = mock.MagicMock()
+ mock_backend.get_property = CMProcessorNodeActivationTest.backend_get_property
+
+ mock_validator = mock.MagicMock()
+ mock_activator = mock.MagicMock()
+ mock_changemonitor = mock.MagicMock()
+ mock_activationstate_handler = mock.MagicMock()
+ mock_snapshot_handler = mock.MagicMock()
+
+ mock_work.return_value.get_result = mock.MagicMock()
+ mock_work.return_value.get_result.return_value = {'test handler': ['test error']}
+
+ mock_work.OPER_NODE = mock.MagicMock()
+
+ processor = CMProcessor(mock_backend, mock_validator, mock_activator,
+ mock_changemonitor, mock_activationstate_handler,
+ mock_snapshot_handler)
+
+ processor.activate_node('node-b')
+
+ mock_work.assert_called_once_with(mock_work.OPER_NODE, 101, {}, 'node-b')
+ mock_activator.add_work.assert_called_once_with(mock_work.return_value)
+ mock_work.return_value.get_result.assert_called_once()
+ mock_sync_node_csn.assert_not_called()
+ mock_alarm.return_value.assert_has_calls(
+ [call.cancel_alarm_for_node('node-b'),
+ call.raise_alarm_for_node('node-b', {'failed activators': ['test error']})])
+
+if __name__ == '__main__':
+ unittest.main()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import unittest
+import mock
+from mock import call
+
+from cmframework.server.cmprocessor import CMProcessor
+from cmframework.apis.cmerror import CMError
+from cmframework.server.cmcsn import CMCSN
+from cmframework.server import cmchangemonitor
+
+
+class CMProcessorSnapshotTest(unittest.TestCase):
+ @mock.patch('cmframework.server.cmprocessor.cmcsn.CMCSN')
+ @mock.patch('cmframework.server.cmprocessor.cmsnapshot.CMSnapshot')
+ @mock.patch('cmframework.server.cmprocessor.logging')
+ def test_get_property(self, mock_logging, mock_cmsnapshot, mock_cmcsn):
+ mock_backend = mock.MagicMock()
+
+ mock_validator = mock.MagicMock()
+ mock_activator = mock.MagicMock()
+ mock_changemonitor = mock.MagicMock()
+ mock_activationstate_handler = mock.MagicMock()
+ mock_snapshot_handler = mock.MagicMock()
+
+ processor = CMProcessor(mock_backend, mock_validator, mock_activator,
+ mock_changemonitor, mock_activationstate_handler,
+ mock_snapshot_handler)
+
+ property = processor.get_property('foo')
+ mock_cmsnapshot.return_value.assert_not_called()
+
+ snapshot_property = processor.get_property('foo', 'snap1')
+ mock_cmsnapshot.return_value.assert_has_calls([call.load('snap1'),
+ call.get_property('foo')],
+ any_order=True)
+
+ @mock.patch('cmframework.server.cmprocessor.cmcsn.CMCSN')
+ @mock.patch('cmframework.server.cmprocessor.cmsnapshot.CMSnapshot')
+ @mock.patch('cmframework.server.cmprocessor.logging')
+ def test_get_properties(self, mock_logging, mock_cmsnapshot, mock_cmcsn):
+ mock_backend = mock.MagicMock()
+
+ mock_validator = mock.MagicMock()
+ mock_activator = mock.MagicMock()
+ mock_changemonitor = mock.MagicMock()
+ mock_activationstate_handler = mock.MagicMock()
+ mock_snapshot_handler = mock.MagicMock()
+
+ processor = CMProcessor(mock_backend, mock_validator, mock_activator,
+ mock_changemonitor, mock_activationstate_handler,
+ mock_snapshot_handler)
+
+ property = processor.get_properties('.*')
+ mock_cmsnapshot.return_value.assert_not_called()
+
+ snapshot_property = processor.get_properties('.*', 'snap1')
+ mock_cmsnapshot.return_value.assert_has_calls([call.load('snap1'),
+ call.get_properties('.*')],
+ any_order=True)
+
+ @mock.patch('cmframework.server.cmprocessor.cmcsn.CMCSN')
+ @mock.patch('cmframework.server.cmprocessor.cmsnapshot.CMSnapshot')
+ @mock.patch('cmframework.server.cmprocessor.logging')
+ def test_create_snapshot(self, mock_logging, mock_cmsnapshot, mock_cmcsn):
+ mock_backend = mock.MagicMock()
+
+ mock_validator = mock.MagicMock()
+ mock_activator = mock.MagicMock()
+ mock_changemonitor = mock.MagicMock()
+ mock_activationstate_handler = mock.MagicMock()
+ mock_snapshot_handler = mock.MagicMock()
+
+ processor = CMProcessor(mock_backend, mock_validator, mock_activator,
+ mock_changemonitor, mock_activationstate_handler,
+ mock_snapshot_handler)
+
+ processor.create_snapshot('snap1')
+
+ mock_cmsnapshot.return_value.create.assert_called_once_with('snap1', mock_backend)
+
+ @mock.patch('cmframework.server.cmprocessor.cmcsn.CMCSN')
+ @mock.patch('cmframework.server.cmprocessor.cmsnapshot.CMSnapshot')
+ @mock.patch('cmframework.server.cmprocessor.logging')
+ def test_delete_snapshot(self, mock_logging, mock_cmsnapshot, mock_cmcsn):
+ mock_backend = mock.MagicMock()
+
+ mock_validator = mock.MagicMock()
+ mock_activator = mock.MagicMock()
+ mock_changemonitor = mock.MagicMock()
+ mock_activationstate_handler = mock.MagicMock()
+ mock_snapshot_handler = mock.MagicMock()
+
+ processor = CMProcessor(mock_backend, mock_validator, mock_activator,
+ mock_changemonitor, mock_activationstate_handler,
+ mock_snapshot_handler)
+
+ processor.delete_snapshot('snap1')
+
+ mock_cmsnapshot.return_value.delete.assert_called_once_with('snap1')
+
+ @mock.patch('cmframework.server.cmprocessor.cmcsn.CMCSN')
+ @mock.patch('cmframework.server.cmprocessor.cmsnapshot.CMSnapshot')
+ @mock.patch('cmframework.server.cmprocessor.logging')
+ def test_list_snapshots(self, mock_logging, mock_cmsnapshot, mock_cmcsn):
+ mock_backend = mock.MagicMock()
+
+ mock_validator = mock.MagicMock()
+ mock_activator = mock.MagicMock()
+ mock_changemonitor = mock.MagicMock()
+ mock_activationstate_handler = mock.MagicMock()
+ mock_snapshot_handler = mock.MagicMock()
+
+ processor = CMProcessor(mock_backend, mock_validator, mock_activator,
+ mock_changemonitor, mock_activationstate_handler,
+ mock_snapshot_handler)
+
+ processor.list_snapshots()
+
+ mock_cmsnapshot.return_value.list.assert_called_once()
+
+ @mock.patch('cmframework.server.cmprocessor.cmactivationwork.CMActivationWork')
+ @mock.patch('cmframework.server.cmprocessor.cmcsn.CMCSN')
+ @mock.patch('cmframework.server.cmprocessor.cmsnapshot.CMSnapshot')
+ @mock.patch('cmframework.server.cmprocessor.logging')
+ def test_restore_snapshot(self,
+ mock_logging,
+ mock_cmsnapshot,
+ mock_cmcsn,
+ mock_cmactivationwork):
+ csn1 = mock.MagicMock()
+ csn2 = mock.MagicMock()
+ mock_cmcsn.side_effect = [csn1, csn2]
+ mock_backend = mock.MagicMock()
+
+ mock_validator = mock.MagicMock()
+ mock_activator = mock.MagicMock()
+ mock_changemonitor = mock.MagicMock()
+ mock_activationstate_handler = mock.MagicMock()
+ mock_snapshot_handler = mock.MagicMock()
+
+ processor = CMProcessor(mock_backend, mock_validator, mock_activator,
+ mock_changemonitor, mock_activationstate_handler,
+ mock_snapshot_handler)
+
+ self.assertEqual(processor.csn, csn1)
+
+ processor.restore_snapshot('snap1')
+
+ self.assertEqual(processor.csn, csn2)
+ mock_cmsnapshot.return_value.load.assert_called_once_with('snap1')
+ mock_validator.validate_set.assert_called_once_with(
+ mock_cmsnapshot.return_value.get_properties.return_value)
+ mock_cmsnapshot.return_value.restore.assert_called_once_with(mock_backend)
+ mock_cmactivationwork.assert_called_once_with(
+ mock_cmactivationwork.OPER_SET,
+ csn2.get.return_value,
+ mock_cmsnapshot.return_value.get_properties.return_value)
+ mock_activator.add_work.assert_called_once_with(mock_cmactivationwork.return_value)
+
+if __name__ == '__main__':
+ unittest.main()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import unittest
+import mock
+from mock import call
+import json
+
+from cmframework.server.cmsnapshot import CMSnapshot
+from cmframework.apis.cmerror import CMError
+
+
+class CMSnapshotTest(unittest.TestCase):
+ @staticmethod
+ def snapshot_list_data(name):
+ return {'snapshot_properties': {'some': 'value'},
+ 'snapshot_metadata': 'meta-{}'.format(name)}
+
+ @mock.patch('cmframework.server.cmsnapshot.logging')
+ def test_new_snapshot_object(self, mock_logging):
+ mock_handler = mock.MagicMock()
+
+ snapshot = CMSnapshot(mock_handler)
+
+ with self.assertRaises(CMError) as context:
+ snapshot.get_property('foo')
+
+ @mock.patch('cmframework.server.cmsnapshot.logging')
+ def test_restore_without_load(self, mock_logging):
+ mock_handler = mock.MagicMock()
+
+ mock_target_backend = mock.MagicMock()
+ mock_target_backend.get_properties.return_value = {"foo": "bar",
+ "some": {"other": "value"}}
+
+ snapshot = CMSnapshot(mock_handler)
+
+ with self.assertRaises(CMError) as context:
+ snapshot.restore(mock_target_backend)
+
+ @mock.patch('cmframework.server.cmsnapshot.datetime.datetime')
+ @mock.patch('cmframework.server.cmsnapshot.logging')
+ def test_create(self, mock_logging, mock_datetime):
+ mock_handler = mock.MagicMock()
+ mock_handler.snapshot_exists.return_value = False
+
+ mock_source_backend = mock.MagicMock()
+ mock_source_backend.get_properties.return_value = {"foo": "bar", "some": {"other": "value"}}
+ mock_source_backend.get_property.return_value = 'some'
+
+ mock_datetime.now = mock.MagicMock()
+ from datetime import datetime
+ mock_datetime.now.return_value = datetime.now()
+ expected_creation_date = mock_datetime.now.return_value.isoformat()
+
+ snapshot = CMSnapshot(mock_handler)
+ snapshot.create('snap1', mock_source_backend)
+
+ mock_handler.set_data.assert_called_once_with(
+ 'snap1',
+ {'snapshot_properties': mock_source_backend.get_properties.return_value,
+ 'snapshot_metadata': {'name': 'snap1',
+ 'creation_date': expected_creation_date,
+ 'custom': None}})
+
+ @mock.patch('cmframework.server.cmsnapshot.logging')
+ def test_create_already_exists(self, mock_logging):
+ mock_handler = mock.MagicMock()
+ mock_handler.snapshot_exists.return_value = True
+
+ mock_backend_handler = mock.MagicMock()
+
+ snapshot = CMSnapshot(mock_handler)
+
+ with self.assertRaises(CMError) as context:
+ snapshot.create('already_exists', mock_backend_handler)
+
+ @mock.patch('cmframework.server.cmsnapshot.logging')
+ def test_load_non_existing(self, mock_logging):
+ mock_handler = mock.MagicMock()
+ mock_handler.snapshot_exists.return_value = False
+
+ snapshot = CMSnapshot(mock_handler)
+
+ with self.assertRaises(CMError) as context:
+ snapshot.load('snap1')
+
+ @mock.patch('cmframework.server.cmsnapshot.logging')
+ def test_load(self, mock_logging):
+ expected_data = {'some': 'value'}
+ expected_metadata = {'name': 'snap1',
+ 'creation_date': 'somedate',
+ 'custom': None}
+
+ mock_handler = mock.MagicMock()
+ mock_handler.snapshot_exists.return_value = True
+ mock_handler.get_data.return_value = {
+ 'snapshot_properties': expected_data,
+ 'snapshot_metadata': expected_metadata}
+
+ snapshot = CMSnapshot(mock_handler)
+
+ snapshot.load('already_exists')
+
+ assert snapshot._data == expected_data
+ assert snapshot._metadata == expected_metadata
+
+ @mock.patch('cmframework.server.cmsnapshot.logging')
+ def test_restore(self, mock_logging):
+ expected_data = {'foo': 'bar', 'some': {'other': 'value'}}
+ expected_metadata = {'name': 'already_exists',
+ 'creation_date': 'somedate',
+ 'custom': None}
+
+ mock_handler = mock.MagicMock()
+ mock_handler.snapshot_exists.return_value = True
+ mock_handler.get_data.return_value = {
+ 'snapshot_properties': expected_data,
+ 'snapshot_metadata': expected_metadata}
+
+ target_backend = mock.MagicMock()
+ target_backend.get_properties.return_value = {"a": "1", "b": "2"}
+
+ snapshot = CMSnapshot(mock_handler)
+ snapshot.load('already_exists')
+
+ snapshot.restore(target_backend)
+
+ assert {'a', 'b'} == set(target_backend.delete_properties.call_args[0][0])
+ target_backend.set_properties.assert_called_once_with(expected_data)
+
+ @mock.patch('cmframework.server.cmsnapshot.logging')
+ def test_restore_only_one_in_target(self, mock_logging):
+ expected_data = {'foo': 'bar', 'some': {'other': 'value'}}
+ expected_metadata = {'name': 'already_exists',
+ 'creation_date': 'somedate',
+ 'custom': None}
+
+ mock_handler = mock.MagicMock()
+ mock_handler.snapshot_exists.return_value = True
+ mock_handler.get_data.return_value = {
+ 'snapshot_properties': expected_data,
+ 'snapshot_metadata': expected_metadata}
+
+ target_backend = mock.MagicMock()
+ target_backend.get_properties.return_value = {"a": "1"}
+
+ snapshot = CMSnapshot(mock_handler)
+ snapshot.load('already_exists')
+
+ snapshot.restore(target_backend)
+
+ target_backend.delete_property.assert_called_once_with('a')
+ target_backend.set_properties.assert_called_once_with(expected_data)
+
+ @mock.patch('cmframework.server.cmsnapshot.logging')
+ def test_list(self, mock_logging):
+ mock_handler = mock.MagicMock()
+ mock_handler.get_data.side_effect = CMSnapshotTest.snapshot_list_data
+ mock_handler.list_snapshots.return_value = {'snap1', 'snap2'}
+
+ expected_snapshot_list = {'meta-snap1', 'meta-snap2'}
+
+ snapshot = CMSnapshot(mock_handler)
+ snapshot_list = snapshot.list()
+
+ assert set(snapshot_list) == expected_snapshot_list
+
+ @mock.patch('cmframework.server.cmsnapshot.logging')
+ def test_delete(self, mock_logging):
+ mock_handler = mock.MagicMock()
+ mock_handler.snapshot_exists.return_value = True
+
+ snapshot = CMSnapshot(mock_handler)
+ snapshot.delete('already_exists')
+
+ mock_handler.delete_snapshot.assert_called_once_with('already_exists')
+
+ @mock.patch('cmframework.server.cmsnapshot.logging')
+ def test_delete_non_existing(self, mock_logging):
+ mock_handler = mock.MagicMock()
+ mock_handler.snapshot_exists.return_value = False
+
+ snapshot = CMSnapshot(mock_handler)
+
+ with self.assertRaises(CMError) as context:
+ snapshot.delete('snap1')
+
+if __name__ == '__main__':
+ unittest.main()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import unittest
+import mock
+from mock import call
+import json
+
+from cmframework.apis.cmupdate import CMUpdate
+from cmframework.lib.cmupdateimpl import CMUpdateImpl
+from cmframework.apis.cmerror import CMError
+
+
+class CMUpdateImplTest(unittest.TestCase):
+ @mock.patch('cmframework.lib.cmupdateimpl.CMPluginLoader')
+ @mock.patch('cmframework.lib.cmupdateimpl.CMManage')
+ @mock.patch('cmframework.lib.cmupdateimpl.logging')
+ def test_init(self, mock_logging, mock_client, mock_pluginloader):
+ mock_pluginloader.return_value.load.return_value = ({}, None)
+
+ updater = CMUpdate('test_plugin_path', 'test_server_ip', 'test_server_port',
+ 'test_client_lib_impl_module', 'test_verbose_logger')
+
+ mock_pluginloader.assert_called_once_with('test_plugin_path')
+ mock_client.assert_called_once_with('test_server_ip',
+ 'test_server_port',
+ 'test_client_lib_impl_module',
+ 'test_verbose_logger')
+
+ @staticmethod
+ def _test__read_dependency_file_incorrect(file_name):
+ if file_name == 'test_plugin_path/test_handler_a.deps':
+ return ([], ['x'])
+ if file_name == 'test_plugin_path/test_handler_b.deps':
+ return (['y'], [])
+
+ @mock.patch.object(CMUpdateImpl, '_read_dependency_file')
+ @mock.patch('cmframework.lib.cmupdateimpl.CMPluginLoader')
+ @mock.patch('cmframework.lib.cmupdateimpl.CMManage')
+ @mock.patch('cmframework.lib.cmupdateimpl.logging')
+ def test_init_missing_handler_in_dependencies(self,
+ mock_logging,
+ mock_client,
+ mock_pluginloader,
+ mock__read_dependency_file):
+ mock_pluginloader.return_value.load.return_value = ({}, None)
+ mock__read_dependency_file.side_effect = \
+ CMUpdateImplTest._test__read_dependency_file_incorrect
+
+ test_handler_a_module = mock.MagicMock()
+ test_handler_a_class = mock.MagicMock()
+ test_handler_a_class.return_value.__str__.return_value = 'test_handler_a'
+ setattr(test_handler_a_module, 'test_handler_a', test_handler_a_class)
+
+ mock_pluginloader.return_value.load.return_value = \
+ ({'test_handler_a': test_handler_a_module}, None)
+
+ with self.assertRaises(CMError) as context:
+ updater = CMUpdate('test_plugin_path', 'test_server_ip', 'test_server_port',
+ 'test_client_lib_impl_module', 'test_verbose_logger')
+
+ test_handler_b_module = mock.MagicMock()
+ test_handler_b_class = mock.MagicMock()
+ test_handler_b_class.return_value.__str__.return_value = 'test_handler_b'
+ setattr(test_handler_b_module, 'test_handler_b', test_handler_b_class)
+
+ mock_pluginloader.return_value.load.return_value = \
+ ({'test_handler_b': test_handler_b_module}, None)
+
+ with self.assertRaises(CMError) as context:
+ updater = CMUpdate('test_plugin_path', 'test_server_ip', 'test_server_port',
+ 'test_client_lib_impl_module', 'test_verbose_logger')
+
+ mock_client.assert_not_called()
+
+ @staticmethod
+ def _test__read_dependency_file(file_name):
+ if file_name == 'test_plugin_path/test_handler_a.deps':
+ return ([], ['test_handler_b'])
+ if file_name == 'test_plugin_path/test_handler_b.deps':
+ return (['test_handler_a'], [])
+ if file_name == 'test_plugin_path/test_handler_c.deps':
+ return (['test_handler_b'], ['test_handler_a'])
+
+ @staticmethod
+ def _test_update_func_a(confman):
+ CMUpdateImplTest._test_update_func_calls.append('test_handler_a')
+ if str(confman) == 'raise exception':
+ raise Exception('test_update_exception')
+
+ @staticmethod
+ def _test_update_func_b(confman):
+ CMUpdateImplTest._test_update_func_calls.append('test_handler_b')
+
+ @staticmethod
+ def _test_update_func_c(confman):
+ confman.get_config.return_value = {'test_properties': '_test_update_func_c properties'}
+ CMUpdateImplTest._test_update_func_calls.append('test_handler_c')
+
+ def _setup_test_handlers(self, mock_pluginloader, mock__read_dependency_file, mock_sorter):
+ CMUpdateImplTest._test_update_func_calls = []
+
+ test_handler_a_module = mock.MagicMock()
+ test_handler_a_class = mock.MagicMock()
+ test_handler_a_class.return_value.__str__.return_value = 'test_handler_a'
+ test_handler_a_class.return_value.update.side_effect = CMUpdateImplTest._test_update_func_a
+ setattr(test_handler_a_module, 'test_handler_a', test_handler_a_class)
+
+ test_handler_b_module = mock.MagicMock()
+ test_handler_b_class = mock.MagicMock()
+ test_handler_b_class.return_value.__str__.return_value = 'test_handler_b'
+ test_handler_b_class.return_value.update.side_effect = CMUpdateImplTest._test_update_func_b
+ setattr(test_handler_b_module, 'test_handler_b', test_handler_b_class)
+
+ test_handler_c_module = mock.MagicMock()
+ test_handler_c_class = mock.MagicMock()
+ test_handler_c_class.return_value.__str__.return_value = 'test_handler_c'
+ test_handler_c_class.return_value.update.side_effect = CMUpdateImplTest._test_update_func_c
+ setattr(test_handler_c_module, 'test_handler_c', test_handler_c_class)
+
+ mock_pluginloader.return_value.load.return_value = \
+ ({'test_handler_a': test_handler_a_module,
+ 'test_handler_b': test_handler_b_module,
+ 'test_handler_c': test_handler_c_module}, None)
+
+ mock__read_dependency_file.side_effect = CMUpdateImplTest._test__read_dependency_file
+
+ mock_sorter.return_value.sort.return_value = ['test_handler_c',
+ 'test_handler_a',
+ 'test_handler_b']
+
+ return (test_handler_a_class, test_handler_b_class, test_handler_c_class)
+
+ @mock.patch.object(CMUpdateImpl, '_read_dependency_file')
+ @mock.patch('cmframework.lib.cmupdateimpl.CMDependencySort')
+ @mock.patch('cmframework.lib.cmupdateimpl.CMPluginLoader')
+ @mock.patch('cmframework.lib.cmupdateimpl.CMManage')
+ @mock.patch('cmframework.lib.cmupdateimpl.logging')
+ def test_update(self,
+ mock_logging,
+ mock_client,
+ mock_pluginloader,
+ mock_sorter,
+ mock__read_dependency_file):
+ test_handler_a_class, test_handler_b_class, test_handler_c_class = \
+ self._setup_test_handlers(mock_pluginloader, mock__read_dependency_file, mock_sorter)
+
+ updater = CMUpdate('test_plugin_path', 'test_server_ip', 'test_server_port',
+ 'test_client_lib_impl_module', 'test_verbose_logger')
+
+ mock_confman = mock.MagicMock()
+ mock_confman.__str__.return_value = 'confman'
+ mock_confman.get_config.return_value = {'test_properties': 'some properties'}
+
+ updater.update(mock_confman)
+
+ sorter_after_graph = mock_sorter.call_args[0][0]
+ assert len(sorter_after_graph) == 3
+ assert 'test_handler_a' in sorter_after_graph
+ assert 'test_handler_b' in sorter_after_graph
+ assert 'test_handler_c' in sorter_after_graph
+ assert sorter_after_graph['test_handler_a'] == ['test_handler_b']
+ assert sorter_after_graph['test_handler_b'] == []
+ assert sorter_after_graph['test_handler_c'] == ['test_handler_a']
+
+ sorter_before_graph = mock_sorter.call_args[0][1]
+ assert len(sorter_before_graph) == 3
+ assert 'test_handler_a' in sorter_before_graph
+ assert 'test_handler_b' in sorter_before_graph
+ assert 'test_handler_c' in sorter_before_graph
+ assert sorter_before_graph['test_handler_a'] == []
+ assert sorter_before_graph['test_handler_b'] == ['test_handler_a']
+ assert sorter_before_graph['test_handler_c'] == ['test_handler_b']
+
+ mock_sorter.return_value.sort.assert_called_once()
+
+ test_handler_a_class.return_value.update.assert_called_once_with(mock_confman)
+ test_handler_b_class.return_value.update.assert_called_once_with(mock_confman)
+ test_handler_c_class.return_value.update.assert_called_once_with(mock_confman)
+
+ assert CMUpdateImplTest._test_update_func_calls == \
+ mock_sorter.return_value.sort.return_value
+
+ mock_client.return_value.set_properties.assert_called_once_with(
+ {'test_properties': '_test_update_func_c properties'}, True)
+
+ @mock.patch.object(CMUpdateImpl, '_read_dependency_file')
+ @mock.patch('cmframework.lib.cmupdateimpl.ConfigManager')
+ @mock.patch('cmframework.lib.cmupdateimpl.CMDependencySort')
+ @mock.patch('cmframework.lib.cmupdateimpl.CMPluginLoader')
+ @mock.patch('cmframework.lib.cmupdateimpl.CMManage')
+ @mock.patch('cmframework.lib.cmupdateimpl.logging')
+ def test_update_no_confman(self,
+ mock_logging,
+ mock_client,
+ mock_pluginloader,
+ mock_sorter,
+ mock_configmanager,
+ mock__read_dependency_file):
+ test_handler_a_class, test_handler_b_class, test_handler_c_class = \
+ self._setup_test_handlers(mock_pluginloader, mock__read_dependency_file, mock_sorter)
+
+ updater = CMUpdate('test_plugin_path', 'test_server_ip', 'test_server_port',
+ 'test_client_lib_impl_module', 'test_verbose_logger')
+
+ updater.update()
+
+ test_handler_a_class.return_value.update.assert_called_once_with(
+ mock_configmanager.return_value)
+ test_handler_b_class.return_value.update.assert_called_once_with(
+ mock_configmanager.return_value)
+ test_handler_c_class.return_value.update.assert_called_once_with(
+ mock_configmanager.return_value)
+
+ '''
+ mock_configmanager.assert_called_once_with(
+ mock_client.return_value.get_properties.return_value)
+ '''
+
+ assert CMUpdateImplTest._test_update_func_calls == \
+ mock_sorter.return_value.sort.return_value
+
+ @mock.patch.object(CMUpdateImpl, '_read_dependency_file')
+ @mock.patch('cmframework.lib.cmupdateimpl.ConfigManager')
+ @mock.patch('cmframework.lib.cmupdateimpl.CMDependencySort')
+ @mock.patch('cmframework.lib.cmupdateimpl.CMPluginLoader')
+ @mock.patch('cmframework.lib.cmupdateimpl.CMManage')
+ @mock.patch('cmframework.lib.cmupdateimpl.logging')
+ def test_update_exception(self,
+ mock_logging,
+ mock_client,
+ mock_pluginloader,
+ mock_sorter,
+ mock_configmanager,
+ mock__read_dependency_file):
+ test_handler_a_class, test_handler_b_class, test_handler_c_class = \
+ self._setup_test_handlers(mock_pluginloader, mock__read_dependency_file, mock_sorter)
+
+ updater = CMUpdate('test_plugin_path', 'test_server_ip', 'test_server_port',
+ 'test_client_lib_impl_module', 'test_verbose_logger')
+
+ mock_confman = mock.MagicMock()
+ mock_confman.__str__.return_value = 'raise exception'
+
+ with self.assertRaises(Exception) as context:
+ updater.update(mock_confman)
+
+ # TODO:verify plugin(s) before exception are called.
+ mock_logging.warning.assert_called_with('Update handler %s failed: %s',
+ 'test_handler_a',
+ 'test_update_exception')
+
+ @mock.patch('cmframework.lib.cmupdateimpl.logging')
+ def test_dependency_files(self, mock_logging):
+ with mock.patch('cmframework.lib.cmupdateimpl.open', create=True) as mock_open:
+ mock_open.return_value = mock.MagicMock(spec=file)
+ file_handle = mock_open.return_value.__enter__.return_value
+
+ file_handle.readline.side_effect = IOError('File not found')
+ before, after = CMUpdateImpl._read_dependency_file('testexception: not existing')
+ assert before == []
+ assert after == []
+ mock_logging.debug.assert_called_with('Dependency file %s not found.',
+ 'testexception: not existing')
+
+ file_handle.readline.side_effect = [None]
+ before, after = CMUpdateImpl._read_dependency_file('./test_deps/1.deps')
+ assert before == []
+ assert after == []
+
+ file_handle.readline.side_effect = ['foo', 'bar', None]
+ before, after = CMUpdateImpl._read_dependency_file('./test_deps/1.deps')
+ assert before == []
+ assert after == []
+
+ file_handle.readline.side_effect = ['Before: a, b', 'After: c, d', None]
+ before, after = CMUpdateImpl._read_dependency_file('./test_deps/1.deps')
+ assert before == ['a', 'b']
+ assert after == ['c', 'd']
+
+ file_handle.readline.side_effect = ['foo',
+ 'Before: a, b',
+ 'bar',
+ 'After: c, d',
+ 'something',
+ None]
+ before, after = CMUpdateImpl._read_dependency_file('./test_deps/1.deps')
+ assert before == ['a', 'b']
+ assert after == ['c', 'd']
+
+ file_handle.readline.side_effect = ['After: c, d', 'Before: a, b', None]
+ before, after = CMUpdateImpl._read_dependency_file('./test_deps/1.deps')
+ assert before == ['a', 'b']
+ assert after == ['c', 'd']
+
+ file_handle.readline.side_effect = ['Before:a,b', 'After:c,d', None]
+ before, after = CMUpdateImpl._read_dependency_file('./test_deps/1.deps')
+ assert before == ['a', 'b']
+ assert after == ['c', 'd']
+
+ file_handle.readline.side_effect = ['Before: a, b ', 'After: c, d ', None]
+ before, after = CMUpdateImpl._read_dependency_file('./test_deps/1.deps')
+ assert before == ['a', 'b']
+ assert after == ['c', 'd']
+
+if __name__ == '__main__':
+ unittest.main()
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+class ConfigManager(object):
+ def __init__(self, _):
+ pass
+
+ def get_config(self):
+ return None
+
+ def get_hosts_config_handler(self):
+ return None
+
+ def get_networking_config_handler(self):
+ return None
+
+ def get_hosts_config_handler(self):
+ return None
+
+ def get_caas_config_handler(self):
+ return None
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+def is_virtualized():
+ return False
+
+
+def get_own_hwmgmt_ip():
+ return '1.2.3.4'
+
+
+def flatten_config_data(jsondata):
+ result = {}
+ for key, value in jsondata.iteritems():
+ try:
+ result[key] = json.dumps(value)
+ except Exception as exp:
+ result[key] = value
+ return result
+
+
+def unflatten_config_data(props):
+ propsjson = {}
+ for name, value in props.iteritems():
+ try:
+ propsjson[name] = json.loads(value)
+ except Exception as exp:
+ propsjson[name] = value
+ return propsjson
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+class Error(Exception):
+ pass
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+class Client(object):
+ def get(self, domain, name):
+ pass
+
+ def get_domain(self, domain):
+ pass
+
+ def set(self, domain, name, value):
+ pass
+
+ def get_domains(self):
+ pass
+
+ def delete(self, domain, name):
+ pass
+
+ def delete_domain(self, domain):
+ pass
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+class AlarmHandler(object):
+ def raise_alarm_with_dn(self, alarmid, origin, supplementary=None, timeout=None):
+ pass
+
+ def cancel_alarm_with_dn(self, alarmid, origin, supplementary=None, timeout=None):
+ pass
--- /dev/null
+[tox]
+envlist = py27-pytest,pylint
+setupdir=src
+
+[testenv]
+
+basepython = python2.7
+setenv =
+ COVERAGE_FILE = .coverage{envname}
+
+commands = /bin/cp -R {toxinidir}/test/mocked_dependencies/fm {toxworkdir}/py27-pytest/lib/python2.7/site-packages/
+ /bin/cp -R {toxinidir}/test/mocked_dependencies/cmdatahandlers {toxworkdir}/py27-pytest/lib/python2.7/site-packages/
+ /bin/cp -R {toxinidir}/test/mocked_dependencies/dss {toxworkdir}/py27-pytest/lib/python2.7/site-packages/
+ pytest -vv \
+ --basetemp={envtmpdir} \
+ --pep8 \
+ --cov cmframework \
+ --cov-branch \
+ --cov-report term \
+ --cov-report html:htmlcov \
+ {posargs:.}
+
+deps=pytest
+ mock
+ pytest-cov
+ pytest-pep8
+ pytest-flakes
+ eventlet
+ requests
+ more-itertools==5.0.0
+
+[pytest]
+cache_dir = .pytest-cache
+pep8maxlinelength = 100
+
+[testenv:pylint]
+commands = /bin/cp -R {toxinidir}/test/mocked_dependencies/fm {toxworkdir}/pylint/lib/python2.7/site-packages/
+ /bin/cp -R {toxinidir}/test/mocked_dependencies/cmdatahandlers {toxworkdir}/pylint/lib/python2.7/site-packages/
+ /bin/cp -R {toxinidir}/test/mocked_dependencies/dss {toxworkdir}/pylint/lib/python2.7/site-packages/
+ -pylint --rcfile={toxinidir}/.pylintrc {posargs:src}
+
+deps=pylint==1.7.4
+ astroid==1.6.5
+ pymongo
+ requests
+ routes
+ eventlet
+ redis
+ pika
+ pyyaml
+ prettytable
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+Name: config-manager
+Version: %{_version}
+Release: 1%{?dist}
+Summary: Contains code for the config manager
+License: %{_platform_licence}
+Source0: %{name}-%{version}.tar.gz
+Vendor: %{_platform_vendor}
+BuildArch: noarch
+
+BuildRequires: python
+BuildRequires: python-setuptools
+
+%description
+This RPM contains source code for the config manager.
+
+%prep
+%autosetup
+
+%install
+mkdir -p %{buildroot}/opt/cmframework/activators
+mkdir -p %{buildroot}/opt/cmframework/validators
+mkdir -p %{buildroot}/opt/cmframework/userconfighandlers
+mkdir -p %{buildroot}/opt/cmframework/inventoryhandlers
+mkdir -p %{buildroot}/etc/cmframework/masks.d
+
+mkdir -p %{buildroot}/opt/cmframework/scripts
+cp cmframework/scripts/*.sh %{buildroot}/opt/cmframework/scripts
+cp cmframework/scripts/cmserver %{buildroot}/opt/cmframework/scripts
+cp cmframework/scripts/cmagent %{buildroot}/opt/cmframework/scripts
+cp cmframework/scripts/redis.conf %{buildroot}/opt/cmframework/scripts
+cp cmframework/config/masks.d/default.cfg %{buildroot}/etc/cmframework/masks.d/
+
+mkdir -p %{buildroot}/usr/lib/systemd/system
+cp cmframework/systemd/config-manager.service %{buildroot}/usr/lib/systemd/system
+cp cmframework/systemd/cmagent.service %{buildroot}/usr/lib/systemd/system
+
+mkdir -p %{buildroot}/%{_python_site_packages_path}/cmframework/
+set -e
+cd cmframework/src && python setup.py install --root %{buildroot} --no-compile --install-purelib %{_python_site_packages_path} --install-scripts %{_platform_bin_path} && cd -
+
+cd cmdatahandlers && python setup.py install --root %{buildroot} --no-compile --install-purelib %{_python_site_packages_path} --install-scripts %{_platform_bin_path} && cd -
+
+mkdir -p %{buildroot}/etc/service-profiles/
+cp serviceprofiles/profiles/*.profile %{buildroot}/etc/service-profiles/
+
+cd serviceprofiles/python && python setup.py install --root %{buildroot} --no-compile --install-purelib %{_python_site_packages_path} && cd -
+
+cd hostcli && python setup.py install --root %{buildroot} --no-compile --install-purelib %{_python_site_packages_path} && cd -
+
+%files
+%defattr(0755,root,root,0755)
+/opt/cmframework
+/usr/lib/systemd/system/config-manager.service
+/usr/lib/systemd/system/cmagent.service
+%{_platform_bin_path}/cmserver
+%{_platform_bin_path}/cmcli
+%{_platform_bin_path}/cmagent
+%dir /etc/cmframework/masks.d
+/etc/cmframework/masks.d/default.cfg
+%{_python_site_packages_path}/cmframework*
+%{_python_site_packages_path}/cmdatahandlers*
+%{_python_site_packages_path}/serviceprofiles*
+%{_python_site_packages_path}/cmcli*
+/etc/service-profiles/*
+
+%pre
+
+%post
+ln -sf /opt/cmframework/scripts/inventory.sh /opt/openstack-ansible/inventory/
+chmod -x /usr/lib/systemd/system/config-manager.service
+chmod -x /usr/lib/systemd/system/cmagent.service
+
+%preun
+
+%postun
+
+%clean
+rm -rf %{buildroot}
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#!/usr/bin/env python
+
+PROJECT = 'cmcli'
+
+VERSION = '0.2'
+
+from setuptools import setup, find_packages
+
+setup(
+ name=PROJECT,
+ version=VERSION,
+ description='config-manager CLI',
+ author='Baha Mesleh',
+ author_email='baha.mesleh@nokia.com',
+ platforms=['Any'],
+ scripts=[],
+ provides=[],
+ install_requires=['hostcli'],
+ namespace_packages=['cmcli'],
+ packages=find_packages('src'),
+ include_package_data=True,
+ package_dir={'': 'src'},
+ entry_points={
+ 'hostcli.commands': [
+ 'config-manager show property = cmcli.cm:ShowProperty',
+ 'config-manager list properties = cmcli.cm:ListProperties',
+ 'config-manager delete property = cmcli.cm:DeleteProperty',
+ 'config-manager set property = cmcli.cm:SetProperty',
+ 'config-manager dump-to-file property = cmcli.cm:DumpPropertyToFile'
+ ],
+ },
+ zip_safe=False,
+)
+
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+__import__('pkg_resources').declare_namespace(__name__)
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import logging
+import json
+import os
+import re
+
+from cmdatahandlers.api.configmanager import ConfigManager
+from cmframework.apis import cmmanage
+
+from cliff import command
+from cliff.show import ShowOne
+from cliff.lister import Lister
+from cliff.formatters.table import TableFormatter
+
+
+
+class VerboseLogger(object):
+ def __init__(self, logger):
+ self.logger = logger
+
+ def __call__(self, msg):
+ self.logger.debug(msg)
+
+
+def _add_basic_arguments(parser):
+ parser.add_argument('--cmserver-ip',
+ dest='cmserverip',
+ metavar='IP',
+ required=False,
+ default='config-manager',
+ type=str,
+ action='store')
+ parser.add_argument('--cmserver-port',
+ dest='cmserverport',
+ metavar='PORT',
+ required=False,
+ default=61100,
+ type=str,
+ action='store')
+ parser.add_argument('--client-lib',
+ dest='clientlib',
+ metavar='LIB',
+ required=False,
+ default='cmframework.lib.CMClientImpl',
+ type=str,
+ action='store')
+
+
+class ShowProperty(ShowOne):
+ """A command for showing the value associated with a property"""
+
+ log = logging.getLogger(__name__)
+
+ def get_parser(self, prog_name):
+ parser = super(ShowProperty, self).get_parser(prog_name)
+ _add_basic_arguments(parser)
+ parser.add_argument(
+ 'property',
+ metavar='<property>',
+ help=('Property name'),
+ )
+
+ return parser
+
+ @staticmethod
+ def dumps(data):
+ return json.dumps(data, sort_keys=True, indent=4, separators=(',', ':'))
+
+ def take_action(self, parsed_args):
+ try:
+ logger = VerboseLogger(self.log)
+
+ api = cmmanage.CMManage(parsed_args.cmserverip,
+ parsed_args.cmserverport,
+ parsed_args.clientlib,
+ logger)
+
+ cm_property = parsed_args.property
+
+ data = api.get_properties('')
+
+ d = {}
+ for name, value in data.iteritems():
+ try:
+ d[name] = json.loads(value)
+ except Exception as ex:
+ d[name] = value
+
+ cm = ConfigManager(d)
+ cm.mask_sensitive_data()
+
+ prop = d.get(cm_property)
+ split = None
+ if not prop:
+ split = cm_property.split('.')
+ cm_property = '.'.join(split[:2])
+ prop = d.get(cm_property)
+
+ if prop == '' or prop == None:
+ raise Exception('{} not configured'.format(cm_property))
+
+ d = d[cm_property]
+ if split:
+ if split[2:]:
+ for field in split[2:]:
+ try:
+ d = d[field]
+ except KeyError as ex:
+ raise Exception('{} not found in {}'.format(field, cm_property))
+
+ if isinstance(d, dict):
+ columns = tuple(d.keys())
+ if isinstance(self.formatter, TableFormatter):
+ data = tuple(map(ShowProperty.dumps, d.values()))
+ else:
+ data = tuple(d.values())
+ else:
+ columns = (parsed_args.property, )
+ data = (d, )
+ return (columns, data)
+
+ except Exception:
+ raise
+
+
+class ListProperties(Lister):
+ """A command for showing properties matching some filter"""
+
+ log = logging.getLogger(__name__)
+
+ def get_parser(self, prog_name):
+ parser = super(ListProperties, self).get_parser(prog_name)
+ _add_basic_arguments(parser)
+ parser.add_argument('--matching-filter',
+ dest='filter',
+ metavar='FILTER',
+ required=False,
+ default='.*',
+ type=str,
+ action='store')
+
+ return parser
+
+ def take_action(self, parsed_args):
+ try:
+ logger = VerboseLogger(self.log)
+
+ api = cmmanage.CMManage(parsed_args.cmserverip,
+ parsed_args.cmserverport,
+ parsed_args.clientlib,
+ logger)
+
+ prop_filter = parsed_args.filter
+ data = api.get_properties('')
+
+ header = ('property', 'value')
+
+ columns = ()
+
+ d = {}
+ for name, value in data.iteritems():
+ try:
+ d[name] = json.loads(value)
+ except Exception as ex:
+ d[name] = value
+
+ cm = ConfigManager(d)
+ cm.mask_sensitive_data()
+
+ pattern = re.compile(prop_filter)
+ for name, value in d.iteritems():
+ if not pattern.match(name):
+ continue
+ if isinstance(self.formatter, TableFormatter):
+ try:
+ v = json.dumps(value, sort_keys=True, indent=1, separators=(',', ':'))
+ except Exception:
+ pass
+ else:
+ v = value
+ entry = (name, v)
+ columns = (entry,) + columns
+ if not columns:
+ raise Exception('Not found')
+
+ return (header, columns)
+
+ except Exception:
+ raise
+
+
+class DeleteProperty(command.Command):
+ """A command for deleting a property"""
+
+ log = logging.getLogger(__name__)
+
+ def get_parser(self, prog_name):
+ parser = super(DeleteProperty, self).get_parser(prog_name)
+ _add_basic_arguments(parser)
+
+ parser.add_argument(
+ 'property',
+ metavar='<property>',
+ help=('Property name'),
+ )
+
+ return parser
+
+ def take_action(self, parsed_args):
+ try:
+ logger = VerboseLogger(self.log)
+
+ api = cmmanage.CMManage(parsed_args.cmserverip,
+ parsed_args.cmserverport,
+ parsed_args.clientlib,
+ logger)
+
+
+ api.delete_property(parsed_args.property)
+
+
+ self.app.stdout.write('%s deleted successfully\n' % parsed_args.property)
+
+ except Exception as exp:
+ self.app.stderr.write('Failed with error %s\n' % str(exp))
+
+
+class SetProperty(command.Command):
+ """A command for setting a property"""
+
+ log = logging.getLogger(__name__)
+
+ def get_parser(self, prog_name):
+ parser = super(SetProperty, self).get_parser(prog_name)
+ _add_basic_arguments(parser)
+
+ parser.add_argument(
+ 'property',
+ metavar='<property>',
+ help=('Property name'),
+ )
+
+ parser.add_argument('--value',
+ dest='value',
+ metavar='VALUE',
+ required=False,
+ type=str,
+ action='store')
+
+ parser.add_argument('--file',
+ dest='file',
+ metavar='FILE',
+ required=False,
+ type=str,
+ action='store')
+ return parser
+
+ def take_action(self, parsed_args):
+ try:
+ logger = VerboseLogger(self.log)
+
+ api = cmmanage.CMManage(parsed_args.cmserverip,
+ parsed_args.cmserverport,
+ parsed_args.clientlib,
+ logger)
+
+ if parsed_args.value and parsed_args.file:
+ raise Exception('Either --value or --file needs to be specified')
+
+ if parsed_args.value:
+ api.set_property(parsed_args.property, parsed_args.value)
+ elif parsed_args.file:
+ if not os.path.exists(parsed_args.file):
+ raise Exception('File %s is not valid' % parsed_args.file)
+
+ with open(parsed_args.file, 'r') as f:
+ data = ""
+ try:
+ data = f.read()
+ d = json.loads(data)
+ data = json.dumps(d)
+ except Exception as exp:
+ pass
+
+ api.set_property(parsed_args.property, data)
+ else:
+ raise Exception('--value or --file needs to be specified')
+
+
+ self.app.stdout.write('%s set successfully\n' % parsed_args.property)
+
+ except Exception as exp:
+ self.app.stderr.write('Failed with error %s\n' % str(exp))
+
+
+class DumpPropertyToFile(command.Command):
+ """A command for dumping property value to a file"""
+
+ log = logging.getLogger(__name__)
+
+ def get_parser(self, prog_name):
+ parser = super(DumpPropertyToFile, self).get_parser(prog_name)
+ _add_basic_arguments(parser)
+
+ parser.add_argument(
+ 'property',
+ metavar='<property>',
+ help=('Property name'),
+ )
+
+ parser.add_argument(
+ 'file',
+ metavar='<FILE>',
+ help=('Filename'),
+ )
+
+ return parser
+
+ def take_action(self, parsed_args):
+ try:
+ logger = VerboseLogger(self.log)
+
+ api = cmmanage.CMManage(parsed_args.cmserverip,
+ parsed_args.cmserverport,
+ parsed_args.clientlib,
+ logger)
+
+ cm_property = parsed_args.property
+ config = api.get_properties('')
+
+ if os.path.exists(parsed_args.file):
+ raise Exception('File %s already exists' % parsed_args.file)
+
+ with open(parsed_args.file, 'w') as f:
+ d = {}
+ if cm_property not in config.keys():
+ raise Exception('Property not found')
+ for name, value in config.iteritems():
+ try:
+ d[name] = json.loads(value)
+ except Exception as ex:
+ d[name] = value
+ cm = ConfigManager(d)
+ cm.mask_sensitive_data()
+ d = d.get(cm_property)
+ try:
+ data = json.dumps(d, sort_keys=True, indent=1, separators=(',', ':'))
+ except Exception as exp:
+ data = d
+ if data == 'null' or data == '""':
+ data = ''
+ f.write(data)
+
+ self.app.stdout.write('Completed successfully\n')
+
+ except Exception as exp:
+ self.app.stderr.write('Failed with error %s\n' % str(exp))
--- /dev/null
+name:base
+description:The basic profile containing services to be run in all nodes
--- /dev/null
+name:caas_master
+description:CAAS master node profile
+inherits:management
--- /dev/null
+name:caas_worker
+description:CAAS worker node profile
+inherits:base
--- /dev/null
+name:compute
+description:Openstack compute profile
+inherits:base
--- /dev/null
+name:controller
+description:Openstack controller profile
+inherits:management
--- /dev/null
+name:management
+description:The base profile for management nodes
+inherits:base
--- /dev/null
+name:storage
+description:The storage nodes profile
+inherits:base
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+__import__('pkg_resources').declare_namespace(__name__)
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import os
+import re
+
+class Profile(object):
+ def __init__(self):
+ self.name = None
+ self.description = None
+ self.inherits = []
+ self.included_profiles = []
+ def __str__(self):
+ return 'name:{}\ndescription:{}\ninherits:{}\nincluded_profiles:{}\n'.format(self.name, self.description, self.inherits, self.included_profiles)
+
+class Profiles(object):
+ def __init__(self, location='/etc/service-profiles/'):
+ self.location = location
+ self.profiles = {}
+ self._load_profiles()
+
+ def _load_profiles(self):
+ files = self._get_profiles_files()
+ for f in files:
+ self._profile_from_file(f)
+
+ #update the included profiles
+ for name, profile in self.profiles.iteritems():
+ included_profiles = []
+ self._update_included_profiles(profile, included_profiles)
+ profile.included_profiles = included_profiles
+
+ def _update_included_profiles(self, profile, included_profiles):
+ included_profiles.append(profile.name)
+ for b in profile.inherits:
+ self._update_included_profiles(self.profiles[b], included_profiles)
+
+ def _get_profiles_files(self):
+ files = os.listdir(self.location)
+ pattern = re.compile('.*[.]profile$')
+ result = []
+ for f in files:
+ fullpath = self.location + '/' + f
+ if os.path.isfile(fullpath) and pattern.match(f):
+ result.append(fullpath)
+ return result
+
+ def _profile_from_file(self, filename):
+ profile = Profile()
+ with open(filename) as f:
+ lines=f.read().splitlines()
+ for l in lines:
+ data = l.split(':')
+ if len(data) != 2:
+ raise Exception('Invalid line %s in file %s' % (l, filename))
+ elif data[0] == 'name':
+ profile.name = data[1]
+ elif data[0] == 'description':
+ profile.description = data[1]
+ elif data[0] == 'inherits':
+ profile.inherits = data[1].split(',')
+ else:
+ raise Exception('Invalid line %s in file %s' % (l, filename))
+ self.profiles[profile.name] = profile
+
+ def get_included_profiles(self, name):
+ return self.profiles[name].included_profiles
+
+ def get_profiles(self):
+ return self.profiles
+
+ def get_children_profiles(self, name):
+ ret = []
+ for pfname, profile in self.profiles.iteritems():
+ if name in profile.inherits:
+ ret.append(pfname)
+ return ret
+
+ def get_service_profiles(self):
+ #profiles_files = self._get_profiles_files()
+ #profiles_names = [profile_file[:-len('.profile')] for profile_file in profiles_files]
+ profiles_names = self.profiles.keys()
+
+ return profiles_names
+
+ def get_node_service_profiles(self, name):
+ path = '/etc/service-profiles/config.ini'
+ profiles = []
+ with open(path) as f:
+ content = f.readlines()
+ for line in content:
+ tmp = line.strip()
+ node = tmp.split(':')[0]
+ if node == name:
+ profiles = tmp.split(':')[1].split(',')
+ break
+ return profiles
+
+ @staticmethod
+ def get_management_service_profile():
+ return 'management'
+
+ @staticmethod
+ def get_base_service_profile():
+ return 'base'
+
+ @staticmethod
+ def get_controller_service_profile():
+ return 'controller'
+
+ @staticmethod
+ def get_caasmaster_service_profile():
+ return 'caas_master'
+
+ @staticmethod
+ def get_caasworker_service_profile():
+ return 'caas_worker'
+
+ @staticmethod
+ def get_compute_service_profile():
+ return 'compute'
+
+ @staticmethod
+ def get_storage_service_profile():
+ return 'storage'
+
+
+if __name__ == '__main__':
+ import sys
+ import traceback
+ import argparse
+
+ parser = argparse.ArgumentParser(description='Test service profiles',
+ prog=sys.argv[0])
+
+ parser.add_argument('--location',
+ required=True,
+ metavar='LOCATION',
+ dest='location',
+ help='The location for service profile files',
+ type=str,
+ action='store')
+
+ parser.add_argument('--get-included-profiles',
+ dest='get_included_profiles',
+ help='Get the profiles included in some profile name',
+ action='store_true')
+
+ parser.add_argument('--get-all-profiles',
+ help='Get the profiles list',
+ dest='get_all_profiles',
+ action='store_true')
+
+ parser.add_argument('--get-children-profiles',
+ dest='get_children_profiles',
+ help='Get the children of a profile',
+ action='store_true')
+
+ parser.add_argument('--name',
+ metavar='NAME',
+ dest='name',
+ help='The name of the profile',
+ type=str,
+ action='store')
+ try:
+ args = parser.parse_args(sys.argv[1:])
+ location = args.location
+ profiles = Profiles(location)
+ if args.get_included_profiles or args.get_children_profiles:
+ if not args.name:
+ raise Exception('Missing profile name')
+
+ if args.get_included_profiles:
+ included_profiles = profiles.get_included_profiles(args.name)
+ print('Included profiles')
+ for p in included_profiles:
+ print(p)
+ if args.get_children_profiles:
+ children_profiles = profiles.get_children_profiles(args.name)
+ print('Children profiles')
+ for p in children_profiles:
+ print(p)
+ elif args.get_all_profiles:
+ all = profiles.get_profiles()
+ for name, p in all.iteritems():
+ print(p)
+ except Exception as exp:
+ print('Failed with error %s' % exp)
+ traceback.print_exc()
+ sys.exit(1)
--- /dev/null
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from setuptools import setup, find_packages
+setup(
+ name='serviceprofiles',
+ version='1.0',
+ license='Apache-2.0',
+ long_description='serviceprofiles',
+ author='Baha Mesleh',
+ author_email='baha.mesleh@nokia.com',
+ namespace_packages=['serviceprofiles'],
+ packages=find_packages(),
+ include_package_data=True,
+ url='gerrite1.ext.net.nokia.com:8282/service-profiles',
+ description='service profiles library',
+ zip_safe=False,
+ )
--- /dev/null
+---
+# yamllint disable rule:comments rule:comments-indentation rule:line-length
+
+# Copyright 2019 Nokia
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+### Version numbering:
+### X.0.0
+### - Major structural changes compared to the previous version.
+### - Requires all users to update their user configuration to
+### the new template
+### a.X.0
+### - Significant changes in the template within current structure
+### (e.g. new mandatory attributes)
+### - Requires all users to update their user configuration according
+### to the new template (e.g. add new mandatory attributes)
+### a.b.X
+### - Minor changes in template (e.g. new optional attributes or
+### changes in possible values, value ranges or default values)
+### - Backwards compatible
+version: 2.0.0
+
+### Cloud name can consist of lower case letters, digits and dash (-).
+### Name must start and end with a letter or a digit.
+name: <VALUE>
+
+### Cloud description
+description: <VALUE>
+
+### Time related configuration
+time:
+ ### A list of NTP server IP addresses.
+ ntp_servers: [VALUE1, VALUE2, ...]
+
+ ### linux time zone name (e.g. Europe/Helsinki or Asia/Shanghai)
+ zone: <VALUE>
+
+ ### supported values for authentication method of NTP:
+ ### crypto, symmetric, none
+ auth_type: none
+
+ ### If you are using authenticated NTP you must provide the url of the keys used for authentication
+ serverkeys_path:
+
+### User related configuration
+users:
+ ### Admin user details
+ admin_user_name: <VALUE>
+ ### Example how to create SHA512 password hash that can be given as
+ ### the admin password:
+ ### python -c "from passlib.hash import sha512_crypt; import getpass; print sha512_crypt.using(rounds=5000).hash(getpass.getpass())"
+ admin_user_password: <VALUE>
+
+ ### User details for the initial user (gets user_management_admin role)
+ initial_user_name: <VALUE>
+ initial_user_password: <VALUE>
+
+ ### For CaaS deployments
+ ### keystone admin users password (at least 8 characters; at least one letter)
+ admin_password: <VALUE>
+
+### Networking configuration
+networking:
+ ### A list of DNS server IP addresses.
+ ### Max two addresses supported.
+ dns: [VALUE1, VALUE2]
+
+ ### Optional. Default network device mtu.
+ ### Valid value range: 1280 - 9000
+ ### When not set, defaults to 1500
+ #mtu: <VALUE>
+
+ infra_external:
+ ### Optional network mtu
+ ### If not defined default value is used.
+ #mtu: <VALUE>
+
+ ### Network domains
+ network_domains:
+ ### User defined name for network domain
+ rack-1:
+ ### Network address in CIDR format
+ cidr: <VALUE>
+
+ ### Optional vlan id
+ #vlan: <VALUE>
+
+ ### IP address of the gateway for default route
+ gateway: <VALUE>
+
+ ### Range for external IPs
+ ### - First IP address of the range is reserved for vip
+ ### (Public API access)
+ ### - following addresses are reserved for cmanagement hosts
+ ### (one address per management hosts)
+ ip_range_start: <VALUE>
+ ip_range_end: <VALUE>
+
+ ### Optional.
+ ### This configuration is required if there are storage hosts in
+ ### the configuration. This network is used for OSD Replication.
+ #infra_storage_cluster:
+ ### Optional network mtu
+ ### If not defined default value is used.
+ #mtu: <VALUE>
+
+ ### Network domains
+ #network_domains:
+ ### User defined name for network domain
+ #rack-1:
+ ### Network address in CIDR format (e.g. 192.168.4.0/26)
+ #cidr: <VALUE>
+
+ ### Optional vlan id
+ #vlan: <VALUE>
+
+ ### Optional IP range from the CIDR to limit IP addresses to use
+ #ip_range_start: <VALUE>
+ #ip_range_end: <VALUE>
+
+ ### Optional static routes
+ #routes:
+ # - {to: <CIDR>, via: <IP>}
+
+ ### This network is used for:
+ ### - Internal communication/API
+ ### - SSH between hosts
+ ### - Internal services
+ ### - NTP between hosts
+ infra_internal:
+ ### Optional network mtu
+ ### If not defined default value is used.
+ #mtu: <VALUE>
+
+ ### Network domains
+ network_domains:
+ ### User defined name for network domain
+ rack-1:
+ ### Network address in CIDR format
+ cidr: 192.168.12.0/26
+
+ ### Optional vlan id
+ #vlan: <VALUE>
+
+ ### Optional IP range from the CIDR to limit IP addresses to use
+ #ip_range_start: <VALUE>
+ #ip_range_end: <VALUE>
+
+ ### Optional static routes
+ #routes:
+ # - {to: 192.168.12.0/22, via: 192.168.12.1}
+ ### Use above structure for all the other network domains
+ #rack-2:
+ #cidr: 192.168.12.64/26
+ #vlan: <VALUE>
+ #ip_range_start: 192.168.12.68
+ #ip_range_end: 192.168.12.126
+ #routes:
+ # - {to: 192.168.12.0/22, via: 192.168.12.65}
+
+ ### Provider networks
+ ### Provider network to physical interface mapping is done
+ ### in the network profile configuration
+ #provider_networks:
+ ### Any number of provider network names
+ #<provider_network_name1>:
+ ### Optional. Set provider network mtu.
+ ### If not defined default value is used.
+ #mtu: <VALUE>
+
+ ### Provider network vlan ranges
+ #vlan_ranges: "<VID_START1>:<VID_END1>,<VID_START2>:<VID_END2>,..."
+
+ ### Use above structure for all the other provider networks
+ #<provider_network_name2>:
+ #...
+
+### Needed for non-CaaS deployments
+#openstack:
+ ### keystone admin user password (at least 8 characters; at least one letter)
+ #admin_password: <VALUE>
+
+### Caas configuration
+caas:
+ ### This parameter globally sets a maximum allowed writable disk space quota for every container,
+ ### on all caas related hosts. The quota physically forbids any containers from storing data more
+ ### than the allowed size on its own rootfs.
+ ### These ephemeral disks are allocated from the Docker Cinder volume attached to all hosts,
+ ### and as such are limited in size. The quota protects the containers from possible noisy neighbours
+ ### by limiting their maximum consumption, and thus assuring that no one faulty container
+ ### can eat up the disk space of a whole container execution host.
+ ### Mandatory
+ docker_size_quota: "2G"
+
+ ### This parameter, if provided, will be set into the configuration of the CaaS cluster's
+ ### internal DNS server's configuration. Whenever a DNS query cannot be served by the default server,
+ ### it will be forwarded to the configured address, regardless which sub-domain the query belongs to.
+ ### Please note, that in case the address points out of the infrastructure,
+ ### connectivity between the infrastructure and the external DNS server needs to be separately set-up.
+ #upstream_nameserver: "10.74.3.252"
+
+ ### This parameter, if provided, will be set into the configuration of the CaaS cluster's
+ ### internal DNS server's configuration. Whenever a DNS query cannot be served by the default server,
+ ### it might be forwarded to the address set into the "stub_domain_ip" parameter.
+ ### However, forwarding only happens if "stub_domain_name" matches the domain name in the DNS query.
+ ### Please note, that in case the address points out of the infrastructure, connectivity between the
+ ### infrastructure and the external DNS server needs to be separately set-up.
+ #stub_domain:
+ # name: "nokia.com"
+ # ip: "10.74.3.252"
+
+ ### This parameter, if provided, controls how long a Helm install procedure waits before exiting with a timeout error.
+ ### Value is interpreted in minutes.
+ #helm_operation_timeout: "900"
+
+ ### The Docker container run-time engine creates a Linux network bridge by default, and provisions
+ ### a /24 IPv4 network on top of it. Even though this bridge is not used within CaaS subsytem,
+ ### the existence of this bridge is not configurable.
+ ### However, in certain customer environments the default IPv4 network of this bridge can collide with
+ ### real customer networks. To avoid IP collision issues in such cases, the application operator can globally set
+ ### the Docker bridge CIDRs of all host via this parameter.
+ #docker0_cidr: "172.17.0.1/16"
+
+ ### Mandatory parameter. All the infrastructure's HTTP servers are secured with TLS.
+ ### The certificates of the servers are created in infrastructure deployment time, and are signed by an externally provided CA certificate.
+ ### This CA certificate can be configured by setting its encrypted format into this configuration parameter.
+ ### Due to CBAM limitation the value of this parameters shall be provided as a one-element list in JSON format
+ ### e.g. ["U2FsdGVkX1+iaWyYk3W01IFpfVdughR5aDKo2NpcBw2USt.."]
+ encrypted_ca: '["<ENCRYPTED_CA>"]'
+
+ ### Manadatory parameter. All the infrastructure's HTTP servers are secured with TLS.
+ ### The certificates of the servers are created in infrastructure deployment time, and are signed by an externally provided CA certificate.
+ ### This CA certificate can be configured by setting its encrypted format into the "encrypted_CA" configuration parameter.
+ ### The key which can be used to decrypt this CA certificate shall be configured into this configuration parameter, but also encrypted.
+ ###This key shall be encrypted by the super-secret, static key, known only by infrastructure developers, and cloud operators.
+ ### Due to CBAM limitation the value of this parameters shall be provided as a one-element list in JSON format
+ ### e.g. ["U2FsdGVkX1+WlNST+W.."]
+ encrypted_ca_key: '["<ENCRYPTED_CA_KEY>"]'
+
+
+### Storage configuration
+storage:
+ #backends:
+ ### Configuration of supported storage backends.
+ ### At least one backend must be onfigured and only one backend can be enabled.
+ ### If more than one backend is configured then one should be enabled (enabled:true)
+ ### and the others should be disabled (enabled: false).
+
+ #ceph:
+ ### The ceph can be enbled only in a multi node configuration.
+ #enabled: <true/false>
+
+ ### The OSD replica count.
+ ### The number of replicas for objects in the pool.
+ ### Valid value range for any production environment: 2 - 3
+ ### (for testing purposes only, in environments with very limited
+ ### storage resource, value 1 can be used as well)
+ ### Required if there are ceph nodes.
+ #osd_pool_default_size: <VALUE>
+
+
+### Network profiles
+network_profiles:
+ ### Users can define multiple network profiles depending on the hardware.
+ #<profile_name>:
+ ### Compulsory if bonding interfaces used for infra networks.
+ ### Bonding options for linux bonding interfaces used for infra
+ ### networks.
+ ### Supported options: "mode=lacp" and "mode=active-backup"
+ ### In "mode=lacp" both nics are active simultaniously.
+ ### In "mode=active-backup" only one slave in the bond is active and
+ ### the another slave becomes active only if the active slave fails.
+ #linux_bonding_options: <VALUE>
+
+ ### Optional bonding interfaces
+ #bonding_interfaces:
+ ### Any number of bonding interface names.
+ ### Bonding interface name syntax must be bond[n]
+ ### where n is a number.
+ ### Numbers in bonding interface names must be
+ ### consecutive natural numbers starting from 0
+ ### (bond0, bond1, bond2, ...)
+ ###
+ ### Value is a list of at least two physical interface names
+ ### (e.g. bond0: [eno3, eno4])
+ #<bonding interface name>: [<VALUE1>, <VALUE2>, ...]
+
+ ### Interface-subnet mapping
+ ### Any number of (name: value) pairs to map interfaces
+ ### (bonding or physical interface name) to subnets
+ ### Value is list of subnets
+ ### (e.g. bond0: [infra_internal, infra_storage_cluster] or
+ ### eno3: [infra_external])
+ ### An interface can be mapped to at most one non-vlan subnet
+ interface_net_mapping:
+ #<interface_name>: [<VALUE1>, <VALUE2>, ...]
+
+ ### Optional provider network interface
+ #provider_network_interfaces:
+ ### Provider network physical interface.
+ ### Either Ethernet or bonding interface.
+ #<interface_name1>:
+ ### Provider networks on this interface.
+ ### Provider networks must be defined also in the networking:
+ ### provider_networks: configuration.
+ #provider_networks: [<VALUE1>,<VALUE2>,...]
+ ### Use above structure for all the provider network interfaces
+ ### in this profile
+ #<interface_name2>:
+ #...
+
+ ### Optional SR-IOV provider networks
+ #sriov_provider_networks:
+ ### Provider network name.
+ ### Must be defined also in the
+ ### networking: provider_networks: configuration.
+ #<provider_network_name1>:
+ ### SR-IOV physical function interfaces
+ ### Multiple Ethernet interfaces can be mapped to implement one
+ ### logical network.
+ ### SR-IOV interfaces can be used also for the infra networks
+ ### but only if network card type supports that
+ ### (for example Mellanox ConnectX-4 Lx
+ ### does and Intel Niantic doesn't). Another restriction is that
+ ### bond option cannot be "mode=lacp" if SR-IOV interfaces are
+ ### also bonding slave interfaces.
+ #interfaces: [<VALUE1>, <VALUE2>, ...]
+
+ ### Optional VF count per physical PF interface
+ ### If this parameter is not defined, default is to create
+ ### maximum supported amount of VF interfaces. In case of
+ ### Mellanox NIC (mlx5_core driver) given VF count will be
+ ### configured to the NIC HW as a maximum VF count.
+ #vf_count: <COUNT>
+
+ ### Optional VF trusted mode setting
+ ### If enabled, PF can accept some priviledged operations from
+ ### the VF. See the NIC manufacturer documentation for more
+ ### details.
+ ### Default: false
+ #trusted: [true|false]
+ ### Use above structure for all the SR-IOV provider networks in
+ ### this profile
+ #<provider_network_name2>
+ #...
+
+### Performance profiles
+performance_profiles:
+ #<profile_name>:
+ ### The parameters specified here are affected by the type
+ ### of network profile selected for the node as follows:
+ ### The following types are supported:
+ ### SR-IOV: no mandatory parameters, but following can be used:
+ ### - default_hugepagesz
+ ### - hugepagesz
+ ### - hugepages
+
+ ### Configuration for huge page usage.
+ ### Notice: Huge page values must be in balance with RAM available
+ ### in any node.
+ ###
+ ### Default huge page size. Valid values are 2M and 1G.
+ #default_hugepagesz: <VALUE>
+ ### Huge page size selection parameter. Valid values are 2M and 1G.
+ #hugepagesz: <VALUE>
+ ### The number of allocated persistent huge pages
+ #hugepages: <VALUE>
+
+ ### Host CPU allocations.
+ ### Any host CPUs that are not allocated for some specific purpose
+ ### here will be automatically assigned by the system:
+ ### - All remaining CPUs are allocated for the host platform.
+
+ ### Optional. Allocate CPUs for the host platform.
+ ### The configured counts determine the number of full CPU cores to
+ ### allocate from each specified NUMA node. If hyperthreading is
+ ### enabled, all sibling threads are automatically grouped together
+ ### and counted as one CPU core. The actual configurable range
+ ### depends on target hardware CPU topology and desired performance
+ ### configuration.
+ ### Notice: The host platform must always have have at least one CPU
+ ### core from NUMA node 0.
+ #platform_cpus:
+ #numa0: <COUNT>
+ #numa1: <COUNT>
+
+### Storage profiles
+storage_profiles:
+ ### The storage_profiles section name is part of mandatory configuration.
+ ###
+ ### There must always be at least one profile defined when ceph or lvm
+ ### have been configured and enabled as the backend in the storage section.
+ ### This profile represents the enabled backend in question.
+ ###
+ ### In addition the user can optionally configure storage instance profiles
+ ### in this section.
+
+ #<profile_name>:
+ ### Name of the storage backend. The allowed values for the backend are
+ ### - ceph
+ ### - bare_lvm
+ ###
+ #backend: <VALUE>
+
+ ### Backend specific attributes - see examples of supported backend
+ ### specific attributes in the following storage profile templates.
+ #...
+
+ #ceph_backend_profile:
+ ### Mandatory
+ ### A storage profile for ceph backend. This storage profile is linked
+ ### to all of the storage hosts. The ceph profile is possible only with
+ ### a multihost configuration with three (3) management hosts.
+ ###
+ #backend: ceph
+
+ ### Mandatory
+ ### Number of devices that should be used as osd disks in one node.
+ ### This is a mandatory attribute for ceph storage hosts.
+ ### Max number of ceph osd disks is 3.
+ #nr_of_ceph_osd_disks: <VALUE>
+
+ ### Optional
+ ### The share ratio between the Openstack & CaaS subsystems for
+ ### the available Ceph storage. Expected to be in ratio format (A:B),
+ ### where the first number is for Openstack, the second one is for CaaS subsystem.
+ ### Always quote the value! Default value is "1:0".
+ #ceph_pg_openstack_caas_share_ratio: "<VALUE>"
+
+ #bare_lvm_profile
+ ### Mandatory
+ ### A storage profile to create bare lvm volumes.
+ ###
+ ### This profile can be used to create an LVM volume that will be
+ ### available under the defined directory for any further use.
+ ###
+ ### This profile is mandatory for caas_worker hosts and should be
+ ### mounted to /var/lib/docker.
+ ###
+ #backend: bare_lvm
+
+ ### Mandatory
+ ### This paramater contains which partitions to be used
+ ### for instance volume group.
+ #lvm_instance_storage_partitions: [<VALUE1>, <VALUE2>, ...]
+
+ ### Mandatory
+ ### This paramater defines bare_lvm how much space should take
+ ### from LVM pool.
+ ### Note that this option left for compatibility reasons, actual value
+ ### dynamically calculated.
+ ### calculated.
+ #bare_lvm_storage_percentage: <VALUE>
+
+ ### Mandatory
+ ### This parameter contains the name for the created LVM volume.
+ #lv_name: <VALUE>
+
+ ### Mandatory
+ ### This parameter contains the directory where to mount
+ ### the backend of this profile.
+ #mount_dir: <VALUE>
+
+ ### Optional
+ ### This parameter contains the mount options used to mount
+ ### the backend. The format must be a valid fstab format.
+ ### By default it is empty.
+ #mount_options: <VALUE>
+
+host_os:
+ ### The value of this parameter is used to protect the entire GRUB 2 menu structure of all the infrastructure nodes.
+ ### The configured value should be a properly salted PBKDF2 (Password-Based Key Derivation Function 2) hash.
+ ### Interactive tool "grub2-mkpasswd-pbkdf2" can be used to create the hash.
+ ### Operators will be only able to make changes in the GRUB menu, if the
+ ### hashed version of the typed-in password matches with the value of this parameter.
+ ###
+ #grub2_password: "<VALUE>"
+ ### User lockout parameters are set with failed_login_attempts (default is 5)
+ ### and lockout_time (default is 300 seconds (5 minutes))
+ #failed_login_attempts: <VALUE>
+ #lockout_time: <VALUE>
+
+### Cloud hosts
+hosts:
+ #<node-name>:
+ ### The service profiles for this node. Valid values are the following:
+ ### management/base/storage/caas_master/caas_worker
+ ### Currently supported service profile combinations:
+ ### 1 Any permutations of: management/base/storage e.g: [ manangement, storage ]
+ ### 2 Either or both [management, caas_master] e.g.: [ management, caas_master ]
+ ### 3 caas_worker can't be combined with any other profile: e.g.: [ caas_worker ]
+ service_profiles: [<VALUE1>, <VALUE2>, ...]
+
+ ### The network profiles for this node, the value used in the list
+ ### should match a profile from the network_profiles section.
+ ### Only one network profile per host supported at the moment.
+ network_profiles: [profile1]
+
+ ### The storage profiles for this node, the value used in the list
+ ### should match a profile from the storage_profiles section.
+ #storage_profiles: [profile1]
+
+ ### The performance profiles for this node, the value used in the list
+ ### should match a profile from the performance_profiles section.
+ ### Only one performance profile per host supported at the moment.
+ #performance_profiles: [profile1]
+
+ ### The kubernetes label set of the node, you can define an arbitrary set of key-value pairs.
+ ### These key-value pairs will be provisioned to the corresponding
+ ### Kubernetes node object as kubernetes labels.
+ ### Optional parameter, only interpreted when the node has a CaaS subsystem related service profile.
+ ### For any other node this attribute will be silently ignored.
+ ### The keys under "labels" can be anything, except: 'name', 'nodetype', 'nodeindex', 'nodename'
+ ### These labels are reserved for infrastructure usage
+ #labels:
+ # type: "performance"
+ # cpu: "turboboost"
+ # hyperthreading: "off"
+ # ...
+
+ ### Network domain for this node
+ ### Value should match some network domain in networking section.
+ network_domain: rack-1
+
+ ### HW management (e.g. IPMI or iLO) address and credentials
+ hwmgmt:
+ address: <VALUE>
+ user: <VALUE>
+ password: <VALUE>
+
+ ### Optional parameter needed for virtual deployment to identify the
+ ### nodes the mac address for the provisioning interface
+ #mgmt_mac: [<VALUE1>, <VALUE2>, ...]
+
+...
+
Code Review / ta / config-manager.git / commitdiff