From: Baha Mesleh Date: Wed, 20 Mar 2019 08:12:14 +0000 (+0200) Subject: Initial commit X-Git-Url: https://gerrit.akraino.org/r/gitweb?p=ta%2Fdistributed-state-server.git;a=commitdiff_plain;h=bd5a0a173f1ae9c64782fbf47565cc26ed23b448 Initial commit Change-Id: I8ecf950bc187b3fac82dd7adfef359485778b158 Signed-off-by: Baha Mesleh --- diff --git a/.gitreview b/.gitreview new file mode 100644 index 0000000..f70bc43 --- /dev/null +++ b/.gitreview @@ -0,0 +1,5 @@ +[gerrit] +host=gerrit.akraino.org +port=29418 +project=ta/distributed-state-server +defaultremote=origin diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..d645695 --- /dev/null +++ b/LICENSE @@ -0,0 +1,202 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/README.md b/README.md new file mode 100644 index 0000000..e69de29 diff --git a/distributed-state-server.spec b/distributed-state-server.spec new file mode 100644 index 0000000..da71473 --- /dev/null +++ b/distributed-state-server.spec @@ -0,0 +1,65 @@ +# Copyright 2019 Nokia + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +Name: distributed-state-server +Version: %{_version} +Release: 1%{?dist} +Summary: Distributed State Server +Group: %{_platform_group} +License: %{_platform_license} +Source0: %{name}-%{version}.tar.gz +Vendor: %{_platform_vendor} + +BuildArch: noarch + +BuildRequires: python +BuildRequires: python-setuptools + +Requires: etcd +Requires: python2-python-etcd +Requires: python-dns + + +%description +This RPM contains source code for the distributed state server and its plugins + +%prep +%autosetup + +%install +mkdir -p %{buildroot}/%{_python_site_packages_path}/dss +mkdir -p %{buildroot}/opt/dss-server/plugins +cd src && python setup.py install --root %{buildroot} --no-compile --install-purelib %{_python_site_packages_path} --install-scripts %{_platform_bin_path} && cd - + +cp plugins/*.py %{buildroot}/opt/dss-server/plugins/ + + +%files +%{_python_site_packages_path}/dss* +%{_python_site_packages_path}/dss* +%{_platform_bin_path}/dsscli +%{_platform_bin_path}/dss +/opt/dss-server/plugins/*.py* + +%pre + +%post +echo "distributed-state-server succesfully installed" + + +%preun + +%postun + +%clean +rm -rf %{buildroot} diff --git a/docs/dss.asciio b/docs/dss.asciio new file mode 100644 index 0000000..544c536 Binary files /dev/null and b/docs/dss.asciio differ diff --git a/docs/dss.txt b/docs/dss.txt new file mode 100644 index 0000000..6020c60 --- /dev/null +++ b/docs/dss.txt @@ -0,0 +1,56 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + + + .--------------------------------------------------------------------------. + | .-------------. .------------. .---------. | + | | <> | | <> |<---------| <> | | + | | dssserver | | dssclient | | dsscli | | + | '-------------' '------------' '---------' | + | | | | + | | | set | + | | v get | + | '--------------> O uds (json-rpc) get-domain | + | ^ get-domains | + | | delete | + | ___ | delete-domain | + | | |\ read .------------. | + | | '-|<-----------| <> | | + | | | | dssserver | | + | |_____| '------------' | + | config.ini | | + | .----------'-----------. | + | | load-plugin | | + | | | | + | | | | + | v v | + | .-------------. .-------------. | + | for multi | <> | | <> | for single | + | management | etcd-plugin | | file-plugin | management | + | '-------------' '-------------' | + | | | | + | | | | + | | | | + | | | | + | v __v | + | .------------. | |\ | + | | etcd | | '-| | + | '------------' | | | + | |_____| | + | fs | + | | + | | + | management-x | + '--------------------------------------------------------------------------' diff --git a/plugins/dssetcd.py b/plugins/dssetcd.py new file mode 100644 index 0000000..049dff8 --- /dev/null +++ b/plugins/dssetcd.py @@ -0,0 +1,121 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +from dss.api import dss_plugin +from dss.api import dss_error + +import ConfigParser +import logging +import os +import etcd + +class dssetcd(dss_plugin.DSSPlugin): + """ + Read the ini file. The structure is as follows + [etcd] + host = controller-1 + port = 2380 + """ + def __init__(self, config_file): + super(dssetcd, self).__init__(config_file) + self.host = None + self.port = None + self.connected = False + try: + config = ConfigParser.ConfigParser() + config.read([config_file]) + self.host = config.get('etcd', 'host') + self.port = int(config.get('etcd', 'port')) + self._connect() + except Exception as exp: + pass + + + def _connect(self): + try: + self.client = etcd.Client(self.host, self.port) + self.connected = True + except Exception as exp: + self.connected = False + raise dss_error.Error(exp) + + def get(self, domain, name): + if not self.connected: + self._connect() + + try: + value = self.client.read('/'+domain+'/'+name) + return value.value + except Exception as exp: + raise dss_error.Error(exp) + + def get_domain(self, domain): + if not self.connected: + self._connect() + + ret = {} + try: + values = self.client.read('/'+domain) + for value in values._children: + if 'dir' not in value and 'key' in value: + k = value['key'] + n = k.split('/')[2] + v = value['value'] + ret[n] = v + return ret + except Exception as exp: + raise dss_error.Error(exp) + + def get_domains(self): + if not self.connected: + self._connect() + ret = [] + try: + domains = self.client.read('/') + for domain in domains._children: + if 'dir' in domain and domain['dir'] and 'key' in domain: + d = domain['key'] + v = d.split('/')[1] + ret.append(v) + return ret + except Exception as exp: + raise dss_error.Error(exp) + + def set(self, domain, name, value): + if not self.connected: + self._connect() + + try: + self.client.write('/'+domain+'/'+name, value) + except Exception as exp: + raise dss_error.Error(exp) + + def delete(self, domain, name): + if not self.connected: + self._connect() + + try: + self.client.delete('/'+domain+'/'+name) + except Exception as exp: + raise dss_error.Error(exp) + + def delete_domain(self, domain): + if not self.connected: + self._connect() + + try: + self.client.delete('/'+domain, recursive=True, dir=True) + except Exception as exp: + raise dss_error.Error(exp) diff --git a/plugins/dssfile.py b/plugins/dssfile.py new file mode 100644 index 0000000..841e566 --- /dev/null +++ b/plugins/dssfile.py @@ -0,0 +1,147 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +from dss.api import dss_plugin +from dss.api import dss_error + +import ConfigParser +import logging +import os + +class dssfile(dss_plugin.DSSPlugin): + """ + Read the ini file. The structure is as follows + [default] + directory = + """ + def __init__(self, config_file): + super(dssfile, self).__init__(config_file) + self.directory = None + try: + config = ConfigParser.ConfigParser() + config.read([config_file]) + self.directory = config.get('default', 'directory') + except Exception as exp: + raise dss_error.Error('Failed to parse configuration, got error %s' % exp) + + if not os.path.isdir(self.directory): + raise dss_error.Error('%s is not a valid directory name' % self.directory) + + def read_file(self, domain): + filename = self.directory + '/' + domain + if not os.path.isfile(filename): + raise dss_error.Error('Key not found %s' % domain) + + try: + with open(filename) as f: + return f.read().splitlines() + except Exception as exp: + raise dss_error.Error('Failed with error %s' % exp) + + def write_file(self, domain, lines): + filename = self.directory + '/' + domain + try: + with open(filename, 'w') as f: + for line in lines: + f.write(line+'\n') + except Exception as exp: + raise dss_error.Error('Failed with error %s' % exp) + + def get(self, domain, name): + lines = self.read_file(domain) + try: + for line in lines: + index=line.find('=') + part0 = line[:index] + part1 = line[index+1:] + if name == part0: + return part1 + except Exception as exp: + raise dss_error.Error('Failed with error %s' % exp) + + raise dss_error.Error('Key not found %s/%s' % (domain, name)) + + def get_domain(self, domain): + lines = self.read_file(domain) + ret = {} + for line in lines: + index=line.find('=') + part0 = line[:index] + part1 = line[index+1:] + ret[part0] = part1 + return ret + + def get_domains(self): + ret = [] + try: + entries = os.listdir(self.directory) + for entry in entries: + filename = self.directory + '/' + entry + if os.path.isfile(filename): + ret.append(entry) + if not ret: + raise dss_error.Error('No domains found') + + return ret + except Exception as exp: + raise dss_error.Error('Failed with error %s' % exp) + + def set(self, domain, name, value): + filename = self.directory + '/' + domain + lines = [] + if os.path.isfile(filename): + lines = self.read_file(domain) + try: + found = False + for index, line in enumerate(lines): + i=line.find('=') + part0 = line[:i] + part1 = line[i+1:] + if part0 == name: + found = True + lines[index] = name+'='+value + break + if not found: + lines.append(name+'='+value) + self.write_file(domain, lines) + except Exception as exp: + raise dss_error.Error('Failed with error %s' % exp) + + def delete(self, domain, name): + lines = self.read_file(domain) + found_index = -1 + try: + for index, line in enumerate(lines): + i=line.find('=') + part0 = line[:i] + part1 = line[i+1:] + if part0 == name: + found_index = index + break + if found_index != -1: + del lines[found_index] + self.write_file(domain, lines) + except Exception as exp: + raise dss_error.Error('Failed with error %s' % exp) + + if found_index == -1: + raise dss_error.Error('%s/%s not found' % (domain, name)) + + def delete_domain(self, domain): + filename = self.directory + '/' + domain + if os.path.isfile(filename): + os.remove(filename) + else: + raise dss_error.Error('Invalid domain %s' % domain) diff --git a/src/__init__.py b/src/__init__.py new file mode 100644 index 0000000..1383fde --- /dev/null +++ b/src/__init__.py @@ -0,0 +1,16 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +__import__('pkg_resources').declare_namespace(__name__) diff --git a/src/dss/__init__.py b/src/dss/__init__.py new file mode 100644 index 0000000..f035b4a --- /dev/null +++ b/src/dss/__init__.py @@ -0,0 +1,15 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + diff --git a/src/dss/api/__init__.py b/src/dss/api/__init__.py new file mode 100644 index 0000000..f035b4a --- /dev/null +++ b/src/dss/api/__init__.py @@ -0,0 +1,15 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + diff --git a/src/dss/api/dss_delete_domain_rpc.py b/src/dss/api/dss_delete_domain_rpc.py new file mode 100644 index 0000000..2ace9b2 --- /dev/null +++ b/src/dss/api/dss_delete_domain_rpc.py @@ -0,0 +1,39 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +''' +Delete Domain RPC +Req: + { + "domain": + } + +Reply: +{} +''' +class DeleteDomainRPC(object): + @staticmethod + def get_name(): + return "delete-domain" + + @staticmethod + def create_req_payload(domain): + payload = {} + payload['domain'] = domain + return payload + + @staticmethod + def get_domain_from_req_payload(payload): + return payload['domain'] diff --git a/src/dss/api/dss_delete_rpc.py b/src/dss/api/dss_delete_rpc.py new file mode 100644 index 0000000..47f2dd5 --- /dev/null +++ b/src/dss/api/dss_delete_rpc.py @@ -0,0 +1,45 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +''' +Delete RPC +Req: + { + "domain": , + "name": + } + +Reply: +{} +''' +class DeleteRPC(object): + @staticmethod + def get_name(): + return "delete" + + @staticmethod + def create_req_payload(domain, name): + payload = {} + payload['domain'] = domain + payload['name'] = name + return payload + + @staticmethod + def get_domain_from_req_payload(payload): + return payload['domain'] + + @staticmethod + def get_name_from_req_payload(payload): + return payload['name'] diff --git a/src/dss/api/dss_error.py b/src/dss/api/dss_error.py new file mode 100644 index 0000000..221d4c0 --- /dev/null +++ b/src/dss/api/dss_error.py @@ -0,0 +1,57 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import sys + + +class Error(Exception): + def __init__(self, description): + self.description = description + + def get_description(self): + return self.description + + def __str__(self): + return '%s' % self.description + +def handle_exceptions(func): + def wrapper(self, *arg, **kwargs): + try: + return func(self, *arg, **kwargs) + except Error as exp: + raise + except Exception as exp: + raise Error(str(exp)) + return wrapper + + +if __name__ == '__main__': + class Test: + @handle_exceptions + def test_raise(self): + raise Exception('Some error') + + try: + test = Test() + test.test_raise() + except Error as exp: + print('Got exception %s' % exp) + + try: + raise Error(sys.argv[1]) + except Error as error: + print('Got exception %s' % str(error)) + except Exception as exp: + print('Got exception %s' % str(exp)) diff --git a/src/dss/api/dss_get_domain_rpc.py b/src/dss/api/dss_get_domain_rpc.py new file mode 100644 index 0000000..e22744e --- /dev/null +++ b/src/dss/api/dss_get_domain_rpc.py @@ -0,0 +1,52 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +''' +Get Domain RPC +Req: + { + "domain": + } + +Reply: +{ + "data": { + "name1": "value1", + "name2": "value2", + ... + } +} +''' + +class GetDomainRPC(object): + @staticmethod + def get_name(): + return "get-domain" + + @staticmethod + def create_req_payload(domain): + return {'domain': domain} + + @staticmethod + def create_rep_payload(value): + return { "data": value } + + @staticmethod + def get_domain_from_req_payload(payload): + return payload['domain'] + + @staticmethod + def get_data_from_rep_payload(payload): + return payload['data'] diff --git a/src/dss/api/dss_get_domains_rpc.py b/src/dss/api/dss_get_domains_rpc.py new file mode 100644 index 0000000..db0adec --- /dev/null +++ b/src/dss/api/dss_get_domains_rpc.py @@ -0,0 +1,43 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +''' +Get Domains RPC +Req: + { + } + +Reply: +{ + "data": [ + , + , + ... + ] +} +''' + +class GetDomainsRPC(object): + @staticmethod + def get_name(): + return "get-domains" + + @staticmethod + def create_rep_payload(domains): + return { "data": domains } + + @staticmethod + def get_data_from_rep_payload(payload): + return payload['data'] diff --git a/src/dss/api/dss_get_rpc.py b/src/dss/api/dss_get_rpc.py new file mode 100644 index 0000000..9996971 --- /dev/null +++ b/src/dss/api/dss_get_rpc.py @@ -0,0 +1,53 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +''' +Get RPC +Req: + { + "domain": , + "name": + } + +Reply: +{ + "value": +} +''' + +class GetRPC(object): + @staticmethod + def get_name(): + return "get" + + @staticmethod + def create_req_payload(domain, name): + return {'domain': domain, 'name': name} + + @staticmethod + def create_rep_payload(value): + return {'value': value} + + @staticmethod + def get_domain_from_req_payload(payload): + return payload['domain'] + + @staticmethod + def get_name_from_req_payload(payload): + return payload['name'] + + @staticmethod + def get_value_from_rep_payload(payload): + return payload['value'] diff --git a/src/dss/api/dss_msg.py b/src/dss/api/dss_msg.py new file mode 100644 index 0000000..5815061 --- /dev/null +++ b/src/dss/api/dss_msg.py @@ -0,0 +1,67 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import json +from collections import OrderedDict + +from dss.api import dss_error + +''' +{ + "name": , + "id": , + "result": , + "payload": +} +''' +class Msg(object): + def __init__(self, name=None, id=None, payload={}, result='OK'): + self.name = name + self.id = id + self.payload = payload + self.result = result + + def serialize(self): + msg = OrderedDict([('name', self.name), ('id', self.id), ('result', self.result), ('payload', self.payload) ]) + #msg = {} + #msg['name'] = self.name + #msg['id'] = self.id + #msg['result'] = self.result + #msg['payload'] = self.payload + + return json.dumps(msg, separators=(',', ':')) + + def deserialize(self, msg): + data = json.loads(msg, object_pairs_hook=OrderedDict) + self.name = data['name'] + self.id = data['id'] + self.result = data['result'] + self.payload = data['payload'] + + if self.result != 'OK': + raise dss_error.Error('Request failed with error %s' % self.result) + + def get_name(self): + return self.name + + def get_id(self): + return self.id + + def get_result(self): + return self.result + + def get_payload(self): + return self.payload + diff --git a/src/dss/api/dss_plugin.py b/src/dss/api/dss_plugin.py new file mode 100644 index 0000000..21b146b --- /dev/null +++ b/src/dss/api/dss_plugin.py @@ -0,0 +1,84 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +from dss.api import dss_error + +class DSSPlugin(object): + def __init__(self, config_file): + self.config_file = config_file + + + def get(self, domain, name): + """get the value associated with an attribute + + Arguments: + + domain: The domain name. + + name: The attribute name. + + Return: + + The value + + Raise: + + dss_error.Error can be raised in-case of an error. + """ + raise dss_error.Error('Not implemented') + + def set(self, domain, name, value): + """set an attribute to some value + + Arguments: + + domain: The domain name. + + name: The attribute name. + + value: The value. + + Raise: + + dss_error.Error can be raised in-case of an error. + """ + raise dss_error.Error('Not implemented') + + def get_domain(self, domain): + """Get all the attributes in a domain + + Arguments: + + Return: A dictionary. + + Raise: + + dss_error.Error can raised in case of an error + """ + raise dss_error.Error('Not implemented') + + def get_domains(self): + """Get the domains list + + Arguments: + + Return: A list containing domain names + + Raise: + + dss_error.Error can raised in case of an error + """ + raise dss_error.Error('Not implemented') + diff --git a/src/dss/api/dss_set_rpc.py b/src/dss/api/dss_set_rpc.py new file mode 100644 index 0000000..0750fdd --- /dev/null +++ b/src/dss/api/dss_set_rpc.py @@ -0,0 +1,51 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +''' +Set RPC +Req: + { + "domain": , + "name": , + "value": + } + +Reply: +{} +''' +class SetRPC(object): + @staticmethod + def get_name(): + return "set" + + @staticmethod + def create_req_payload(domain, name, value): + payload = {} + payload['domain'] = domain + payload['name'] = name + payload['value'] = value + return payload + + @staticmethod + def get_domain_from_req_payload(payload): + return payload['domain'] + + @staticmethod + def get_name_from_req_payload(payload): + return payload['name'] + + @staticmethod + def get_value_from_req_payload(payload): + return payload['value'] diff --git a/src/dss/cli/__init__.py b/src/dss/cli/__init__.py new file mode 100644 index 0000000..f035b4a --- /dev/null +++ b/src/dss/cli/__init__.py @@ -0,0 +1,15 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + diff --git a/src/dss/cli/dss_clihandlers.py b/src/dss/cli/dss_clihandlers.py new file mode 100644 index 0000000..84e65bf --- /dev/null +++ b/src/dss/cli/dss_clihandlers.py @@ -0,0 +1,174 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import socket +import sys +import json + +from dss.api import dss_error +from dss.client import dss_client + +class VerboseLogger: + def __call__(self, msg): + print(msg) + +class CLIHandler: + def __init__(self): + self.sock = None + self.verbose_logger = VerboseLogger() + + def _init_api(self, server, verbose): + logger = None + if verbose: + logger = self.verbose_logger + + self.client = dss_client.Client(server) + + def set_handler(self, subparser): + subparser.set_defaults(handler=self) + + def init_subparser(self, subparsers): + raise dss_error.Error('Not implemented') + + def __call__(self, args): + raise dss_error.Error('Not implemented') + + +class CLIGetHandler(CLIHandler): + def init_subparser(self, subparsers): + subparser = subparsers.add_parser('get', help='Get the value of an attribute') + subparser.add_argument('--domain', + required=True, + dest='domain', + metavar='DOMAIN', + action='store') + + subparser.add_argument('--name', + required=True, + dest='name', + metavar='NAME', + action='store') + + self.set_handler(subparser) + + def __call__(self, args): + self._init_api(args.server, args.verbose) + value = self.client.get(args.domain, args.name) + print('%s' % value) + +class CLIGetDomainHandler(CLIHandler): + def init_subparser(self, subparsers): + subparser = subparsers.add_parser('get-domain', help='Get the domain attributes') + subparser.add_argument('--domain', + required=True, + dest='domain', + metavar='DOMAIN', + action='store') + + self.set_handler(subparser) + + def __call__(self, args): + self._init_api(args.server, args.verbose) + attrs = self.client.get_domain(args.domain) + for name, value in attrs.iteritems(): + print('%s = %s' % (name, value)) + +class CLIGetDomainsHandler(CLIHandler): + def init_subparser(self, subparsers): + subparser = subparsers.add_parser('get-domains', help='Get the domains') + self.set_handler(subparser) + + def __call__(self, args): + self._init_api(args.server, args.verbose) + domains = self.client.get_domains() + for domain in domains: + print('%s' % domain) + +class CLISetHandler(CLIHandler): + def init_subparser(self, subparsers): + subparser = subparsers.add_parser('set', help='Set an attribute to some value') + subparser.add_argument('--domain', + required=True, + dest='domain', + metavar='DOMAIN', + action='store') + subparser.add_argument('--name', + required=True, + dest='name', + metavar='NAME', + action='store') + subparser.add_argument('--value', + required=True, + dest='value', + metavar='VALUE', + action='store') + + self.set_handler(subparser) + + def __call__(self, args): + self._init_api(args.server, args.verbose) + self.client.set(args.domain, args.name, args.value) + +class CLIDeleteHandler(CLIHandler): + def init_subparser(self, subparsers): + subparser = subparsers.add_parser('delete', help='Delete an attribute') + subparser.add_argument('--domain', + required=True, + dest='domain', + metavar='DOMAIN', + action='store') + subparser.add_argument('--name', + required=True, + dest='name', + metavar='NAME', + action='store') + + self.set_handler(subparser) + + def __call__(self, args): + self._init_api(args.server, args.verbose) + self.client.delete(args.domain, args.name) + +class CLIDeleteDomainHandler(CLIHandler): + def init_subparser(self, subparsers): + subparser = subparsers.add_parser('delete-domain', help='Delete a domain') + subparser.add_argument('--domain', + required=True, + dest='domain', + metavar='DOMAIN', + action='store') + + self.set_handler(subparser) + + def __call__(self, args): + self._init_api(args.server, args.verbose) + self.client.delete_domain(args.domain) + +import sys +import inspect + +def get_handlers_list(): + handlers = [] + for name, obj in inspect.getmembers(sys.modules[__name__]): + if inspect.isclass(obj): + if name is not 'CLIHandler': + if issubclass(obj, CLIHandler): + handlers.append(obj()) + return handlers + +if __name__ == '__main__': + handlers = get_handlers_list() + for handler in handlers: + print('handler is ', handler) diff --git a/src/dss/cli/dss_cliprocessors.py b/src/dss/cli/dss_cliprocessors.py new file mode 100644 index 0000000..85b8a3d --- /dev/null +++ b/src/dss/cli/dss_cliprocessors.py @@ -0,0 +1,57 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import sys +import argparse + +from dss.cli import dss_clihandlers + +class CLIProcessor: + def __init__(self, prog): + self.prog = prog + + def __call__(self, args): + parser = argparse.ArgumentParser(description='dss CLI', prog=self.prog) + parser.add_argument('--server', + dest='server', + metavar='SERVER', + default='/var/run/.dss-server', + type=str, + action='store') + + parser.add_argument('--verbose', + required=False, + default=False, + action='store_true') + + subparsers = parser.add_subparsers() + handlers = dss_clihandlers.get_handlers_list() + for handler in handlers: + handler.init_subparser(subparsers) + + parse_result = parser.parse_args(args) + + parse_result.handler(parse_result) + + +if __name__ == '__main__': + processor = CLIProcessor('dsscli') + args = sys.argv[1:] + try: + processor(args) + except Exception as error: + print('Got error %s' % str(error)) + sys.exit(1) + diff --git a/src/dss/cli/dsscli.py b/src/dss/cli/dsscli.py new file mode 100755 index 0000000..57695bb --- /dev/null +++ b/src/dss/cli/dsscli.py @@ -0,0 +1,35 @@ +#! /usr/bin/python +# +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import sys +import traceback + +from dss.cli import dss_cliprocessors + +def main(): + try: + processor = dss_cliprocessors.CLIProcessor('dsscli') + args = sys.argv[1:] + processor(args) + except Exception as exp: + print('Failed with error: %s' % str(exp)) + #traceback.print_exc() + return 1 + return 0 + +if __name__ == '__main__': + sys.exit(main()) diff --git a/src/dss/client/__init__.py b/src/dss/client/__init__.py new file mode 100644 index 0000000..f035b4a --- /dev/null +++ b/src/dss/client/__init__.py @@ -0,0 +1,15 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + diff --git a/src/dss/client/dss_client.py b/src/dss/client/dss_client.py new file mode 100644 index 0000000..7277804 --- /dev/null +++ b/src/dss/client/dss_client.py @@ -0,0 +1,97 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +from dss.api import dss_error +from dss.api import dss_msg +from dss.api import dss_get_rpc +from dss.api import dss_set_rpc +from dss.api import dss_get_domain_rpc +from dss.api import dss_get_domains_rpc +from dss.api import dss_delete_rpc +from dss.api import dss_delete_domain_rpc + +import socket + +class Client(object): + def __init__(self, uds): + self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) + + self.server_address = uds + self.connect = True + self.fd = None + self.id = 0 + + def _connect(self): + if self.connect: + self.sock.connect(self.server_address) + self.fd = self.sock.makefile('rw') + self.connect = False + + def _call_rpc(self, msg): + self.id += 1 + repmsg = dss_msg.Msg() + try: + self._connect() + req = msg.serialize() + self.sock.sendall(req+'\n') + rep = self.fd.readline() + except Exception as exp: + try: + self.sock.close() + except: + pass + self.connect = True + raise dss_error.Error(str(exp)) + repmsg.deserialize(rep) + return repmsg + + def get(self, domain, name): + reqpayload = dss_get_rpc.GetRPC.create_req_payload(domain, name) + reqmsg = dss_msg.Msg(dss_get_rpc.GetRPC.get_name(), self.id, reqpayload) + repmsg = self._call_rpc(reqmsg) + reppayload = repmsg.get_payload() + value = dss_get_rpc.GetRPC.get_value_from_rep_payload(reppayload) + return value + + def get_domain(self, domain): + reqpayload = dss_get_domain_rpc.GetDomainRPC.create_req_payload(domain) + reqmsg = dss_msg.Msg(dss_get_domain_rpc.GetDomainRPC.get_name(), self.id, reqpayload) + repmsg = self._call_rpc(reqmsg) + reppayload = repmsg.get_payload() + attrs = dss_get_domain_rpc.GetDomainRPC.get_data_from_rep_payload(reppayload) + return attrs + + + def set(self, domain, name, value): + reqpayload = dss_set_rpc.SetRPC.create_req_payload(domain, name, value) + reqmsg = dss_msg.Msg(dss_set_rpc.SetRPC.get_name(), self.id, reqpayload) + repmsg = self._call_rpc(reqmsg) + + def get_domains(self): + reqmsg = dss_msg.Msg(dss_get_domains_rpc.GetDomainsRPC.get_name(), self.id, None) + repmsg = self._call_rpc(reqmsg) + reppayload = repmsg.get_payload() + domains = dss_get_domains_rpc.GetDomainsRPC.get_data_from_rep_payload(reppayload) + return domains + + def delete(self, domain, name): + reqpayload = dss_delete_rpc.DeleteRPC.create_req_payload(domain, name) + reqmsg = dss_msg.Msg(dss_delete_rpc.DeleteRPC.get_name(), self.id, reqpayload) + repmsg = self._call_rpc(reqmsg) + + def delete_domain(self, domain): + reqpayload = dss_delete_domain_rpc.DeleteDomainRPC.create_req_payload(domain) + reqmsg = dss_msg.Msg(dss_delete_domain_rpc.DeleteDomainRPC.get_name(), self.id, reqpayload) + repmsg = self._call_rpc(reqmsg) diff --git a/src/dss/server/__init__.py b/src/dss/server/__init__.py new file mode 100644 index 0000000..f035b4a --- /dev/null +++ b/src/dss/server/__init__.py @@ -0,0 +1,15 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + diff --git a/src/dss/server/dss_config.py b/src/dss/server/dss_config.py new file mode 100644 index 0000000..0ac41c8 --- /dev/null +++ b/src/dss/server/dss_config.py @@ -0,0 +1,93 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import ConfigParser +from dss.api import dss_error + +SERVER_SECTION = "server" +PLUGIN_SECTION = "plugin" +DEFAULT_CONFIG_FILE="/etc/dss/dss-server/config.ini" + +class Config(object): + """ + Read the ini file. The structure of the file is as follows: + [server] + logging_level = info + logging_destination = console + verbose = true + listening_uds = /var/run/.dss-server + transport_type = dgram + [plugin] + plugin = /opt/dss-server/etcd.py + config = /etc/dss-server/etcd.ini + """ + + def __init__(self, config_file = DEFAULT_CONFIG_FILE): + try: + # server + self.logging_level = None + self.logging_destination = None + self.verbose = None + self.listening_uds = None + self.transport_type = None + + # plugin + self.plugin = None + self.plugin_config = None + + config = ConfigParser.ConfigParser() + config.read([config_file]) + + self.logging_level = config.get(SERVER_SECTION, "logging_level") + self.logging_destination = config.get(SERVER_SECTION, "logging_destination") + self.verbose = config.get(SERVER_SECTION, "verbose") + self.listening_uds = config.get(SERVER_SECTION, "listening_uds") + self.transport_type = config.get(SERVER_SECTION, "transport_type") + + self.plugin = config.get(PLUGIN_SECTION, "plugin") + self.plugin_config = config.get(PLUGIN_SECTION, "config") + + + except Exception as exp: + raise dss_error.Error(str(exp)) + + def get_logging_level(self): + return self.logging_level + + def get_logging_destination(self): + return self.logging_destination + + def get_verbose(self): + return self.verbose + + def get_listening_uds(self): + return self.listening_uds + + def get_plugin(self): + return self.plugin + + def get_plugin_config(self): + return self.plugin_config + + def get_transport_type(self): + return self.transport_type + + +if __name__ == "__main__": + import sys + try: + config = Config(sys.argv[1]) + except dss_error.Error as exp: + print("Got exception %s" % exp) diff --git a/src/dss/server/dss_connection.py b/src/dss/server/dss_connection.py new file mode 100644 index 0000000..df732a9 --- /dev/null +++ b/src/dss/server/dss_connection.py @@ -0,0 +1,105 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import logging +import json +import Queue +import socket +import errno +import time + +from dss.server import dss_rpc_processor + +class Connection(object): + def __init__(self, sock, address, rpc_processor): + self.rpc_processor = rpc_processor + self.address = address + self.sock = sock + self.fd = self.sock.makefile('rw', bufsize=0) + self.recvbuffer = "" + logging.info("Received connection from %r %r" % (address, self)) + + def fileno(self): + return self.sock.fileno() + + def send_all(self, rep): + totalsent = 0 + msglen = len(rep) + while True: + try: + logging.info("Sending %s %r" % (rep, self)) + total = self.sock.send(rep[totalsent:]) + if total == 0: + raise RuntimeError('connection closed!') + totalsent += total + if totalsent == msglen: + logging.info("Sent successfully %r" % self) + return + except socket.error as e: + if e.argus[0] == errno.EWOULDBLOCK: + time.sleep(1) + else: + raise + + def process_recv_buffer(self, data): + self.recvbuffer += data + #check for '\n' + c = self.recvbuffer.count('\n') + parts = None + if c: + parts = self.recvbuffer.split('\n') + if self.recvbuffer.endswith('\n'): + self.recvbuffer = "" + else: + self.recvbuffer = parts[c] + + for i in range(0, c): + msg = parts[i] + logging.debug("Received %s %r" % (msg, self)) + # strip the req + msg = msg.lstrip('\x00').rstrip('\x00') + rep = self.rpc_processor.process(msg) + self.send_all(rep+'\n') + + + + + def recv(self): + data = None + try: + data = self.sock.recv(1024) + if not data: + logging.info("Client %r %r disconnected" % (self.address, self)) + return False + self.process_recv_buffer(data) + except socket.error as e: + if e.args[0] == errno.EWOULDBLOCK: + return True + except Exception as exp: + logging.warning('Failed when processing %s got exp %s' % (data, exp)) + return False + + return True + + @staticmethod + def recv_dgram(sock, req, address, rpc_processor): + logging.debug("Received %s address is %r" % (req, address)) + try: + # convert to json structure + rep = rpc_processor.process(req) + # add '/n' + sock.sendto(rep+'\n', address) + except Exception as exp: + logging.warning('Failed when processing %s got exp %s' % (req, exp)) diff --git a/src/dss/server/dss_delete_domain_handler.py b/src/dss/server/dss_delete_domain_handler.py new file mode 100644 index 0000000..edb412f --- /dev/null +++ b/src/dss/server/dss_delete_domain_handler.py @@ -0,0 +1,30 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import logging +from dss.server import dss_rpc_handler +from dss.server import dss_plugin_loader +from dss.api import dss_delete_domain_rpc + +class DeleteDomainHandler(dss_rpc_handler.RPCHandler): + def __init__(self, plugin): + super(DeleteDomainHandler, self).__init__(dss_delete_domain_rpc.DeleteDomainRPC.get_name()) + self.plugin = plugin + + def handle(self, reqpayload): + domain = dss_delete_domain_rpc.DeleteDomainRPC.get_domain_from_req_payload(reqpayload) + logging.info("Deleting domain %s" % (domain)) + self.plugin.delete_domain(domain) + return {} diff --git a/src/dss/server/dss_delete_handler.py b/src/dss/server/dss_delete_handler.py new file mode 100644 index 0000000..cc106aa --- /dev/null +++ b/src/dss/server/dss_delete_handler.py @@ -0,0 +1,31 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import logging +from dss.server import dss_rpc_handler +from dss.server import dss_plugin_loader +from dss.api import dss_delete_rpc + +class DeleteHandler(dss_rpc_handler.RPCHandler): + def __init__(self, plugin): + super(DeleteHandler, self).__init__(dss_delete_rpc.DeleteRPC.get_name()) + self.plugin = plugin + + def handle(self, reqpayload): + domain = dss_delete_rpc.DeleteRPC.get_domain_from_req_payload(reqpayload) + name = dss_delete_rpc.DeleteRPC.get_name_from_req_payload(reqpayload) + logging.info("Deleting %s/%s" % (domain, name)) + self.plugin.delete(domain, name) + return {} diff --git a/src/dss/server/dss_get_domain_handler.py b/src/dss/server/dss_get_domain_handler.py new file mode 100644 index 0000000..46339d3 --- /dev/null +++ b/src/dss/server/dss_get_domain_handler.py @@ -0,0 +1,32 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import logging +from dss.server import dss_rpc_handler +from dss.server import dss_plugin_loader +from dss.api import dss_get_domain_rpc + +class GetDomainHandler(dss_rpc_handler.RPCHandler): + def __init__(self, plugin): + super(GetDomainHandler, self).__init__(dss_get_domain_rpc.GetDomainRPC.get_name()) + self.plugin = plugin + + def handle(self, reqpayload): + domain = dss_get_domain_rpc.GetDomainRPC.get_domain_from_req_payload(reqpayload) + logging.info("Handling getting domain %s" % domain) + attrs = self.plugin.get_domain(domain) + logging.info("attrs = %r" % attrs) + reppayload = dss_get_domain_rpc.GetDomainRPC.create_rep_payload(attrs) + return reppayload diff --git a/src/dss/server/dss_get_domains_handler.py b/src/dss/server/dss_get_domains_handler.py new file mode 100644 index 0000000..5dab705 --- /dev/null +++ b/src/dss/server/dss_get_domains_handler.py @@ -0,0 +1,30 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import logging +from dss.server import dss_rpc_handler +from dss.server import dss_plugin_loader +from dss.api import dss_get_domains_rpc + +class GetDomainsHandler(dss_rpc_handler.RPCHandler): + def __init__(self, plugin): + super(GetDomainsHandler, self).__init__(dss_get_domains_rpc.GetDomainsRPC.get_name()) + self.plugin = plugin + + def handle(self, reqpayload): + logging.info("Handling getting domains") + domains = self.plugin.get_domains() + reppayload = dss_get_domains_rpc.GetDomainsRPC.create_rep_payload(domains) + return reppayload diff --git a/src/dss/server/dss_get_handler.py b/src/dss/server/dss_get_handler.py new file mode 100644 index 0000000..0331d9f --- /dev/null +++ b/src/dss/server/dss_get_handler.py @@ -0,0 +1,32 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import logging +from dss.server import dss_rpc_handler +from dss.server import dss_plugin_loader +from dss.api import dss_get_rpc + +class GetHandler(dss_rpc_handler.RPCHandler): + def __init__(self, plugin): + super(GetHandler, self).__init__(dss_get_rpc.GetRPC.get_name()) + self.plugin = plugin + + def handle(self, reqpayload): + domain = dss_get_rpc.GetRPC.get_domain_from_req_payload(reqpayload) + name = dss_get_rpc.GetRPC.get_name_from_req_payload(reqpayload) + logging.info("Handling getting %s/%s" % (domain, name)) + value = self.plugin.get(domain, name) + reppayload = dss_get_rpc.GetRPC.create_rep_payload(value) + return reppayload diff --git a/src/dss/server/dss_logger.py b/src/dss/server/dss_logger.py new file mode 100644 index 0000000..f726bd9 --- /dev/null +++ b/src/dss/server/dss_logger.py @@ -0,0 +1,113 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import sys +import logging +import logging.handlers + +from dss.api import dss_error + +class Logger: + + levels = {'debug': logging.DEBUG, + 'info': logging.INFO, + 'warning': logging.WARNING, + 'error': logging.error} + + DEST_CONSOLE = 1 + DEST_SYSLOG = 2 + dests = {'console': DEST_CONSOLE, + 'syslog': DEST_SYSLOG} + + def __init__(self, dest, verbose, level): + self.verbose = verbose + self.dest = Logger.str_to_dest(dest) + self.level = Logger.str_to_level(level) + self.init() + + def init(self): + args = {} + + if self.level not in Logger.levels.values(): + raise dss_error.Error('Invalid level value, possible values are %s' % str(Logger.levels)) + + if self.dest not in Logger.dests.values(): + raise dss_error.Error('Invalid destination value, possible values are %s' % str(Logger.dests)) + + if self.verbose: + if self.dest is Logger.DEST_CONSOLE: + args['format'] = '[%(asctime)s %(levelname)7s %(module)s(%(lineno)3s)] %(message)s' + else: + args['format'] = '[%(module)s(%(lineno)3s)] %(message)s' + else: + args['format'] = '%(message)s' + + if self.dest is Logger.DEST_CONSOLE: + args['stream'] = sys.stdout + elif self.dest is Logger.DEST_SYSLOG: + logging.getLogger('').addHandler(logging.handlers.SysLogHandler(address='/dev/log')) + + args['level'] = self.level + logging.basicConfig(**args) + + def set_level(self, level): + self.level = Logger.str_to_level(level) + self.init() + + def set_dest(self, dest): + self.dest = Logger.str_to_dest(dest) + self.init() + + @staticmethod + def str_to_level(level): + ret = None + try: + ret = Logger.levels[level] + except KeyError as exp: + raise dss_error.Error('Invalid log level, possible values %s' % str(Logger.levels.keys())) + return ret + + @staticmethod + def str_to_dest(dest): + ret = None + try: + ret = Logger.dests[dest] + except KeyError as exp: + raise dss_error.Error('Invalid destination, possible values %s' % str(Logger.dests.keys())) + return ret + + @staticmethod + def level_to_str(level): + for key, value in Logger.levels.iteritems(): + if value is level: + return key + return None + + @staticmethod + def dest_to_str(dest): + for key, value in Logger.dests.iteritems(): + if value is dest: + return key + return None + +if __name__ == '__main__': + dest = Logger.str_to_dest('console') + level = Logger.str_to_level('debug') + logger = Logger(dest, True, level) + world='world' + logging.error('hello %s!' % world) + logging.warn('hello %s!' % world) + logging.info('hello %s!' % world) + logging.debug('hello %s!' % world) diff --git a/src/dss/server/dss_main.py b/src/dss/server/dss_main.py new file mode 100644 index 0000000..feeaecb --- /dev/null +++ b/src/dss/server/dss_main.py @@ -0,0 +1,104 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +from dss.server import dss_config +from dss.server import dss_server +from dss.server import dss_logger +from dss.server import dss_rpc_processor +from dss.server import dss_get_handler +from dss.server import dss_get_domain_handler +from dss.server import dss_get_domains_handler +from dss.server import dss_set_handler +from dss.server import dss_delete_handler +from dss.server import dss_delete_domain_handler +from dss.server import dss_plugin_loader + +import logging + + +def _main(config): + #initialize config + conf = dss_config.Config(config) + + #initialize logger + logger = dss_logger.Logger(conf.get_logging_destination(), + conf.get_verbose(), + conf.get_logging_level()) + + #initialize plugin + logging.info('Initializing dss plugin') + plugin = dss_plugin_loader.DSSPluginLoader(conf) + + #initialize rpc processor + logging.info('Initializing rpc processor') + rpcprocessor = dss_rpc_processor.RPCProcessor() + + #adding rpc handlers + logging.info('Adding RPC handlers') + gethandler = dss_get_handler.GetHandler(plugin) + rpcprocessor.add_handler(gethandler) + getdomainhandler = dss_get_domain_handler.GetDomainHandler(plugin) + rpcprocessor.add_handler(getdomainhandler) + getdomainshandler = dss_get_domains_handler.GetDomainsHandler(plugin) + rpcprocessor.add_handler(getdomainshandler) + sethandler = dss_set_handler.SetHandler(plugin) + rpcprocessor.add_handler(sethandler) + deletehandler = dss_delete_handler.DeleteHandler(plugin) + rpcprocessor.add_handler(deletehandler) + deletedomainhandler = dss_delete_domain_handler.DeleteDomainHandler(plugin) + rpcprocessor.add_handler(deletedomainhandler) + + #initialize tcp server + logging.info("Initializing tcp server") + server = dss_server.Server(conf, rpcprocessor) + + logging.info('Waiting for TCP requests') + server.start() + + logging.info('TCP server stopped') + + logging.info('Exiting, bye bye...') + + +def main(): + import sys + import traceback + import argparse + + parser = argparse.ArgumentParser(description='dss-server', + prog=sys.argv[0]) + + parser.add_argument('--config', + required=False, + dest='config', + metavar='CONFIG', + default='/etc/dss-server/config.ini', + help='The dss server configuration file', + type=str, + action='store') + + try: + result = parser.parse_args(sys.argv[1:]) + _main(result.config) + except Exception as exp: + print("Failed with error %s" % exp) + traceback.print_exc() + sys.exit(1) + + print("Exiting...") + sys.exit(0) + +if __name__ == '__main__': + main() diff --git a/src/dss/server/dss_plugin_loader.py b/src/dss/server/dss_plugin_loader.py new file mode 100644 index 0000000..21b3ff6 --- /dev/null +++ b/src/dss/server/dss_plugin_loader.py @@ -0,0 +1,81 @@ +# vim: tabstop=8 expandtab shiftwidth=4 softtabstop=4 +# +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import os +import imp +import sys +import logging +from dss.api import dss_error +from dss.server import dss_config + +class DSSPluginLoader(object): + def __init__(self, config): + try: + plugin_file = config.get_plugin() + location = os.path.dirname(plugin_file) + pluginname = os.path.basename(plugin_file).replace(".py", "") + sys.path.append(location) + self.plugin = None + self._load_plugin(pluginname, config.get_plugin_config()) + except Exception as exp: + raise dss_error.Error(str(exp)) + + def _load_plugin(self, pluginname, configfile): + logging.info('Loading plugin %s' % pluginname) + fp, pathname, description = imp.find_module(pluginname) + try: + pluginmodule = imp.load_module(pluginname, fp, pathname, description) + class_name = getattr(pluginmodule, pluginname) + self.plugin = class_name(configfile) + except Exception as exp: + logging.error('Failed to load plugin %s, got exp %s' % (pluginname, str(exp))) + raise + finally: + if fp: + fp.close() + + def set(self, domain, name, value): + logging.info("Setting %s/%s to %s" % (domain, name, value)) + self.plugin.set(domain, name, value) + + def get(self, domain, name): + logging.info("Getting attribute %s/%s" % (domain, name)) + value = self.plugin.get(domain, name) + logging.info("Value of %s/%s is %s" % (domain, name, value)) + return value + + def get_domain(self, domain): + logging.info("Getting the attributes of domain %s" % domain) + attrs = self.plugin.get_domain(domain) + return attrs + + def get_domains(self): + logging.info("Getting the domains") + domains = self.plugin.get_domains() + return domains + + def delete(self, domain, name): + logging.info("Deleting %s/%s" % (domain, name)) + self.plugin.delete(domain, name) + + def delete_domain(self, domain): + logging.info("Delete domain %s" % domain) + self.plugin.delete_domain(domain) + +if __name__ == '__main__': + pl = DSSPluginLoader("configfile") + pl.set("domain1", "name1", "value1") diff --git a/src/dss/server/dss_rpc_handler.py b/src/dss/server/dss_rpc_handler.py new file mode 100644 index 0000000..821afa5 --- /dev/null +++ b/src/dss/server/dss_rpc_handler.py @@ -0,0 +1,35 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +from dss.api import dss_error + +class RPCHandler(object): + def __init__(self, name): + self.name = name + + def get_name(self): + return self.name + + ''' + handle request + Arguments: + req: request payload + + return reply payload + raise oha_error.Error in-case of error + ''' + def handle(self, reqmsg): + raise dss_error.Error('Not implemented') + diff --git a/src/dss/server/dss_rpc_processor.py b/src/dss/server/dss_rpc_processor.py new file mode 100644 index 0000000..05908c7 --- /dev/null +++ b/src/dss/server/dss_rpc_processor.py @@ -0,0 +1,66 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import logging +from dss.api import dss_msg +from dss.server import dss_rpc_handler + +class RPCProcessor(object): + def __init__(self): + self.handlers = [] + + def add_handler(self, handler): + self.handlers.append(handler) + + def get_handler(self, name): + for handler in self.handlers: + if handler.get_name() == name: + return handler + return None + + ''' + process the request + Arguments: + reqstring the request string + Return: + repstring the reply string + ''' + def process(self, reqstring): + reppayload = {} + result = 'OK' + id = None + name = None + try: + # deserialize req string + reqmsg = dss_msg.Msg() + reqmsg.deserialize(reqstring) + name = reqmsg.get_name() + + logging.info("Processing %s" % name) + + id = reqmsg.get_id() + + # find suitable handler + handler = self.get_handler(name) + + # process message + reppayload = handler.handle(reqmsg.get_payload()) + except Exception as exp: + logging.warning('Failed when processing %s' % reqstring) + result = str(exp) + + repmsg = dss_msg.Msg(name, id, reppayload, result) + + return repmsg.serialize() diff --git a/src/dss/server/dss_server.py b/src/dss/server/dss_server.py new file mode 100644 index 0000000..1660f5b --- /dev/null +++ b/src/dss/server/dss_server.py @@ -0,0 +1,79 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import socket +import logging +import select +import errno +import os + +from dss.server import dss_connection +from dss.server import dss_config +from dss.server import dss_rpc_processor + +class Server(object): + def __init__(self, conf, rpc_processor): + self.uds = conf.get_listening_uds() + self.rpc_processor = rpc_processor + self.transport_type = conf.get_transport_type() + self.stopped = False + if (self.transport_type == 'stream'): + self.server = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) + else: + self.server = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM) + self.server.setblocking(0) + self.remove_uds() + + def start(self): + logging.info("Server listening at %s" % self.uds) + self.server.bind(self.uds) + if self.transport_type == 'stream': + self.server.listen(1) + inputs = [self.server] + while not self.stopped: + try: + readable, writable, errored = select.select(inputs, [], []) + for s in readable: + if s is self.server: + if self.transport_type == 'stream': + client, address = self.server.accept() + client.setblocking(0) #pylint: disable=no-member + conn = dss_connection.Connection(client, address, self.rpc_processor) + inputs.append(conn) + logging.info("Accepted connection from %r" % address) + else: + data, address = self.server.recvfrom(4096) + dss_connection.Connection.recv_dgram(self.server, data, address, self.rpc_processor) + else: + result = s.recv() + if not result: + inputs.remove(s) + except (SystemExit, KeyboardInterrupt): + break + except select.error as e: + if e.args[0] == errno.EINTR: + break + logging.info("Server stopping") + self.remove_uds() + + def remove_uds(self): + try: + os.unlink(self.uds) + except OSError: + pass + + def shutdown(self): + logging.info("Shutting down tcp server") + self.stopped = True diff --git a/src/dss/server/dss_set_handler.py b/src/dss/server/dss_set_handler.py new file mode 100644 index 0000000..b1ca210 --- /dev/null +++ b/src/dss/server/dss_set_handler.py @@ -0,0 +1,32 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import logging +from dss.server import dss_rpc_handler +from dss.server import dss_plugin_loader +from dss.api import dss_set_rpc + +class SetHandler(dss_rpc_handler.RPCHandler): + def __init__(self, plugin): + super(SetHandler, self).__init__(dss_set_rpc.SetRPC.get_name()) + self.plugin = plugin + + def handle(self, reqpayload): + domain = dss_set_rpc.SetRPC.get_domain_from_req_payload(reqpayload) + name = dss_set_rpc.SetRPC.get_name_from_req_payload(reqpayload) + value = dss_set_rpc.SetRPC.get_value_from_req_payload(reqpayload) + logging.info("Setting state of %s/%s to %s" % (domain, name, value)) + self.plugin.set(domain, name, value) + return {} diff --git a/src/dss/server/dss_shutdown_handler.py b/src/dss/server/dss_shutdown_handler.py new file mode 100644 index 0000000..55d0eb6 --- /dev/null +++ b/src/dss/server/dss_shutdown_handler.py @@ -0,0 +1,41 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import signal +import logging +from dss.api import dss_error + +class ShutdownHandlerBase(object): + def __init__(self): + pass + + def shutdown(self): + raise dss_error.Error('Not implemented') + +class ShutdownHandler(object): + def __init__(self): + signal.signal(signal.SIGINT, self.stop) + signal.signal(signal.SIGTERM, self.stop) + self.handlers = [] + + def add_handler(self, handler): + logging.info("Added handler %r" % handler) + self.handlers.append(handler) + + def stop(self, signum, frame): + for handler in self.handlers: + logging.info("Calling shutdown on %r" % handler) + handler.shutdown() + logging.info("shutdown of %r returned" % handler) diff --git a/src/dss/tst/dss-server-config.ini b/src/dss/tst/dss-server-config.ini new file mode 100644 index 0000000..0eab660 --- /dev/null +++ b/src/dss/tst/dss-server-config.ini @@ -0,0 +1,24 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +[server] +logging_level = debug +logging_destination = console +verbose = true +listening_uds = /var/run/.dss-server +transport_type = stream +[plugin] +plugin = PLUGIN_FILE +config = PLUGIN_CONF diff --git a/src/dss/tst/plugin.ini b/src/dss/tst/plugin.ini new file mode 100644 index 0000000..812c58c --- /dev/null +++ b/src/dss/tst/plugin.ini @@ -0,0 +1,17 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +[default] +directory = DIR diff --git a/src/dss/tst/start-cli.sh b/src/dss/tst/start-cli.sh new file mode 100755 index 0000000..bd8f91f --- /dev/null +++ b/src/dss/tst/start-cli.sh @@ -0,0 +1,22 @@ +#! /bin/bash +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + + +cwd=$(pwd) + +export PYTHONPATH=$cwd/../../../ + +exec python $cwd/../cli/dsscli.py $* diff --git a/src/dss/tst/start-server.sh b/src/dss/tst/start-server.sh new file mode 100755 index 0000000..4825cb1 --- /dev/null +++ b/src/dss/tst/start-server.sh @@ -0,0 +1,73 @@ +#! /bin/bash +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + + +cwd=$(pwd) + +if [ $# -ne 1 ]; then + echo "Usage: $0 " +fi + +workdir=$1 + +echo "Creating work dir $workdir" +mkdir $workdir +if [ $? -ne 0 ]; then + exit 1 +fi + +echo "Copying dss-server-config.ini file" +cp $cwd/dss-server-config.ini $workdir/ +if [ $? -ne 0 ]; then + exit 1 +fi + +echo "Copying plugin.ini file" +cp $cwd/plugin.ini $workdir/ +if [ $? -ne 0 ]; then + exit 1 +fi + +echo "Copying plugin" +cp ../../../../plugins/dssfile.py $workdir/ + +echo "Updating dss-server-config.ini" +pluginfile=$workdir/dssfile.py +escapepluginfile=$(echo $pluginfile | sed 's/\//\\\//g') +sed -i "s/PLUGIN_FILE/$escapepluginfile/g" $workdir/dss-server-config.ini +if [ $? -ne 0 ]; then + exit 1 +fi + +echo "Updating plugin.ini" +pluginconf=$workdir/plugin.ini +escapepluginconf=$(echo $pluginconf | sed 's/\//\\\//g') +sed -i "s/PLUGIN_CONF/$escapepluginconf/g" $workdir/dss-server-config.ini + +files=$workdir/files +escapefiles=$(echo $files | sed 's/\//\\\//g') +mkdir $files +if [ $? -ne 0 ]; then + exit 1 +fi + +sed -i "s/DIR/$escapefiles/g" $workdir/plugin.ini + + + +export PYTHONPATH=$cwd/../../../ + +exec python $cwd/../server/dss_main.py --config $workdir/dss-server-config.ini diff --git a/src/setup.py b/src/setup.py new file mode 100644 index 0000000..2f24871 --- /dev/null +++ b/src/setup.py @@ -0,0 +1,36 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +from setuptools import setup, find_packages +setup( + name='dss', + version='1.0', + license='Apache v2', + long_description='distributed state server', + author='Baha Mesleh', + author_email='baha.mesleh@nokia.com', + namespace_packages=[], + packages=find_packages(), + include_package_data=True, + url='akraino/distributed-state-server', + description='distributed state server', + entry_points={ + 'console_scripts': [ + 'dss = dss.server.dss_main:main', + 'dsscli = dss.cli.dsscli:main' + ], + }, + zip_safe=False, + ) diff --git a/systemd/dss-server.service b/systemd/dss-server.service new file mode 100644 index 0000000..0daef9c --- /dev/null +++ b/systemd/dss-server.service @@ -0,0 +1,27 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +[Unit] +Description=distributed state server +After=network.target +After=etcd.service + +[Service] +Type=simple +ExecStart=/usr/local/bin/dss --config /etc/dss/distributed-state-server/config.ini +Restart=always + +[Install] +WantedBy=multi-user.target