From: Janne Suominen Date: Wed, 8 May 2019 12:37:37 +0000 (+0300) Subject: Seed code for yarf X-Git-Url: https://gerrit.akraino.org/r/gitweb?p=ta%2Fyarf.git;a=commitdiff_plain;h=51e80b41a9ba507b2e877f93ea3037e92ee3f78e Seed code for yarf Seed code for yarf Change-Id: I220af8eba8ef38dda2a40f72b38220c50a40a446 Signed-off-by: Janne Suominen --- diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..d645695 --- /dev/null +++ b/LICENSE @@ -0,0 +1,202 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/README.rst b/README.rst new file mode 100644 index 0000000..707039a --- /dev/null +++ b/README.rst @@ -0,0 +1,402 @@ +:: + + Copyright 2019 Nokia + + Licensed under the Apache License, Version 2.0 (the "License"); + + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + + +========================================= +YARF REST framework +========================================= + +:Author: Janne Suominen + +.. raw:: pdf + + PageBreak + +.. sectnum:: + +.. contents:: + +.. raw:: pdf + + PageBreak + + +Introduction +============ + +What is REST +------------ +**Representational state transfer** (REST) or **RESTful** web services is a way of providing +interoperability between computer systems on the Internet. REST-compliant Web services allow +requesting systems to access and manipulate textual representations of Web resources using a +uniform and predefined set of stateless operations. + +About this project +------------------ + +YARF --- Yet Another Restful Framework + +This project provides an implementation for a generic rest framework The +framework is based on +`Flask-RESTful `__ + +| + +*The framework provides the following:* + +* A generic rest server for whole cloud + +* A plugin based interface for creating rest apis. + +| + +*At the high level, the framework serves the following purposes*: + +* Provides a unified interface for creating rest interfaces. + +* Single point of entry with plugin based authentication for apis. + +* Define the rest response and query format. + +| + +Requirements +------------ + +* Framework shall provide easy way to integrate new REST apis via plugins + +* Framework shall be integrated to the CM and provide means to automatically configure itself + +* Framework shall listen to management address of controller on all the controllers + +* Framework shall listen to external and internal load balancer address + +* Framework shall provide means to configure SSL certification + +* Framework shall be integrated to Authentication mechanism provided by the platform + +* Framework shall provide means to validate the parameters given as part of the request + +* Framework shall return an error code in case of: + + * Internal failure; Any failure within the module is considered as internal failure + + * Authentication failure; When authentication has failed or missing credentials + + * Not found; When the object is not found + +Structure of data +================= + +The implementation of this framework promotes returning JSON format +objects with key value pairs. + +Structure of the operation +-------------------------- + +The framework supports adding function calls for any HTTP requests. The +request can either: + +* Have a request in the body of the message as JSON + +* Have a request embedded in the url + +Structure of response +--------------------- + +This framework does not enforce any special structure for the response, +but **it's strongly** encouraged to use the following formatting for the +response: + +.. code:: json + + { + code: + description: + data: + } + +Where: + +* code is the return value of the api in question. + + * 0 means no error and anything other is considered as failure + +* description is the description of the possible failure (can be left empty in case there is no failure) + +* data is the data returned by the api. The data should be in dictionary (JSON) format + +The reasoning for the quite strict guidelines is: + +* Uniqueness of the response makes it easier for the upper level to check the response + +**Note**: The framework will return HTTP status code that is not 200 in +case of: + +* 500: Internal failure (ie. Uncaught exception from the plugin) + +* 401: In case of authentication failure (if authentication is defined) + +* 404: In case the object requested is not found + +High level architecture of the restful framework +================================================ + +At the high level, there is a layer built on top of flask-restful to be +able to: + +* Isolation of the framework implementation details. + +* To be able to provide more specific implementation to fit to our needs. + +* To be able to make a single point of entry to the clusters rest api + +* Flexibility to change different parts of the implementation without affecting the users of the framework. + +* Provide unique responses to caller for easy parsing + + +.. figure:: ./design/architecture.jpeg + :alt: architecture + + architecture + +Restful framework interfaces +============================ + +RestResource +------------ + +All the plugins have to inherit from this class. This class is the basis +of the plugin framework. All the plugins that inherit from this object +and are defined in plugin specific inifile will be automatically +imported. + +The http requests will be converted to corresponding lowercase +functions. For example: request method GET will call *get()* function and +POST will call *post()*. + +The resource should also define endpoints where it want's to register +these calls. This is done by setting the *endpoint* class variable list. + + +For decorating functions with decorators *extra_wrappers* can be used. +The function must return either the function or a dictionary that is of +the same format defined in `Structure of Response`_. + +To have authentication for the module adding class variable named +*authentication_method* needs to be defined. For production environment +this should be left untouched since the authentication should be controlled +centrally by the framework. + +For logging there is a class variable called *logger* that works like a +normal logger. + +An example of a plugin can look like this: + +.. _test_rest: +.. code:: python + + class TestRest(RestResource): + endpoints = ['test'] + def get(self): + self.logger.debug("Got get request") + return {"code": 0, "description": "", "data": "Foobar"} + +Arguments: +~~~~~~~~~~ + +For parsing the arguments from the message body there function defined in `RestResource`_ +called *get_args*. This function will return the arguments that are in the request. +The parser needs to be initialized with *parser_arguments* variable that is a list of +variables your module want's to parse. + +If one needs to define more complex type of an argument it can be done with the help of *RequestArgument*. +This class provides the means of: + + * Setting a default value + + * Validation of the value by callback function + + * The type of the value + +When this type of argument is passed as one (or more) of the values. The validation +will be automatically triggered when calling the *get_args* from the *RestResource*. + + +BaseAuth +-------- + +This class defines the Base for the authentication. + +The class needs to define function *is_authenticated*. The function +gets the request as an argument. + +This function will be called when a plugin has specified the +authentication method as a derived class of BaseAuth. + +Here is an example of a very simple authentication class. + +.. code:: python + + from base_auth import BaseAuthMethod + + class TextBase(BaseAuthMethod): + def __init__(self): + super(TextBase, self).__init__() + self.user = '' + self.password = '' + with open('/tmp/foo') as f: + self.user, self.password = f.read().strip().split(':') + + def is_authenticated(self, request): + if request.authorization and request.authorization.username == self.user and request.authorization.password == self.password: + return True + return False + +Keystone +~~~~~~~~ + +For keystone additional configuration is needed: + +* User with admin role needs to be added (or admin used) + +* The config.ini has to contain the credentials and the url of keystone + +The following configuration needs to be added to config.ini + +.. code:: ini + + [keystone] + user=restful + password=foobar + auth_uri=http://192.168.1.15:5000 + +After the configuration is done and the authentication will be needed +then the http headers have to contain token with admin privileges as +X-Auth-Token. + +Restful framework binary +======================== + +The framework has only one binary. It's called restapi. The server will +be automatically started during the deployment. + +restapi config file +------------------- + +The default configuration file for the restful server is located at +/etc/yarf/config.ini + +To override the default config file, restapi can be started with command +line parameter --config. This allows testing plugins without +interference to the rest of the system. + +The config file contains the following parameters: + +.. code:: ini + + [restframe] + + #The port that the restful app will listen DEFAULT:61200 + port=61200 + #The IP address that the restful app will bind to DEFAULT:127.0.0.1 + ip_address=127.0.0.1 + + + #Use SSL or not + #If true then private key and certificate has to be also given DEFAULT:False + use_ssl=false + #ssl_private_key=PATHTOKEY/KEY.key + #ssl_certificate=PATHTOCERTIFICATE/CERT.crt + + #The directory where the handlers are + #Defaults to /opt/yarf/handlers + handler_directory=/opt/yarf/handlers + +The configuration file will be generated with an ansible module that will configure the framework. +Restapi service will run on all the controllers and listen to the controller internal management IP. +HAProxy will be configured so that clients can take a connection to the internal loadbalancer address +(or internal VIP) or external loadbalancer address (external VIP). +The framework will listen to port 61200. + +Creating plugins +================ + +First of all you need to have your own Class defined like the described +in `RestResource`_. The second thing needed is an ini file that describes +the handlers for different requests and the api version of the handler. + +The plugins should be placed in their own directory under : + +/opt/yarf/handlers. + +The thing that needs to be remembered that the object lifetime of *RestResource* is +and will be only the duration of the query. Any data stored by that query that is needed +cannot be stored in the internal variables of the module. This is anyway against the +*statelessness* nature of rest. + +Ini file for plugin +------------------- + +The recommendation is to have your own directory (although not mandatory) +per plugin. Within that directory you have to create an inifile that is +of the following format: + +.. code:: ini + + [] + handlers= + +Where the name of the inifile will the first part of your path on the +rest server. API_VERSION the second And the handler endpoint(s) the +third. + +For example if one would create an inifile for the `test_rest`_ resource +it would look like this: testing.ini: + +.. code:: ini + + [v1] + handlers=TestRest + +Then if you want to test your api you could do that with curl: + +curl http://testing/test/v1/test + +.. code:: json + + { + "code": 0, + "description": "u''" + "data": "u'Foobar'" + } + +There is also a helper to check the apis and their locations: + +curl http://testing/test/apis + +.. code:: json + + [ + { + "href": "http://testing/test/v1", + "id": "v1" + } + ] + + diff --git a/conf/config.ini b/conf/config.ini new file mode 100644 index 0000000..509e7ac --- /dev/null +++ b/conf/config.ini @@ -0,0 +1,43 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +[restframe] + +#The port that the restful app will listen DEFAULT:61200 +port=61200 +#The IP address that the restful app will bind to DEFAULT:127.0.0.1 +ip_address=127.0.0.1 + +# This option tells the restful app to run in debug mode or not. Defaults to True +debug=true + +#Use SSL or not +#If true then private key and certificate has to be also given DEFAULT:False +use_ssl=false +#ssl_private_key=PATHTOKEY/KEY.key +#ssl_certificate=PATHTOCERTIFICATE/CERT.crt + +#Authentication method. +auth_method=yarf.authentication.keystone.KeystoneAuth + +#The directory where the handlers are +#Defaults to /opt/yarf/handlers +#handler_directory=/opt/yarf/handlers + +[keystone] +user= +password= +auth_uri= + diff --git a/design/architecture.dia b/design/architecture.dia new file mode 100644 index 0000000..5a53855 Binary files /dev/null and b/design/architecture.dia differ diff --git a/design/architecture.jpeg b/design/architecture.jpeg new file mode 100644 index 0000000..69a30c8 Binary files /dev/null and b/design/architecture.jpeg differ diff --git a/required-secrets/restful.yaml b/required-secrets/restful.yaml new file mode 100644 index 0000000..2f0af0d --- /dev/null +++ b/required-secrets/restful.yaml @@ -0,0 +1,17 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +#Restful keystone secrets +restful_service_password: diff --git a/src/setup.py b/src/setup.py new file mode 100644 index 0000000..570e7fd --- /dev/null +++ b/src/setup.py @@ -0,0 +1,34 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +from setuptools import setup, find_packages +setup( + name='yarf', + version='1.0', + license='Commercial', + author='Janne Suominen', + author_email='janne.suominen@nokia.com', + packages=find_packages(), + include_package_data=True, + url='https://gitlab.fp.nsn-rdnet.net/jannsuom/restfulframework', + description='Yet Another Restful Framework', + install_requires=['flask', 'flask-restful'], + entry_points={ + 'console_scripts': [ + 'restapi = yarf.app:main', + ], + }, + zip_safe=False, + ) diff --git a/src/yarf/__init__.py b/src/yarf/__init__.py new file mode 100644 index 0000000..f035b4a --- /dev/null +++ b/src/yarf/__init__.py @@ -0,0 +1,15 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + diff --git a/src/yarf/apihandler.py b/src/yarf/apihandler.py new file mode 100644 index 0000000..d70a467 --- /dev/null +++ b/src/yarf/apihandler.py @@ -0,0 +1,32 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +from flask import request +from yarf.restresource import RestResource + + +class APIHandler(RestResource): + apis = set() + authentication_method = None + def create_api_reference(self): + ref = [] + for api in self.apis: + ver = {} + ver['api'] = api + ver['href'] = "%s%s" %(request.host_url, api) + ref.append(ver) + return ref + def get(self): + return self.create_api_reference() diff --git a/src/yarf/app.py b/src/yarf/app.py new file mode 100644 index 0000000..f6c5a83 --- /dev/null +++ b/src/yarf/app.py @@ -0,0 +1,154 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import sys +import logging +import socket +from OpenSSL import SSL +from flask import Flask, request +from flask_restful import Api +from werkzeug.exceptions import InternalServerError +from yarf.handlers.pluginhandler import PluginLoader +from yarf.iniloader import ConfigError +import yarf.restfulargs as restfulconfig +import yarf.restfullogger as restlog +from yarf.helpers import remove_secrets + +CRIT_RESP_LEN = 150000 + +app = Flask(__name__) +api = Api(app) +auth_method = None + +def handle_excp(failure): + if isinstance(failure, socket.error): + app.logger.warning("Socket error, ignoring") + return + elif failure: + app.logger.error("Internal error: %s ", failure) + else: + app.logger.info("Failure not defined... Ignoring.") + return + raise InternalServerError() + +def get_config(args, logger): + try: + config = restfulconfig.RestConfig() + if args: + config.parse(sys.argv[1:]) + else: + config.parse() + except ConfigError as error: + logger.error("Failed to start %s" % error) + return None + return config + +def request_logger(): + app.logger.info('Request: remote_addr: %s method: %s endpoint: %s, user: %s', request.remote_addr, + request.method, remove_secrets(request.full_path), get_username()) + +def response_logger(response): + app.logger.info('Response: status: %s (Associated Request: remote_addr: %s, method: %s, endpoint: %s, user: %s)', + response.status, request.remote_addr, request.method, + remove_secrets(request.full_path), get_username()) + + if len(response.data) > CRIT_RESP_LEN: + app.logger.debug('Response\'s data is too big, truncating!') + app.logger.debug('Response\'s truncated data: %s', response.data[:CRIT_RESP_LEN]) + else: + app.logger.debug('Response\'s data: %s', response.data) + + response.headers["Server"] = "Restapi" + + return response + +def get_username(): + try: + return auth_method.get_authentication(request)[1] + except Exception as err: # pylint: disable=broad-except + app.logger.warn("Failed to get username from request returning empty. Err: %s", str(err)) + return '' + + +def initialize(config, logger): + logger.info("Initializing...") + loglevel = logging.INFO if not config.get_debug() else logging.DEBUG + app.logger.setLevel(loglevel) + app.register_error_handler(Exception, handle_excp) + app.before_request(request_logger) + app.after_request(response_logger) + logger.error("%s", config.get_handler_dir()) + p = PluginLoader(config.get_handler_dir(), api, config.get_auth_method()) + auth_handler = p.get_auth_method() + handlers = p.get_modules() + for handler in handlers: + p.init_handler(handler) + + for handler in restlog.get_log_handlers(): + app.logger.addHandler(handler) + p.create_api_versionhandlers(handlers) + logger.info("Starting up...") + + +def get_wsgi_application(): + logger = restlog.get_logger() + config = get_config(None, logger) + initialize(config, logger) + return app + +def main(): + logger = restlog.get_logger() + config = get_config(sys.argv[1:], logger) + if not config: + raise ConfigError("Failed to read config file") + initialize(config, logger) + run_params = {} + run_params["debug"] = config.get_debug() + run_params["port"] = config.get_port() + run_params["host"] = config.get_ip() + # When this https://github.com/pallets/werkzeug/issues/954 is fixed then the error handling + # can be done in the error handler of app level + passthrough_errors = config.get_passthrough_errors() + run_params["passthrough_errors"] = passthrough_errors + run_params["threaded"] = config.is_threaded() + logger.debug("%s %s %s", run_params["debug"], run_params["port"], run_params["threaded"]) + if config.use_ssl(): + context = SSL.Context(SSL.SSLv23_METHOD) + context.use_privatekey_file(config.get_private_key()) + context.use_certificate_file(config.get_certificate()) + run_params['ssl_context'] = context + while True: + try: + app.run(**run_params) + except Exception as err: # pylint: disable=broad-except + logger.warning("Caught exception but starting again %s", err) + if passthrough_errors: + handle_excp(err) + else: + raise err + logger.warning("Die in piece %s", err) + func = request.environ.get('werkzeug.server.shutdown') + if func is None: + raise RuntimeError('Not running with the Werkzeug Server') + func() + + return 0 + +if __name__ == '__main__': + try: + sys.exit(main()) + except Exception as error:# pylint: disable=broad-except + print "Failure: %s" % error + sys.exit(255) diff --git a/src/yarf/authentication/__init__.py b/src/yarf/authentication/__init__.py new file mode 100644 index 0000000..f035b4a --- /dev/null +++ b/src/yarf/authentication/__init__.py @@ -0,0 +1,15 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + diff --git a/src/yarf/authentication/base_auth.py b/src/yarf/authentication/base_auth.py new file mode 100644 index 0000000..e675b1e --- /dev/null +++ b/src/yarf/authentication/base_auth.py @@ -0,0 +1,37 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +from flask import abort, request +from functools import wraps + +def login_required(func): + @wraps(func) + def wrapper(*args, **kwargs): + auth_method = func.__self__.authentication_method + if auth_method is None: + return func(*args, **kwargs) + + if isinstance(auth_method, BaseAuthMethod) and auth_method.get_authentication(request)[0]: + return func(*args, **kwargs) + else: + abort(401) + return None + return wrapper + +class BaseAuthMethod(object): + def __init__(self): + pass + def get_authentication(self, req): + raise NotImplementedError("Function get_authentication not implemented") diff --git a/src/yarf/authentication/keystone.py b/src/yarf/authentication/keystone.py new file mode 100644 index 0000000..0d0c8fd --- /dev/null +++ b/src/yarf/authentication/keystone.py @@ -0,0 +1,72 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +from keystoneauth1.identity import v3 +from keystoneauth1 import session +from keystoneclient.v3 import client +from keystoneclient.v3.tokens import TokenManager +from keystoneauth1.exceptions.http import Unauthorized, NotFound + +import yarf.restfullogger as logger + +from yarf.authentication.base_auth import BaseAuthMethod +from yarf.restfulargs import RestConfig + + +class KeystoneAuth(BaseAuthMethod): + def __init__(self): + super(KeystoneAuth, self).__init__() + self.logger = logger.get_logger() + config = RestConfig() + config.parse() + conf = config.get_section("keystone", format='dict') + try: + self.user = conf["user"] + self.password = conf["password"] + self.uri = conf["auth_uri"] + '/v3' + self.domain = "default" + except KeyError as error: + self.logger.error("Failed to find all the needed parameters. Authentication with Keystone not possible: {}" + .format(str(error))) + self.auth = v3.Password(auth_url=self.uri, + username=self.user, + password=self.password, + user_domain_id=self.domain) + self.sess = session.Session(auth=self.auth) + self.keystone = client.Client(session=self.sess) + self.tokenmanager = TokenManager(self.keystone) + + def get_authentication(self, req): + try: + token = req.headers.get("X-Auth-Token", type=str) + except KeyError: + self.logger.error("Failed to get the authentication token from request") + return (False, "") + + try: + tokeninfo = self.tokenmanager.validate(token) + except Unauthorized: + self.logger.error("Failed to authenticate with given credentials") + return (False, "") + except NotFound: + self.logger.error("Unauthorized token") + return (False, "") + except Exception as error: + self.logger.error("Failure: {}".format(str(error))) + return (False, "") + + if 'admin' in tokeninfo.role_names: + return (True, 'admin') + return (False, "") diff --git a/src/yarf/authentication/text_base.py b/src/yarf/authentication/text_base.py new file mode 100644 index 0000000..cf493b0 --- /dev/null +++ b/src/yarf/authentication/text_base.py @@ -0,0 +1,29 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +from yarf.authentication.base_auth import BaseAuthMethod + +class TextBase(BaseAuthMethod): + def __init__(self): + super(TextBase, self).__init__() + self.user = '' + self.password = '' + with open('/tmp/foo') as f: + self.user, self.password = f.read().strip().split(':') + + def get_authentication(self, req): + if req.authorization and req.authorization.username == self.user and req.authorization.password == self.password: + return (True, "") + return (False, "") diff --git a/src/yarf/baseresource.py b/src/yarf/baseresource.py new file mode 100644 index 0000000..14350be --- /dev/null +++ b/src/yarf/baseresource.py @@ -0,0 +1,46 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +from flask_restful import Resource, reqparse +from yarf.authentication.base_auth import login_required + +class BaseResource(Resource): + # THESE VALUES ARE FILLED BY THE FRAMEWORK + method_decorators = None +# authentication_method = None + api_versions = [] + subarea = "none" + parser = None + logger = None + + # USED INTERNALLY ONLY + @classmethod + def add_wrappers(cls): + cls.method_decorators = [login_required] + for extra_wrapper in cls.extra_wrappers: + if extra_wrapper not in cls.method_decorators: + cls.logger.debug("Adding wrapper %s", extra_wrapper) + cls.method_decorators.append(extra_wrapper) + else: + cls.logger.debug("Not added %s", extra_wrapper) + + @classmethod + def add_parser_arguments(cls): + cls.parser = reqparse.RequestParser() + for argument in cls.parser_arguments: + if isinstance(argument, cls.int_arg_class): + cls.parser.add_argument(argument.argument_class) + else: + cls.parser.add_argument(argument) diff --git a/src/yarf/config_defaults.py b/src/yarf/config_defaults.py new file mode 100644 index 0000000..e6712ed --- /dev/null +++ b/src/yarf/config_defaults.py @@ -0,0 +1,18 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +default_config_file = "/etc/yarf/config.ini" +default_section = "restframe" +config_defaults = {"port":"61200", "ip_address": "127.0.0.1", "use_ssl": "False", "handler_directory": '/usr/lib/python2.7/site-packages/yarf/handlers/', "threaded": "True", "passthrough_errors": "True"} diff --git a/src/yarf/exceptions.py b/src/yarf/exceptions.py new file mode 100644 index 0000000..c6675a4 --- /dev/null +++ b/src/yarf/exceptions.py @@ -0,0 +1,18 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +class ConfigError(Exception): + def __init__(self, message): + super(ConfigError, self).__init__(message) diff --git a/src/yarf/handlers/__init__.py b/src/yarf/handlers/__init__.py new file mode 100644 index 0000000..f035b4a --- /dev/null +++ b/src/yarf/handlers/__init__.py @@ -0,0 +1,15 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + diff --git a/src/yarf/handlers/pluginhandler.py b/src/yarf/handlers/pluginhandler.py new file mode 100644 index 0000000..aa145a6 --- /dev/null +++ b/src/yarf/handlers/pluginhandler.py @@ -0,0 +1,173 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import os +import os.path +import sys +import inspect +from yarf.iniloader import INILoader +from yarf.restresource import RestResource +from yarf.versionhandler import VersionHandler +from yarf.authentication.base_auth import BaseAuthMethod +from yarf.exceptions import ConfigError +import yarf.restfullogger as restlog + +class PluginLoader(object): + def __init__(self, path, api, auth_method): + self.logger = restlog.get_logger() + self.plugin_class_type = RestResource + self.auth_method = self._get_auth_method(auth_method) + self.path = path + self.api = api + + def get_module_dirs(self): + files = os.listdir(self.path) + modules = [] + for f in files: + if os.path.isdir("%s/%s"%(self.path, f)): + modules.append("%s/%s"%(self.path, f)) + return modules + + def _get_auth_method(self, authmethod): + auth_class_module = None + class_name = None + try: + auth_class_module, class_name = authmethod.rsplit('.', 1) + except ValueError: + error = "Cannot decode the authentication method from configuration file" + self.logger.error(error) + raise ConfigError(error) + auth_classes = self._get_classes_wanted_classes(auth_class_module, [class_name], BaseAuthMethod) + if auth_classes is None or auth_classes == []: + error = "Cannot find the authentication class in provided module %s %s" % (auth_class_module, class_name) + raise ConfigError(error) + return auth_classes[0] + + def _get_classes_wanted_classes(self, module_name, wanted_modules, class_type): + classes = [] + try: + __import__(module_name) + except ImportError: + self.logger.error("Failed import in %s, skipping", module_name) + return None + module = sys.modules[module_name] + for obj_name in dir(module): + # Skip objects that are meant to be private. + if obj_name.startswith('_'): + continue + # Skip the same name that base class has + elif obj_name == class_type.__name__: + continue + elif obj_name not in wanted_modules: + continue + itm = getattr(module, obj_name) + if inspect.isclass(itm) and issubclass(itm, class_type): + classes.append(itm) + return classes + + def get_classes_from_dir(self, directory, wanted_modules): + classes = [] + if directory not in sys.path: + sys.path.append(directory) + for fname in os.listdir(directory): + root, ext = os.path.splitext(fname) + if ext != '.py' or root == '__init__': + continue + module_name = "%s" % (root) + + mod_classes = self._get_classes_wanted_classes(module_name, wanted_modules, self.plugin_class_type) + if mod_classes: + classes.extend(mod_classes) + return classes + + def get_modules_from_dir(self, module_dir): + modules = {} + for f in os.listdir(module_dir): + if not f.endswith(".ini"): + continue + root, _ = os.path.splitext(f) + loader = INILoader("%s/%s" %(module_dir, f)) + sections = loader.get_sections() + modules[root] = {} + for section in sections: + handlers = loader.get_handlers(section) + if handlers: + modules[root][section] = handlers + else: + self.logger.error("Problem in the configuration file %s in section %s: No handlers found", f, section) + return modules + + def get_auth_method(self): + return self.auth_method() + + def get_modules(self): + dirs = self.get_module_dirs() + auth_class = self.auth_method() + modules = [] + for d in dirs: + wanted_modules = self.get_modules_from_dir(d) + for mod in wanted_modules.keys(): + for api_version in wanted_modules[mod].keys(): + classes = self.get_classes_from_dir(d, wanted_modules[mod][api_version]) + if not classes: + continue + for c in classes: + setattr(c, "subarea", mod) + if getattr(c, "authentication_method", "EMPTY") == "EMPTY": + setattr(c, "authentication_method", auth_class) + if getattr(c, "api_versions", None): + c.api_versions.append(api_version) + else: + setattr(c, "api_versions", [api_version]) + for cls in classes: + if cls not in modules: + modules.append(cls) + return modules + + def create_endpoints(self, handler): + endpoint_list = [] + for endpoint in handler.endpoints: + for api_version in handler.api_versions: + self.logger.debug("Registering /%s/%s/%s for %s", handler.subarea, api_version, endpoint, handler.__name__) + endpoint_list.append("/%s/%s/%s"% (handler.subarea, api_version, endpoint)) + self.api.add_resource(handler, *(endpoint_list)) + + def add_logger(self, handler): + self.logger.info("Adding logger to: %s", handler.__name__) + handler.logger = self.logger + + def init_handler(self, handler): + + self.add_logger(handler) + handler.add_wrappers() + self.create_endpoints(handler) + handler.add_parser_arguments() + + def create_api_versionhandlers(self, handlers): + apiversions = {} + endpoint_list = [] + for handler in handlers: + subarea = handler.subarea + if apiversions.get(subarea, False): + for hapiversion in handler.api_versions: + if hapiversion not in apiversions[subarea]: + apiversions[subarea].append(hapiversion) + else: + apiversions[subarea] = handler.api_versions + self.logger.debug("Registering /%s/apis for %s", subarea, subarea) + endpoint_list.append("/%s/apis" % subarea) + setattr(VersionHandler, "versions", apiversions) + setattr(VersionHandler, "method_decorators", []) + self.api.add_resource(VersionHandler, *(endpoint_list)) diff --git a/src/yarf/helpers.py b/src/yarf/helpers.py new file mode 100644 index 0000000..917a506 --- /dev/null +++ b/src/yarf/helpers.py @@ -0,0 +1,27 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import re + +SECRETS = ["password", "community-string"] + + +def remove_secrets(endpoint): + for secret in SECRETS: + endpoint = re.sub(r'%s=(.[^&]*)' % secret, "%s=*****" % secret, endpoint) + return endpoint + + + diff --git a/src/yarf/iniloader.py b/src/yarf/iniloader.py new file mode 100644 index 0000000..3c762ea --- /dev/null +++ b/src/yarf/iniloader.py @@ -0,0 +1,66 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import configparser +from yarf.exceptions import ConfigError + + +class INILoader(dict): + def __init__(self, inifile, defaults=None, defaultsection=None): + super(INILoader, self).__init__(self) + self.inifile = inifile + self.handlers = 'handlers' + self.configparser = configparser.ConfigParser(defaults) + self.config = self.configparser.read(inifile) + self.defaultsection = defaultsection + if inifile not in self.config: + raise ConfigError("Failed to read config file: %s" % inifile) + + def get_sections(self): + return self.configparser.sections() + + def get_handlers(self, section): + return self[section][self.handlers].split(',') + + def __getitem__(self, key): + try: + return self.configparser[key] + except KeyError: + raise ConfigError("No such key %s" % key) + + def get(self, key, section=None, type_of_value=str): + if section is None and self.defaultsection is not None: + section = self.defaultsection + else: + return None + + if type_of_value is int: + return self.configparser.getint(section, key) + elif type_of_value is bool: + return self.configparser.getboolean(section, key) + elif type_of_value is float: + return self.configparser.getfloat(section, key) + return self.configparser.get(section, key) + + def keys(self): + return self.configparser.sections() + + def get_section(self, section, format='list'): + if section in self.keys(): + items = self.configparser.items(section) + if format == 'dict': + return dict(items) + return items + return None diff --git a/src/yarf/restfulargs.py b/src/yarf/restfulargs.py new file mode 100644 index 0000000..5624b5c --- /dev/null +++ b/src/yarf/restfulargs.py @@ -0,0 +1,119 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import argparse +import sys + +import yarf.config_defaults as config_defaults +import yarf.restfullogger as restfullogger + +from yarf.iniloader import INILoader +from yarf.exceptions import ConfigError + + +def exception_handler(func): + def exception_wrapper(*args, **kwargs): + try: + restlogger = restfullogger.get_logger() + restlogger.debug("calling {}".format(func.__name__)) + ret = func(*args, **kwargs) + return ret + except Exception as error: + restlogger.info("Exception from function {} (error: {})".format(func.__name__, str(error))) + if isinstance(error, ConfigError): + raise error + else: + raise ConfigError(str(error)) + return exception_wrapper + + +class RestConfig(object): + __restinstance = None + + def __new__(cls): + if RestConfig.__restinstance is None: + RestConfig.__restinstance = object.__new__(cls) + return RestConfig.__restinstance + + def __init__(self): + self.default_section = config_defaults.default_section + self.config_default = config_defaults.config_defaults + self.default_config_file = config_defaults.default_config_file + + self.config = None + self.config_file = None + + @exception_handler + def parse(self, args=sys.argv[1:]): + parser = argparse.ArgumentParser(description='Restful server') + parser.add_argument('--config', + type=str, + default=self.default_config_file, + help="Configuration file", + dest='config_file') + + args = parser.parse_args(args) + self.config_file = args.config_file + self.config = INILoader(self.config_file, defaults=self.config_default, defaultsection=self.default_section) + + @exception_handler + def get_port(self): + return self.config.get('port', type_of_value=int) + + @exception_handler + def get_ip(self): + return self.config.get('ip_address') + + @exception_handler + def use_ssl(self): + return self.config.get('use_ssl', type_of_value=bool) + + @exception_handler + def get_private_key(self): + if self.use_ssl(): + return self.config.get('ssl_private_key') + return None + + @exception_handler + def get_certificate(self): + if self.use_ssl(): + return self.config.get('ssl_certificate') + return None + + @exception_handler + def get_handler_dir(self): + return self.config.get('handler_directory') + + def get_section(self, section, format='list'): + return self.config.get_section(section, format=format) + + def get_debug(self): + return self.config.get('debug', type_of_value=bool) + + @exception_handler + def get_passthrough_errors(self): + return self.config.get('passthrough_errors', type_of_value=bool) + + @exception_handler + def is_threaded(self): + return self.config.get('threaded', type_of_value=bool) + + @exception_handler + def get_auth_method(self): + return self.config.get('auth_method') + + +def get_config(): + return RestConfig() diff --git a/src/yarf/restfullogger.py b/src/yarf/restfullogger.py new file mode 100644 index 0000000..44fb074 --- /dev/null +++ b/src/yarf/restfullogger.py @@ -0,0 +1,67 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import logging +import logging.handlers +restlogger = None + + +class RestfulLogger(object): + def __init__(self): + self.logger = logging.getLogger("Restfulserver") + self.logger.setLevel(logging.DEBUG) + # werkzug logs out endpoint in DEBUG level, and aaa feature endpoint contains password + # in clear text, so log level setting is needed to avoid showing password in journalctl + logging.getLogger('werkzeug').setLevel(logging.WARNING) + self.handlers = [] + self.sysloghandler = self._get_syslog_handler() + self.handlers.append(self.sysloghandler) + self._add_handlers() + + def __del__(self): + handlers = self.logger.handlers[:] + for handler in handlers: + handler.close() + self.logger.removeHandler(handler) + + @staticmethod + def _get_syslog_handler(): + sh = logging.handlers.SysLogHandler(address='/dev/log') + formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') + sh.setFormatter(formatter) + sh.setLevel(logging.NOTSET) + return sh + + def _add_handlers(self): + for handler in self.handlers: + self.logger.addHandler(handler) + + def get_handlers(self): + return self.handlers + + def get_logger(self): + return self.logger + +def get_logger(): + global restlogger + if not restlogger: + restlogger = RestfulLogger() + return restlogger.get_logger() + +def get_log_handlers(): + global restlogger + if not restlogger: + restlogger = RestfulLogger() + return restlogger.get_handlers() diff --git a/src/yarf/restresource.py b/src/yarf/restresource.py new file mode 100644 index 0000000..4e8af83 --- /dev/null +++ b/src/yarf/restresource.py @@ -0,0 +1,93 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +import inspect +import six +from flask import request +from flask_restful import reqparse +from werkzeug.exceptions import BadRequest + +from yarf.baseresource import BaseResource + +class RequestArgument(object): + """ More advanced arguments + Parameters: + name: Name of the argument + default: The default value if not defined + validate: function pointer to a validation function that will + be called to validate the argument. + The function should return tuple containing + status: (boolean) True if validation passed + False if not + reason: (string) Setting the reasoning for the + failure + typeof: The typeof the argument. The argument will be converted + to the type you define. Should be a function pointer + """ + def __init__(self, name, default=None, validate=None, typeof=lambda x: six.text_type(x)): + self.argument_class = reqparse.Argument(name=name, default=default, type=typeof) + if validate and inspect.isfunction(validate): + self.validate_func = validate + else: + self.validate_func = None + self.name = name + + def validate(self, value): + if not self.validate_func: + return + status, reason = self.validate_func(value) + if not status: + raise BadRequest(description=reason) + +class RestResource(BaseResource): + """ Class from which the plugins should inherit + Variables: + extra_wrappers: are function wrappers that will + be executed when any function is + executed by the frame (for example + get) + parser_arguments: Are the arguments that can be defined + if you need arguments for your plugin + these arguments can be fetched with + get_args + """ + extra_wrappers = [] + parser_arguments = [] + endpoints = None + int_arg_class = RequestArgument + + """ Function to get arguments from request + The function will call validate to the + arguments if they are of type RequestArgument + Returns: A dictionary of the arguments + """ + @classmethod + def get_args(cls): + args = cls.parser.parse_args() + for arg in args.keys(): + for parg in cls.parser_arguments: + if isinstance(parg, cls.int_arg_class) and parg.name == arg: + parg.validate(args[arg]) + return args + + @classmethod + def get_token(cls): + token = "" + try: + token = request.headers.get("X-Auth-Token", type=str) + except KeyError as err: + cls.logger.info("Failed to get auth token from request.") + return token + diff --git a/src/yarf/versionhandler.py b/src/yarf/versionhandler.py new file mode 100644 index 0000000..5937817 --- /dev/null +++ b/src/yarf/versionhandler.py @@ -0,0 +1,38 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +from flask import request +from yarf.restresource import RestResource + +def get_api(req): + splitted = req.full_path.split("/") + domain = splitted[1] + return domain + +class VersionHandler(RestResource): + versions = {} + + def create_api_reference(self, api): + ref = [] + for version in self.versions[api]: + ver = {} + ver['id'] = version + ver['href'] = "%s%s/%s" %(request.host_url, api, version) + ref.append(ver) + return ref + + def get(self): + api = get_api(request) + return self.create_api_reference(api) diff --git a/systemd/restapi.service b/systemd/restapi.service new file mode 100644 index 0000000..f3156d2 --- /dev/null +++ b/systemd/restapi.service @@ -0,0 +1,27 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +[Unit] +Description=Restful api server +DefaultDependencies=no + +[Service] +Restart=on-failure +RestartSec=3 +ExecStart=/usr/local/bin/restapi +User=restapi + +[Install] +WantedBy=multi-user.target diff --git a/yarf.spec b/yarf.spec new file mode 100644 index 0000000..70dd0c8 --- /dev/null +++ b/yarf.spec @@ -0,0 +1,78 @@ +# Copyright 2019 Nokia +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +Name: yarf +Version: %{_version} +Release: 1%{?dist} +Summary: Yet Another Restfulframework +License: %{_platform_licence} +Source0: %{name}-%{version}.tar.gz +Vendor: %{_platform_vendor} + +Requires: python-flask, python2-flask-restful, python2-configparser, python2-requests, mod_wsgi, python2-six +BuildRequires: python +BuildRequires: python-setuptools + +%description +Yet Another Restfulframework. + +%prep +#./autogen.sh +%autosetup + +%build + +%install +rm -rf $RPM_BUILD_ROOT +mkdir -p %{buildroot}%{_platform_etc_path}/yarf +mkdir -p %{buildroot}%{_platform_etc_path}/required-secrets +mkdir -p %{buildroot}%{_unitdir}/ +mkdir -p %{buildroot}/var/log/restapi +cp required-secrets/*.yaml %{buildroot}/%{_platform_etc_path}/required-secrets +#mkdir -p {buildroot}/etc/httpd/conf.d/ +#mkdir -p {buildroot}/var/www/yarf/ + +cd src && python setup.py install --root %{buildroot} --no-compile --install-purelib %{_python_site_packages_path} --install-scripts %{_platform_bin_path} && cd - + +rsync -rv systemd/* %{buildroot}%{_unitdir}/ + +%files +%defattr(0755,root,root,0755) +%{_python_site_packages_path}/yarf* +%attr(0755,restapi, restapi) %{_platform_etc_path}/yarf/ +%{_platform_etc_path}/required-secrets/restful.yaml +#/etc/ansible/roles/restful +#/opt/openstack-ansible/playbooks/yarf.yml +%attr(0755,root, root) %{_platform_bin_path}/restapi +# %attr(0644,root, root) %{_unitdir}/restapi.service +%attr(0644,root, root) %{_unitdir}/* +%dir %attr(0770, restapi,restapi) /var/log/restapi + +%pre +/usr/bin/getent passwd restapi > /dev/null||/usr/sbin/useradd -r -s /sbin/nologin -M restapi + +%post +if [ $1 -eq 2 ]; then + if [ -f %{_platform_etc_path}/restful/config.ini ]; then + sudo /usr/bin/systemctl restart restapi + fi +fi + +%postun + +#Uninstall +if [ $1 -eq 0 ];then + /usr/sbin/userdel restapi +fi