Seed code for yarf

Seed code for yarf

Change-Id: I220af8eba8ef38dda2a40f72b38220c50a40a446
Signed-off-by: Janne Suominen <janne.suominen@nokia.com>

diff --git a/LICENSE b/LICENSE
new file mode 100644 (file)
index 0000000..d645695
--- /dev/null
+++ b/LICENSE
@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

diff --git a/README.rst b/README.rst
new file mode 100644 (file)
index 0000000..707039a
--- /dev/null
+++ b/README.rst
@@ -0,0 +1,402 @@
+::
+
+   Copyright 2019 Nokia
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
+
+
+=========================================
+YARF REST framework
+=========================================
+
+:Author: Janne Suominen
+
+.. raw:: pdf
+
+    PageBreak
+
+.. sectnum::
+
+.. contents::
+
+.. raw:: pdf
+
+    PageBreak
+
+
+Introduction
+============
+
+What is REST
+------------
+**Representational state transfer** (REST) or **RESTful** web services is a way of providing 
+interoperability between computer systems on the Internet. REST-compliant Web services allow 
+requesting systems to access and manipulate textual representations of Web resources using a 
+uniform and predefined set of stateless operations.
+
+About this project
+------------------
+
+YARF --- Yet Another Restful Framework
+
+This project provides an implementation for a generic rest framework The
+framework is based on
+`Flask-RESTful <https://flask-restful.readthedocs.io/en/0.3.5/>`__ 
+
+|  
+
+*The framework provides the following:*
+
+* A generic rest server for whole cloud 
+
+* A plugin based interface for creating rest apis.
+
+|  
+
+*At the high level, the framework serves the following purposes*:
+
+* Provides a unified interface for creating rest interfaces. 
+
+* Single point of entry with plugin based authentication for apis. 
+
+* Define the rest response and query format.
+
+| 
+
+Requirements
+------------
+
+* Framework shall provide easy way to integrate new REST apis via plugins
+
+* Framework shall be integrated to the CM and provide means to automatically configure itself
+
+* Framework shall listen to management address of controller on all the controllers
+
+* Framework shall listen to external and internal load balancer address
+
+* Framework shall provide means to configure SSL certification
+
+* Framework shall be integrated to Authentication mechanism provided by the platform
+
+* Framework shall provide means to validate the parameters given as part of the request
+
+* Framework shall return an error code in case of:
+    
+    * Internal failure; Any failure within the module is considered as internal failure
+
+    * Authentication failure; When authentication has failed or missing credentials
+
+    * Not found; When the object is not found
+
+Structure of data
+=================
+
+The implementation of this framework promotes returning JSON format
+objects with key value pairs.
+
+Structure of the operation
+--------------------------
+
+The framework supports adding function calls for any HTTP requests. The
+request can either: 
+
+* Have a request in the body of the message as JSON
+
+* Have a request embedded in the url
+
+Structure of response
+---------------------
+
+This framework does not enforce any special structure for the response,
+but **it's strongly** encouraged to use the following formatting for the
+response:
+
+.. code:: json
+
+     { 
+       code: <INT>
+       description: <STR>
+       data: <DICT>
+     }
+
+Where: 
+
+* code is the return value of the api in question. 
+
+  * 0 means no error and anything other is considered as failure 
+
+* description is the description of the possible failure (can be left empty in case there is no failure) 
+
+* data is the data returned by the api. The data should be in dictionary (JSON) format
+
+The reasoning for the quite strict guidelines is: 
+
+* Uniqueness of the response makes it easier for the upper level to check the response
+
+**Note**: The framework will return HTTP status code that is not 200 in
+case of: 
+
+* 500: Internal failure (ie. Uncaught exception from the plugin) 
+
+* 401: In case of authentication failure (if authentication is defined)
+
+* 404: In case the object requested is not found
+
+High level architecture of the restful framework
+================================================
+
+At the high level, there is a layer built on top of flask-restful to be
+able to: 
+
+* Isolation of the framework implementation details. 
+
+* To be able to provide more specific implementation to fit to our needs. 
+
+* To be able to make a single point of entry to the clusters rest api 
+
+* Flexibility to change different parts of the implementation without affecting the users of the framework. 
+
+* Provide unique responses to caller for easy parsing
+
+
+.. figure:: ./design/architecture.jpeg
+   :alt: architecture
+
+   architecture
+
+Restful framework interfaces
+============================
+
+RestResource
+------------
+
+All the plugins have to inherit from this class. This class is the basis
+of the plugin framework. All the plugins that inherit from this object
+and are defined in plugin specific inifile will be automatically
+imported.
+
+The http requests will be converted to corresponding lowercase
+functions. For example: request method GET will call *get()* function and
+POST will call *post()*.
+
+The resource should also define endpoints where it want's to register
+these calls. This is done by setting the *endpoint* class variable list.
+
+
+For decorating functions with decorators *extra_wrappers* can be used.
+The function must return either the function or a dictionary that is of
+the same format defined in `Structure of Response`_.
+
+To have authentication for the module adding class variable named
+*authentication_method* needs to be defined. For production environment
+this should be left untouched since the authentication should be controlled
+centrally by the framework.
+
+For logging there is a class variable called *logger* that works like a
+normal logger.
+
+An example of a plugin can look like this:
+
+.. _test_rest:
+.. code:: python
+
+    class TestRest(RestResource):
+        endpoints = ['test'] 
+        def get(self):
+            self.logger.debug("Got get request")
+            return {"code": 0, "description": "", "data": "Foobar"}
+
+Arguments:
+~~~~~~~~~~
+
+For parsing the arguments from the message body there function defined in `RestResource`_
+called *get_args*. This function will return the arguments that are in the request.
+The parser needs to be initialized with *parser_arguments* variable that is a list of
+variables your module want's to parse. 
+
+If one needs to define more complex type of an argument it can be done with the help of *RequestArgument*.
+This class provides the means of:
+    
+    * Setting a default value
+    
+    * Validation of the value by callback function
+    
+    * The type of the value
+
+When this type of argument is passed as one (or more) of the values. The validation
+will be automatically triggered when calling the *get_args* from the *RestResource*.
+
+
+BaseAuth
+--------
+
+This class defines the Base for the authentication.
+
+The class needs to define function *is_authenticated*. The function
+gets the request as an argument.
+
+This function will be called when a plugin has specified the
+authentication method as a derived class of BaseAuth.
+
+Here is an example of a very simple authentication class.
+
+.. code:: python
+
+    from base_auth import BaseAuthMethod
+
+    class TextBase(BaseAuthMethod):
+        def __init__(self):
+            super(TextBase, self).__init__()
+            self.user = ''
+            self.password = ''
+            with open('/tmp/foo') as f:
+                self.user, self.password = f.read().strip().split(':')
+
+        def is_authenticated(self, request):
+            if request.authorization and request.authorization.username == self.user and request.authorization.password == self.password:
+                return True
+            return False
+
+Keystone
+~~~~~~~~
+
+For keystone additional configuration is needed: 
+
+* User with admin role needs to be added (or admin used)
+
+* The config.ini has to contain the credentials and the url of keystone
+
+The following configuration needs to be added to config.ini
+
+.. code:: ini
+
+    [keystone]
+    user=restful
+    password=foobar
+    auth_uri=http://192.168.1.15:5000
+
+After the configuration is done and the authentication will be needed
+then the http headers have to contain token with admin privileges as
+X-Auth-Token.
+
+Restful framework binary
+========================
+
+The framework has only one binary. It's called restapi. The server will
+be automatically started during the deployment.
+
+restapi config file
+-------------------
+
+The default configuration file for the restful server is located at
+/etc/yarf/config.ini
+
+To override the default config file, restapi can be started with command
+line parameter --config. This allows testing plugins without
+interference to the rest of the system.
+
+The config file contains the following parameters:
+
+.. code:: ini
+
+    [restframe]
+
+    #The port that the restful app will listen DEFAULT:61200
+    port=61200
+    #The IP address that the restful app will bind to DEFAULT:127.0.0.1
+    ip_address=127.0.0.1
+
+
+    #Use SSL or not
+    #If true then private key and certificate has to be also given DEFAULT:False
+    use_ssl=false
+    #ssl_private_key=PATHTOKEY/KEY.key
+    #ssl_certificate=PATHTOCERTIFICATE/CERT.crt
+
+    #The directory where the handlers are 
+    #Defaults to /opt/yarf/handlers
+    handler_directory=/opt/yarf/handlers
+
+The configuration file will be generated with an ansible module that will configure the framework.
+Restapi service will run on all the controllers and listen to the controller internal management IP.
+HAProxy will be configured so that clients can take a connection to the internal loadbalancer address 
+(or internal VIP) or external loadbalancer address (external VIP). 
+The framework will listen to port 61200.
+
+Creating plugins
+================
+
+First of all you need to have your own Class defined like the described
+in `RestResource`_. The second thing needed is an ini file that describes
+the handlers for different requests and the api version of the handler.
+
+The plugins should be placed in their own directory under :
+
+/opt/yarf/handlers.
+
+The thing that needs to be remembered that the object lifetime of *RestResource* is
+and will be only the duration of the query. Any data stored by that query that is needed
+cannot be stored in the internal variables of the module. This is anyway against the 
+*statelessness* nature of rest.
+
+Ini file for plugin
+-------------------
+
+The recommendation is to have your own directory (although not mandatory)
+per plugin. Within that directory you have to create an inifile that is
+of the following format:
+
+.. code:: ini
+
+    [<API_VERSION>]
+    handlers=<YOUR_HANDLER>
+
+Where the name of the inifile will the first part of your path on the
+rest server. API_VERSION the second And the handler endpoint(s) the
+third.
+
+For example if one would create an inifile for the `test_rest`_ resource
+it would look like this: testing.ini:
+
+.. code:: ini
+
+    [v1]
+    handlers=TestRest
+
+Then if you want to test your api you could do that with curl:
+
+curl http://testing/test/v1/test
+
+.. code:: json
+
+    {
+        "code": 0, 
+        "description": "u''"
+        "data": "u'Foobar'"
+    }
+
+There is also a helper to check the apis and their locations:
+
+curl http://testing/test/apis
+
+.. code:: json
+
+    [
+      {
+        "href": "http://testing/test/v1", 
+        "id": "v1"
+      } 
+    ]
+
+

diff --git a/conf/config.ini b/conf/config.ini
new file mode 100644 (file)
index 0000000..509e7ac
--- /dev/null
+++ b/conf/config.ini
@@ -0,0 +1,43 @@
+# Copyright 2019 Nokia
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+[restframe]
+
+#The port that the restful app will listen DEFAULT:61200
+port=61200
+#The IP address that the restful app will bind to DEFAULT:127.0.0.1
+ip_address=127.0.0.1
+
+# This option tells the restful app to run in debug mode or not. Defaults to True
+debug=true
+
+#Use SSL or not
+#If true then private key and certificate has to be also given DEFAULT:False
+use_ssl=false
+#ssl_private_key=PATHTOKEY/KEY.key
+#ssl_certificate=PATHTOCERTIFICATE/CERT.crt
+
+#Authentication method.
+auth_method=yarf.authentication.keystone.KeystoneAuth
+
+#The directory where the handlers are 
+#Defaults to /opt/yarf/handlers
+#handler_directory=/opt/yarf/handlers
+
+[keystone]
+user=<USER>
+password=<PASSWORD>
+auth_uri=<URL:PORT>
+

diff --git a/design/architecture.dia b/design/architecture.dia
new file mode 100644 (file)
index 0000000..5a53855
Binary files /dev/null and b/design/architecture.dia differ

diff --git a/design/architecture.jpeg b/design/architecture.jpeg
new file mode 100644 (file)
index 0000000..69a30c8
Binary files /dev/null and b/design/architecture.jpeg differ

diff --git a/required-secrets/restful.yaml b/required-secrets/restful.yaml
new file mode 100644 (file)
index 0000000..2f0af0d
--- /dev/null
+++ b/required-secrets/restful.yaml
@@ -0,0 +1,17 @@
+# Copyright 2019 Nokia
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+#Restful keystone secrets
+restful_service_password:

diff --git a/src/setup.py b/src/setup.py
new file mode 100644 (file)
index 0000000..570e7fd
--- /dev/null
+++ b/src/setup.py
@@ -0,0 +1,34 @@
+# Copyright 2019 Nokia
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+from setuptools import setup, find_packages
+setup(
+    name='yarf',
+    version='1.0',
+    license='Commercial',
+    author='Janne Suominen',
+    author_email='janne.suominen@nokia.com',
+    packages=find_packages(),
+    include_package_data=True,
+    url='https://gitlab.fp.nsn-rdnet.net/jannsuom/restfulframework',
+    description='Yet Another Restful Framework',
+    install_requires=['flask', 'flask-restful'],
+    entry_points={
+        'console_scripts': [
+            'restapi = yarf.app:main',
+        ],
+    },
+    zip_safe=False,
+    )

diff --git a/src/yarf/__init__.py b/src/yarf/__init__.py
new file mode 100644 (file)
index 0000000..f035b4a
--- /dev/null
+++ b/src/yarf/__init__.py
@@ -0,0 +1,15 @@
+# Copyright 2019 Nokia
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+

diff --git a/src/yarf/apihandler.py b/src/yarf/apihandler.py
new file mode 100644 (file)
index 0000000..d70a467
--- /dev/null
+++ b/src/yarf/apihandler.py
@@ -0,0 +1,32 @@
+# Copyright 2019 Nokia
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+from flask import request
+from yarf.restresource import RestResource
+
+
+class APIHandler(RestResource):
+    apis = set()
+    authentication_method = None
+    def create_api_reference(self):
+        ref = []
+        for api in self.apis:
+            ver = {}
+            ver['api'] = api
+            ver['href'] = "%s%s" %(request.host_url, api)
+            ref.append(ver)
+        return ref
+    def get(self):
+        return self.create_api_reference()

diff --git a/src/yarf/app.py b/src/yarf/app.py
new file mode 100644 (file)
index 0000000..f6c5a83
--- /dev/null
+++ b/src/yarf/app.py
@@ -0,0 +1,154 @@
+# Copyright 2019 Nokia
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+import sys
+import logging
+import socket
+from OpenSSL import SSL
+from flask import Flask, request
+from flask_restful import Api
+from werkzeug.exceptions import InternalServerError
+from yarf.handlers.pluginhandler import PluginLoader
+from yarf.iniloader import ConfigError
+import yarf.restfulargs as restfulconfig
+import yarf.restfullogger as restlog
+from yarf.helpers import remove_secrets
+
+CRIT_RESP_LEN = 150000
+
+app = Flask(__name__)
+api = Api(app)
+auth_method = None
+
+def handle_excp(failure):
+    if isinstance(failure, socket.error):
+        app.logger.warning("Socket error, ignoring")
+        return
+    elif failure:
+        app.logger.error("Internal error: %s ", failure)
+    else:
+        app.logger.info("Failure not defined... Ignoring.")
+        return
+    raise InternalServerError()
+
+def get_config(args, logger):
+    try:
+        config = restfulconfig.RestConfig()
+        if args:
+            config.parse(sys.argv[1:])
+        else:
+            config.parse()
+    except ConfigError as error:
+        logger.error("Failed to start %s" % error)
+        return None
+    return config
+
+def request_logger():
+    app.logger.info('Request: remote_addr: %s method: %s endpoint: %s, user: %s', request.remote_addr,
+                    request.method, remove_secrets(request.full_path), get_username())
+
+def response_logger(response):
+    app.logger.info('Response: status: %s (Associated Request: remote_addr: %s, method: %s, endpoint: %s, user: %s)',
+                    response.status, request.remote_addr, request.method,
+                    remove_secrets(request.full_path), get_username())
+
+    if len(response.data) > CRIT_RESP_LEN:
+        app.logger.debug('Response\'s data is too big, truncating!')
+        app.logger.debug('Response\'s truncated data: %s', response.data[:CRIT_RESP_LEN])
+    else:
+        app.logger.debug('Response\'s data: %s', response.data)
+
+    response.headers["Server"] = "Restapi"
+
+    return response
+
+def get_username():
+    try:
+        return auth_method.get_authentication(request)[1]
+    except Exception as err: # pylint: disable=broad-except
+        app.logger.warn("Failed to get username from request returning empty. Err: %s", str(err))
+    return ''
+
+
+def initialize(config, logger):
+    logger.info("Initializing...")
+    loglevel = logging.INFO if not config.get_debug() else logging.DEBUG
+    app.logger.setLevel(loglevel)
+    app.register_error_handler(Exception, handle_excp)
+    app.before_request(request_logger)
+    app.after_request(response_logger)
+    logger.error("%s", config.get_handler_dir())
+    p = PluginLoader(config.get_handler_dir(), api, config.get_auth_method())
+    auth_handler = p.get_auth_method()
+    handlers = p.get_modules()
+    for handler in handlers:
+        p.init_handler(handler)
+
+    for handler in restlog.get_log_handlers():
+        app.logger.addHandler(handler)
+    p.create_api_versionhandlers(handlers)
+    logger.info("Starting up...")
+
+
+def get_wsgi_application():
+    logger = restlog.get_logger()
+    config = get_config(None, logger)
+    initialize(config, logger)
+    return app
+
+def main():
+    logger = restlog.get_logger()
+    config = get_config(sys.argv[1:], logger)
+    if not config:
+        raise ConfigError("Failed to read config file")
+    initialize(config, logger)
+    run_params = {}
+    run_params["debug"] = config.get_debug()
+    run_params["port"] = config.get_port()
+    run_params["host"] = config.get_ip()
+    # When this https://github.com/pallets/werkzeug/issues/954 is fixed then the error handling
+    # can be done in the error handler of app level
+    passthrough_errors = config.get_passthrough_errors()
+    run_params["passthrough_errors"] = passthrough_errors
+    run_params["threaded"] = config.is_threaded()
+    logger.debug("%s %s %s", run_params["debug"], run_params["port"], run_params["threaded"])
+    if config.use_ssl():
+        context = SSL.Context(SSL.SSLv23_METHOD)
+        context.use_privatekey_file(config.get_private_key())
+        context.use_certificate_file(config.get_certificate())
+        run_params['ssl_context'] = context
+    while True:
+        try:
+            app.run(**run_params)
+        except Exception as err: # pylint: disable=broad-except
+            logger.warning("Caught exception but starting again %s", err)
+            if passthrough_errors:
+                handle_excp(err)
+            else:
+                raise err
+            logger.warning("Die in piece %s", err)
+            func = request.environ.get('werkzeug.server.shutdown')
+            if func is None:
+               raise RuntimeError('Not running with the Werkzeug Server')
+            func()
+
+    return 0
+
+if __name__ == '__main__':
+    try:
+        sys.exit(main())
+    except Exception as error:# pylint: disable=broad-except
+        print "Failure: %s" % error
+        sys.exit(255)

diff --git a/src/yarf/authentication/__init__.py b/src/yarf/authentication/__init__.py
new file mode 100644 (file)
index 0000000..f035b4a
--- /dev/null
+++ b/src/yarf/authentication/__init__.py
@@ -0,0 +1,15 @@
+# Copyright 2019 Nokia
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+

diff --git a/src/yarf/authentication/base_auth.py b/src/yarf/authentication/base_auth.py
new file mode 100644 (file)
index 0000000..e675b1e
--- /dev/null
+++ b/src/yarf/authentication/base_auth.py
@@ -0,0 +1,37 @@
+# Copyright 2019 Nokia
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+from flask import abort, request
+from functools import wraps
+
+def login_required(func):
+    @wraps(func)
+    def wrapper(*args, **kwargs):
+        auth_method = func.__self__.authentication_method
+        if auth_method is None:
+            return func(*args, **kwargs)
+
+        if isinstance(auth_method, BaseAuthMethod) and auth_method.get_authentication(request)[0]:
+            return func(*args, **kwargs)
+        else:
+            abort(401)
+            return None
+    return wrapper
+
+class BaseAuthMethod(object):
+    def __init__(self):
+        pass
+    def get_authentication(self, req):
+        raise NotImplementedError("Function get_authentication not implemented")

diff --git a/src/yarf/authentication/keystone.py b/src/yarf/authentication/keystone.py
new file mode 100644 (file)
index 0000000..0d0c8fd
--- /dev/null
+++ b/src/yarf/authentication/keystone.py
@@ -0,0 +1,72 @@
+# Copyright 2019 Nokia
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+from keystoneauth1.identity import v3
+from keystoneauth1 import session
+from keystoneclient.v3 import client
+from keystoneclient.v3.tokens import TokenManager
+from keystoneauth1.exceptions.http import Unauthorized, NotFound
+
+import yarf.restfullogger as logger
+
+from yarf.authentication.base_auth import BaseAuthMethod
+from yarf.restfulargs import RestConfig
+
+
+class KeystoneAuth(BaseAuthMethod):
+    def __init__(self):
+        super(KeystoneAuth, self).__init__()
+        self.logger = logger.get_logger()
+        config = RestConfig()
+        config.parse()
+        conf = config.get_section("keystone", format='dict')
+        try:
+            self.user = conf["user"]
+            self.password = conf["password"]
+            self.uri = conf["auth_uri"] + '/v3'
+            self.domain = "default"
+        except KeyError as error:
+            self.logger.error("Failed to find all the needed parameters. Authentication with Keystone not possible: {}"
+                              .format(str(error)))
+        self.auth = v3.Password(auth_url=self.uri,
+                                username=self.user,
+                                password=self.password,
+                                user_domain_id=self.domain)
+        self.sess = session.Session(auth=self.auth)
+        self.keystone = client.Client(session=self.sess)
+        self.tokenmanager = TokenManager(self.keystone)
+
+    def get_authentication(self, req):
+        try:
+            token = req.headers.get("X-Auth-Token", type=str)
+        except KeyError:
+            self.logger.error("Failed to get the authentication token from request")
+            return (False, "")
+
+        try:
+            tokeninfo = self.tokenmanager.validate(token)
+        except Unauthorized:
+            self.logger.error("Failed to authenticate with given credentials")
+            return (False, "")
+        except NotFound:
+            self.logger.error("Unauthorized token")
+            return (False, "")
+        except Exception as error:
+            self.logger.error("Failure: {}".format(str(error)))
+            return (False, "")
+
+        if 'admin' in tokeninfo.role_names:
+            return (True, 'admin')
+        return (False, "")

diff --git a/src/yarf/authentication/text_base.py b/src/yarf/authentication/text_base.py
new file mode 100644 (file)
index 0000000..cf493b0
--- /dev/null
+++ b/src/yarf/authentication/text_base.py
@@ -0,0 +1,29 @@
+# Copyright 2019 Nokia
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+from yarf.authentication.base_auth import BaseAuthMethod
+
+class TextBase(BaseAuthMethod):
+    def __init__(self):
+        super(TextBase, self).__init__()
+        self.user = ''
+        self.password = ''
+        with open('/tmp/foo') as f:
+            self.user, self.password = f.read().strip().split(':')
+
+    def get_authentication(self, req):
+        if req.authorization and req.authorization.username == self.user and req.authorization.password == self.password:
+            return (True, "")
+        return (False, "")

diff --git a/src/yarf/baseresource.py b/src/yarf/baseresource.py
new file mode 100644 (file)
index 0000000..14350be
--- /dev/null
+++ b/src/yarf/baseresource.py
@@ -0,0 +1,46 @@
+# Copyright 2019 Nokia
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+from flask_restful import Resource, reqparse
+from yarf.authentication.base_auth import login_required
+
+class BaseResource(Resource):
+    # THESE VALUES ARE FILLED BY THE FRAMEWORK
+    method_decorators = None
+#   authentication_method = None
+    api_versions = []
+    subarea = "none"
+    parser = None
+    logger = None
+    
+    # USED INTERNALLY ONLY
+    @classmethod
+    def add_wrappers(cls):
+        cls.method_decorators = [login_required]
+        for extra_wrapper in cls.extra_wrappers:
+            if extra_wrapper not in cls.method_decorators:
+                cls.logger.debug("Adding wrapper %s", extra_wrapper)
+                cls.method_decorators.append(extra_wrapper)
+            else:
+                cls.logger.debug("Not added %s", extra_wrapper)
+
+    @classmethod
+    def add_parser_arguments(cls):
+        cls.parser = reqparse.RequestParser()
+        for argument in cls.parser_arguments:
+            if isinstance(argument, cls.int_arg_class):
+                cls.parser.add_argument(argument.argument_class)
+            else:
+                cls.parser.add_argument(argument)

diff --git a/src/yarf/config_defaults.py b/src/yarf/config_defaults.py
new file mode 100644 (file)
index 0000000..e6712ed
--- /dev/null
+++ b/src/yarf/config_defaults.py
@@ -0,0 +1,18 @@
+# Copyright 2019 Nokia
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+default_config_file = "/etc/yarf/config.ini"
+default_section = "restframe"
+config_defaults = {"port":"61200", "ip_address": "127.0.0.1", "use_ssl": "False", "handler_directory": '/usr/lib/python2.7/site-packages/yarf/handlers/', "threaded": "True", "passthrough_errors": "True"}

diff --git a/src/yarf/exceptions.py b/src/yarf/exceptions.py
new file mode 100644 (file)
index 0000000..c6675a4
--- /dev/null
+++ b/src/yarf/exceptions.py
@@ -0,0 +1,18 @@
+# Copyright 2019 Nokia
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+class ConfigError(Exception):
+    def __init__(self, message):
+        super(ConfigError, self).__init__(message)

diff --git a/src/yarf/handlers/__init__.py b/src/yarf/handlers/__init__.py
new file mode 100644 (file)
index 0000000..f035b4a
--- /dev/null
+++ b/src/yarf/handlers/__init__.py
@@ -0,0 +1,15 @@
+# Copyright 2019 Nokia
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+

diff --git a/src/yarf/handlers/pluginhandler.py b/src/yarf/handlers/pluginhandler.py
new file mode 100644 (file)
index 0000000..aa145a6
--- /dev/null
+++ b/src/yarf/handlers/pluginhandler.py
@@ -0,0 +1,173 @@
+# Copyright 2019 Nokia
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+import os
+import os.path
+import sys
+import inspect
+from yarf.iniloader import INILoader
+from yarf.restresource import RestResource
+from yarf.versionhandler import VersionHandler
+from yarf.authentication.base_auth import BaseAuthMethod
+from yarf.exceptions import ConfigError
+import yarf.restfullogger as restlog
+
+class PluginLoader(object):
+    def __init__(self, path, api, auth_method):
+        self.logger = restlog.get_logger()
+        self.plugin_class_type = RestResource
+        self.auth_method = self._get_auth_method(auth_method)
+        self.path = path
+        self.api = api
+
+    def get_module_dirs(self):
+        files = os.listdir(self.path)
+        modules = []
+        for f in files:
+            if os.path.isdir("%s/%s"%(self.path, f)):
+                modules.append("%s/%s"%(self.path, f))
+        return modules
+
+    def _get_auth_method(self, authmethod):
+        auth_class_module = None
+        class_name = None
+        try:
+            auth_class_module, class_name = authmethod.rsplit('.', 1)
+        except ValueError:
+            error = "Cannot decode the authentication method from configuration file"
+            self.logger.error(error)
+            raise ConfigError(error)
+        auth_classes = self._get_classes_wanted_classes(auth_class_module, [class_name], BaseAuthMethod)
+        if auth_classes is None or auth_classes == []:
+            error = "Cannot find the authentication class in provided module %s %s" % (auth_class_module, class_name)
+            raise ConfigError(error)
+        return auth_classes[0]
+
+    def _get_classes_wanted_classes(self, module_name, wanted_modules, class_type):
+        classes = []
+        try:
+            __import__(module_name)
+        except ImportError:
+            self.logger.error("Failed import in %s, skipping", module_name)
+            return None
+        module = sys.modules[module_name]
+        for obj_name in dir(module):
+            # Skip objects that are meant to be private.
+            if obj_name.startswith('_'):
+                continue
+            # Skip the same name that base class has
+            elif obj_name == class_type.__name__:
+                continue
+            elif obj_name not in wanted_modules:
+                continue
+            itm = getattr(module, obj_name)
+            if inspect.isclass(itm) and issubclass(itm, class_type):
+                classes.append(itm)
+        return classes
+
+    def get_classes_from_dir(self, directory, wanted_modules):
+        classes = []
+        if directory not in sys.path:
+            sys.path.append(directory)
+        for fname in os.listdir(directory):
+            root, ext = os.path.splitext(fname)
+            if ext != '.py' or root == '__init__':
+                continue
+            module_name = "%s" % (root)
+
+            mod_classes = self._get_classes_wanted_classes(module_name, wanted_modules, self.plugin_class_type)
+            if mod_classes:
+                classes.extend(mod_classes)
+        return classes
+
+    def get_modules_from_dir(self, module_dir):
+        modules = {}
+        for f in os.listdir(module_dir):
+            if not f.endswith(".ini"):
+                continue
+            root, _ = os.path.splitext(f)
+            loader = INILoader("%s/%s" %(module_dir, f))
+            sections = loader.get_sections()
+            modules[root] = {}
+            for section in sections:
+                handlers = loader.get_handlers(section)
+                if handlers:
+                    modules[root][section] = handlers
+                else:
+                    self.logger.error("Problem in the configuration file %s in section %s: No handlers found", f, section)
+        return modules
+
+    def get_auth_method(self):
+        return self.auth_method()
+
+    def get_modules(self):
+        dirs = self.get_module_dirs()
+        auth_class = self.auth_method()
+        modules = []
+        for d in dirs:
+            wanted_modules = self.get_modules_from_dir(d)
+            for mod in wanted_modules.keys():
+                for api_version in wanted_modules[mod].keys():
+                    classes = self.get_classes_from_dir(d, wanted_modules[mod][api_version])
+                    if not classes:
+                        continue
+                    for c in classes:
+                        setattr(c, "subarea", mod)
+                        if getattr(c, "authentication_method", "EMPTY") == "EMPTY":
+                            setattr(c, "authentication_method", auth_class)
+                        if getattr(c, "api_versions", None):
+                            c.api_versions.append(api_version)
+                        else:
+                            setattr(c, "api_versions", [api_version])
+                    for cls in classes:
+                        if cls not in modules:
+                            modules.append(cls)
+        return modules
+
+    def create_endpoints(self, handler):
+        endpoint_list = []
+        for endpoint in handler.endpoints:
+            for api_version in handler.api_versions:
+                self.logger.debug("Registering /%s/%s/%s for %s", handler.subarea, api_version, endpoint, handler.__name__)
+                endpoint_list.append("/%s/%s/%s"% (handler.subarea, api_version, endpoint))
+        self.api.add_resource(handler, *(endpoint_list))
+
+    def add_logger(self, handler):
+        self.logger.info("Adding logger to: %s", handler.__name__)
+        handler.logger = self.logger
+
+    def init_handler(self, handler):
+
+        self.add_logger(handler)
+        handler.add_wrappers()
+        self.create_endpoints(handler)
+        handler.add_parser_arguments()
+
+    def create_api_versionhandlers(self, handlers):
+        apiversions = {}
+        endpoint_list = []
+        for handler in handlers:
+            subarea = handler.subarea
+            if apiversions.get(subarea, False):
+                for hapiversion in handler.api_versions:
+                    if hapiversion not in apiversions[subarea]:
+                        apiversions[subarea].append(hapiversion)
+            else:
+                apiversions[subarea] = handler.api_versions
+                self.logger.debug("Registering /%s/apis for %s", subarea, subarea)
+                endpoint_list.append("/%s/apis" % subarea)
+        setattr(VersionHandler, "versions", apiversions)
+        setattr(VersionHandler, "method_decorators", [])
+        self.api.add_resource(VersionHandler, *(endpoint_list))

diff --git a/src/yarf/helpers.py b/src/yarf/helpers.py
new file mode 100644 (file)
index 0000000..917a506
--- /dev/null
+++ b/src/yarf/helpers.py
@@ -0,0 +1,27 @@
+# Copyright 2019 Nokia
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+import re
+
+SECRETS = ["password", "community-string"]
+
+
+def remove_secrets(endpoint):
+    for secret in SECRETS:
+        endpoint = re.sub(r'%s=(.[^&]*)' % secret, "%s=*****" % secret, endpoint)
+    return endpoint
+
+
+

diff --git a/src/yarf/iniloader.py b/src/yarf/iniloader.py
new file mode 100644 (file)
index 0000000..3c762ea
--- /dev/null
+++ b/src/yarf/iniloader.py
@@ -0,0 +1,66 @@
+# Copyright 2019 Nokia
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+import configparser
+from yarf.exceptions import ConfigError
+
+
+class INILoader(dict):
+    def __init__(self, inifile, defaults=None, defaultsection=None):
+        super(INILoader, self).__init__(self)
+        self.inifile = inifile
+        self.handlers = 'handlers'
+        self.configparser = configparser.ConfigParser(defaults)
+        self.config = self.configparser.read(inifile)
+        self.defaultsection = defaultsection
+        if inifile not in self.config:
+            raise ConfigError("Failed to read config file: %s" % inifile)
+
+    def get_sections(self):
+        return self.configparser.sections()
+
+    def get_handlers(self, section):
+        return self[section][self.handlers].split(',')
+
+    def __getitem__(self, key):
+        try:
+            return self.configparser[key]
+        except KeyError:
+            raise ConfigError("No such key %s" % key)
+
+    def get(self, key, section=None, type_of_value=str):
+        if section is None and self.defaultsection is not None:
+            section = self.defaultsection
+        else:
+            return None
+
+        if type_of_value is int:
+            return self.configparser.getint(section, key)
+        elif type_of_value is bool:
+            return self.configparser.getboolean(section, key)
+        elif type_of_value is float:
+            return self.configparser.getfloat(section, key)
+        return self.configparser.get(section, key)
+
+    def keys(self):
+        return self.configparser.sections()
+
+    def get_section(self, section, format='list'):
+        if section in self.keys():
+            items = self.configparser.items(section)
+            if format == 'dict':
+                return dict(items)
+            return items
+        return None

diff --git a/src/yarf/restfulargs.py b/src/yarf/restfulargs.py
new file mode 100644 (file)
index 0000000..5624b5c
--- /dev/null
+++ b/src/yarf/restfulargs.py
@@ -0,0 +1,119 @@
+# Copyright 2019 Nokia
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+import argparse
+import sys
+
+import yarf.config_defaults as config_defaults
+import yarf.restfullogger as restfullogger
+
+from yarf.iniloader import INILoader
+from yarf.exceptions import ConfigError
+
+
+def exception_handler(func):
+    def exception_wrapper(*args, **kwargs):
+        try:
+            restlogger = restfullogger.get_logger()
+            restlogger.debug("calling {}".format(func.__name__))
+            ret = func(*args, **kwargs)
+            return ret
+        except Exception as error:
+            restlogger.info("Exception from function {} (error: {})".format(func.__name__, str(error)))
+            if isinstance(error, ConfigError):
+                raise error
+            else:
+                raise ConfigError(str(error))
+    return exception_wrapper
+
+
+class RestConfig(object):
+    __restinstance = None
+
+    def __new__(cls):
+        if RestConfig.__restinstance is None:
+            RestConfig.__restinstance = object.__new__(cls)
+        return RestConfig.__restinstance
+
+    def __init__(self):
+        self.default_section = config_defaults.default_section
+        self.config_default = config_defaults.config_defaults
+        self.default_config_file = config_defaults.default_config_file
+
+        self.config = None
+        self.config_file = None
+
+    @exception_handler
+    def parse(self, args=sys.argv[1:]):
+        parser = argparse.ArgumentParser(description='Restful server')
+        parser.add_argument('--config',
+                            type=str,
+                            default=self.default_config_file,
+                            help="Configuration file",
+                            dest='config_file')
+
+        args = parser.parse_args(args)
+        self.config_file = args.config_file
+        self.config = INILoader(self.config_file, defaults=self.config_default, defaultsection=self.default_section)
+
+    @exception_handler
+    def get_port(self):
+        return self.config.get('port', type_of_value=int)
+
+    @exception_handler
+    def get_ip(self):
+        return self.config.get('ip_address')
+
+    @exception_handler
+    def use_ssl(self):
+        return self.config.get('use_ssl', type_of_value=bool)
+
+    @exception_handler
+    def get_private_key(self):
+        if self.use_ssl():
+            return self.config.get('ssl_private_key')
+        return None
+
+    @exception_handler
+    def get_certificate(self):
+        if self.use_ssl():
+            return self.config.get('ssl_certificate')
+        return None
+
+    @exception_handler
+    def get_handler_dir(self):
+        return self.config.get('handler_directory')
+
+    def get_section(self, section, format='list'):
+        return self.config.get_section(section, format=format)
+
+    def get_debug(self):
+        return self.config.get('debug', type_of_value=bool)
+
+    @exception_handler
+    def get_passthrough_errors(self):
+        return self.config.get('passthrough_errors', type_of_value=bool)
+
+    @exception_handler
+    def is_threaded(self):
+        return self.config.get('threaded', type_of_value=bool)
+
+    @exception_handler
+    def get_auth_method(self):
+        return self.config.get('auth_method')
+
+
+def get_config():
+    return RestConfig()

diff --git a/src/yarf/restfullogger.py b/src/yarf/restfullogger.py
new file mode 100644 (file)
index 0000000..44fb074
--- /dev/null
+++ b/src/yarf/restfullogger.py
@@ -0,0 +1,67 @@
+# Copyright 2019 Nokia
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+import logging
+import logging.handlers
+restlogger = None
+
+
+class RestfulLogger(object):
+    def __init__(self):
+        self.logger = logging.getLogger("Restfulserver")
+        self.logger.setLevel(logging.DEBUG)
+        # werkzug logs out endpoint in DEBUG level, and aaa feature endpoint contains password
+        # in clear text, so log level setting is needed to avoid showing password in journalctl
+        logging.getLogger('werkzeug').setLevel(logging.WARNING)
+        self.handlers = []
+        self.sysloghandler = self._get_syslog_handler()
+        self.handlers.append(self.sysloghandler)
+        self._add_handlers()
+
+    def __del__(self):
+        handlers = self.logger.handlers[:]
+        for handler in handlers:
+            handler.close()
+            self.logger.removeHandler(handler)
+
+    @staticmethod
+    def _get_syslog_handler():
+        sh = logging.handlers.SysLogHandler(address='/dev/log')
+        formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
+        sh.setFormatter(formatter)
+        sh.setLevel(logging.NOTSET)
+        return sh
+
+    def _add_handlers(self):
+        for handler in self.handlers:
+            self.logger.addHandler(handler)
+
+    def get_handlers(self):
+        return self.handlers
+
+    def get_logger(self):
+        return self.logger
+
+def get_logger():
+    global restlogger
+    if not restlogger:
+        restlogger = RestfulLogger()
+    return restlogger.get_logger()
+
+def get_log_handlers():
+    global restlogger
+    if not restlogger:
+        restlogger = RestfulLogger()
+    return restlogger.get_handlers()

diff --git a/src/yarf/restresource.py b/src/yarf/restresource.py
new file mode 100644 (file)
index 0000000..4e8af83
--- /dev/null
+++ b/src/yarf/restresource.py
@@ -0,0 +1,93 @@
+# Copyright 2019 Nokia
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+import inspect
+import six
+from flask import request
+from flask_restful import reqparse
+from werkzeug.exceptions import BadRequest
+
+from yarf.baseresource import BaseResource
+
+class RequestArgument(object):
+    """ More advanced arguments
+        Parameters:
+            name: Name of the argument
+            default: The default value if not defined
+            validate: function pointer to a validation function that will
+                      be called to validate the argument.
+                      The function should return tuple containing
+                      status: (boolean) True if validation passed
+                                        False if not
+                      reason: (string) Setting the reasoning for the
+                                       failure
+            typeof: The typeof the argument. The argument will be converted
+                    to the type you define. Should be a function pointer
+    """
+    def __init__(self, name, default=None, validate=None, typeof=lambda x: six.text_type(x)):
+        self.argument_class = reqparse.Argument(name=name, default=default, type=typeof)
+        if validate and inspect.isfunction(validate):
+            self.validate_func = validate
+        else:
+            self.validate_func = None
+        self.name = name
+
+    def validate(self, value):
+        if not self.validate_func:
+            return
+        status, reason = self.validate_func(value)
+        if not status:
+            raise BadRequest(description=reason)
+
+class RestResource(BaseResource):
+    """ Class from which the plugins should inherit
+        Variables:
+        extra_wrappers: are function wrappers that will
+                        be executed when any function is
+                        executed by the frame (for example
+                        get)
+        parser_arguments: Are the arguments that can be defined
+                          if you need arguments for your plugin
+                          these arguments can be fetched with
+                          get_args
+    """
+    extra_wrappers = []
+    parser_arguments = []
+    endpoints = None
+    int_arg_class = RequestArgument
+
+    """ Function to get arguments from request
+        The function will call validate to the
+        arguments if they are of type RequestArgument
+        Returns: A dictionary of the arguments
+    """
+    @classmethod
+    def get_args(cls):
+        args = cls.parser.parse_args()
+        for arg in args.keys():
+            for parg in cls.parser_arguments:
+                if isinstance(parg, cls.int_arg_class) and parg.name == arg:
+                    parg.validate(args[arg])
+        return args
+
+    @classmethod
+    def get_token(cls):
+        token = ""
+        try:
+            token = request.headers.get("X-Auth-Token", type=str)
+        except KeyError as err:
+            cls.logger.info("Failed to get auth token from request.")
+        return token
+

diff --git a/src/yarf/versionhandler.py b/src/yarf/versionhandler.py
new file mode 100644 (file)
index 0000000..5937817
--- /dev/null
+++ b/src/yarf/versionhandler.py
@@ -0,0 +1,38 @@
+# Copyright 2019 Nokia
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+from flask import request
+from yarf.restresource import RestResource
+
+def get_api(req):
+    splitted = req.full_path.split("/")
+    domain = splitted[1]
+    return domain
+
+class VersionHandler(RestResource):
+    versions = {}
+
+    def create_api_reference(self, api):
+        ref = []
+        for version in self.versions[api]:
+            ver = {}
+            ver['id'] = version
+            ver['href'] = "%s%s/%s" %(request.host_url, api, version)
+            ref.append(ver)
+        return ref
+
+    def get(self):
+        api = get_api(request)
+        return self.create_api_reference(api)

diff --git a/systemd/restapi.service b/systemd/restapi.service
new file mode 100644 (file)
index 0000000..f3156d2
--- /dev/null
+++ b/systemd/restapi.service
@@ -0,0 +1,27 @@
+# Copyright 2019 Nokia
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+[Unit]
+Description=Restful api server
+DefaultDependencies=no
+
+[Service]
+Restart=on-failure
+RestartSec=3
+ExecStart=/usr/local/bin/restapi
+User=restapi
+
+[Install]
+WantedBy=multi-user.target

diff --git a/yarf.spec b/yarf.spec
new file mode 100644 (file)
index 0000000..70dd0c8
--- /dev/null
+++ b/yarf.spec
@@ -0,0 +1,78 @@
+# Copyright 2019 Nokia
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+Name:           yarf
+Version:        %{_version}
+Release:        1%{?dist}
+Summary:        Yet Another Restfulframework
+License:        %{_platform_licence}
+Source0:        %{name}-%{version}.tar.gz
+Vendor:         %{_platform_vendor}
+
+Requires: python-flask, python2-flask-restful, python2-configparser, python2-requests, mod_wsgi, python2-six 
+BuildRequires: python
+BuildRequires: python-setuptools
+
+%description
+Yet Another Restfulframework.
+
+%prep
+#./autogen.sh
+%autosetup
+
+%build
+
+%install
+rm -rf $RPM_BUILD_ROOT
+mkdir -p %{buildroot}%{_platform_etc_path}/yarf
+mkdir -p %{buildroot}%{_platform_etc_path}/required-secrets
+mkdir -p %{buildroot}%{_unitdir}/
+mkdir -p %{buildroot}/var/log/restapi
+cp required-secrets/*.yaml %{buildroot}/%{_platform_etc_path}/required-secrets
+#mkdir -p {buildroot}/etc/httpd/conf.d/
+#mkdir -p {buildroot}/var/www/yarf/
+
+cd src && python setup.py install --root %{buildroot} --no-compile --install-purelib %{_python_site_packages_path} --install-scripts %{_platform_bin_path} && cd -
+
+rsync -rv systemd/* %{buildroot}%{_unitdir}/
+
+%files
+%defattr(0755,root,root,0755)
+%{_python_site_packages_path}/yarf*
+%attr(0755,restapi, restapi) %{_platform_etc_path}/yarf/
+%{_platform_etc_path}/required-secrets/restful.yaml
+#/etc/ansible/roles/restful
+#/opt/openstack-ansible/playbooks/yarf.yml
+%attr(0755,root, root) %{_platform_bin_path}/restapi
+# %attr(0644,root, root) %{_unitdir}/restapi.service
+%attr(0644,root, root) %{_unitdir}/* 
+%dir %attr(0770, restapi,restapi) /var/log/restapi
+
+%pre
+/usr/bin/getent passwd restapi > /dev/null||/usr/sbin/useradd -r -s /sbin/nologin -M restapi
+
+%post
+if [ $1 -eq 2 ]; then
+    if [ -f %{_platform_etc_path}/restful/config.ini ]; then
+        sudo /usr/bin/systemctl restart restapi
+    fi
+fi
+
+%postun
+
+#Uninstall
+if [ $1 -eq 0 ];then
+    /usr/sbin/userdel restapi
+fi