:: Copyright 2019 Nokia Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. ========================================= YARF REST framework ========================================= :Author: Janne Suominen .. raw:: pdf PageBreak .. sectnum:: .. contents:: .. raw:: pdf PageBreak Introduction ============ What is REST ------------ **Representational state transfer** (REST) or **RESTful** web services is a way of providing interoperability between computer systems on the Internet. REST-compliant Web services allow requesting systems to access and manipulate textual representations of Web resources using a uniform and predefined set of stateless operations. About this project ------------------ YARF --- Yet Another Restful Framework This project provides an implementation for a generic rest framework The framework is based on `Flask-RESTful `__ | *The framework provides the following:* * A generic rest server for whole cloud * A plugin based interface for creating rest apis. | *At the high level, the framework serves the following purposes*: * Provides a unified interface for creating rest interfaces. * Single point of entry with plugin based authentication for apis. * Define the rest response and query format. | Requirements ------------ * Framework shall provide easy way to integrate new REST apis via plugins * Framework shall be integrated to the CM and provide means to automatically configure itself * Framework shall listen to management address of controller on all the controllers * Framework shall listen to external and internal load balancer address * Framework shall provide means to configure SSL certification * Framework shall be integrated to Authentication mechanism provided by the platform * Framework shall provide means to validate the parameters given as part of the request * Framework shall return an error code in case of: * Internal failure; Any failure within the module is considered as internal failure * Authentication failure; When authentication has failed or missing credentials * Not found; When the object is not found Structure of data ================= The implementation of this framework promotes returning JSON format objects with key value pairs. Structure of the operation -------------------------- The framework supports adding function calls for any HTTP requests. The request can either: * Have a request in the body of the message as JSON * Have a request embedded in the url Structure of response --------------------- This framework does not enforce any special structure for the response, but **it's strongly** encouraged to use the following formatting for the response: .. code:: json { code: description: data: } Where: * code is the return value of the api in question. * 0 means no error and anything other is considered as failure * description is the description of the possible failure (can be left empty in case there is no failure) * data is the data returned by the api. The data should be in dictionary (JSON) format The reasoning for the quite strict guidelines is: * Uniqueness of the response makes it easier for the upper level to check the response **Note**: The framework will return HTTP status code that is not 200 in case of: * 500: Internal failure (ie. Uncaught exception from the plugin) * 401: In case of authentication failure (if authentication is defined) * 404: In case the object requested is not found High level architecture of the restful framework ================================================ At the high level, there is a layer built on top of flask-restful to be able to: * Isolation of the framework implementation details. * To be able to provide more specific implementation to fit to our needs. * To be able to make a single point of entry to the clusters rest api * Flexibility to change different parts of the implementation without affecting the users of the framework. * Provide unique responses to caller for easy parsing .. figure:: ./design/architecture.jpeg :alt: architecture architecture Restful framework interfaces ============================ RestResource ------------ All the plugins have to inherit from this class. This class is the basis of the plugin framework. All the plugins that inherit from this object and are defined in plugin specific inifile will be automatically imported. The http requests will be converted to corresponding lowercase functions. For example: request method GET will call *get()* function and POST will call *post()*. The resource should also define endpoints where it want's to register these calls. This is done by setting the *endpoint* class variable list. For decorating functions with decorators *extra_wrappers* can be used. The function must return either the function or a dictionary that is of the same format defined in `Structure of Response`_. To have authentication for the module adding class variable named *authentication_method* needs to be defined. For production environment this should be left untouched since the authentication should be controlled centrally by the framework. For logging there is a class variable called *logger* that works like a normal logger. An example of a plugin can look like this: .. _test_rest: .. code:: python class TestRest(RestResource): endpoints = ['test'] def get(self): self.logger.debug("Got get request") return {"code": 0, "description": "", "data": "Foobar"} Arguments: ~~~~~~~~~~ For parsing the arguments from the message body there function defined in `RestResource`_ called *get_args*. This function will return the arguments that are in the request. The parser needs to be initialized with *parser_arguments* variable that is a list of variables your module want's to parse. If one needs to define more complex type of an argument it can be done with the help of *RequestArgument*. This class provides the means of: * Setting a default value * Validation of the value by callback function * The type of the value When this type of argument is passed as one (or more) of the values. The validation will be automatically triggered when calling the *get_args* from the *RestResource*. BaseAuth -------- This class defines the Base for the authentication. The class needs to define function *is_authenticated*. The function gets the request as an argument. This function will be called when a plugin has specified the authentication method as a derived class of BaseAuth. Here is an example of a very simple authentication class. .. code:: python from base_auth import BaseAuthMethod class TextBase(BaseAuthMethod): def __init__(self): super(TextBase, self).__init__() self.user = '' self.password = '' with open('/tmp/foo') as f: self.user, self.password = f.read().strip().split(':') def is_authenticated(self, request): if request.authorization and request.authorization.username == self.user and request.authorization.password == self.password: return True return False Keystone ~~~~~~~~ For keystone additional configuration is needed: * User with admin role needs to be added (or admin used) * The config.ini has to contain the credentials and the url of keystone The following configuration needs to be added to config.ini .. code:: ini [keystone] user=restful password=foobar auth_uri=http://192.168.1.15:5000 After the configuration is done and the authentication will be needed then the http headers have to contain token with admin privileges as X-Auth-Token. Restful framework binary ======================== The framework has only one binary. It's called restapi. The server will be automatically started during the deployment. restapi config file ------------------- The default configuration file for the restful server is located at /etc/yarf/config.ini To override the default config file, restapi can be started with command line parameter --config. This allows testing plugins without interference to the rest of the system. The config file contains the following parameters: .. code:: ini [restframe] #The port that the restful app will listen DEFAULT:61200 port=61200 #The IP address that the restful app will bind to DEFAULT:127.0.0.1 ip_address=127.0.0.1 #Use SSL or not #If true then private key and certificate has to be also given DEFAULT:False use_ssl=false #ssl_private_key=PATHTOKEY/KEY.key #ssl_certificate=PATHTOCERTIFICATE/CERT.crt #The directory where the handlers are #Defaults to /opt/yarf/handlers handler_directory=/opt/yarf/handlers The configuration file will be generated with an ansible module that will configure the framework. Restapi service will run on all the controllers and listen to the controller internal management IP. HAProxy will be configured so that clients can take a connection to the internal loadbalancer address (or internal VIP) or external loadbalancer address (external VIP). The framework will listen to port 61200. Creating plugins ================ First of all you need to have your own Class defined like the described in `RestResource`_. The second thing needed is an ini file that describes the handlers for different requests and the api version of the handler. The plugins should be placed in their own directory under : /opt/yarf/handlers. The thing that needs to be remembered that the object lifetime of *RestResource* is and will be only the duration of the query. Any data stored by that query that is needed cannot be stored in the internal variables of the module. This is anyway against the *statelessness* nature of rest. Ini file for plugin ------------------- The recommendation is to have your own directory (although not mandatory) per plugin. Within that directory you have to create an inifile that is of the following format: .. code:: ini [] handlers= Where the name of the inifile will the first part of your path on the rest server. API_VERSION the second And the handler endpoint(s) the third. For example if one would create an inifile for the `test_rest`_ resource it would look like this: testing.ini: .. code:: ini [v1] handlers=TestRest Then if you want to test your api you could do that with curl: curl http://testing/test/v1/test .. code:: json { "code": 0, "description": "u''" "data": "u'Foobar'" } There is also a helper to check the apis and their locations: curl http://testing/test/apis .. code:: json [ { "href": "http://testing/test/v1", "id": "v1" } ]