The source code structure is as follows:
1 bbox33:servera $ find . -type f -name \\*.py 2 ./modules/__init__.py 3 ./modules/admin/__init__.py 4 ./modules/admin/disk.py 5 ./rhbase.py 6 ./server.py
Here’s a brief demo:
1 bbox33:servera $ python server.py 2 ** 2007-05-30 00:15:18,628 INFO - starting server.. 3 !! please type in a req in HTTP-GET format (or 'q' to quit) 4 >>> http://xyz.net/admin/disk/purge?path=/tmp 5 ** 2007-05-30 00:15:34,889 INFO - do_purge() called for params: 'path=/tmp' 6 ** 2007-05-30 00:15:34,889 INFO - S: http://xyz.net/admin/disk/purge?path=/tmp 7 !! please type in a req in HTTP-GET format (or 'q' to quit) 8 >>> http://xyz.net/admin/disk/list?path=/usr 9 ** 2007-05-30 00:15:48,791 ERROR - no 'do_list' function in '<class 'modules.admin.disk.ReqH'>' 10 ** 2007-05-30 00:15:48,791 INFO - F: http://xyz.net/admin/disk/list?path=/usr 11 !! please type in a req in HTTP-GET format (or 'q' to quit) 12 >>> q 13 ** 2007-05-30 00:15:51,695 INFO - server terminated
And here’s what actually went on:
- the server is invoked (line 1) and is given a request URI (line 4) to serve
- based on the request URI path (
/admin/disk/purge) an appropriate request handler is loaded dynamically (
- the request handler’s
dispatch()function is called
- by looking at the last segment of the request URI path (
purge) the dispatch function guesses that the request should actually be handled by a method called
do_purge()is invoked, logs the request parameters (line 5) and returns with a value indicating success (zero)
- after the request was handled the server logs the URI and the outcome (line 6 (‘S’ stands for success))
- on line 8 the user typed in another request URI but this time no handler function could be found (resulting in the error logged on line 10 (‘F’ stands for failure))
The server is meant to operate on
HTTP GET style requests. The
serve() method (on line 9) accepts a
req string parameter that contains a
HTTP GET style URI.
dfuncs dictionary (defined on line 7) is a request handler function cache. The key is the path portion of the request URI. The value holds the corresponding request handler function.
The scheme + authority portion of the URI are ignored by the dispatch mechanism. Any URIs with identical path portions map to the same request handler module and function.
For example the following URIs both map to the module
The last segment of the URI’s path portion is taken to be the request action (i.e.
sleep in the URI path above).
Serving of requests
1 #!/usr/bin/env python 2 # encoding: utf-8 3 import logging, urlparse 4 5 class Server(object): 6 def __init__(self): 7 self.dfuncs = dict() 8 self.modules = dict() 9 def serve(self, req): 10 result = -1 11 req_path = urlparse.urlsplit(req) 12 if req_path in self.dfuncs: dfunc = self.dfuncs[req_path] # in cache 13 else: dfunc = self.load(req_path) # handler function not in cache 14 if dfunc: result = dfunc(req) 15 return(result)
The server first checks whether the handler function for a request is already in its cache (line 12). If this is not the case (line 13), the
load() method is called in order to load the appropriate request handler module and function.
The request handler function — if available — is invoked on line 14. Please note that a zero is returned in case of success and any non-zero return value indicates a failure.
Dynamic code loading
The load method below constructs the name of the request handler module (line 19) and tries to import it (line 23).
For more detail on dynamic code loading in Python
- see Mark Pilgrim’s Dynamically importing modules or
- go the Python Cookbook, scroll down to chapter 15 and click on the “Importing from a Module Whose Name Is Determined at Runtime” heading
16 def load(self, req_path): 17 logging.debug("load() called for '%s'" % req_path) 18 m = dfunc = None 19 mpath = 'modules.%s' % '.'.join(filter(None, req_path.split('/')[:-1])) 20 # already have the module needed? 21 if mpath in self.modules: m = self.modules[mpath] # yes! 22 else: # no, try import 23 try: m = __import__(mpath, globals(), locals(), ['ReqH']) 24 except ImportError, e: logging.error(str(e)) 25 else: self.modules[m.__name__] = m # cache it
If the request handler module was imported successfully, the loading logic checks whether the module has a
ReqH class (line 28) and whether the latter has a
dispatch attribute that may be called (line 30).
If all checks succeed the
dispatch attribute is returned as the request handler function (line 34).
26 if m: 27 # got the module, does it have a 'ReqH' class? 28 if hasattr(m, 'ReqH'): 29 # yes, does the 'ReqH' class have a dispatch function? 30 if hasattr(m.ReqH, 'dispatch') and callable(m.ReqH.dispatch): 31 self.dfuncs[req_path] = dfunc = m.ReqH.dispatch 32 else: logging.error("no dispatch function in '%s'" % mpath) 33 else: logging.error("no request handler class in '%s'" % mpath) 34 return dfunc
A proper server needs to log errors as well as any requests received. A
StreamHandler (logging to
stderr by default) is set up as the root logger. For more detail on Python logging see
35 if __name__ == '__main__': 36 logf = logging.Formatter('** %(asctime)s %(levelname)s - %(message)s') 37 lcon = logging.StreamHandler() 38 lcon.setFormatter(logf) 39 logging.getLogger('').addHandler(lcon) 40 logging.getLogger('').setLevel(logging.INFO) 41 logging.info('starting server..')
The section of code below is a minimal input loop facilitating the experimentation with the server.
42 server = Server() 43 while 1: 44 req = raw_input("!! please type in a req in HTTP-GET format " \\ 45 "(or 'q' to quit)\\n>>> ") 46 if req == 'q': break 47 result = server.serve(req) 48 if result == 0: logging.info('S: %s' % req) 49 else: logging.info('F: %s' % req) 50 logging.info('server terminated')
As noted above, a zero return value is taken to be an indication of success (line 48) whereas a non-zero value signifies failure (line 49).
It is an established practice to split the back-end logic into a number of modules that are loaded by the server on demand (as shown above).
In our example each request handler module needs to have a
dispatch() method that takes the URI string as its only argument.
In order to factor out the boiler plate code, a request handler base class that takes care of the request action dispatching is put into place.
Please note that the
dispatch() method is a class method i.e. it receives a class object as its implicit first argument (lines 7-8 below). This is different from a static method that receives no implicit first argument whatsoever.
1 #!/usr/bin/env python 2 # encoding: utf-8 3 4 import logging, urlparse 5 6 class ReqHBase(object): 7 @classmethod 8 def dispatch(cls, req): 9 result = -1 10 targetf = 'do_%s' % urlparse.urlsplit(req).split('/')[-1] 11 12 try: handler = getattr(cls, targetf) 13 except AttributeError: 14 logging.error("no '%s' function in '%s'" % (targetf, str(cls))) 15 else: # is it a function? 16 if callable(handler): 17 try: result = handler(req) 18 except Exception, e: logging.exception(str(e)) 19 else: 20 logging.error("'%s' not callable in '%s'" % (targetf, str(cls))) 21 return(result)
The dispatch code above takes the last segment of the URI path (i.e. the request action) and expects to find a static method called
do_<last_segment> (line 10)
The expected target function must both be present and callable to be invoked. Otherwise the corresponding error messages are logged (line 14 and 20).
Example request handler module
Please note: more sophisticated request handler modules may be derived from
object and implement their own
dispatch() methods as needed.
1 #!/usr/bin/env python 2 # encoding: utf-8 3 4 import logging, urlparse 5 from urlparse import urlsplit 6 from rhbase import ReqHBase 7 8 class ReqH(ReqHBase): 9 do_nothing = 1 10 @staticmethod 11 def do_purge(req): 12 logging.info("do_purge() called for params: '%s'" % urlsplit(req)) 13 return(0)
The server presented in this article is quite simple (it handles all requests in serial fashion, no threading is used etc.). Nevertheless, it clearly demonstrates the potency and the productivity of the Python programming environment.
A little code goes a long way 🙂