Roll your own server in 50 lines of code

Introduction

Just in case you wondered why there are so many frameworks in Python land, here’s a basic server (including a request dispatch mechanism) in only 50 lines of code (syntax highlighted code here).

The source code structure is as follows:

 1 bbox33:servera $ find . -type f -name \\*.py
 2 ./modules/__init__.py
 3 ./modules/admin/__init__.py
 4 ./modules/admin/disk.py
 5 ./rhbase.py
 6 ./server.py

Here's a brief demo:

  1 bbox33:servera $ python server.py
  2 ** 2007-05-30 00:15:18,628 INFO - starting server..
  3 !! please type in a req in HTTP-GET format (or 'q' to quit)
  4  >>> http://xyz.net/admin/disk/purge?path=/tmp
  5 ** 2007-05-30 00:15:34,889 INFO - do_purge() called for params: 'path=/tmp'
  6 ** 2007-05-30 00:15:34,889 INFO - S: http://xyz.net/admin/disk/purge?path=/tmp
  7 !! please type in a req in HTTP-GET format (or 'q' to quit)
  8  >>> http://xyz.net/admin/disk/list?path=/usr
  9 ** 2007-05-30 00:15:48,791 ERROR - no 'do_list' function in '<class 'modules.admin.disk.ReqH'>'
 10 ** 2007-05-30 00:15:48,791 INFO - F: http://xyz.net/admin/disk/list?path=/usr
 11 !! please type in a req in HTTP-GET format (or 'q' to quit)
 12  >>> q
 13 ** 2007-05-30 00:15:51,695 INFO - server terminated

And here's what actually went on:

  1. the server is invoked (line 1) and is given a request URI (line 4) to serve
  2. based on the request URI path (/admin/disk/purge) an appropriate request handler is loaded dynamically (modules.admin.disk)
  3. the request handler's dispatch() function is called
  4. by looking at the last segment of the request URI path (purge) the dispatch function guesses that the request should actually be handled by a method called do_purge()
  5. do_purge() is invoked, logs the request parameters (line 5) and returns with a value indicating success (zero)
  6. after the request was handled the server logs the URI and the outcome (line 6 ('S' stands for success))
  7. on line 8 the user typed in another request URI but this time no handler function could be found (resulting in the error logged on line 10 ('F' stands for failure))

The server

The server is meant to operate on HTTP GET style requests. The serve() method (on line 9) accepts a req string parameter that contains a HTTP GET style URI.

The dfuncs dictionary (defined on line 7) is a request handler function cache. The key is the path portion of the request URI. The value holds the corresponding request handler function.
The scheme + authority portion of the URI are ignored by the dispatch mechanism. Any URIs with identical path portions map to the same request handler module and function.

For example the following URIs both map to the module modules.admin.disk, class ReqH, method dispatch():

The last segment of the URI's path portion is taken to be the request action (i.e. sleep in the URI path above).

Serving of requests

  1 #!/usr/bin/env python
  2 # encoding: utf-8
  3 import logging, urlparse
  4 
  5 class Server(object):
  6     def __init__(self):
  7         self.dfuncs = dict()
  8         self.modules = dict()
  9     def serve(self, req):
 10         result = -1
 11         req_path = urlparse.urlsplit(req)[2]
 12         if req_path in self.dfuncs: dfunc = self.dfuncs[req_path] # in cache
 13         else: dfunc = self.load(req_path)  # handler function not in cache
 14         if dfunc: result = dfunc(req)
 15         return(result)

The server first checks whether the handler function for a request is already in its cache (line 12). If this is not the case (line 13), the load() method is called in order to load the appropriate request handler module and function.

The request handler function -- if available -- is invoked on line 14. Please note that a zero is returned in case of success and any non-zero return value indicates a failure.

Dynamic code loading

The load method below constructs the name of the request handler module (line 19) and tries to import it (line 23).

For more detail on dynamic code loading in Python

 16     def load(self, req_path):
 17         logging.debug("load() called for '%s'" % req_path)
 18         m = dfunc = None
 19         mpath = 'modules.%s' % '.'.join(filter(None, req_path.split('/')[:-1]))
 20         # already have the module needed?
 21         if mpath in self.modules: m = self.modules[mpath] # yes!
 22         else:   # no, try import
 23             try: m = __import__(mpath, globals(), locals(), ['ReqH'])
 24             except ImportError, e: logging.error(str(e))
 25             else: self.modules[m.__name__] = m  # cache it

If the request handler module was imported successfully, the loading logic checks whether the module has a ReqH class (line 28) and whether the latter has a dispatch attribute that may be called (line 30).

If all checks succeed the dispatch attribute is returned as the request handler function (line 34).

 26         if m:
 27             # got the module, does it have a 'ReqH' class?
 28             if hasattr(m, 'ReqH'):
 29                 # yes, does the 'ReqH' class have a dispatch function?
 30                 if hasattr(m.ReqH, 'dispatch') and callable(m.ReqH.dispatch):
 31                     self.dfuncs[req_path] = dfunc = m.ReqH.dispatch
 32                 else: logging.error("no dispatch function in '%s'" % mpath)
 33             else: logging.error("no request handler class in '%s'" % mpath)
 34         return dfunc

Logging setup

A proper server needs to log errors as well as any requests received. A StreamHandler (logging to stderr by default) is set up as the root logger. For more detail on Python logging see

 35 if __name__ == '__main__':
 36     logf = logging.Formatter('** %(asctime)s %(levelname)s - %(message)s')
 37     lcon = logging.StreamHandler()
 38     lcon.setFormatter(logf)
 39     logging.getLogger('').addHandler(lcon)
 40     logging.getLogger('').setLevel(logging.INFO)
 41     logging.info('starting server..')

Minimum scaffolding

The section of code below is a minimal input loop facilitating the experimentation with the server.

 42     server = Server()
 43     while 1:
 44         req = raw_input("!! please type in a req in HTTP-GET format " \\
 45                         "(or 'q' to quit)\\n>>> ")
 46         if req == 'q': break
 47         result = server.serve(req)
 48         if result == 0: logging.info('S: %s' % req)
 49         else: logging.info('F: %s' % req)
 50     logging.info('server terminated')

As noted above, a zero return value is taken to be an indication of success (line 48) whereas a non-zero value signifies failure (line 49).

Request handling

It is an established practice to split the back-end logic into a number of modules that are loaded by the server on demand (as shown above).

In our example each request handler module needs to have a dispatch() method that takes the URI string as its only argument.

DRY

In order to factor out the boiler plate code, a request handler base class that takes care of the request action dispatching is put into place.

Please note that the dispatch() method is a class method i.e. it receives a class object as its implicit first argument (lines 7-8 below). This is different from a static method that receives no implicit first argument whatsoever.

  1 #!/usr/bin/env python
  2 # encoding: utf-8
  3 
  4 import logging, urlparse
  5 
  6 class ReqHBase(object):
  7     @classmethod
  8     def dispatch(cls, req):
  9         result = -1
 10         targetf = 'do_%s' % urlparse.urlsplit(req)[2].split('/')[-1]
 11 
 12         try: handler = getattr(cls, targetf)
 13         except AttributeError:
 14             logging.error("no '%s' function in '%s'" % (targetf, str(cls)))
 15         else: # is it a function?
 16             if callable(handler):
 17                 try: result = handler(req)
 18                 except Exception, e: logging.exception(str(e))
 19             else:
 20                 logging.error("'%s' not callable in '%s'" % (targetf, str(cls)))
 21         return(result)

The dispatch code above takes the last segment of the URI path (i.e. the request action) and expects to find a static method called do_<last_segment> (line 10)

The expected target function must both be present and callable to be invoked. Otherwise the corresponding error messages are logged (line 14 and 20).

Example request handler module

What follows is a "minimalistic" request handler module. It is derived from the ReqHBase class and hence inherits the latter's dispatch() method.

Please note: more sophisticated request handler modules may be derived from object and implement their own dispatch() methods as needed.

  1 #!/usr/bin/env python
  2 # encoding: utf-8
  3 
  4 import logging, urlparse
  5 from urlparse import urlsplit
  6 from rhbase import ReqHBase
  7 
  8 class ReqH(ReqHBase):
  9     do_nothing = 1
 10     @staticmethod
 11     def do_purge(req):
 12         logging.info("do_purge() called for params: '%s'" % urlsplit(req)[3])
 13         return(0)

In conclusion

The server presented in this article is quite simple (it handles all requests in serial fashion, no threading is used etc.). Nevertheless, it clearly demonstrates the potency and the productivity of the Python programming environment.

A little code goes a long way :-)