Cherrypy use plugin in _cp_dispatch/popargs

Question

I'm using cherrypy with sqlalchemy in order to build an headless (only CLI client) restful server.

I've used the following receipe to bind sqlalchemy to cherrypy engine: https://bitbucket.org/Lawouach/cherrypy-recipes/src/c8290261eefb/web/database/sql_alchemy/ The receipe is slightly modified in order to build the database if it doesn' exists.

The server expose several uri such as clients, articles, stores...

• GET /clients get the list of clients
• GET /clients/1 get the client id #1
• GET /clients/foo get the client named foo
• PUT /clients/1 update the client #1
• DELETE /client/foo delete the clien named foo ...

I intend to use the decorator popargs with _cp_dispatch in order to converted my resource name to their id ahead of the handling. I use cherrypy.dispatch.MethodDispatcher as dispatcher (so I can code GET/POST/PUT/DELETE method)

I can access the plugin from any of GET/POST/PUT/DELETE method but I can't access it from _cp_dispatch.

Any idea how I can converted resource name to their id before entrering GET/POST/PUT/DELETE method?

Here is a reproducer of my problem

$ tree
.
├── __init__.py
├── models.py
├── my.db
├── root.py
├── saplugin.py
├── saplugin.py.original
└── satool.py

My sqlalchemy models (I've got several of them, only one is necessary to reproduce the problem)

$ cat models.py
# -*- coding: utf-8 -*-
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, String
Base = declarative_base()

class User(Base):

    __tablename__ = 'users'

    id = Column(Integer, primary_key=True)
    name = Column(String)

Here is the main script of the server

$ cat root.py
# -*- coding: utf-8 -*-

import cherrypy
from models import User
from saplugin import SAEnginePlugin
from satool import SATool

def name_to_id(param, table):
    if not param.isdigit():
        param = cherrypy.request.db.query(table.id).\
                filter(table.name == param).one()
    return param

class Root(object):

    exposed = True

    def __init__(self):
        self.users = Users()

    def GET(self):
        # Get the SQLAlchemy session associated
        # with this request.
        # It'll be released once the request
        # processing terminates
        return "Hello World"


class Users(object):

    exposed = True

    @cherrypy.popargs('user_id')
    def GET(self, user_id=None, **kwargs):
        user_id = name_to_id(user_id, User)
        return "I'm resource %s" % user_id

if __name__ == '__main__':
    # Register the SQLAlchemy plugin
    SAEnginePlugin(cherrypy.engine).subscribe()

    # Register the SQLAlchemy tool
    cherrypy.tools.db = SATool()

    cherrypy.quickstart(Root(), '', {'/': {'tools.db.on': True,
                                        "request.dispatch": cherrypy.dispatch.MethodDispatcher()}})

Here is the modification applied to the sqlalchemy plugin receipe

$ diff -up saplugin.py.original saplugin.py
--- saplugin.py.original    2015-06-15 18:14:45.469706863 +0200
+++ saplugin.py 2015-06-15 18:14:37.042741785 +0200
@@ -3,6 +3,7 @@ import cherrypy
from cherrypy.process import wspbus, plugins
from sqlalchemy import create_engine
from sqlalchemy.orm import scoped_session, sessionmaker
+from models import User, Base

__all__ = ['SAEnginePlugin']

@@ -26,7 +27,16 @@ class SAEnginePlugin(plugins.SimplePlugi
        self.sa_engine = create_engine('sqlite:///my.db', echo=False)
        self.bus.subscribe("bind-session", self.bind)
        self.bus.subscribe("commit-session", self.commit)
- 
+        # Creating the database
+        self.bus.log('Creating database')
+        self.bus.subscribe("create-all", self.create_all)
+        try:
+            self.create_all()
+        except Exception as err:
+            logging.error("Can't start")
+            logging.error(err.message)
+            sys.exit(1)
+
    def stop(self):
        self.bus.log('Stopping down DB access')
        self.bus.unsubscribe("bind-session", self.bind)
@@ -59,4 +69,8 @@ class SAEnginePlugin(plugins.SimplePlugi
            raise
        finally:
            self.session.remove()
-    
+
+    def create_all(self):
+        """ create database structure """
+        self.bus.log('Creating database')
+        Base.metadata.create_all(self.sa_engine)

Here is the log of the server, when I convert the name at the beginning of the GET method.

$ python root.py 
[15/Jun/2015:18:16:23] ENGINE Listening for SIGHUP.
[15/Jun/2015:18:16:23] ENGINE Listening for SIGTERM.
[15/Jun/2015:18:16:23] ENGINE Listening for SIGUSR1.
[15/Jun/2015:18:16:23] ENGINE Bus STARTING
[15/Jun/2015:18:16:23] ENGINE Starting up DB access
[15/Jun/2015:18:16:23] ENGINE Creating database
[15/Jun/2015:18:16:23] ENGINE Creating database
[15/Jun/2015:18:16:23] ENGINE Started monitor thread 'Autoreloader'.
[15/Jun/2015:18:16:23] ENGINE Started monitor thread '_TimeoutMonitor'.
[15/Jun/2015:18:16:23] ENGINE Serving on 127.0.0.1:8080
[15/Jun/2015:18:16:23] ENGINE Bus STARTED
127.0.0.1 - - [15/Jun/2015:18:16:26] "GET /users/1 HTTP/1.1" 200 14 "" "curl/7.29.0"
127.0.0.1 - - [15/Jun/2015:18:16:28] "GET /users/foo HTTP/1.1" 200 14 "" "curl/7.29.0"

Here is the query for the previous logs

$ curl 127.0.0.1:8080/users/1; echo
I'm resource 1
$ curl 127.0.0.1:8080/users/foo; echo
I'm resource 1

Here is the content of the database

$ sqlite3 my.db 
sqlite> .schema
CREATE TABLE users (
    id INTEGER NOT NULL, 
    name VARCHAR, 
    PRIMARY KEY (id)
);
sqlite> select * from users;
1|foo
2|bar

I can't use both the decorator popargs and the _cp_dispatch method. I haven't found how to work with my url segment with the decorator. When I try to only use _cp_dispatch method, I end up with the following error:

File "/usr/lib/python2.7/site-packages/cherrypy/_cprequest.py", line 628, in respond
    self.get_resource(path_info)
File "/usr/lib/python2.7/site-packages/cherrypy/_cprequest.py", line 744, in get_resource
    dispatch(path)
File "/usr/lib/python2.7/site-packages/cherrypy/_cpdispatch.py", line 423, in __call__
    resource, vpath = self.find_handler(path_info)
File "/usr/lib/python2.7/site-packages/cherrypy/_cpdispatch.py", line 311, in find_handler
    subnode = dispatch(vpath=iternames)
File "root.py", line 49, in _cp_dispatch
    vpath[0] = name_to_id(vpath[0], Users)
File "root.py", line 10, in name_to_id
    param = cherrypy.request.db.query(table.id).\
File "/usr/lib/python2.7/site-packages/cherrypy/__init__.py", line 208, in __getattr__
    return getattr(child, name)
AttributeError: 'Request' object has no attribute 'db'

Here is the modification applied to the Users class:

class Users(object):

    # [....]

    def _cp_dispatch(self, vpath):
        if len(vpath) > 1:
            raise
        vpath[0] = name_to_id(vpath[0], Users)
        return vpath

I'm using the following version (I'm working on an Centos 7 environnement and can't change using those from pip): $ rpm -qa | egrep "cherrypy|sqlalchemy" python-cherrypy-3.2.2-4.el7.noarch python-sqlalchemy-0.9.7-3.el7.x86_64

Your help would be very much appreciated!!!!

Answer 1

The solution to this problem it's described in the documentation.

I've added the following tool to the root script:

class UserManager(cherrypy.Tool):
    def __init__(self):
        cherrypy.Tool.__init__(self, 'before_handler',
                            self.load, priority=10)

    def load(self, **kwargs):
        req = cherrypy.request

        # let's assume we have a db session
        # attached to the request somehow 
        db = req.db

        # retrieve the user id and remove it
        # from the request parameters
        if 'user' in req.params:
            user = req.params.pop('user')
            if not user.isdigit():
                req.params['user'] = db.query(User).\
                        filter(User.name == user).one()
            else:
                req.params['user'] = db.query(User).\
                        filter(User.id == user).one()

cherrypy.tools.user = UserManager()

And rewrite the Users class like follow:

@cherrypy.popargs("user")
class Users(object):

    exposed = True

    @cherrypy.tools.user()
    def GET(self, user=None, **kwargs):
        if user:
            return "I'm resource %s" % user.id
        else:
            return "All users"

The decorator @cherrypy.popargs("user") tells to cherrypy to take the first segment in the url and store it into the key "user". The decorator @cherrypy.tools.user() tells to cherrypy to call the tool user before entering the GET method. In the load method of the UserManager class, I convert either an user's id or a name to the sqlalchemy model.