Why does using FastCGI with lighttpd trigger a 404 when the routing is not changed?

Question

I have a bottle-based web application which I would like to move to FastCGI:

# coding=utf-8
import bottle

class Webserver:
    def __init__(self):
        bottle.route('/api', ['GET', 'OPTIONS'], self.data)
        bottle.run(host='127.0.0.1', port=8081, debug=True, quiet=False)

    def data(self):
        return "hello"

    @bottle.error(404)
    def error404(error):
        print("404 Not Found from {0} @ {1}".format(bottle.request.remote_addr, bottle.request.url))

Webserver()

It works as expected.

Moving it to FastCGI, I modified the run() call:

# coding=utf-8
import bottle

class Webserver:
    def __init__(self):
        bottle.route('/api', ['GET', 'OPTIONS'], self.data)
        bottle.run(server='flup', host='127.0.0.1', port=8081, debug=True, quiet=False)

    def data(self):
        return "hello"

    @bottle.error(404)
    def error404(error):
        print("404 Not Found from {0} @ {1}".format(bottle.request.remote_addr, bottle.request.url))

Webserver()

The frontal web server is lighttpd and its full configuration is:

root@domotique /e/lighttpd# cat lighttpd.conf
server.modules = (
        "mod_access",
        "mod_alias",
        "mod_compress",
        "mod_redirect",
#       "mod_rewrite",
        "mod_fastcgi",
        "mod_accesslog",
)


server.document-root        = "/var/www/html"
server.upload-dirs          = ( "/var/cache/lighttpd/uploads" )
server.errorlog             = "/var/log/lighttpd/error.log"
server.pid-file             = "/var/run/lighttpd.pid"
server.username             = "www-data"
server.groupname            = "www-data"
server.port                 = 80
accesslog.filename          = "/var/log/lighttpd/access.log"

index-file.names            = ( "index.php", "index.html", "index.lighttpd.html" )
url.access-deny             = ( "~", ".inc" )
static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" )

compress.cache-dir          = "/var/cache/lighttpd/compress/"
compress.filetype           = ( "application/javascript", "text/css", "text/html", "text/plain" )

# default listening port for IPv6 falls back to the IPv4 port
## Use ipv6 if available
#include_shell "/usr/share/lighttpd/use-ipv6.pl " + server.port
include_shell "/usr/share/lighttpd/create-mime.assign.pl"
include_shell "/usr/share/lighttpd/include-conf-enabled.pl"

fastcgi.server = (
    "/api" =>
        (
                (
                    "host" => "127.0.0.1",
                    "port" => 8081,
                    "check-local" => "disable"
                )
        )
)

When both lighttpd and my Python script are running on machine 10.200.0.7, I get the following when issuing a http call:

root@srv ~# http 10.200.0.7/api
HTTP/1.1 404 Not Found
Content-Length: 0
Content-Type: text/html; charset=UTF-8
Date: Thu, 22 Dec 2016 08:02:20 GMT
Server: lighttpd/1.4.35

From the perspective of lighttpd:

[email protected]# cat /var/log/lighttpd/access.log
10.200.0.1 10.200.0.7 - [22/Dec/2016:08:02:20 +0000] "GET /api HTTP/1.1" 404 0 "-" "HTTPie/0.9.2"

and from the python script one:

[email protected]# python3 wtest.py
Bottle v0.12.7 server starting up (using FlupFCGIServer())...
Listening on http://127.0.0.1:8081/
Hit Ctrl-C to quit.

404 Not Found from 10.200.0.1 @ http://10.200.0.7/api/

So the call goes though lighttpd, reaches wtest.py, who in turn issues a 404 as if the path was not known. On the other hand, when started directly (without flup and with the same routing), it returns hello.

Where does this difference in behaviour come from?

Answer 1

Newer versions of lighttpd support the uwsgi protocol, so that is probably a better solution than FastCGI to run your bottle app: https://redmine.lighttpd.net/projects/lighttpd/wiki/HowToPythonWSGI

In your case, have you configured flup, used by bottle, to expect the FastCGI protocol? flup can speak multiple different protocols, and you have to configure it to use FastCGI. The first few hits might help: https://www.google.com/#q=configuring+bottle+fastcgi+lighttpd