Using Twisted's reactor to run a DieselFlask app doesn't work

[qoh]

(Using Ubuntu 12.10 and packages from the official apt repositories)

When using WSGIResource/Site in order to run a DieselFlask app (in these examples, "app"), you'll always end up with getting some exception from Diesel and Twisted's internal code.

from twisted.internet import reactor
from twisted.web.wsgi import WSGIResource
from twisted.web.server import Site

resource = WSGIResource(reactor, reactor.getThreadPool(), app)
site = Site(resource)

reactor.listenTCP(8080, site)
reactor.run()

With this method (app is a simple "hello world" route on /), going to / throws this exception:

WSGI application error
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/twisted/python/threadpool.py", line 167, in _worker
    result = context.call(ctx, function, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 118, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 81, in callWithContext
    return func(*args,**kw)
  File "/usr/lib/python2.7/dist-packages/twisted/web/wsgi.py", line 332, in run
    self.reactor.callFromThread(wsgiError, self.started, *exc_info())
--- <exception caught here> ---
  File "/usr/lib/python2.7/dist-packages/twisted/web/wsgi.py", line 315, in run
    appIterator = self.application(self.environ, self.startResponse)
  File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1518, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1502, in wsgi_app
    with self.request_context(environ):
  File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1463, in request_context
    return RequestContext(self, environ)
  File "/usr/lib/python2.7/dist-packages/flask/ctx.py", line 87, in __init__
    self.url_adapter = app.create_url_adapter(self.request)
  File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1348, in create_url_adapter
    return self.url_map.bind_to_environ(request.environ,
exceptions.AttributeError: 'dict' object has no attribute 'environ'

After several tries, @dontcare4free managed to make a WSGI middleware that fixes that case. More specifically:

from twisted.internet import reactor
from twisted.web.wsgi import WSGIResource
from twisted.web.server import Site

from flask.wrappers import Request
from UserDict import UserDict

class _RequestDict(UserDict, Request):
    def __init__(self, environ):
        Request.__init__(self, environ)
        UserDict.__init__(self)

    @property
    def data(self):
        return self.environ

    @data.setter
    def data(self, data):
        pass  # Bad, but required for this

def environFixerMiddleware(f):
    def wrapper(environ, start_response):
        return f(_RequestDict(environ), start_response)

    return wrapper

resource = WSGIResource(reactor, reactor.getThreadPool(), environFixerMiddleware(app))
site = Site(resource)

reactor.listenTCP(8080, site)
reactor.run()

This works for a simple "hello world" example. However, when throwing more complicated stuff like WebSockets and such (taken from examples) into the cocktail, another exception arises, which ends up in the actual code that handles exceptions:

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/twisted/python/threadpool.py", line 167, in _worker
    result = context.call(ctx, function, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 118, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 81, in callWithContext
    return func(*args,**kw)
  File "/usr/lib/python2.7/dist-packages/twisted/web/wsgi.py", line 332, in run
    self.reactor.callFromThread(wsgiError, self.started, *exc_info())
--- <exception caught here> ---
  File "/usr/lib/python2.7/dist-packages/twisted/web/wsgi.py", line 315, in run
    appIterator = self.application(self.environ, self.startResponse)
  File "avenir-api/hack.py", line 19, in wrapper
    return f(_RequestDict(environ), start_response)
  File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1518, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1506, in wsgi_app
    response = self.make_response(self.handle_exception(e))
  File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1194, in handle_exception
    self.log_exception((exc_type, exc_value, tb))
  File "/usr/local/lib/python2.7/dist-packages/diesel/web.py", line 61, in log_exception
    self._logger.trace().error('Exception on {0} [{1}]',
    exceptions.AttributeError: 'NoneType' object has no attribute 'trace

Not really sure whether this is an issue with Diesel or Twisted, though.

[Christian Wyglendowski]

I'm not surprised that you're running into trouble here. Twisted and diesel are two different answers to the same problem, and probably aren't compatible.

I don't see any diesel code in your first traceback. Not sure what's going on there.

The second traceback is happening because DieselFlask instances are meant to run within diesel, by calling either the .run(...) or .make_service(...) methods. You can try to workaround that particular traceback by calling make_logger(...) on the DieselFlask instance (app in your examples) but WebSockets aren't going to work - they depend on diesel owning the event loop, and you are using Twisted's event loop.