./manage.py runworker randomly crashes at startup

[Vincent MONTÉCOT]

Hello!

I'm trying to use asgi_ipc as channel backends but if sometimes it works :

$ ./manage.py runworker
2016-09-29 12:31:35,258 - INFO - runworker - Running worker against channel layer default (asgi_ipc.IPCChannelLayer)
2016-09-29 12:31:35,259 - INFO - worker - Listening on channels http.request, websocket.connect, websocket.disconnect, websocket.receive

But sometimes (3 times out of 4) it crashes :

$ ./manage.py runworker
Traceback (most recent call last):
  File "/path/to/site-packages/asgi_ipc.py", line 208, in __init__
    size=self.size,
ValueError: The size is invalid or the memory is read-only

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/path/to/site-packages/django/core/management/__init__.py", line 353, in execute_from_command_line
    utility.execute()
  File "/path/to/site-packages/django/core/management/__init__.py", line 345, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/path/to/site-packages/django/core/management/base.py", line 348, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/path/to/site-packages/django/core/management/base.py", line 399, in execute
    output = self.handle(*args, **options)
  File "/path/to/site-packages/channels/management/commands/runworker.py", line 36, in handle
    self.channel_layer = channel_layers[options.get("layer", DEFAULT_CHANNEL_LAYER)]
  File "/path/to/site-packages/channels/asgi.py", line 53, in __getitem__
    self.backends[key] = self.make_backend(key)
  File "/path/to/site-packages/channels/asgi.py", line 44, in make_backend
    asgi_layer = backend_class(**self.configs[name].get("CONFIG", {}))
  File "/path/to/site-packages/asgi_ipc.py", line 43, in __init__
    self.channel_store = MemoryDict("/%s-chan" % self.prefix, size=channel_memory)
  File "/path/to/site-packages/asgi_ipc.py", line 213, in __init__
    "Error was: %s" % e
ValueError: Unable to allocate shared memory segment (potentially out of memory).
Error was: The size is invalid or the memory is read-only
Exception ignored in: <bound method MemoryDatastructure.__del__ of <asgi_ipc.MemoryDict object at 0x104d48438>>
Traceback (most recent call last):
  File "/path/to/site-packages/asgi_ipc.py", line 294, in __del__
    self.mmap.close()
AttributeError: 'MemoryDict' object has no attribute 'mmap'

I used the following CHANNEL_LAYERS value :

CHANNEL_LAYERS = {
    "default" : {
        "BACKEND" : "asgi_ipc.IPCChannelLayer",
        "CONFIG" : {
            "prefix" : "ws",
        },
        "ROUTING": "ws.routing.channel_routing",
    },
}

And installed asgi_ipc from sources, the last commit being :

f22c60a

I've seen the issue #4 but I don't know it it is related. Anyway the PR used to solve this one doesn't prevent this problem for me.

I have no problem if I use an other channel backend (tested with asgi_redis).

[Andrew Godwin]

As the exception ways, it's failing to allocate the shared memory segment, probably due to RAM pressure (are you close to using all of your memory?) - there's not much the code can do if mmap fails, it means the system rejected us for some reason.

If you're very short on memory, you can try changing the channel_memory and group_memory config values down below the defaults of 100MB and 20MB - see the readme for more info on that.

[Jonathan Stray]

I get this every single time I try daphne cjworkbench.asgi:channel_layer

Traceback (most recent call last):
  File "/Users/jonathanstray/anaconda/lib/python3.5/site-packages/asgi_ipc.py", line 208, in __init__
    size=self.size,
ValueError: The size is invalid or the memory is read-only

And then several other exceptions while handling that one. This is on Mac OS. Maybe is mmap not supported on this OS?

My settings:

CHANNEL_LAYERS = {
    "default": {
        "BACKEND": "asgi_ipc.IPCChannelLayer",
        "ROUTING": "cjworkbench.routing.channel_routing",
        "CONFIG": {
            "prefix": "cjworkbench",
        },
    },
}

[Andrew Godwin]

Above comment addressed in #19.