Using ZeroMQ in Kuberenetes?

[Patrik]

We're trying to write a micro service architectured batch worker. We're working towards using ZeroMQ for communicating between the producers and consumers, we have multiple of both and we auto scale based on the amount of work to be processed.

Do you know of any issues in using ZeroMQ in Kubernetes setups? Anything specific we need to keep in mind while doing this?

We are exposing ports through svc nodes for our broker (dealer + router according to the zeromq book).

Thanks in advance for any and all feedback you might find relevant.

// Patrik

[Peter Kleiweg]

I don't know anything about working with Kubernetes.

This from the README may be relevant:

This package provides a default context. This is what will be used by the functions without a context receiver, that create a socket or manipulate the context. Package developers that import this package should probably not use the default context with its associated functions, but create their own context(s).

[Patrik]

In which README file did you find that? I cannot find it in the READme for pebbe/zmq4.

What's in this context?
What might i have to change and/or implement in this context?

// Patrik

[Peter Kleiweg]

My mistake. It is not the README. It's at the top of the package doc:

go doc github.com/pebbe/zmq4

zmq4.Context is a wrapper for a ZeroMQ context.

There is a problem when someone imports two packages that both import zmq4, if those packages use the default context. Changing something in the default context from one package effects the default context used from the other package.

Solution: don't use the default context (unless you write a stand-alone program that doesn't import any other packages). Have your package create its own context with zmq4.NewContext(), and use that one for all other operations.

Don't use any function that works on the default context. When a function works on the default context, it is mentioned in the doc.

Instead of this:

zmq4.SetIpv6(true) // this changes a setting in the default context for any package that imports zmq4
socket, _ := zmq4.NewSocket(zmq4.REQ) // uses settings from the default context

_ = zmq4.Term() // Terminates the default context!

...use this:

ctx, _ := zmq4.NewContext() // get your own context
ctx.SetIpv6(true) // change some something in ctx
socket, _ := ctx.NewSocket(zmq4.REQ) // create socket with settings from ctx

_ = ctx.Term()

[Patrik]

What's the difference between the default context and the one I would get from NewContext()? Patrik Iselind Den tors 9 aug. 2018 16:47Peter Kleiweg <[e-mail]> skrev:

…

My mistake. It is not the README. It's at the top of the package doc: go doc github.com/pebbe/zmq4 zmq4.Context is a wrapper for a ZeroMQ context. There is a problem when someone imports two packages that both import zmq4, if those packages use the default context. Changing something in the default context from one package effects the default context used from the other package. Solution: don't use the default context (unless you write a stand-alone program that doesn't import any other packages). Have your package create its own context with zmq4.NewContext(), and use that one for all other operations. Don't use any function that works on the default context. When a function works on the default context, it is mentioned in the doc. Instead of this: zmq4.SetIpv6(true) // this changes a setting in the default context for any package that imports zmq4socket, _ := zmq4.NewSocket(zmq4.REQ) // uses settings from the default context _ = zmq4.Term() // Terminates the default context! ...use this: ctx, _ := zmq4.NewContext() // get your own context ctx.SetIpv6(true) // change some something in ctxsocket, _ := ctx.NewSocket(zmq4.REQ) // create socket with settings from ctx _ = ctx.Term() — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#139 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AUIB0XOYqjIOEKY79sImGmww7y4yqy3Sks5uPEuQgaJpZM4V1k2A> .

[Peter Kleiweg]

Anyone (any package) can change the default context. Only you can change the context you made.

[Patrik]

But isn't the new context based on the default one? Patrik Iselind Den tors 9 aug. 2018 18:24Peter Kleiweg <[e-mail]> skrev:

…

Anyone (any package) can change the default context. Only you can change the context you made. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#139 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AUIB0aAuH3oHEJbVPh0ubzsaqYConLkIks5uPGJagaJpZM4V1k2A> .

[Peter Kleiweg]

The new context is completely separate from the default context.

At first, there was only the default context. I thought that would simplify things. I hadden't thought it through. The default context is a bad idea. Just pretend it's not there. It only exists for backward compatability.

[Patrik]

So I'm back at square one, more or less. I shouldn't poke around with the context then? Patrik Iselind Den tors 9 aug. 2018 18:35Peter Kleiweg <[e-mail]> skrev:

…

The new context is completely separate from the default context. At first, there was only the default context. I thought that would simplify things. I hadden't thought it through. The default context is a bad idea. Just pretend it's not there. It only exists for backward compatability. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#139 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AUIB0fRqTKDQwIUgjaAnCjyWbCRT36yDks5uPGTggaJpZM4V1k2A> .

[Patrik]

I'll try with rabbitmq instead. Might be easier. Thanks for your time.