Goals/scope of the project

[Christoph Buchner]

We should agree on a set of goals and non-goals, and use cases that we want to cover with this protocol, to avoid getting "in too deep" and getting bogged down by expanding scope.
Questions like:

1. What use-cases do we want to have covered in principle
2. What use-cases are already covered by other, "bigger" frameworks
3. How do we want to separate "our niche" from their established frameworks, what is a realistic frame/scope for this

I already understood some goals are

• be simple to start with -- download a package, write some code to connect instruments and datahandlers, maybe a procedure, start a logger, go!
• Scope is one or more computation nodes (on the same network), with maybe a couple dozen instruments max.

[Christoph Buchner]

"LEGO!-like modularity is important -- reassemble, change combinations of devices, easily adapt to changing experiments, data rate needs, etc.

Benedit writes:

Let us focus on the basic building blocks and let the people create their castle, house, villa, or farm. Therefore I propose to minimize the GUI and data analysis / measurement control part. Instead, we give the tools (the communication) to connect everything.
For that we need:

• Instrument drivers (already present)
• An instrument controller, which is the communication part for an instrument (user has to glue it to the instrument driver)
• A data sink controller (to which you can connect different data sinks)
• a simple data sink to get started
• A communication interface for the "mastermind", which may control the instrument controllers and data sink controller
• A simple gui to show / control data

[Christoph Buchner]

We want to be asynchronous: The controller might want to send several commands (to different devices) at the same time without waiting for the answer of a (slow) instrument before writing to the next one.

[Christoph Buchner]

I write: Just to pipe in on this small detail: I will push back a lot before I'd greenlight adding async code to PyMeasure. I absolutely don't want the headache of the red/green function problem, and in our context there are much easier solutions than reaching for async.

I had exactly the same problem (of a slow instrument), and could solve this nicely using concurrent.futures -- every request to any number of instruments (e.g. in a poll cycle refreshing all connected instruments) were submitted to a ThreadPoolExecutor. The futures patiently wait until the instruments reply, and you can collect them as they arrive (as_completed). Communication was locked with threading.Lock for every instrument instance individually, so there was no possibility that two requests to one instrument wreak havoc.

In effect, my loop time went from N * average response time to being capped by the slowest (a vacuum pressure sensor).

[Benedikt Burger]

A use case I'd like to have covered:

• Several (10ish) instrumental nodes
• A few (2ish) data acquisition nodes
• Regular readout of the instrument nodes into the data acquisition
• Control the instrument and acquisition nodes from some control node (maybe even more than one)
• Mix and match the used instruments and the acquired data easily (changing the setup several times a day)
• A failing node should not corrupt the measurement data acquired until then for important measurements (most measurements are for alignment purposes and the data is only needed for the alignment and not afterwards, therefore corrupted data does not matter)

More general goals:

• (from our discussion:) Some (or any) node may be written in another language (LabView, C, Fortran...)
• Good and simple protocol definition, that anybody might write their own software talking this protocol.
• Simple setup: Start two communication nodes, start the coordinator and send a message from one node to the other.

My vision:

• You can send a message to any participant by specifying its (human readable) name, similar to email (transfer part of our protocol)
• We have a default interpretation of the message content to implement remote procedure call (RPC), i.e. to set/read properties or call methods. (control part of our protocol)
• Taking advantage of zmq's PUB-SUB structure, we have a broadcasting channel for regular data readouts (and log entries).

Scope:

I think this repository should contain the protocol definition and their reference implementation in python.

Instrumental and data acquisition nodes contain at one hand a protocol implementation (which should reside in here) and on the other hand threading and data storage parts etc., which might be better situated at pymeasure itself. Or we could put these "ready-to-use" parts into a special folder to separate them from the protocol itself.

[Christoph Buchner]

Non-goal: Get involved with unit conversion or computations with units (Units are possible and optional, but the spec will not deal with unit conversion of computation; #15).