Pods for bluesky(-adaptive)
This is a set of buildah and podman scripts that will stand up a pod that can run a Bluesky session and an out-of-core adaptive plan
Get podman
Podman and buildah are packaged on many Linux distributions. Refer to
the official installation guide
for specific instructions. These instructions cover how to install podman
.
Also install buildah
in exactly the same fashion.
Enable "rootless" usage
Unlike Docker, podman and buildah can be used without elevated privileges (i.e.
without root
or a docker
group). Podman only needs access to a range of uids
and gids to run processes in the container as a range of different "users".
Enable that like so:
sudo usermod --add-subuids 200000-201000 --add-subgids 200000-201000 $USER
podman system migrate
For additional details and troubleshooting, see the rootless tutorial.
Configure for display over SSH
If the machine where you will be running podman is one you are connected to via
SSH, then you will need to configure the SSH daemon to accept connections routed
through podman---specifically, connections to its IP address rather than
localhost
.
Add this line to /etc/ssh/sshd_config
.
X11UseLocalhost no
If podman is running on the machine you are sitting in front of, no action is required.
Build the containers
# this is fedora + some heavy weight Python
bash image_builders/build_bluesky_base_image.sh
# installs the rest of our stack on top of the base image
bash image_builders/build_bluesky_image.sh
# build an image with caproto installed
bash image_builders/build_caproto_image.sh
# build an image for the databroker server
bash image_builders/build_databroker_server_image.sh
# build an image with pydm / typhos installed
bash image_builders/build_typhos_image.sh
If you are feeling brave (and have the dependencies checked out as peers of this directory) build a "snapshot" image via
bash image_builders/build_bluesky_snapshot.sh
run the pod
# this sarts up caproto, mongo, zmqproxy, and redis
bash start_core_pod.sh
.. manually
Run
bash launch_bluesky.sh
in a terminal or
bash launch_bluesky.sh bluesky-dev
to get the snapshot version.
...and watch from the outside
On your host machine run:
pip install -r bluesky_config/scripts/requirements.txt
python bluesky_config/scripts/kafka_echo_consumer.py
...adaptively
Start the adaptive server:
bash start_adaptive_server.sh
Running in the shell should
from ophyd.sim import *
RE(adaptive_plan([det], {motor: 0}, to_brains=to_brains, from_brains=from_brains))
should now take 17 runs stepping the motor by 1.5. The data flow is
| ---> kafka to the edge --- /exposed ports on edge/ --> external consumers
| ---> mongo
| ---> live table
|
^
RE ---- kafka broker -----> adaptive_server
^ |
| < -------- redis --------<-----< |
Maybe redis should be replaced by kafka?
The extra imports are because the motor and det that are set up by 00-base.py do not have the keys that the adatptive code is expecting.
...queuely
and run
RE(queue_server_plan())
On your host machine post to that end point (via httpie in this example):
http POST localhost:60607/qs/add_to_queue 'plan:={"plan":"scan", "args":[["pinhole"], "motor_ph", -10, 10, 25]}'
and watch the scans run!
The data flow is
| ---> kafka to the edge ----------- /exposed ports on edge/ ---> external consumers
| ---> mongo |
| ---> live table |
^ ↓
RE < --- http --- queueserver < ---- / http from edge / <-------- http POST {json}