Welcome to the Research Software Engineering Operations (RSE-ops) landscape! As you might guess, RSE-ops is a flavor of DevOps. If DevOps is the intersection of "Developer" and "Operations," and we want to map this idea to research software engineering and technologies that are more likely found in high performance computing and scientific software development, then we can define RSE-ops as the intersection of Research Software Engineering and Operations. Research Software Engineers (RSEs) are the individuals writing code for scientific software, and supporting researchers to use codes on high performance computing systems. RSE-ops, then, generally refers to best practices for development and operations of scientific software. It's important to note that HPC is only a subset of research software engineering. However, because so many modern applications are web-based (or extend beyond HPC) it's important to consider that set as part of a larger scientific, or research software engineering, universe.
This landscape and the RSE-ops community is a community initiative started at LLNL by @vsoch and under the umbrella of the RADIUSS initiatives.
To learn more about RSE-ops, we invite you to visit the landscape.
Build the container, if needed.
$ make buildUpdate the landscape after making changes to the tex files in src (requires docker)
$ makeRun the jekyll server (requires jekyll installed)
$ make runOpen to http://localhost:4000/landscape/
The site here renders from LaTex, which we convert to markdown. This will happen in continuous integration, but you can also do it locally! E.g., you might first want to make changes to LaTeX files in the src folder, and then build a container to compile the PDF:
$ docker build -t latex2md .And then enter the container, binding the present working directory with code:
$ docker run -it -v $PWD:/code --entrypoint bash latex2mdThese commands are represented with make build and make shown earlier. The
commands above will do the following:
- Take each file defined in includes.txt included in the src folder and render into _includes/paper
- Update the newly generated markdown to be in a reference format that is rendered by jekyll-scholar, including a bibliography.
- Also render a new PDF of the entire document.
The cool thing is that we are rendering content into a website (Jekyll) from LaTex, and that some content is also rendered into a traditional looking paper. Awesome! The first is more fun and readable, and the second is mostly expected for any kind of academic endeavour.
You'll have pandoc on the path:
# which pandoc
/usr/bin/pandocAnd you can use it to compile the entire paper into a PDF:
$ cd /code/src
$ pandoc -s main.tex -o "rse-ops.pdf" --pdf-engine=xelatex --bibliography rseops.bib --citeprocNote that the output format isn't perfect - I'm still working on figuring out how to get from Overleaf to manual here!
For each page, the basic logic to convert from latex to markdown is:
$ pandoc -s monitoring.tex -o monitoring.mdThis is the basic logic that we use to render pages of content for the Jekyll site, and we render them to be "includes" to make it easy to insert snippets where we need them!
Thus, we can combine the two things above to automatically build the site - meaning that we render latex based on a list in includes.txt to generate markdown that is then included in the pages. After building the container, do the following:
$ docker run -it -v $PWD:/code latex2mdFor each category of technology, we have a "chooser" interface, or a place where you can go (typically as a new user) to answer questions relevant to your use case, and get a recommendation of a technology. You can read more about the chooser here.
Thank you to contributors to the original Overleaf document that are not included in the version history, including:
This is a community effort across national labs, academic institutions, and interested parties in industry, and we encourage you to participate!
Copyright (c) 2017-2021, Lawrence Livermore National Security, LLC and contributors.
This project is licensed under the MIT license LICENSE.
Copyrights and patents in the RSE-ops project are retained by contributors. No copyright assignment is required to contribute to this project.
This work was produced under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344.
LLNL-TR-824197