Beyond the limits of the CBS RA environment: efficient programming and the ODISSEI secure supercomputer
Presentation and code for CBS microdata meeting on May 16th, 2022.
What to do when your CBS microdata analysis takes too many computational resources to run on the remote access environment? In this meeting, Erik-Jan van Kesteren (Utrecht University) will talk about solutions to this problem. It will be an accessible introduction to a variety of ways in which you can programme more efficiently when using microdata in your research. Furthermore, it will discuss when you should and should not move your project to the ODISSEI Secure Supercomputer.
The introduction will include some live coding, exploring different options for project organisation, speeding up code, benchmarking, profiling, and reducing memory requirements. During his talk, Van Kesteren will also touch upon topics such as "embarassingly parallel", scientific programming, data pipelines, open source, and open science. Although the presentation will center around data analysis with R, these principles also hold for other languages, such as Python or Julia.
This project is developed and maintained by the ODISSEI Social Data Science (SoDa) team.
Do you have questions, suggestions, or remarks? File an issue in the issue tracker or feel free to contact Erik-Jan van Kesteren (@ejvankesteren)