EAGLE-XL is the next generation of cosmological simulations of periodic cubic volumes, created with the SWIFT hydrodynamical code and its derivatives: Velociraptor for halo finding and Swiftsimio for handling initial conditions and reading outputs.
In order to clone this repository locally, just run
git clone https://github.com/edoaltamura/xl-zooms.git
This will download all the top-level codes for zooms analysis. The ic-zooms directory links to an external project for the generation
of initial conditions of zoom simulations. After cloning xl-zooms, ic-zooms will still be empty, as Git does not import submodules
by default. If you have read privileges on the project linked to ic-zooms, run
git submodule update --init --recursive
in the cloned xl-zooms repository to clone the submodules recursively. If your want to update xl-zooms, you can do so with
git pull
and for updating all submodules
git submodule update --recursive --remote
Access to the mirrored ic-zooms repository may be granted on a case-by-case basis after consulting the contributing co-authors of the project.
Zoom-in simulations are high-resolution runs of a given cosmological volume, where the particle resolution is highly enhanced in correspondence of objects of interests (typically clusters or galaxy groups) and lowered elsewhere. The zoomed halos are chosen to be isolated objects in one of the EAGLE-XL boxes at present redshift. Three halos are currently selected within the mass range 1013 ≤ M200/M⊙ ≤ 1014.
Zooms are run with a single-node configuration and preferably on the cosma7 computer cluster. The table below summarises
the properties of the computing clusters available to the Virgo Consortium. Visit the COSMA-DiRAC pages
for more info.
| Machine name | Memory/node | Cores/node | CPU info | Total nodes |
|---|---|---|---|---|
| Cosma 5 | 128 GB | 16 | 2x Intel Xeon E5-2670 0 @ 2.60GHz | 302 |
| Cosma 6 | 128 GB | 16 | 2x Intel Xeon E5-2670 0 @ 2.60GHz | 675 |
| Cosma 7 | 512 GB | 28 | 2x Intel Xeon Gold 5120 @ 2.20GHz | 452 |
| Cosma 8 (compute) | 1 TB | 64 | 2x AMD EPYC 7H12 @ 2.6GHz | 32 |
| Cosma 8 (monolith) | 4 TB | 64 | 2x AMD EPYC 7702 @ 2.6GHz | 1 |
An important task of this project is to assess the possibility to run high-resolution zooms with SWIFT on one single node,
first with dark matter only, then with baryonic physics integrated. Due to the intensive use of memory, the final version
of the hydro simulations might make use of the cosma8's large capacity nodes.
The process of simulating useful zooms of a galaxy cluster (or group) comprises several stages, some of which are listed below.
- Generate of the (dark matter-only) cosmological parent box and the relative structure-finder output.
- Select of the (dark matter-only) halos from the parent box meeting specific criteria.
- Generate if the initial conditions for the zoom simulation:
- Particle load: Mask the high-resolution region;
- Particle load: Make the high-resolution grid;
- Particle load: Populate the high-resolution grid with particles using glass files;
- Particle load: Generate outer particles of low-resolution grid with growing skins;
- Construct the Panphasian parameter file and submission script;
- Run Panphasia on the particle load file to generate the Gaussian white noise field.
- Configure SWIFT and Velociraptor for dark matter-only zooms.
- Submit the initial condition file to SWIFT (with Velociraptor embedded) and run to the present redshift (dark matter-only).
- Analyse the dark matter-only snapshots and diagnose possible inconsistency between the parent and zoomed halos.
- Configure SWIFT and Velociraptor for zooms with hydrodynamics and subgrid.
- Submit the initial condition file to SWIFT (with Velociraptor embedded) and run to the present redshift (hydro and subgrid).
- The full hydro version of the zoom simulation is complete and ready to be analysed.
An alternative route consists in running SWIFT without Velociraptor embedded and then launch the structure finder on the snapshots in the post-production phase. Since the SWIFT/Velociraptor integration for hydrodynamic runs is still under development, the current zoom pipeline operates with the stand-alone version of Velociraptor, called once the SWIFT job is completed.
Before running a high-resolution zoom simulation of a galaxy cluster (or group), we firstly need to select it from a cosmological environment.
EAGLE-XL's testing program includes the production of medium-sized dark matter-only cosmological boxes (size 300 Mpc) and the first set of
halos are selected from one of such boxes: L0300N0564.
Dark matter field-halos are identified using Velociraptor. Then, we select those which are found to be in a relatively isolated region of
the box, defined by specific isolation criteria.
The /xl-zooms/performance directory contains a suite of scripts for a simple diagnostics of the runtime. They are adapted from
Josh's xl-pipeline and from SWIFT's tools repository.
For further information about the usage of codes in this repository, please contact edoardo.altamura@manchester.ac.uk. These programs are provided and distributed without any warranty.
If you are a fan of dark modes when viewing code, you can enable a dark theme for GitHub by adding this extension to your favourite web browser. The EAGLE-XL logo displayed on this page will also give a much more pleasant outlook with dark-mode enabled.
Logo design by Edo Altamura © 2020