Writing RAMSES outputs

Rewrite a RAMSES output for any number of MPI ranks, or write one from Mera data, so RAMSES can restart the run. The walkthrough with every check and figure is Restart RAMSES with any number of CPUs.

you havefunctionresult
the RAMSES outputrepartitionevery value bit-identical
complete Mera data or a mera filewriteramsesleaves bit-identical, refined cells rebuilt as RAMSES builds them; gravity ones kept as stored with getgravity(info; parents=true)

Both write ordering='hilbert' or 'bisection', plain or in IO groups (IOGROUPSIZEREP), and read older formats: outputs without a namelist, legacy particle files, gravity files with the forces only.

From RAMSES files

Mera.repartition — Function
repartition(output, path, ncpu; outpath, ordering=:auto, iogroupsize=0, overload=1,
            balance_particles=true, ngridmax=:auto, nexpand_bound=:auto, overwrite=false, verbose=true)
repartition(info::InfoType, ncpu; outpath, ...)

Rewrite a RAMSES output for ncpu MPI ranks, so the simulation can be restarted with mpirun -np ncpu (namelist nrestart = output). RAMSES itself only restarts with the number of ranks that wrote the output.

path is the simulation folder holding output_XXXXX, as for getinfo. The new output_XXXXX is written into outpath, which is normally the new run directory.

What is done, in order:

  1. the global octree is read from all old amr files;
  2. the Hilbert curve is split into ncpu domains with equal work (cells, plus particles when balance_particles=true, so no rank exceeds npartmax);
  3. every new rank's octree is built as RAMSES would hold it in memory: its own grids, the ghost grids around them and the boundary grids, with all pointers;
  4. the amr, hydro, gravity and RT files are written with the values copied bit for bit; particles move to the rank that owns the cell around them; clump tables are re-dealt;
  5. info_*.txt gets the new ncpu and domain table; files that do not depend on ncpu (header, namelist.txt, sink_*.csv, descriptors, …) are copied.

Returns a report with the grid and particle counts per rank. Its ngridmax_needed and npart_max are the minimum ngridmax/npartmax for the restart namelist.

ordering is the domain decomposition of the new files: :auto keeps the output's own, :hilbert or :bisection converts (the restart namelist must then name it). iogroupsize = n writes the files of every n ranks into one group_GGGGG folder, as RAMSES with IOGROUPSIZEREP = n; outputs in group folders are read either way.

Supported: 3-D outputs with ordering='hilbert' or 'bisection', plain or in IO groups, periodic or simple physical boundaries (also without a namelist), hydro, MHD, RT, gravity (also the older forces-only files), particles (also the legacy format), sinks and clumps. Not supported: QUADHILBERT builds, and 1-D or 2-D runs (Mera is 3-D).

r = repartition(6, "/sims/run_8cpu", 3; outpath="/sims/run_3cpu")
# then:  cd /sims/run_3cpu && mpirun -np 3 ramses3d run.nml 6

See also compareramses to check a conversion, and writeramses to write an output from Mera data instead of from RAMSES files.

Mera.compareramses — Function
compareramses(output, path_a, path_b; leaves_only=false)

Compare two RAMSES outputs with the same output number, written with any numbers of CPUs: whether they hold the same octree, the largest difference of every hydro, gravity and RT variable (relative to that variable's largest value), and the particles (matched by identity) and sinks.

Use it to check a repartition (the values must be identical), a round trip A → B → A, or two restarts of the same run on different numbers of ranks. With leaves_only=true only leaf cells are compared, which is the right test for an output written by writeramses, whose refined cells are rebuilt from their children.

From Mera data

Mera.writeramses — Function
writeramses(ncpu; hydro, gravity=nothing, rt=nothing, particles=nothing, sinks=nothing,
            outpath, overwrite=false, verbose=true, force=false, iogroupsize=0,
            ordering=:auto) -> RepartitionReport

Write a restartable RAMSES output with ncpu files from Mera data (loaded with gethydro/getgravity/getrt/getparticles/getsinks, or from mera files with loaddata). The data must be complete (restart_completeness). Refined cells are rebuilt from the leaves, with RAMSES's own averaging rules. Files that leaf data cannot carry (info_rt, stellar_*.csv, …) are copied from the original output if it still exists.

Refused: incomplete data, particle files older than part_file_descriptor.txt (use repartition on the original files), writing into the output the data came from, and a cosmological run from an old mera file whose simulation is gone. force=true writes the last one anyway, and also incomplete data; such an output is for inspection, not for a faithful restart.

iogroupsize = n > 0 writes the files of every n ranks into one group_GGGGG folder, as RAMSES does with IOGROUPSIZEREP = n; RAMSES must then run with the same value. ordering is the domain decomposition, :hilbert or :bisection; :auto takes the run's own (its namelist, or the original output), which the restart namelist names.

Mera.restart_completeness — Function
restart_completeness(gas; gravity=nothing, rt=nothing, particles=nothing, sinks=nothing)
    -> (ok::Bool, report::Vector{Pair{String,String}})

Proves, from the data itself, that the Mera objects hold the complete state of the simulation and can be written as a restartable RAMSES output: full box, all levels and variables, no clamping, a closed octree covering the box exactly, gravity/RT on the same cells, every particle and sink.

Mera.upgrade_merafile — Function
upgrade_merafile(output, path; datatypes=[:hydro, :gravity, :particles, :rt, :clumps, :sinks],
                 simulation=nothing, parents=true, verbose=true)

Brings a mera file saved by an older Mera up to date: fills grid_info.amr_header from the original simulation (simulation = its directory, default the path recorded in the file) and saves every data type of the file again. With parents=true complete gravity data also gets the stored values of its refined cells (see getgravity(...; parents=true)). Old files load fine without this; the upgrade makes them self-contained, e.g. for writing a RAMSES restart after the simulation is deleted.