Data Loading API Reference

loadall(path, output; components, myargs, verbose, kwargs...) -> NamedTuple

Read several components of one snapshot in a single call, and return them named.

Replaces the usual opening of an analysis, a getinfo followed by one getter per component each repeating the same selection, with one line:

(; hydro, gravity, particles) = loadall("/path/to/sim", 300)

The returned NamedTuple also carries info, so nothing is lost:

d = loadall(path, 300)
d.info.levelmax

Keywords

• components: which to read, as symbols, e.g. (:hydro, :particles). Defaults to every component the snapshot actually has, so a run without gravity simply returns fewer fields rather than failing.
• myargs: an ArgumentsType bundle, applied to every component. This is the point of pairing the two: set lmax, xrange and center once and every read honours them.
• verbose: passed through; defaults to Mera's global setting.
• any further keyword is forwarded to each getter, so lmax=10 or xrange=[-10,10] work directly without a bundle.

Examples

# everything the snapshot has
(; hydro, gravity, particles) = loadall(path, 300)

# one bundle, applied to every component
args = ArgumentsType(lmax=10, xrange=[-10., 10.], yrange=[-10., 10.],
                     center=[:bc], range_unit=:kpc)
(; hydro, particles) = loadall(path, 300; components=(:hydro, :particles), myargs=args)

# or the keywords inline
d = loadall(path, 300; components=(:hydro,), lmax=9)

A component that fails to read is reported and returned as nothing rather than taking the whole call down, so one missing file does not cost you the rest of the snapshot.

See also: @loadall, getinfo, ArgumentsType.

@loadall path output component...

Bind several components of one snapshot to variables of those names, in one line.

@loadall path 300 hydro gravity particles
# hydro, gravity and particles are now bound, and so is info

This is a thin wrapper over loadall; it expands to a destructuring assignment and adds nothing of its own.

(; hydro, gravity, particles) = loadall(path, 300)

Keyword arguments are passed through:

@loadall path 300 hydro particles lmax=10 range_unit=:kpc

Returns the same named tuple as loadall, so nt = @loadall path 300 hydro also works.

withargs(args; kwargs...) -> ArgumentsType

A copy of args with some fields changed, leaving the original untouched.

Deriving a variant of a bundle otherwise means deepcopy and then assignment, two lines and a mutable original one step away from being edited by accident:

base = ArgumentsType(lmax=10, xrange=[-10., 10.], center=[:bc], range_unit=:kpc)
coarse = withargs(base; lmax=7)          # same region, fewer levels
zoom   = withargs(base; xrange=[-2., 2.], yrange=[-2., 2.])

base is unchanged in both cases.

See also: ArgumentsType, loadall.

gethydro(info::InfoType; vars=[:all], kwargs...) -> HydroDataType
gethydro(info::InfoType, var::Symbol; kwargs...) -> HydroDataType
gethydro(info::InfoType, vars::Vector{Symbol}; kwargs...) -> HydroDataType

Load the hydro cells of a RAMSES output into a table in memory.

Each row is one leaf cell: its refinement level, its integer position on that level, and one column per hydro variable, in code units. Cells refined beyond lmax are returned at level lmax, with the values RAMSES stores for them. The output files are read in parallel, one group of files per thread.

Arguments

• info::InfoType: the output to read, from getinfo.
• var, vars: the variables to load, in any order. :all (the default) loads every hydro variable: :rho, :vx, :vy, :vz, :p, the magnetic field of MHD runs, and the passive scalars. Scalars carry the names from hydro_file_descriptor.txt when the output has one (e.g. :metallicity), otherwise positional names (:var6, :var7, ...); positional names always work. :cpu adds the number of the file each cell came from.

Keywords

Selection:

• lmax::Real = info.levelmax: the deepest level to read.
• xrange, yrange, zrange: [min, max] of the region to read, in range_unit and relative to center. missing (the default) means the full extent.
• center = [0, 0, 0]: the origin of the ranges, in range_unit. [:bc] is the box centre; single components can be :bc, e.g. [:bc, :bc, 12.0].
• range_unit::Symbol = :standard: the unit of the ranges and the centre. :standard means fractions of the box (0 to 1); any length unit in info.scale works, e.g. :kpc or :pc.

Value checks:

• smallr::Real = 0: raise densities below this value to it, in code units. 0 switches it off.
• smallc::Real = 0: the same floor for the thermal pressure (the name follows RAMSES).
• check_negvalues::Bool = false: count negative densities and pressures and report them. Used only where the matching floor is off.

Output and performance:

• max_threads::Int = Threads.nthreads(): the most threads to use for reading.
• verbose::Bool = true: print a summary of what is read.
• show_progress::Bool = true: show a progress bar.
• print_filenames::Bool = false: print each file name as it is read.
• myargs::ArgumentsType: set lmax, the ranges, center, range_unit, verbose and show_progress in one object. A keyword given directly takes precedence.

Returns

A HydroDataType with

• data: the table, with the columns :level, :cx, :cy, :cz and then the variables;
• info, scale: the simulation information and its unit scales;
• lmin, lmax, boxlen, ranges: the levels and the region that were read;
• selected_hydrovars, used_descriptors, smallr, smallc: the options used.

Derived quantities, positions and units come from getvar, not from the table.

Examples

julia> info = getinfo(300, "path/to/simulation");

julia> gas = gethydro(info);                       # every variable, the whole box

julia> gas = gethydro(info, [:rho, :p]; lmax=8);   # two variables, down to level 8

julia> propertynames(gas.data.columns)
(:level, :cx, :cy, :cz, :rho, :p)

julia> gas = gethydro(info; xrange=[-10, 10], yrange=[-10, 10], zrange=[-2, 2],
                      center=[:bc], range_unit=:kpc);   # a 20 x 20 x 4 kpc slab

julia> getvar(gas, :mass, :Msol)                   # cell masses in solar masses

See also getinfo, getvar, subregion, projection, savedata.

getparticles(info::InfoType; vars=[:all], kwargs...) -> PartDataType
getparticles(info::InfoType, var::Symbol; kwargs...) -> PartDataType
getparticles(info::InfoType, vars::Vector{Symbol}; kwargs...) -> PartDataType

Load the particles of a RAMSES output (dark matter, stars, sinks, tracers) into a table in memory.

Each row is one particle: its position, its identity and one column per particle variable, in code units. The output files are read in parallel, one group of files per thread.

Arguments

• info::InfoType: the output to read, from getinfo.
• var, vars: the variables to load, in any order. :all (the default) loads every particle variable of the output, e.g. :vx, :vy, :vz, :mass, :birth, :metals. :cpu adds the number of the file each particle came from.

Keywords

Selection:

• xrange, yrange, zrange: [min, max] of the region to read, in range_unit and relative to center. missing (the default) means the full extent.
• center = [0, 0, 0]: the origin of the ranges, in range_unit. [:bc] is the box centre; single components can be :bc, e.g. [:bc, :bc, 12.0].
• range_unit::Symbol = :standard: the unit of the ranges and the centre. :standard means fractions of the box (0 to 1); any length unit in info.scale works, e.g. :kpc or :pc.
• stars::Bool = true: false drops the star particles. They are found by :family == 2 in current RAMSES outputs and by :birth > 0 in older ones, so that column must be loaded.
• subsample::Real = 1.0: read only every n-th output file, with n = 1/subsample. Each file holds one part of the box, so the result is a patchy subset, not a random sample: use it for quick looks at very large runs.
• lmax: ignored for particles; a warning says so if it is set.

Output and performance:

• max_threads::Int = Threads.nthreads(): the most threads to use for reading.
• presorted::Bool = true: sort the table by position and identity, which makes later selections faster.
• verbose::Bool = true: print a summary of what is read.
• show_progress::Bool = true: show a progress bar.
• print_filenames::Bool = false: print each file name as it is read.
• myargs::ArgumentsType: set the ranges, center, range_unit, verbose and show_progress in one object. A keyword given directly takes precedence.

Returns

A PartDataType with

• data: the table, with the columns :level, :x, :y, :z, :id (in current outputs also :family and :tag) and then the variables;
• info, scale: the simulation information and its unit scales;
• lmin, lmax, boxlen, ranges: the levels and the region that were read;
• selected_partvars, used_descriptors: the options used.

Derived quantities, ages and units come from getvar, not from the table.

Examples

julia> info = getinfo(300, "path/to/simulation");

julia> part = getparticles(info);                       # every variable, the whole box

julia> part = getparticles(info, [:mass, :birth]);      # two variables

julia> propertynames(part.data.columns)
(:level, :x, :y, :z, :id, :family, :tag, :mass, :birth)

julia> dm = getparticles(info; stars=false, xrange=[-10, 10], yrange=[-10, 10],
                         zrange=[-2, 2], center=[:bc], range_unit=:kpc);   # no stars, a slab

julia> getvar(part, :age, :Myr)                         # stellar ages

See also getinfo, getvar, subregion, projection, gethydro.

getgravity(info::InfoType; vars=[:all], kwargs...) -> GravDataType
getgravity(info::InfoType, var::Symbol; kwargs...) -> GravDataType
getgravity(info::InfoType, vars::Vector{Symbol}; kwargs...) -> GravDataType

Load the gravity cells of a RAMSES output (potential and acceleration) into a table in memory.

Each row is one leaf cell: its refinement level, its integer position on that level, and one column per gravity variable, as stored by RAMSES. Cells refined beyond lmax are returned at level lmax. The output files are read in parallel, one group of files per thread.

Arguments

• info::InfoType: the output to read, from getinfo.
• var, vars: the variables to load, in any order. :all (the default) loads :epot, :ax, :ay and :az; names without a descriptor are positional (:grav5, ...). :cpu adds the number of the file each cell came from.

Keywords

Selection:

• lmax::Real = info.levelmax: the deepest level to read.
• xrange, yrange, zrange: [min, max] of the region to read, in range_unit and relative to center. missing (the default) means the full extent. A cell is kept when its centre lies inside.
• center = [0, 0, 0]: the origin of the ranges, in range_unit. [:bc] is the box centre; single components can be :bc, e.g. [:bc, :bc, 12.0].
• range_unit::Symbol = :standard: the unit of the ranges and the centre. :standard means fractions of the box (0 to 1); any length unit in info.scale works, e.g. :kpc or :pc.

Restarts:

• parents::Bool = false: also keep the values RAMSES stores for refined cells. Needed only to write a restart with writeramses that continues to round-off; it requires the whole box, all levels and all variables.

Output and performance:

• max_threads::Int = Threads.nthreads(): the most threads to use for reading.
• verbose::Bool = true: print a summary of what is read.
• show_progress::Bool = true: show a progress bar.
• print_filenames::Bool = false: print each file name as it is read.
• myargs::ArgumentsType: set lmax, the ranges, center, range_unit, verbose and show_progress in one object. A keyword given directly takes precedence.

Returns

A GravDataType with

• data: the table, with the columns :level, :cx, :cy, :cz and then the variables;
• info, scale: the simulation information and its unit scales;
• lmin, lmax, boxlen, ranges: the levels and the region that were read;
• selected_gravvars, used_descriptors: the options used.

Read the values through getvar, not from the table: RAMSES stores the potential short by a factor boxlen, and getvar(grav, :epot) corrects it.

Examples

julia> info = getinfo(300, "path/to/simulation");

julia> grav = getgravity(info);                     # every variable, the whole box

julia> grav = getgravity(info, :epot; lmax=8);      # the potential, down to level 8

julia> propertynames(grav.data.columns)
(:level, :cx, :cy, :cz, :epot)

julia> grav = getgravity(info; xrange=[-5, 5], yrange=[-5, 5], zrange=[-2, 2],
                         center=[:bc], range_unit=:kpc);   # a 10 x 10 x 4 kpc slab

julia> getvar(grav, :epot)                          # the corrected potential

See also getinfo, getvar, gethydro, subregion.

getclumps(info::InfoType; vars=[:all], kwargs...) -> ClumpDataType
getclumps(info::InfoType, vars::Vector{Symbol}; kwargs...) -> ClumpDataType

Load the clump catalogue that the RAMSES clump finder (PHEW) wrote with an output.

Each row is one clump, with one column per entry of the clump files, parsed as Float64. For clumps found by Mera itself on any data, use clumpfind instead.

Arguments

• info::InfoType: the output to read, from getinfo.
• vars: the columns to read. [:all] (the default) takes the names from the file header, e.g. :index, :lev, :parent, :ncell, :peak_x, :peak_y, :peak_z, :rho_av, :mass_cl. A custom list must follow the order of the file columns; it may be shorter, to read the first columns only, or longer, when the files hold more columns than the header names. Names with a dash are written Symbol("rho-").

Keywords

Selection:

• xrange, yrange, zrange: [min, max] of the region, in range_unit and relative to center. A clump is kept when its peak (:peak_x, :peak_y, :peak_z) lies inside, so these columns must be read; the extent of the clump is not tested.
• center = [0, 0, 0]: the origin of the ranges, in range_unit. [:bc] is the box centre; single components can be :bc, e.g. [:bc, :bc, 12.0].
• range_unit::Symbol = :standard: the unit of the ranges and the centre. :standard means fractions of the box (0 to 1); any length unit in info.scale works, e.g. :kpc or :pc.

Output:

• verbose::Bool = true: print a summary of what is read.
• print_filenames::Bool = false: print each file name as it is read.
• myargs::ArgumentsType: set the ranges, center, range_unit and verbose in one object. A keyword given directly takes precedence.

Returns

A ClumpDataType with

• data: the table, one row per clump, with the columns read;
• info, scale: the simulation information and its unit scales;
• boxlen, ranges: the region that was read;
• selected_clumpvars, used_descriptors: the options used.

Examples

julia> info = getinfo(300, "path/to/simulation");

julia> clumps = getclumps(info);                    # all columns, the whole box

julia> propertynames(clumps.data.columns)
(:index, :halo, :lev, :parent, :ncell, :peak_x, :peak_y, :peak_z, Symbol("rho-"),
 Symbol("rho+"), :rho_av, :mass_cl, :relevance)

julia> clumps = getclumps(info, [:index, :lev, :parent, :ncell, :peak_x, :peak_y, :peak_z]);

julia> clumps = getclumps(info; xrange=[-10, 10], yrange=[-10, 10], zrange=[-2, 2],
                          center=[:bc], range_unit=:kpc);   # clumps peaking in a slab

julia> getvar(clumps, :mass_cl, :Msol)              # clump masses in solar masses

See also getinfo, getvar, clumpfind.

Read the RAMSES sink-particle catalogue into a SinkDataType

getsinks(dataobject::InfoType; vars::Array{Symbol,1}=[:all],
         xrange::Array{<:Any,1}=[missing, missing],
         yrange::Array{<:Any,1}=[missing, missing],
         zrange::Array{<:Any,1}=[missing, missing],
         center::CenterType=[0., 0., 0.],
         range_unit::Symbol=:standard,
         verbose::Bool=true) -> SinkDataType

Sink particles are RAMSES's accreting point masses (black holes, protostars, star-cluster particles). Unlike the AMR data they are written as ONE small CSV per output, sink_NNNNN.csv, not one file per CPU — the catalogue is global and short.

The file carries two header lines: the column names, and the dimensional formula of each column in terms of mass, length and time (m, l, l t**-1, m l**2 t**-1, …). Those formulas are preserved in used_descriptors[:units], so what each column means is not lost.

Returns

A SinkDataType whose data is an IndexedTable of the catalogue, one row per sink. A run with no sinks yet returns an empty table rather than failing — sinks are created during a run, so early outputs legitimately have none.

Arguments

• dataobject: the InfoType from getinfo
• vars: columns to keep; [:all] (default) keeps every column in the file
• xrange/yrange/zrange, center, range_unit: spatial selection on the sink positions (:x, :y, :z), in the same style as getparticles
• verbose: print a short summary

Notes

• Column names are taken from the file, so they follow RAMSES's spelling. Some are not valid Julia identifiers — cs**2 must be written Symbol("cs**2").
• All columns are parsed as Float64, including id and level; cast afterwards if needed.
• Positions are in code units, as written by RAMSES. Use getvar for unit conversion.

Examples

info  = getinfo(2, "path/to/output")
sinks = getsinks(info)
getvar(sinks, :msink, :Msol)          # sink masses in solar masses
sinks.used_descriptors[:units][:msink]  # "m" — the dimensional formula RAMSES recorded

Read RAMSES radiative-transfer (RT) leaf-cells into an RtDataType.

RT data have the same AMR cell structure as hydro. Each photon group g contributes a photon number density Npgand flux componentsFxg/Fyg/Fzg (so nvarrt = 4 * nGroups). Variable names come from info.rt_variable_list.

getrt(dataobject::InfoType;
      lmax::Real=dataobject.levelmax,
      vars::Array{Symbol,1}=[:all],
      xrange=[missing,missing], yrange=[missing,missing], zrange=[missing,missing],
      center=[0.,0.,0.], range_unit::Symbol=:standard,
      print_filenames::Bool=false, verbose::Bool=true, show_progress::Bool=true,
      myargs::ArgumentsType=ArgumentsType(), max_threads::Int=Threads.nthreads())

Returns an RtDataType with data (IndexedTable: position columns :cx,:cy,:cz, optionally :level/:cpu, then the selected RT variables), and info, lmin, lmax, boxlen, ranges, selected_rtvars, used_descriptors, scale.

info = getinfo(2, "…/rt_stromgren")
rt   = getrt(info)                       # all RT variables
rt   = getrt(info, vars=[:Np1, :Fx1])    # selected

download_testdata([names]; dir, force=false, verbose=true)

Download Mera's public RAMSES test simulations. They are small (a few MB each, except Bondi) and every one has a known answer, so they can be used to check that an installation behaves correctly without a large download.

Called with a single name, returns the path to that simulation, which can be handed straight to getinfo:

path = download_testdata("sedov3d_amr")
info = getinfo(path, output=7)     # this run has 7 snapshots; 1 is before the blast
gas  = gethydro(info)

Called with several names or none at all, downloads them and returns the directory holding them. Anything already present is skipped unless force=true.

By default they land in joinpath(DEPOT_PATH[1], "mera_testdata"), alongside Julia's own package depot, so they survive between sessions. Pass dir to put them somewhere else.

The layout matches what Mera's test suite looks for, so the returned directory can be handed straight to it. Set MERA_SMOKE_ONLY too: without it the suite also looks for the maintainer's unpublished runs and fails on missing data.

ENV["MERA_TEST_DATA"] = download_testdata()   # or a subset by name
ENV["MERA_SMOKE_ONLY"] = "1"                  # every test the public data can back
using Pkg; Pkg.test("Mera")

Available simulations, with download sizes:

The Test Simulations page shows each of them with quicklook.

See also: getinfo, synthetic_clumps for a data-free alternative.

covering_grid(obj, var, [unit]; lmax=obj.lmax, center=[0.,0.,0.],
              xrange=[missing,missing], yrange=[missing,missing], zrange=[missing,missing],
              range_unit=:standard, max_bytes=4e9, pos_unit=:standard, verbose=true) -> CoveringGridResult

Resample AMR cell data (HydroDataType, GravDataType, or RtDataType) — or moving-mesh gas carried as particles (PartDataType with a :volume column, i.e. AREPO/GADGET PartType0) — onto a uniform Nx×Ny×Nz grid at refinement level lmax over the (optional) sub-box, every output cell sampled (not integrated). var may be a Symbol or a vector; unit likewise (defaults to code units).

For AMR cells, coarse leaves are replicated and fine leaves volume-averaged. For moving-mesh gas each output cell centre takes the value of the nearest generator that reaches it (KD-tree, capped at (√3/2)·V^(1/3) — the same ownership rule projection's weighting=:voronoi uses, so the two agree about which cell owns a point). Output cells that no cell reaches are NaN in both cases.

Point particles without a :volume column (stars, dark matter) have no extent to resample and raise an ArgumentError; use projection for those. Clumps raise a MethodError.

A uniform grid is dense and can be far larger than the AMR data — call covering_grid_memory first; this errors rather than allocate past max_bytes.

Pass periodic=true to let the window continue around a box face instead of being cut at it, which is what you want for a structure sitting on a boundary. A run that wraps in some directions only takes (x=true, y=true, z=false); getinfo reports which (info.boundaries). Without it the requested range is clamped to the box, so the grid comes back narrower than asked for.

gas = gethydro(getinfo(output, path))
covering_grid_memory(gas, [:rho, :T]; lmax=8)          # check size first
cg  = covering_grid(gas, [:rho, :T], [:nH, :K]; lmax=8) # then build
cg[:rho]                                                # the 3-D array

covering_grid_memory(obj, [vars]; lmax=obj.lmax, center=[0.,0.,0.], xrange=[missing,missing],
                     yrange=[missing,missing], zrange=[missing,missing], range_unit=:standard,
                     verbose=true) -> NamedTuple

Predict the size of the covering_grid before allocating it — a uniform grid is dense and can dwarf the sparse AMR data, so size it first. Returns (; level, dims, ncells, nvars, bytes_per_array, result_bytes, peak_bytes, amr_ncells, blowup): result_bytes is the returned arrays, peak_bytes the transient high-water mark during construction ((nvars+1) arrays — one shared geometric weight), and blowup = output cells ÷ AMR cells. vars only sets nvars (default 1). Pass an InfoType to size without loading data (then amr_ncells/blowup are missing).