MERA.jl Testing Framework

This page is the authoritative reference for the MERA.jl test suite: how to run it, how it is structured, which datasets it uses, and how coverage is measured and published.

Why local testing with heavy data

MERA.jl reads and analyses real RAMSES adaptive-mesh-refinement (AMR) simulation output. Meaningful tests therefore need production-scale data:

• Data scale: realistic RAMSES outputs contain millions of AMR cells across 10+ refinement levels.
• Physical validity: results must obey conservation/decomposition relations (mass, momentum, energy) and reproduce textbook formulas.
• Reader correctness: the binary RAMSES readers have format-specific branches (e.g. legacy vs. family/tag particle formats) that only exercise against actual simulation files.

These datasets are far too large to ship to GitHub Actions runners, so the suite is split into a CI-friendly smoke subset and a full local run.

Running the tests

The suite supports three modes, all driven by environment variables read in test/test_config.jl and test/runtests.jl:

# 1. Smoke run — data-independent only (this is what CI runs):
MERA_SMOKE_ONLY=1 julia --project -e 'using Pkg; Pkg.test("Mera")'

# 2. Full local run — requires RAMSES test data mounted:
julia --project -e 'using Pkg; Pkg.test("Mera")'

# 3. Full local run + coverage + Codecov upload (maintainer):
UPLOAD=1 ./scripts/run_local_coverage.sh

Two additional environment variables help during development:

# Example: run two files in isolation
MERA_FOCUS=07_regions.jl,21_untested_surfaces_tests.jl \
    julia --project -e 'using Pkg; Pkg.test("Mera")'

When the simulation directory is absent (or MERA_SMOKE_ONLY=1 is set), the data-dependent tiers are skipped cleanly, so Pkg.test("Mera") always succeeds, CI and contributors without data still get a valid reduced run.

Full local run: roughly 5 minutes on an Apple-silicon laptop.

GitHub Actions workflows

Four workflows live in .github/workflows/. Only the first runs tests; the others handle docs, dependency maintenance, and releases.

Julia version coverage

The CI.yml matrix pins 1.10 (the LTS, and the package's declared minimum in Project.toml), 1.11, and 1.12 (the current stable). setup-julia resolves each minor-version string to its newest patch release at run time, so the suite always runs against the latest 1.10.x, 1.11.x, and 1.12.x.

Pre-releases (e.g. 1.13, which at time of writing is only a release candidate) are intentionally not included: they are moving targets and upstream rc bugs would produce spurious failures. When 1.13.0 ships as stable: add a '1.13' row (and optionally drop an older series). To get early warning of breakage on an upcoming release: add a non-blocking 'pre' (or 'nightly') matrix entry; fail-fast: false is already set so such a row can fail without failing the whole build.

Test suite structure

test/runtests.jl executes 62 test files in tiered order. Tier 1 is data-independent and always runs; the data-backed tier runs only when simulation data is present and MERA_SMOKE_ONLY is not set.

A per-file map, which file proves what and which simulation backs it, is kept in test/README.md. It is maintained alongside the suite, so this page does not repeat it; a list copied here would silently fall behind as files are added.

Three files are infrastructure rather than tests:

Test datasets

All datasets live under SIMULATION_PATH (test_config.jl), overridable via MERA_TEST_DATA.

Any RAMSES output can be substituted by editing SIMULATION_PATH and the DATASETS dictionary in test_config.jl.

Coverage workflow

Because the data lives only on the maintainer's machine, coverage is measured locally and uploaded to Codecov, CI does not measure it.

scripts/run_local_coverage.sh:

1. Wipes stale *.cov and coverage.lcov files.
2. Runs Pkg.test("Mera"; coverage=true).
3. Aggregates src/**/*.cov into coverage.lcov via scripts/process_coverage.jl (which excludes src/dev/, src/benchmarks/, and src/visualization/, non-library code).
4. When UPLOAD=1 and a CODECOV_TOKEN is available, uploads to Codecov.

The token can be stored in ~/.config/mera/codecov.env (mode 600) instead of being exported manually.

Notebook / tutorial tests (real-world workflows)

The Jupyter tutorial notebooks in the documentation (Mera-Docs/version_1) double as an end-to-end, real-world workflow test tier. Where the unit suite exercises functions in isolation, the notebooks run the full analysis pipelines a user would actually follow: load info → read hydro/particles/gravity/clumps → select regions → derive variables → project → mask/filter → save/load Mera files → export VTK, against the real RAMSES datasets. Two distinct kinds of value:

• Executable regression tests. Every notebook is run headless with jupyter nbconvert --execute and scanned for error cells. A failing cell is a real, user-facing breakage. This catches integration-level bugs the unit suite can miss, e.g. the Julia 1.12 dataoverview crash (Core.TypeName.mt removed) was surfaced by the notebooks while the unit suite had it marked @test_broken.
• Visual verification for authors/reviewers. The executed notebooks preserve their outputs, tables, projection plots, VTK previews, so a maintainer can see that results look physically correct, not merely that no exception was thrown. This is the human-in-the-loop check that pure assertions cannot provide.

Running them

The notebooks use a dedicated Julia 1.12 kernel bound to the docs environment, which devs Mera from this repo (so they test the working tree, not a released version). Create the kernels once via IJulia.installkernel from the notebooks project:

using IJulia
NB = "/path/to/Mera-Docs/version_1"
# plain execution kernel (4 threads)
installkernel("Mera-Docs 1.12 (4t)", "--project=$NB",
              env=Dict("JULIA_NUM_THREADS" => "4"))
# coverage kernel — tracks only this repo's source
installkernel("Mera-Docs 1.12 cov", "--project=$NB",
              "--code-coverage=@/path/to/Mera.jl",
              env=Dict("JULIA_NUM_THREADS" => "4"))

Executed copies are written to version_1/executed/ (gitignored); the originals are left untouched.

Wired into coverage

scripts/run_coverage_with_notebooks.sh produces a combined coverage report from both the unit suite and the notebooks:

1. Wipes stale *.cov.
2. Runs Pkg.test("Mera"; coverage=true) → *.cov next to src/.
3. Executes every tutorial notebook with the coverage kernel (--code-coverage=@<repo>), which appends more *.cov to the same src/ files, so notebook execution counts toward library coverage.
4. Aggregates everything via scripts/process_coverage.jl and (with UPLOAD=1) uploads to Codecov under the local-full-notebooks flag.

Because both phases write *.cov next to the source, the tutorials raise coverage of paths the unit suite under-exercises (display/viewfields, dataoverview, projection and VTK variants). The coverage kernel's @<repo> path restriction keeps *.cov confined to Mera.jl/src, none leak into the notebooks repo (which also gitignores *.cov as a safety net).

What the tests validate

The suite is designed for meaningful coverage, not line-hit padding:

• Physics formulas are re-derived from primitive variables in CGS and compared to Mera's output (sound speed, temperature, Jeans length/mass, free-fall time).
• Region selections validate the extent of the returned cells, not just the count, every subregion/shellregion test checks that selected cells actually lie inside the requested geometry.
• I/O round-trips compare loaded data cell-by-cell against the original RAMSES read, including info metadata and scale factors.
• Error paths are exercised with @test_throws for invalid inputs.
• Partition invariants: subregion plus its inverse must reconstruct the parent dataset exactly.

The suite has been spot-checked with manual mutation testing: deliberately breaking a physics factor, a reader branch, or a region bound each causes the corresponding test to fail, confirming the assertions bite.

For JOSS reviewers

1. CI verification: MERA_SMOKE_ONLY=1 julia --project -e 'using Pkg; Pkg.test("Mera")' passes without simulation data. Aqua.jl checks code quality (no ambiguities, no unbound type parameters, no stale deps, no type piracy).
2. Full verification: with RAMSES data mounted, the full suite runs in ~5 minutes and exercises data I/O, derived quantities, projections, regions, conservation relations, parallel safety, clump analysis, VTK export, and save/load round-trips.
3. Coverage: measured locally and published to Codecov (badge above). Uncovered code is concentrated in rarely-used backends (multi-CPU parallel readers, sink-particle paths) and interactive display methods.
4. Reproducibility: point MERA_TEST_DATA at any RAMSES output and edit the DATASETS dictionary in test_config.jl.
5. Real-world workflows: the tutorial notebooks are executed end-to-end and their rendered outputs (tables, projection plots, VTK previews) can be inspected directly, demonstrating that complete user pipelines work and produce physically sensible results (see Notebook / tutorial tests).

Testing philosophy

MERA.jl's testing ensures both software reliability and scientific validity: traditional software-testing practices (unit, integration, error-path, quality-assurance) combined with the validation needs of astrophysical simulation analysis (conservation relations, AMR boundary handling, coordinate transforms, numerical precision). The two-mode design keeps CI fast and deterministic while the local run delivers full, physically meaningful validation.