Edit on GitHub

Home

DOI

MERA is a package for working with large 3D AMR/uniform-grid and N-body particle data sets from astrophysical simulations. It is entirely written in the language Julia and currently supports the hydrodynamic code RAMSES. With this package, I intend to provide essential functions to load and prepare the simulation data for calculations but try to avoid too high-level abstraction (black boxes).

Note

To get a first impression, look at the Hands-On Session RUM2023 with downloadable simulation examples: https://github.com/ManuelBehrendt/RUM2023

Package Features

  • Easy to install and update
  • Fast and memory lightweight data reading/saving and handling
  • The data is loaded and processed in a database framework JuliaDB.jl
  • Efficient workflow
  • Many functionalities for advanced analysis
  • Easy to extend
  • Interactive and script functionality
  • Many examples and tutorials
  • Mera-files, a significant faster way to read/store the RAMSES data for time sequence analysis

Dependencies

Find the main dependencies on JuliaHub or of the development version listed in the file Project.toml.

Tests

We are developing unit-test and end-to-end testing strategies to encounter bugs like general errors, incorrect data returns, and functionality issues. After new commits are pushed to GitHub, different operating system environments and Julia versions run automated tests, e. g. on outputs from various RAMSES simulations, to ensure important functionalities of MERA. The test folder contains all tests with the main function in the runtest.jl file. Find the current test status of the development version at: https://github.com/ManuelBehrendt/Mera.jl/blob/master/README.md

Julia Installation

  • Binary download + installation instructions: https://julialang.org/downloads/

  • Juliaup, an installer and version manager: https://github.com/JuliaLang/juliaup

  • Apple Silicon: M1/M2 Chips: Julia 1.6.x can be installed without any trouble. But to use PyPlot, it is recommended to install/pin the package PyCall@1.92.3 ! https://pkgdocs.julialang.org/v1.6/managing-packages/#Pinning-a-package . If you encounter any problems with Julia 1.9, try the binary macOS x86 (Intel or Rosetta) instead of macOS (Apple Silicon).

Package Installation

The package is tested against the long-term supported Julia 1.6.x (recommended), 1.7.x, 1.8.x, 1.9.x and can be installed with the Julia package manager: https://pkgdocs.julialang.org/v1/

Julia REPL

From the Julia REPL, type ] to enter the Pkg REPL mode and run:

pkg> add Mera

Jupyter Notebook

Or, equivalently, via the Pkg API in the Jupyter notebook use

using Pkg
Pkg.add("Mera")

Updates

Watch on GitHub. Note: Before updating, always read the release notes. In Pkg REPL mode run:

pkg> update Mera

Or, equivalently,

using Pkg
Pkg.update("Mera")

Reproducibility

Reproducibility is an essential requirement of the scientific process. Therefore, I recommend working with environments. Create independent projects that contain their list of used package dependencies and their versions. The possibility of creating projects ensures reproducibility of your programs on your or other platforms if, e.g. the code is shared (toml-files are added to the project folder). For more information see Julia environments. In order to create a new project "activate" your working directory:

shell> cd MyProject
/Users/you/MyProject

(v1.6) pkg> activate .

Now add packages like Mera and PyPlot in the favored version:

(MyProject) pkg> add Package

Help and Documentation

The exported functions and types in MERA are listed in the API documentation, but can also be accessed in the REPL or Jupyter notebook.

In the REPL use e.g. for the function getinfo:

julia> ? # upon typing ?, the prompt changes (in place) to: help?>

help?> getinfo
search: getinfo SegmentationFault getindex getpositions MissingException

  Get the simulation overview from RAMSES info, descriptor and output header files
  ----------------------------------------------------------------------------------

  getinfo(; output::Real=1, path::String="", namelist::String="", verbose::Bool=verbose_mode)
  return InfoType

  Keyword Arguments
  -------------------

    •    output: timestep number (default=1)

    •    path: the path to the output folder relative to the current folder or absolute path

    •    namelist: give the path to a namelist file (by default the namelist.txt-file in the output-folder is read)

    •    verbose:: informations are printed on the screen by default: gloval variable verbose_mode=true

  Examples
  ----------
...........

In the Jupyter notebook use e.g.:

?getinfo
search: getinfo SegmentationFault getindex getpositions MissingException

  Get the simulation overview from RAMSES info, descriptor and output header files
  ----------------------------------------------------------------------------------

  getinfo(; output::Real=1, path::String="", namelist::String="", verbose::Bool=verbose_mode)
  return InfoType

  Keyword Arguments
  -------------------

    •    output: timestep number (default=1)

    •    path: the path to the output folder relative to the current folder or absolute path

    •    namelist: give the path to a namelist file (by default the namelist.txt-file in the output-folder is read)

    •    verbose:: informations are printed on the screen by default: gloval variable verbose_mode=true

  Examples
  ----------
...........

Get a list of the defined methods of a function:

julia> methods(viewfields)
# 10 methods for generic function "viewfields":
[1] viewfields(object::PhysicalUnitsType) in Mera at /Users/mabe/Documents/Projects/dev/Mera/src/functions/viewfields.jl:181
[2] viewfields(object::Mera.FilesContentType) in Mera at /Users/mabe/Documents/Projects/dev/Mera/src/functions/viewfields.jl:166
[3] viewfields(object::DescriptorType) in Mera at /Users/mabe/Documents/Projects/dev/Mera/src/functions/viewfields.jl:150
[4] viewfields(object::FileNamesType) in Mera at /Users/mabe/Documents/Projects/dev/Mera/src/functions/viewfields.jl:134
[5] viewfields(object::CompilationInfoType) in Mera at /Users/mabe/Documents/Projects/dev/Mera/src/functions/viewfields.jl:116
[6] viewfields(object::GridInfoType) in Mera at /Users/mabe/Documents/Projects/dev/Mera/src/functions/viewfields.jl:90
[7] viewfields(object::PartInfoType) in Mera at /Users/mabe/Documents/Projects/dev/Mera/src/functions/viewfields.jl:73
[8] viewfields(object::ScalesType) in Mera at /Users/mabe/Documents/Projects/dev/Mera/src/functions/viewfields.jl:57
[9] viewfields(object::InfoType) in Mera at /Users/mabe/Documents/Projects/dev/Mera/src/functions/viewfields.jl:12
[10] viewfields(object::DataSetType) in Mera at /Users/mabe/Documents/Projects/dev/Mera/src/functions/viewfields.jl:197

Further Notes

  • To use the Jupyter interactive environment, please install IJulia (see IJulia) and/or the standalone "JupyterLab Desktop" app: https://github.com/jupyterlab/jupyterlab-desktop
  • The tutorials in the documentation can be downloaded from GitHub as Jupyter notebooks
  • To get a first impression, look at the Hands-On Session RUM2023` with downloadable simulation examples: https://github.com/ManuelBehrendt/RUM2023
  • Mera is tested against the RAMSES versions: =< stable-17.09, stable-18-09, stable-19-10
  • The variables from the descriptor-files are currently only read and can be used in a future Mera version
  • For simulations with a uniform grid, the column :level is not created to reduce memory usage

Why Julia?

In scientific computing, we are dealing with a steadily increasing amount of data. Highest performance is required, and therefore, most science-related libraries are written in low-level languages like C or Fortran with relatively long development times. The reduced data is often processed in a high-level language like Python. Julia is a relatively new and modern language, and it combines high-level programming with high-performance numerical computing. The syntax is simple and great for math. The just-in-time compilation allows for interactive coding and to achieve an optimized machine code on the fly. Both enhance prototyping and code readability. Therefore, complex projects can be realized in relatively short development times. ​ Further features:

  • Package manager
  • Runs on multiple platform
  • Multiple dispatch
  • Build-in parallelism
  • Metaprogramming
  • Directly call C, Fortran, Python (e.g. Matplotlib), R libraries, ...

….

Contact for Questions and Contributing

  • If you have any questions about the package, please feel free to write an email to: mera[>]manuelbehrendt.com
  • For bug reports, etc., please submit an issue on GitHub

New ideas, feature requests are very welcome! MERA can be easily extended for other grid-based or N-body based data. Write an email to: mera[>]manuelbehrendt.com

Supporting and Citing

To credit the Mera software, please star the repository on GitHub. If you use the Mera software as part of your research, teaching, or other activities, I would be grateful if you could cite my work. To give proper academic credit, follow the link for BibTeX export: DOI

License

MIT License

Copyright (c) 2019 Manuel Behrendt

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.