MERA is a package for working with large 3D AMR/uniform-grid and N-body particle data sets from astrophysical simulations. It is entirely written in the language Julia and currently supports the hydrodynamic code RAMSES. With this package, I intend to provide essential functions to load and prepare the simulation data for calculations but try to avoid too high-level abstraction (black boxes).
- Easy to install and update
- Fast and memory lightweight data reading/saving and handling
- The data is loaded and processed in a database framework JuliaDB.jl
- Efficient workflow
- Many functionalities for advanced analysis
- Easy to extend
- Interactive and script functionality
- Many examples and tutorials
This first public release includes not all available functions yet. Stable versions of the following functions will be published stepwise:
- Select particle id/family etc. in projection function
- Particle age calculation for cosmological runs
- Mera-files is a significant faster way to read/store the RAMSES data for time sequence analysis
- Create Profiles of quantities and projected data
- Reader for sinks, gravity, rt, ..
- Create 1D, 2D histograms (phase plots)
- More predefined quantities
- Export data into binary files to use with Paraview (volume rendering)
- Tutorials to create 360° equirectangular projections
The package is tested against Julia 1.3 and can be installed with the Julia package manager. From the Julia REPL, type ] to enter the Pkg REPL mode and run:
pkg> add Mera
Or, equivalently, via the Pkg API in the Jupyter notebook use
using Pkg Pkg.add("Mera")
Optionally, precompile the downloaded package and all its dependencies:
In the Jupyter notebook
using Pkg Pkg.precompile()
Download the Linux binary from Julialang.org and untar it in your favored folder on your server. Define an alias in the .bashrc file that is pointing to julia:
Watch on GitHub. Note: Before updating, always read the release notes. In Pkg REPL mode run:
pkg> update Mera
using Pkg Pkg.update("Mera")
Reproducibility is an essential requirement of the scientific process. Therefore, I recommend working with environments. Create independent projects that contain their list of used package dependencies and their versions. The possibility of creating projects ensures reproducibility of your programs on your or other platforms if, e.g. the code is shared (toml-files are added to the project folder). For more information see Julia environments. In order to create a new project "activate" your working directory:
shell> cd MyProject /Users/you/MyProject (v1.3) pkg> activate .
Now add packages like Mera and PyPlot in the favored version:
(MyProject) pkg> add Package
The exported functions and types in MERA are listed in the API documentation, but can also be accessed in the REPL or Jupyter notebook.
In the REPL use e.g. for the function getinfo:
julia> ? # upon typing ?, the prompt changes (in place) to: help?> help?> getinfo search: getinfo SegmentationFault getindex getpositions MissingException Get the simulation overview from RAMSES info, descriptor and output header files ---------------------------------------------------------------------------------- getinfo(; output::Real=1, path::String="", namelist::String="", verbose::Bool=verbose_mode) return InfoType Keyword Arguments ------------------- • output: timestep number (default=1) • path: the path to the output folder relative to the current folder or absolute path • namelist: give the path to a namelist file (by default the namelist.txt-file in the output-folder is read) • verbose:: informations are printed on the screen by default: gloval variable verbose_mode=true Examples ---------- ...........
In the Jupyter notebook use e.g.:
?getinfo search: getinfo SegmentationFault getindex getpositions MissingException Get the simulation overview from RAMSES info, descriptor and output header files ---------------------------------------------------------------------------------- getinfo(; output::Real=1, path::String="", namelist::String="", verbose::Bool=verbose_mode) return InfoType Keyword Arguments ------------------- • output: timestep number (default=1) • path: the path to the output folder relative to the current folder or absolute path • namelist: give the path to a namelist file (by default the namelist.txt-file in the output-folder is read) • verbose:: informations are printed on the screen by default: gloval variable verbose_mode=true Examples ---------- ...........
Get a list of the defined methods of a function:
julia> methods(viewfields) # 10 methods for generic function "viewfields":  viewfields(object::PhysicalUnitsType) in Mera at /Users/mabe/Documents/Projects/dev/Mera/src/functions/viewfields.jl:181  viewfields(object::Mera.FilesContentType) in Mera at /Users/mabe/Documents/Projects/dev/Mera/src/functions/viewfields.jl:166  viewfields(object::DescriptorType) in Mera at /Users/mabe/Documents/Projects/dev/Mera/src/functions/viewfields.jl:150  viewfields(object::FileNamesType) in Mera at /Users/mabe/Documents/Projects/dev/Mera/src/functions/viewfields.jl:134  viewfields(object::CompilationInfoType) in Mera at /Users/mabe/Documents/Projects/dev/Mera/src/functions/viewfields.jl:116  viewfields(object::GridInfoType) in Mera at /Users/mabe/Documents/Projects/dev/Mera/src/functions/viewfields.jl:90  viewfields(object::PartInfoType) in Mera at /Users/mabe/Documents/Projects/dev/Mera/src/functions/viewfields.jl:73  viewfields(object::ScalesType) in Mera at /Users/mabe/Documents/Projects/dev/Mera/src/functions/viewfields.jl:57  viewfields(object::InfoType) in Mera at /Users/mabe/Documents/Projects/dev/Mera/src/functions/viewfields.jl:12  viewfields(object::DataSetType) in Mera at /Users/mabe/Documents/Projects/dev/Mera/src/functions/viewfields.jl:197
- To use the Jupyter interactive environment, please install IJulia (see IJulia)
- The tutorials in the documentation can be downloaded from GitHub as Jupyter notebooks
- Mera is tested against the RAMSES versions: =< stable-17.09, stable-18-09, stable-19-10
- The variables from the descriptor-files are currently only read and can be used in a future Mera version
- For simulations with a uniform grid is the column :level not created to reduce memory usage
In scientific computing, we are dealing with a steadily increasing amount of data. Highest performance is required, and therefore, most science-related libraries are written in low-level languages like C or Fortran with relatively long development times. The reduced data is often processed in a high-level language like Python. Julia is a relatively new and modern language, and it combines high-level programming with high-performance numerical computing. The syntax is simple and great for math. The just-in-time compilation allows for interactive coding and to achieve an optimized machine code on the fly. Both enhance prototyping and code readability. Therefore, complex projects can be realized in relatively short development times. Further features:
- Package manager
- Runs on multiple platform
- Multiple dispatch
- Build-in parallelism
- Directly call C, Fortran, Python (e.g. Matplotlib), R libraries, ...
- Official Julia website
- Learning Julia
- Julia Cheatsheet
- Free book ThinkJulia
- Synthax comparison: MATLAB–Python–Julia
- Julia forum JuliaDiscourse
- Courses on YouTube
- Database framework used in Mera: JuliaDB.jl
- Interesting Packages: JuliaAstro.jl, JuliaObserver.com
- Use Matplotlib in Julia: PyPlot.jl
- Call Python packages/functions from Julia: PyCall.jl
- Visual Studio Code based Julia IDE julia-vscode
- Atom based Julia IDE Juno
- If you have any questions about the package, please feel free to chat in the Gitter channel:
- For bug reports, etc., please submit an issue on GitHub
New ideas, feature requests are very welcome! MERA can be easily extended for other grid-based or N-body based data. Write an email to: mera[>]manuelbehrendt.com
To credit the Mera software, please star the repository on GitHub. If you use the Mera software as part of your research, teaching, or other activities, I would be grateful if you could cite my work. To give proper academic credit, follow the link for BibTeX export:
Copyright (c) 2019 Manuel Behrendt
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.