Datasets

Datasets define the geometry and gas properties of a simulation. Select one via dataset_type and add a matching top-level section in the YAML.

Overview

See also: Configuration Reference for global options.

Spherical Shell (shellmodel)

Spherically symmetric shell centered in the unit cube, with inner/outer radius in box units.

dataset_type: 'shellmodel'

shellmodel:
  boxsize: 3.086e21       # cm (physical box size)
  inner_radius: 0.0       # 0.0–0.5, in box units
  outer_radius: 0.1       # 0.0–0.5, in box units
  temperature: 20000.0    # K
  density: 0.078098       # cm^-3 (optional, see below)
  outflow_velocity: 0.0   # cm/s, value at r = outer_radius

Key points:

• inner_radius, outer_radius are fractions of the box size, limited to [0, 0.5] and must satisfy inner_radius < outer_radius.

• Gas density can be set in three equivalent ways (pick exactly one):

  • density (volume density in cm⁻³),
  • column_density (cm⁻²) together with boxsize,
  • log_NHI (log10 of column density in cm⁻²).

• Temperature can be given directly via temperature or derived from an effective cross‑section using:

  • sigma (line‑center cross‑section; requires the emission line from the driver).

• Dust‑related parameters exist but the current shell implementation initializes dust absorption to zero.

Power‑Law Shell (powerlaw_shellmodel)

Shell with power‑law profiles for density and velocity, building on the same basic parameters as shellmodel.

dataset_type: 'powerlaw_shellmodel'

powerlaw_shellmodel:
  boxsize: 3.086e21
  inner_radius: 0.01
  outer_radius: 0.1
  temperature: 20000.0
  density: 0.078098              # interpreted as mean density

  # Velocity profile v(r) ∝ r^powerlaw_index_velocity
  outflow_velocity: 200e5        # cm/s at r = outer_radius (simple case)
  powerlaw_index_velocity: 1.0

  # Density profile ρ(r) ∝ r^powerlaw_index_density
  powerlaw_index_density: -2.0
  conserve_column_density: true  # renormalize density to match shell column

Key points:

• All shellmodel density/temperature options are available and interpreted as averages before applying the power law.
• powerlaw_index_velocity and powerlaw_index_density control the radial scalings.
• If conserve_column_density: true, the code rescales density so that the line‑of‑sight column matches the corresponding uniform shell.
• Advanced velocity options:
• velocity_ratio: ratio between maximum and minimum velocity; used to infer an effective inner radius (see code comments).
• outflow_velocity_*average: alternative ways to specify the velocity scale; only one of these (or outflow_velocity from the shell) should be used.

Uniform Grid (unigrid)

Cartesian grid with optional domain decomposition and several initialization “loaders”.

dataset_type: 'unigrid'

unigrid:
  ngrid: 64               # cells per dimension (global)
  loader: sphere          # 'sphere', 'cylinder', or 'gadget'
  boxsize: 3.086e21       # cm, required if not implied by the loader

  # Optional: domain decomposition (see configuration.md)
  # domain: [[0.0, 1.0], [0.0, 1.0], [0.0, 1.0]]
  # local_domain_pad: 0.03

Loader: sphere

unigrid:
  ngrid: 64
  loader: sphere
  sphere:
    radius: 0.1           # in box units by default
    density: 0.078098     # cm^-3
    temperature: 20000.0  # K
    velocity: [1.0e5, 0.0, 0.0]  # optional: uniform velocity [vx, vy, vz] in cm/s
    relative_distances: true  # if false, radius is interpreted in physical units

Loader: cylinder (unused/test)

Primarily used for internal/regression tests; not used in shipped example configurations.

unigrid:
  ngrid: 64
  loader: cylinder
  cylinder:
    radius: 0.1           # in box units
    height: 0.5           # extent along x in box units
    density: 0.078098
    temperature: 20000.0

Loader: gadget

Interpolates a Gadget snapshot onto the grid.

unigrid:
  ngrid: 64
  loader: gadget
  gadget:
    path: "./snapdir_099"        # Gadget snapshot root
    interpolator: "ngp"          # 'ngp' (default) or 'sph' (work‑in‑progress)
    boxsize: 1.0e23              # cm, optional override of Gadget header
    pregridded_path: ""          # optional: read pre‑gridded HDF5 instead
    domain_consistency_check: true

Additional options:

• fields: list of field names to load (case‑insensitive, defaults to density/temperature/velocity for MCRT).
• When pregridded_path is non‑empty, the code loads data from that HDF5 file instead of performing interpolation.
• Distributed runs use domain and local_domain_pad from the dataset section, and pbc from the driver, via the common dataset interface.

Infinite Slab (infiniteslab)

One‑dimensional slab along the z‑axis, infinite in x and y (implemented as a single cell in z with special boundary handling).

dataset_type: 'infiniteslab'

infiniteslab:
  boxsize: 3.086e21      # cm
  temperature: 20000.0   # K

  # choose exactly one of:
  density: 0.078098      # cm^-3
  # tau: 1.0             # optical depth (requires emission line)

  # dust (optional)
  tau_dust: 0.0          # effective dust optical depth across half the slab
  density_fluff: 0.0     # density outside the slab (currently unused in main loop)

Key points:

• Exactly one of density or tau must be specified (the helper enforces a single choice among density, column_density, tau; for slabs column_density is currently not supported and will error if > 0).
• If tau is used, the code derives density from the line cross‑section and boxsize (requires a valid emission line in the driver).
• tau_dust sets a constant dust absorption coefficient inside the slab.

Point Cloud (pointcloud_voronoi and variants)

Voronoi tessellation of particles, typically from a Gadget simulation.

dataset_type: 'pointcloud_voronoi'

pointcloud_voronoi:
  hilbert_bits: 8            # 1–10, controls spatial indexing resolution
  loader: "gadget"

  gadget:
    path: "./snapdir_099"                # Gadget snapshot root
    catalog_path: "./groups_099"         # FOF/subfind catalog (required for zoom_target)
    fields: ["PositionX", "PositionY", "PositionZ", "Temperature", "Density"]
    zoom_target:
      hid: 0                             # Halo ID (0 = most massive)
      rvir: 2.5                          # Cutout radius in multiples of virial radius
    # zoom_box: [[xmin, xmax], [ymin, ymax], [zmin, zmax]]  # alternative to zoom_target

  # Optional: Cloudy ionization/emission tables
  cloudy:
    enabled: true
    emission_table_path: "/path/to/grid_emissivities.hdf5"
    ion_table_path: "/path/to/grid_ions.hdf5"
    redshift_interp: true
    interpolation_order: 1
    redshift_tolerance: 0.05   # max |z_table - z_dataset| when snapping to a slice
    interpolate_redshift: false  # blend the two bracketing slices instead of snapping

Key points:

• Supported variants (same YAML structure, different interpolator under the hood):

  • pointcloud_voronoi (default, experimental)
  • pointcloud_voronoi_tree (tree‑accelerated, do not use)
  • pointcloud_voronoi_approx (approximate Voronoi, do not use)
  • pointcloud_sph (SPH‑style interpolation; used by driver_type: sphprojector for stellar-particle projection across all eight AGORA codes — see Stellar Particle Loading)

• hilbert_bits controls the resolution of the Hilbert grid used to accelerate neighbor lookups (must be between 1 and 10).

• hilbert_sort (default true) shuffles all particles into Hilbert space-filling-curve order at load time; the KNN lookup grid requires it. As a consequence, THOR's internal particle order (and any mesh cache written from it) is not snapshot order. Mesh caches record a per-row identity column to map back.
• Loaders: gadget (HDF5 / Gadget-family snapshots, including AREPO and TNG), tipsy (CHANGA/PKDGRAV binary format), and sphere (synthetic uniform-sphere generator). Selected via pointcloud_voronoi.loader.
• Snapshots whose HDF5 header lacks unit or cosmology attributes (e.g. GADGET-4, which writes them only to param.txt) can supply UnitLength_in_cm, UnitMass_in_g, UnitVelocity_in_cm_per_s, Omega0, OmegaLambda, and HubbleParam directly in the gadget: subsection — see Unit Normalization.
• For pointcloud_sph datasets, three additional loaders source stellar particles from AMR snapshots: ramses_stars, enzo_stars, and art_stars. See Stellar Particle Loading below.
• Voronoi construction-specific options are configured under a construction: subsection and documented in detail here. CGAL remains the production default; construction.voronoi_backend can opt in to the host-only Voro++ backend or the experimental SYCL-native (Ray et al. 2018) backend.

Gadget Loader Options

Zoom Configuration

The Gadget loader supports two mutually exclusive ways to define a spatial cutout. You must use exactly one — specifying both will produce an error.

Option A: zoom_target — halo-centric cutout (requires catalog_path):

Internally, zoom_target is converted to a zoom_box centered on the halo position with half-width rvir * R_vir.

Option B: zoom_box — explicit bounding box in normalized coordinates [0, 1]:

zoom_box: [[0.45, 0.55], [0.45, 0.55], [0.45, 0.55]]

zoom_box can be placed either under the gadget: subsection or at the dataset level.

Frame. zoom_box is in full-box normalized coordinates ([0, 1]³), matching the simulation's native frame (comoving for cosmological runs). Particles that survive the filter are remapped to [0, 1]³ inside the zoomed dataset, so downstream code sees a unit cube even when only a tiny sub-region was loaded.

Derived Field Cache (field_cache)

Point-cloud datasets can persist expensive derived particle fields for reuse. Configure field_cache at the dataset level, as a sibling of the loader subsection:

pointcloud_sph:
  field_cache:
    mode: auto
    path: /scratch/.../frame_0307.fields.hdf5
    request_fingerprint: <source-sha256-plus-recipe-config>
    fields: [MaxDist]
  gadget:
    path: ...
    particle_type: PartType4
    smoothing_length: {mode: knn, knn_k: 32}

The modes have deliberately distinct failure and overwrite behavior:

• disabled bypasses the cache and performs no cache I/O.
• load requires an existing compatible cache. A missing file, fingerprint mismatch, particle mismatch, or missing requested field is an error.
• save computes the requested fields through the normal loader and field recipes, then atomically replaces path; it does not reuse an existing cache.
• auto loads a compatible cache when present. A missing file or metadata mismatch causes it to compute the fields and attempt an atomic replacement. A malformed cache or an inexact particle-ID join is an error. If only the replacement write fails, the run continues with the computed fields.

Cache rows retain Gadget ParticleIDs as exact HDF5 uint64 values. IDs are handled as integers rather than passing through a floating-point field type, so large 64-bit IDs do not lose precision. On load, fields are joined by ID and must cover every selected particle exactly; missing or duplicate IDs abort the run. Cached fields are scalar values in THOR's canonical, post-conversion representation. A loaded value is injected as already canonical and is not unit-converted a second time.

Caching MaxDist is useful for stellar SPH projections: the first run performs the configured smoothing-length derivation (for example, the kNN calculation), while later load runs or auto cache hits reuse it instead of deriving the smoothing lengths again.

request_fingerprint is opaque to THOR and is compared exactly; THOR does not hash the source snapshot or recipe configuration for you. The config author is responsible for constructing it from the source identity and every setting that can affect particle selection or field values, including recipe and smoothing-length configuration. Change the fingerprint whenever any of those inputs change; keeping the same fingerprint incorrectly declares the old cache valid.

The v1 field-cache path supports single-rank Gadget point-cloud loads only. Writers publish through an atomic rename, so another run observes either the previous complete cache or the new complete cache, not a partially written file.

This cache is a THOR-managed, regenerable optimization. In contrast, gadget.aux_fields imports externally produced, authoritative data and joins it by ParticleIDs; THOR neither derives nor writes those auxiliary values.

Auxiliary Fields (aux_fields)

Load externally-computed per-particle fields from a companion HDF5, joined to the loaded particles by ParticleIDs — the snapshot itself is untouched. The typical use is projecting a derived quantity through the SPHProjector, e.g. the per-star attenuated luminosity from a source_points obscuration run.

pointcloud_sph:
  gadget:
    path: snap.hdf5
    particle_type: PartType4
    zoom_box: [[...], [...], [...]]
    aux_fields:
      path: ./stars_atten.hdf5   # companion file; snapshot untouched
      min_coverage: 0.999        # error if fewer loaded particles match a sidecar ID
      # missing_value: 0.0       # value for loaded particles absent from the sidecar
      # validate_zoom_box: true  # error if /Meta.zoom_box != this run's zoom_box
      # group: PartType4         # sidecar HDF5 group (defaults to particle_type)

Sidecar layout (a single file, keyed by ParticleIDs):

stars_atten.hdf5
  /Meta                            attrs: zoom_box (3×2), snapshot, thor_ref  (optional)
  /PartType4/ParticleIDs           uint64[N]        # the join key
  /PartType4/AttenuatedLuminosity  float64[N]       # projectable field(s)
  /PartType4/<other>               float64[N]

Every dataset except ParticleIDs is injected as a field of the same name; select it in a projector's fields: list. The join happens before THOR's internal Hilbert reordering, so the values stay aligned with their particles. If fewer than min_coverage of the loaded particles match a sidecar ID — or the /Meta.zoom_box stamp disagrees with the run — the reader aborts rather than project a stale sidecar. The matching key is the ParticleIDs that source_points writes (tracers/source_particle_ids).

Cloudy Tables

When cloudy.enabled: true, Thor loads pre-computed Cloudy ionization and emission grids to derive line emissivities and absorption coefficients from the gas properties.

Stellar Particle Loading

Stellar particles project through pointcloud_sph + driver_type: sphprojector for all eight AGORA codes. Two routes:

Gadget-family HDF5 snapshots (AREPO, GADGET-3, GIZMO, GEAR) — use the existing loader: gadget with one extra knob:

dataset_type: pointcloud_sph
pointcloud_sph:
  loader: gadget
  gadget:
    path: ./input/snap.hdf5
    particle_type: PartType4         # PartType4 = stars (AREPO/Gadget3/GIZMO);
                                     # PartType1 = stars in GEAR; default PartType0 = gas
    fields: [Masses, Metallicity, StellarAge]
    zoom_box: [[xmin,xmax], [ymin,ymax], [zmin,zmax]]
    smoothing_length:                # see "Smoothing Length" below
      mode: knn
      knn_k: 32
      clamp_percentile: 95.0

ChaNGa Tipsy uses loader: tipsy with particle_type: stars. AMR codes (RAMSES / ENZO / ART) use dedicated loaders that bypass the octree path entirely and feed the per-format star catalog reader straight into a pointcloud_sph dataset:

All three accept the same zoom_box and smoothing_length subsections as the gadget loader.

Projection normalization

driver_type: sphprojector turns the per-particle kernel splat into a pixel value according to operators.projections.normalization:

In mean mode the weight: field chooses the weighting: any field name (default Masses) gives a field-weighted average, while weight: none gives an unweighted average Σ qᵢ W / Σ W (each particle counted only by kernel coverage; the Masses field is not required). weight: is ignored by the two sum-modes.

sphprojector:
  operators:
    projections:
      fields: [Masses]
      normalization: mass_projection   # Σ★ surface density in g/cm²

The older use_weighting/perform_averaging booleans are still accepted (deprecated): use_weighting: true, perform_averaging: true → mean; use_weighting: false, perform_averaging: true → mean + weight: none (unweighted average); use_weighting: false, perform_averaging: false → mass_projection.

Metallicity convention. Each AMR star reader returns per-particle metallicity as a mass fraction Z = M_metals / M_total (dimensionless, typical range 0 – a few percent). RAMSES reads record 20 of part_*.out directly. ENZO reads the metallicity_fraction HDF5 dataset. ART sums the SNII and SNIa metal mass fractions (zstII + zstIa from records 6 and 7 of stars_*.dat). Downstream recipes that expect Z/Z☉ must divide by Z_sun ≈ 0.0134 themselves.

Smoothing Length

Stars don't carry an SPH MaxDist field natively, so the pointcloud_sph loaders compute one. Configure under <loader>.smoothing_length:

mode: knn is the default and is auto-invoked at the end of init_gadget for SPH datasets (or any dataset that supplies a smoothing_length: subsection). For VORONOI datasets compiled with use_hilbert_knn_for_voronoi=ON, only mode: constant works (HilbertGridKNN supports K=1 queries only).