Skip to content

Compiler Settings

THOR builds with a SYCL compiler that targets CPUs and GPUs across vendors. THOR supports two SYCL implementations: AdaptiveCpp (primary development target) and Intel oneAPI DPC++/C++ Compiler.

Choosing a SYCL Compiler

For installation see AdaptiveCpp or Intel oneAPI. Development is done against AdaptiveCpp; newer features may lag in oneAPI.

Select the compiler by setting CXX and CC before building.

For AdaptiveCpp:

export CXX=acpp
export CC=clang

For Intel oneAPI DPC++/C++ Compiler:

export CXX=icpx
export CC=icx
The build helper selects oneAPI explicitly and keeps its artifacts separate from AdaptiveCpp builds:

printf 'debugger=exclude\n' > /tmp/thor-oneapi.conf
source /opt/intel/oneapi/setvars.sh --config=/tmp/thor-oneapi.conf
./build.sh --intel --cmake-arg -DTHOR_OPTIMIZE_ARCHITECTURE=OFF
ONEAPI_DEVICE_SELECTOR=opencl:cpu ./cmake-build-release-intel/src/thor config.yaml

Use device: cpu in that YAML for the OpenCL CPU runtime. On supported GPUs, select the installed oneAPI GPU backend and use device: gpu. GPU availability and validation depend on the installed driver/runtime. PointCloud sorting requires oneDPL headers. Disabling architecture optimization avoids Intel's -xHost host-executable restriction on AMD CPUs. --intel cannot be combined with AdaptiveCpp's --omp or --generic options. The configuration excludes the debugger component because its bundled yaml-cpp can override THOR's library through LD_LIBRARY_PATH. See Intel's setvars configuration documentation.

Source-photon MCRT transport automatically uses one work-item per work-group when a oneAPI build submits to a CPU queue. Independent photon histories can diverge substantially, making the CPU runtime's default vectorized launch expensive. The policy is selected from the submitting queue's device; no YAML option or global vectorizer override is needed. Dataset preparation, peeling transport, GPU queues and AdaptiveCpp use their existing launch policies. Scalar and vectorized floating-point execution can produce different individual Monte Carlo histories, so comparisons use scientific tolerances rather than requiring bit-identical results across those launch policies.

Ahead-of-Time (AOT) vs. Just-in-Time (JIT) Compilation

SYCL kernels can be compiled at build time (AOT) or at runtime (JIT). THOR prefers JIT because:

  • THOR has many kernel variants — AOT compilation of all of them is slow.
  • The hardware target need not be known at build time, which suits mixed-vendor HPC environments.
  • Runtime constants can be folded during JIT, enabling further optimization.

JIT is the default for both AdaptiveCpp and Intel oneAPI. To force AOT under AdaptiveCpp, pass -DACPP_TARGETS="<targets>" to CMake (see the AdaptiveCpp compilation docs).

AdaptiveCpp target selection

The supported configuration is the generic SSCP target (AdaptiveCpp's default): one build serves CPU and GPU, with the backend chosen at runtime via the YAML device: key and ACPP_VISIBILITY_MASK. Two caveats:

  • Do not combine generic;omp unless you know what you are doing. There usually is no reason to combine those — generic alone already covers CPU and GPU. The combination has been observed to produce builds that compile cleanly but fail on GPU at kernel launch with Unresolved extern function ....
  • -DACPP_TARGETS="omp" alone is a valid CPU-only AOT build (used for regression testing), but it has no GPU support.

Avoid hardware-specific AOT targets such as cuda:sm_XX or cuda.explicit-multipass — generic is the convention.

Floating-point semantics

Use -O3 with -fno-fast-math. THOR needs NaN/Inf handling for validation, field masks, range reductions, and sentinels. FMA contraction is allowed.

CMake rejects fast/finite-only flags in CMAKE_CXX_FLAGS and its configuration and initialization variants, plus CMAKE_CXX_FLAGS_FORCE and THOR_CXX_FLAGS. It also checks CXXFLAGS when CMake uses it to initialize CMAKE_CXX_FLAGS. This covers aggressive, custom names such as CMAKE_CXX_FLAGS_FOO-BAR, and forwarded options such as -Xclang=-ffast-math. The guard cannot inspect custom wrappers, response files, target options, or external runtime JIT options; those must preserve NaN/Inf handling too.

Remove -ffast-math and -Ofast entirely. Some AdaptiveCpp versions enable JIT fast-math even when a later -fno-fast-math cancels it, so CMake rejects that combination too.

Run the checks with:

./build.sh --omp --target thor_floating_point_smoke
ACPP_VISIBILITY_MASK=omp ./cmake-build-release-omp/tests/thor_floating_point_smoke
./build.sh --generic --target thor_floating_point_smoke
ACPP_VISIBILITY_MASK=omp ./cmake-build-release-generic/tests/thor_floating_point_smoke
# On a CUDA host, also run the generic executable with ACPP_VISIBILITY_MASK=cuda.
python3 scripts/tests/test_floating_point_flags.py

The smoke test uses production flags and integer expectations to check runtime finite, infinite, and NaN values on the host and selected device. It tests float and double, skipping device double without fp64 support. Both checks are registered with CTest; the flag tests require Python. [AI-Codex]