This document describes specific items for Q3 codebase architecture and conventions for AI coding agents.
NOTE: also include ./AGENTS-generic.md - it describes common conventions for all the projects. This document describes project specific conventions.
An add-on mirrors core's layout under its own addons/<name>/ path segment in src/, tests/ and examples/, and reaches core only through core's registries (input formats, pseudopotential formats, PAW one-center conventions, volumetric formats; see docs/architecture.md). Core never names an add-on: no core code, test or doc includes, imports or links into one, and no core directory may be named addons.
Each layer's identifiers (class, function, parameter, member names) should only use concepts from its own layer or below. Includes must never point upward: layer_N code must not #include a layer_(N+k) header — when a lower layer needs behavior chosen at a higher layer, define an interface at the lower layer and implement it above (dependency inversion; see FFTGridFactory in layer 3, implemented by PlanewaveExecutionPolicy in layer 4, and QuantumSystemExecutionPolicy in layer 2 giving band meaning to layer-0 process groups).
Concept ownership by layer:
| Layer | Owns (examples) | Must NOT appear in identifiers |
|---|---|---|
| 0 | types, math, process groups, devices, logging | bands, k-points, plane waves, spin |
| 1 | crystal, symmetry, k-points, reciprocal space | bands, wavefunctions, pseudopotential |
| 2 | bands, spin, occupations, wavefunctions (abstract) | plane waves, G-vectors, FFT grids |
| 3 | plane waves, G-vectors, FFT, pseudopotential tables | SCF iteration, diagonalizer choice |
| 4 | SCF, diagonalizers, mixing, smearing, pools | — |
| 5 | derived properties (forces, stress, DOS, projections) | — |
Accepted exceptions (the "not 100%" part): the centralized constants registry layer_0_core/math/tolerances.hpp deliberately names physics-specific thresholds in its dedicated section; comments may mention higher-layer use cases for context (identifiers may not); io/ is a vertical slice and may reference any layer.
The Python package follows the same downward-only rule as the C++ layers, for the same reason: an upward import becomes an import cycle, and a cycle gets "fixed" by hiding the import inside a function, which buries the design defect instead of resolving it. See AGENTS-generic.md § Imports for the general rule.
A module may import from its own tier or any tier below it, never above:
| Tier | Modules | Owns |
|---|---|---|
| 3 | q3.workflows, q3.jax | orchestration — multi-step runs, differentiation |
| 2 | q3.calculators, q3.flavors | one run of the engine, driven through an API |
| 1 | q3.binary, q3.io, q3.generated, q3.structure, q3.properties | process, format, and structure adapters; models generated from the schemas; fits of derived properties |
| 0 | q3.constants, q3.errors | leaves — import nothing from q3 |
q3/__init__.py and q3/__main__.py are untiered entry points: they sit above everything and are never imported by package code.
Enforcement is automatic, at two levels:
PLC0415 is enabled in ruff (pyproject.toml), so a function-level import anywhere — package, tests or scripts — fails the lint job, and the no-function-level-imports pre-commit hook fails any noqa or per-file-ignores entry that would exempt one. There are no accepted exceptions: ASE is a base dependency imported at the top, and the optional q3_core extension and Phonopy are imported at the top behind a module-level try/except ImportError that sets the name to None.scripts/repo/check_python_import_layering.py (pre-commit hook python-import-layering) enforces the tier table itself, catching upward imports that are legal at runtime and same-tier cycles. It scans the whole package on every run, so a cycle cannot slip through by having its two halves land in different commits.Adding a subpackage under q3/ means adding it to TIERS in that script; tests/python/test_check_python_import_layering.py fails if you forget.
Worked example: material_json_to_ase_atoms used to live in q3.workflows.relax (tier 3) while q3.calculators.ase_calculator (tier 2) needed it, so the calculator imported the workflow from inside a method body and the two formed a cycle. The fix was to move the adapter down to q3.structure.ase_atoms (tier 1), which both sides now import at top level.
.hpp file must contain exactly one class definition. Do NOT place multiple classes in the same file..hpp) so that each class is fully self-contained in a single file. Use separate .cpp files only when necessary (e.g., very large implementations that significantly impact compile times, GPU vendor backends, entry points).RealSpaceMixer → real_space_mixer.hpp).init.hpp as the package-entry file for a directory's primary class (~79 files, Pythonic __init__ convention) rather than class-named files. Whether to bless init.hpp in this rule or migrate to class-named files is an open decision — see TODOs.md §3.3. Until decided, follow the convention of the directory you are editing.if/else) to handle different physical approximations (e.g., LDA vs GGA, NC vs US, with/without NLCC) inside a single calculator class. Instead, decompose into separate calculator classes — one per physical context — using inheritance or composition, and assemble them in the orchestrator. Each calculator should handle exactly one concern.if (quantum_system.has_nonlinear_core_correction()) { ... } inside a stress calculatorif (has_core_charge) { ... } flag-based branchingExchangeCorrelationStressTensorCalculator (no PP dependency) with a virtual build_total_charge_density_g() method, and a PseudopotentialExchangeCorrelationStressTensorCalculator subclass that overrides it to include NLCC core chargeconstants.hpp<prefix>_results.json, <prefix>_interpolation_results.json, the Python layer's result dictionaries, companion JSON files) that ESSE already has a property schema for MUST be written as that ESSE property record (name, units, value or values), and its Q3 schema MUST $ref the ESSE schema rather than restate it. Look in schemas/esse/properties_directory/, and on ESSE's dev branch, before adding a key. results.json carries these under properties.records (set_record() in src/io/output/results.hpp)._ry, _ev, _kbar, _bohr, _per_bohr, ...). A quantity carries its unit in its own units field, the way ESSE's records do: $ref the ESSE schema for it (core/reusable/energy, structural/atomic_forces, non_scalar/stress_tensor, ...) directly from the output schema. Do not copy or re-narrow ESSE schemas into a Q3 file. Readers check the unit where they read the value and refuse any other.plan/. Until it lands and the mat3ra-esse pin is bumped, the value is written under a Q3 key beside properties.records, as a Q3 schema file of its own built from ESSE pieces (core/primitive/scalar plus an ESSE unit). Never invent an ESSE-shaped record with a Q3-made name, and never add a Q3-local schema for a physical property ESSE could hold.schemas/manifest.yaml, src/generated/) rather than building JSON by hand, and a Q3 schema that $refs an ESSE schema listed there gets ESSE's struct, not a copy of it under a Q3 name; Python reads records through the pydantic models in mat3ra.esse.models (q3.io.results types each record as ESSE's model). Never write a helper class or module that re-describes a record ESSE already models.schemas/q3/ is for what is genuinely Q3's own (the input manifest, execution topology, provenance) and $refs ESSE, by a relative path into schemas/esse, for every piece ESSE already defines. One schema per file: a document's blocks live in a directory named after it (results/runtime_log.json), and a Q3 schema that needs more than ESSE states extends the ESSE schema with allOf rather than copying it.results.json is grouped by domain: structure, provenance, properties (physical results) and runtime_log (how the calculation ran). A new key goes in the group it belongs to, never at the top level.tests/regression/test_results_schema.py enforces the results side: every record under properties.records must validate against an ESSE schema the results schema lists, and no key may end in a unit suffix.tests/unit/ MUST mirror the directory structure under src/. A test file for src/layer_X/foo/bar.hpp must be placed at tests/unit/layer_X/foo/test_bar.cpp. Do NOT create ad-hoc directories like tests/unit/factories/ or flatten the hierarchy.tests/regression/ (Python, validate binary output)requires_qe marker from tests/helpers.py – never a locally re-derived binary path, which is how one of them lost its guard and reddened the scheduled tier-2 job for two cycles. helpers.py finds the build under any ancestor of the repository root, so it is still found from a git worktree, and $Q3_QE_BIN_DIR overrides the search. tests/validation/test_qe_reference_binary_guards.py enforces this per pull request rather than per schedule.The project docs have distinct, non-overlapping roles. Keep facts in exactly one place:
SUMMARY.md — narrative: what Q3 is and why it exists. No capability tables or task lists.FUNCTIONALITY.md — present-state capabilities, organized into three user-facing areas: Science (§1), Performance (§2), Engineering (§3). Only list what exists today.TODOs.md — planned/in-progress work, using the same three areas and numbering as FUNCTIONALITY.md, with [S]/[M]/[L] horizon tags.STATISTICS.md — generated by scripts/repo/generate_statistics.py, and refreshed on main by CI. It counts the core only: each build-time add-on has its own STATISTICS.md in its src/addons/<name>/ directory, generated and guarded the same way, and every export recounts the tree it publishes. Never edit them, and never regenerate them in a pull request — see "Leave STATISTICS.md alone" below.Rules:
TODOs.md (planned) or FUNCTIONALITY.md (done) — never both.FUNCTIONALITY.md area (Science/Performance/Engineering) and remove the corresponding item from the same area in TODOs.md.FUNCTIONALITY.md; link to the relevant TODOs.md section instead.**Note:** …). > marks a quotation, and GitHub and the docs site both render it as one.STATISTICS.md alone: do not edit it, and do not run the generator to "keep it fresh" before committing. It counts lines and test cases across the whole tree, so it moved with almost every commit that touched source — twenty-two of the last forty before this rule — and always on the same adjacent rows of a nine-row table, which git cannot merge. Two open branches therefore conflicted on a file neither was allowed to author. CI now regenerates it on main after every push; a pull request that carries it fails scripts/repo/check_statistics_untouched.py, at commit time through the statistics-untouched hook and again on the runner. The one exception is a change to the generator itself — a new metric, a renamed row — where both files move together and the checker asks that they agree.main. When starting new work mid-session, main is very likely stale — this repo routinely stacks several open, unmerged feature PRs on top of each other (check gh pr list --state open before assuming main is current). Branching from main silently drops whatever the session's current branch already has.main instead when explicitly told to, or when main is confirmed to already contain the tip of the current work (e.g. the stacked PR chain has actually merged — verify with git log --oneline main..<branch> before assuming, don't infer merge state from CI status alone).Read PROVENANCE.md before implementing anything another computational materials science code also does. It explains what does and does not count as a derivative work, and gives the step-by-step procedure. The rules below are the summary; that document is the reasoning and the recipe.
Q3 ships under Apache-2.0 and its public tree must contain only content we have the right to distribute. These rules bind every contributor and every coding agent, in the raw repository and in the published tree:
AGENTS.md.Q3 is developed in a private repository (the raw side) and published through camsai/q3 (the clean side), which receives automated, content-gated exports. What that means if you are contributing:
camsai/q3. They are reviewed and merged there, then replayed into the raw repository with git cherry-pick -x, which preserves your authorship.sync: commits, each a gated snapshot of the raw repository; a direct push would be overwritten by the next export.THIRD_PARTY_NOTICES.md) names a commit of it, and that offer has to hold for as long as the image is distributed. A mistake in published history is corrected by a new commit, never by a force-push.Core libraries fetched via CMake FetchContent — no manual installation needed except FFTW3 and LAPACK.
| Library | Purpose | Link target |
|---|---|---|
| LibXC | XC functionals | Libxc::xc |
| spglib | Symmetry | Spglib::symspg |
| spdlog | Logging | spdlog::spdlog_header_only |
| yaml-cpp | YAML | yaml-cpp::yaml-cpp |
| nlohmann_json | JSON | nlohmann_json::nlohmann_json |
| GoogleTest | Testing | GTest::gtest_main |
Parallel/GPU builds additionally require system packages per opt-in CMake flag (MPI + FFTW3-MPI + optional ScaLAPACK, CUDA Toolkit, ROCm, oneMKL, or the Apple Accelerate/Metal frameworks) — see FUNCTIONALITY.md §3.3 for the full table.
1.9.8