Skip to content

BP4 silently drops attribute-only iterations written through the steps API #1912

Description

@ax3l

Verified in 0.17.1 on BP4, group- and variable-based alike: only the first attribute-only iteration survives, and neither adios2.use_group_table nor empty/constant records anchor the step.
We should probably error out if variable-based encoding is selected for BP4?

Reproducer:

#!/usr/bin/env python3
# ADIOS2 BP4: attribute-only iterations are dropped when written via I/O steps.
# Verified with openPMD-api 0.16.1 and 0.17.1 (ADIOS2 2.12.1).
# HDF5 and BP5 (in all encodings) are unaffected.
# Writing via random access (series.iterations[i])
# instead of write_iterations() also preserves all iterations on BP4.
import openpmd_api as io

for ext in ("bp4", "h5"):
    series = io.Series(f"attr_only.{ext}", io.Access.create)
    for i in (1, 3):
        it = series.write_iterations()[i]
        it.set_attribute("foo", 1.0 * i)  # attributes only, no datasets
        it.close()
    series.close()

    read = io.Series(f"attr_only.{ext}", io.Access.read_only)
    print(f"{ext}: iterations found = {list(read.iterations)}  (expected [1, 3])")
    read.close()

Verified output in your env (openPMD-api 0.17.1, ADIOS2 2.12.1, group-based encoding):

  bp4: iterations found = [1]  (expected [1, 3])
  h5:  iterations found = [1, 3]  (expected [1, 3])
  • Only the first attribute-only iteration survives; iteration 3's attributes are absent from the BP4
    file entirely (confirmed with bpls -A: only /data/1/... paths exist).
  • Same behavior with variable-based encoding on BP4 (read via READ_LINEAR); BP5 is unaffected in
    every encoding, consistent with BP5 scoping attributes per step while BP4 writes them once into
    global metadata.
  • Workarounds that do not help: {"adios2": {"use_group_table": true}}, make_empty() records, and
    zero-extent constant records — attributes never anchor a BP4 step.
  • Workaround that does: writing through the random-access API (series.iterations[i] +
    iteration.close()) instead of write_iterations() preserves all iterations on BP4 — that's what
    ImpactX now does for its moments-only monitor output.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions