OMMX Python SDK 3.0.x

Contents

OMMX Python SDK 3.0.x#

Note

Python SDK 3.0.0 contains breaking API changes. A migration guide is available in the Python SDK v2 to v3 Migration Guide.

Unreleased#

Changes merged after the most recent release will be appended here as they land, and promoted to a new version section when the next release is cut.

3.0.0 Alpha 5#

Static Badge

See the GitHub Release above for full details. The following summarizes the main changes. This is a pre-release version. APIs may change before the final release.

🆕 Run-scoped Experiment trace storage (#910, #916)#

Experiment, with_temp_local_registry(), and fork() now accept store_trace=True. When enabled, each with experiment.run() context captures the OpenTelemetry spans emitted inside that Run and stores one trace on the closed SealedRun. The stored trace is returned as TraceResult from trace, and is carried through commit, load, and fork.

See Tracing and Profiling for the full tracing workflow, renderers, and OpenTelemetry setup notes.

from ommx.experiment import Experiment
from ommx.tracing import render_text_tree
from ommx_highs_adapter import OMMXHighsAdapter

with Experiment.with_temp_local_registry(store_trace=True) as experiment:
    with experiment.run() as run:
        run.log_solve(OMMXHighsAdapter, instance)

loaded = Experiment.from_artifact(experiment.artifact)
trace = loaded.runs[0].trace
if trace is not None:
    print(render_text_tree(trace))

The stored payload is OTLP protobuf, so TraceResult now owns the exported request, exposes flattened spans, and can round-trip with otlp_protobuf() / from_otlp_protobuf(). Text and Chrome trace renderers also use domain-oriented span names such as Run, solve, convert, call, and decode, and surface instrumentation scope while hiding debug-only source attributes.

⚠ Experiment attachments are now name-indexed (#924)#

Experiment and Run attachments are now stored as name-indexed tables in the Experiment config. The public Python API is name-oriented: use attachment_names, attachment_media_type(name), get_attachment(name), the typed getters such as get_json(name) and get_instance(name), get_blob(name), get_with_codec(...), and write_attachment(...).

loaded = Experiment.from_artifact(experiment.artifact)

for name in loaded.attachment_names:
    print(name, loaded.attachment_media_type(name))
    value = loaded.get_attachment(name)

Descriptor-oriented attachment views from earlier 3.0 alphas, including Experiment.experiment_attachments and SealedRun.attachments, are removed. Registry-backed descriptors remain internal so attachment names, media types, file export names, and checkpoint metadata stay in the Experiment config instead of descriptor annotations.

🆕 Experiment checkpoints and restore from interrupted sessions (#917)#

Experiment now publishes local checkpoints for partial experiment state. Closing a Run writes a best-effort draft checkpoint, and exiting an Experiment with an exception writes a failed or interrupted checkpoint instead of advancing the successful Experiment image reference. Closed Runs keep their attachments, solves, traces, and run parameters, including Runs closed as "failed" or "interrupted" after exceptions such as KeyboardInterrupt.

See Experiment Recovery and Cleanup for Run close boundaries, checkpoint restoration, and Local Registry cleanup behavior.

Use restore_from_checkpoint() with the original Experiment image name to resume from the latest checkpoint:

from ommx.experiment import Experiment

image_name = "ghcr.io/example/team/experiment:notebook"

try:
    with Experiment(image_name) as experiment:
        with experiment.run() as run:
            run.log_parameter("solver", "highs")
            raise KeyboardInterrupt
except KeyboardInterrupt:
    pass

experiment = Experiment.restore_from_checkpoint(image_name)
assert experiment.image_name == image_name

Successful commit() still publishes only the requested image reference and removes the local checkpoint when present. Checkpoint Artifact handles and checkpoint image names are intentionally not exposed in the Python API; users restore by remembering the original Experiment image name.

🆕 Local Registry cleanup (#919)#

The ommx CLI now provides Local Registry maintenance commands for the SQLite-backed Artifact registry. Use ommx gc to report blobs that are unreachable from SQLite refs, including Experiment checkpoint refs. The command protects recently written unreachable blobs with a grace period so active Experiment writes are not deleted accidentally.

Destructive cleanup commands report by default and mutate the registry only when --delete is passed:

ommx prune-anonymous
ommx gc
ommx prune-anonymous --delete
ommx gc --delete

Normal reports show counts and sizes rather than raw digests. Pass --show-digests when low-level diagnostics are needed.

The same cleanup operations are also exposed from the Python SDK as ommx.artifact.prune_anonymous() and ommx.artifact.gc(). These functions are report-only by default, mutate the registry with delete=True, and return structured report objects for notebook and script use.

🆕 Typed attachment codecs for Experiments (#921)#

The new ommx.experiment.attachments.AttachmentCodec protocol lets packages that own Python payload types define how those values are stored as Experiment attachments. A codec class provides a media type plus encode / decode methods, and OMMX calls it through log_with_codec and get_with_codec on both Experiment-level and Run-level attachments.

See the Attachable Data Formats section of the Experiment management tutorial for a JijModeling Problem codec example.

from ommx.experiment import Experiment


class TextCodec:
    media_type = "text/plain"

    @staticmethod
    def encode(value: str) -> bytes:
        return value.encode()

    @staticmethod
    def decode(data: bytes) -> str:
        return data.decode()


with Experiment.with_temp_local_registry() as experiment:
    experiment.log_with_codec(TextCodec, "note", "created outside OMMX")

loaded = Experiment.from_artifact(experiment.artifact)
assert loaded.get_with_codec(TextCodec, "note") == "created outside OMMX"

The stored attachment media type is validated before decoding, so using the wrong codec for an attachment fails before the codec’s decode method is called.

🆕 File attachments for Experiments (#922)#

Experiment and Run can now attach files that were produced outside OMMX. Use log_file to copy an existing file into the Experiment Artifact. OMMX stores the file bytes as an attachment blob, records the original basename for later export, and uses an explicitly provided media type or Rust SDK content-based inference with an application/octet-stream fallback.

Committed experiment and run views now also provide write_attachment to restore an attachment blob back to disk. For libraries that accept a binary file-like object, wrap the existing get_blob result with io.BytesIO.

import io
from pathlib import Path

from ommx.experiment import Experiment

with Experiment.with_temp_local_registry() as experiment:
    experiment.log_file("input-spreadsheet", "input.xlsx")

loaded = Experiment.from_artifact(experiment.artifact)
spreadsheet_file = io.BytesIO(loaded.get_blob("input-spreadsheet"))
Path("restored").mkdir(parents=True, exist_ok=True)
loaded.write_attachment("input-spreadsheet", "restored/input.xlsx")

3.0.0 Alpha 4#

Static Badge

See the GitHub Release above for full details. The following summarizes the main changes. This is a pre-release version. APIs may change before the final release.

⚠ SQLite-based Local Registry (#871, #872)#

In v3, local Artifact storage is organized around the SQLite-based Local Registry. Artifact blobs are stored in content-addressed storage, while image-name references and registry metadata are managed in SQLite. APIs that depended on the old disk OCI directory cache are removed; the user-facing flow is now to commit an Artifact into the Local Registry, then save / push / load that committed Artifact.

Alongside this storage model and the new Experiment API, the old ArtifactBuilder is reshaped as ArtifactDraft. An ArtifactDraft represents an uncommitted Artifact draft; after it is committed to the Local Registry, the resulting Artifact can be saved or pushed. .ommx archives are import/export exchange formats for the Local Registry. The main breaking changes are:

  • ArtifactBuilder.new_archive → ArtifactDraft.new + Artifact.save (new method).

  • ArtifactBuilder.new_archive_unnamed → ArtifactDraft.new_anonymous + Artifact.save(path). In v2, an unnamed archive literally had no image name and was read back as None. In v3, an anonymous Artifact gets an automatically generated <registry-id8>.ommx.local/anonymous:<timestamp>-<nonce> image name from the Local Registry, so it can still be saved, loaded again, and cleaned up.

  • Artifact.load_archive raises a migration error pointing at the two replacement methods: Artifact.import_archive (imports the archive into the user’s persistent SQLite Local Registry — the v3 successor with registry-write semantics) and Artifact.inspect_archive (side-effect-free read of the manifest + layer descriptors, returns a new ArchiveManifest view). v2’s load_archive opened archives in place with no registry side effect, so the rename makes the semantic shift explicit instead of silently writing into the registry on upgrade. import_archive accepts v2 archives produced by ArtifactBuilder.new_archive_unnamed (no org.opencontainers.image.ref.name annotation) by synthesizing an anonymous name on the fly; inspect_archive reads such archives back with ArchiveManifest.image_name = None (no registry context for synthesis).

  • CLI ommx push <archive> and ommx push <oci-dir> removed — load into the registry first, then push by image name.

  • New CLI ommx prune-anonymous [--delete] reports accumulated anonymous-commit entries by default and removes them only when --delete is passed.

  • ommx.get_image_dir(...) and the CLI ommx image-dir <name> subcommand are removed. The return value was a v2 disk-cache path (<root>/<image_name>/<tag>/) that no longer corresponds to any v3 storage location — the SQLite Local Registry stores blobs content-addressed and refs in SQLite — so pointing users at it was actively misleading. Existing v2 caches still migrate via ommx import-legacy.

See the Python SDK v2→v3 Migration Guide §13 for the full before/after code and migration checklist.

🆕 Artifact-backed experiment management API: ommx.experiment (#882, #885, #886, #903)#

The new ommx.experiment module records experiment inputs, run conditions, and Solver/Sampler results as one OMMX Artifact. Use Experiment, Run, and Solve to store per-run comparison parameters, attachments, and solve input/output data in the Local Registry.

See the Experiment management tutorial for the basic workflow, sharing an Experiment, loading a committed Experiment, and creating derived experiments with fork.

🆕 Run.log_solve records solve input/output and adapter options (#902)#

log_solve() is now available. Pass a subclass of ommx.adapter.SolverAdapter and an Instance; OMMX calls the adapter’s solve, then stores the input Instance, output Solution, adapter class name, and JSON-serializable keyword arguments as a Solve.

from ommx.experiment import Experiment
from ommx_highs_adapter import OMMXHighsAdapter
from ommx.v1 import Instance, Solution

with Experiment() as experiment:
    with experiment.run() as run:
        solution = run.log_solve(OMMXHighsAdapter, instance, verbose=False)
        run.log_parameter("objective", solution.objective)

solve = experiment.runs[0].solves[0]
assert solve.adapter.endswith("OMMXHighsAdapter")
assert isinstance(solve.input, Instance)
assert isinstance(solve.output, Solution)
assert solve.output.feasible
assert solve.adapter_options == {"verbose": False}

Adapter options are solve-scoped metadata, so they do not appear in run_parameters_df(), which is the table for comparing runs. Record values explicitly with log_parameter() when you want them in that DataFrame.

🆕 Experiment fork and lineage (#905)#

fork() starts a new uncommitted Experiment from a committed one. The child inherits the parent’s attachments, Runs, Solves, and Run parameters, while the parent remains unchanged. When the child is committed after adding new Runs or attachments, the parent manifest descriptor is recorded as the OCI subject.

from ommx.experiment import Experiment
from ommx_highs_adapter import OMMXHighsAdapter

loaded = Experiment.load("ghcr.io/jij-inc/ommx/tutorial/experiment:baseline")

with loaded.fork("ghcr.io/jij-inc/ommx/tutorial/experiment:capacity-64") as child:
    with child.run() as run:
        run.log_parameter("capacity", 64)
        run.log_solve(OMMXHighsAdapter, instance, verbose=False)

Forking creates a new Artifact Manifest, but Instance / Solution / attachment payloads continue to reference content-addressed blobs in the Local Registry, so the data bodies are not duplicated. Saving or pushing the fork shares the complete forked Experiment, including Runs and Solves inherited from the parent.

🆕 Instance.substitute / ParametricInstance.substitute (#891, #897)#

substitute() and substitute() are now available from Python. Pass a dictionary from decision-variable IDs to replacement Function expressions; OMMX rewrites those variables in the objective and active constraints in-place. This exposes the general substitution mechanism behind log_encode, so users can implement custom variable transformations such as unary or one-hot encodings.

from ommx.v1 import DecisionVariable, Instance

x = DecisionVariable.integer(0, lower=0, upper=3)
b = [DecisionVariable.binary(i) for i in (1, 2)]
instance = Instance.from_components(
    decision_variables=[x, *b],
    objective=x,
    constraints={},
    sense=Instance.MAXIMIZE,
)

instance.substitute({0: b[0] + 2 * b[1]})
assert str(instance.objective) == "Function(x1 + 2*x2)"

This API is an algebraic rewrite. It does not translate the substituted variable’s kind / lower / upper into constraints on the replacement expression. To preserve the optimization problem, use a domain-preserving encoding or add the required linking / bound constraints yourself. ParametricInstance.substitute may leave parameters in replacement expressions, so symbolic variable transformations can be applied before concrete values are supplied with with_parameters.

3.0.0 Alpha 3#

Static Badge

See the GitHub Release above for full details. The following summarizes the main changes. This is a pre-release version. APIs may change before the final release.

⚠ *_df accessors are methods + include= filter + sidecar DataFrames (#846)#

Every *_df accessor on Instance / ParametricInstance / Solution / SampleSet is now a regular method instead of a #[getter] property. Existing call sites need parentheses:

# Before
df = solution.constraints_df

# After
df = solution.constraints_df()

The wide *_df methods take an include argument that gates the metadata / parameters column families. The default include=("metadata", "parameters") preserves the v2-equivalent wide shape:

solution.decision_variables_df()                       # core + metadata + parameters
solution.decision_variables_df(include=[])             # core only
solution.decision_variables_df(include=["metadata"])   # core + metadata
solution.decision_variables_df(include=["parameters"]) # core + parameters

Six new long-format / id-indexed sidecar accessors read directly from the SoA metadata stores. kind= selects the constraint family ("regular" / "indicator" / "one_hot" / "sos1", default "regular"):

  • constraint_metadata_df(kind=...) — id-indexed (name / subscripts / description)

  • constraint_parameters_df(kind=...) — long format ({kind}_constraint_id / key / value)

  • constraint_provenance_df(kind=...) — long format ({kind}_constraint_id / step / source_kind / source_id)

  • constraint_removed_reasons_df(kind=...) — long format ({kind}_constraint_id / reason / key / value)

  • variable_metadata_df() — id-indexed

  • variable_parameters_df() — long format

Sidecar index names are kind-qualified (regular_constraint_id / indicator_constraint_id / one_hot_constraint_id / sos1_constraint_id / variable_id) so accidental cross-id-space df.join() mistakes surface in df.head() and friends. Long-format *_parameters_df / *_removed_reasons_df rows are sorted by (id, key), and empty long-format DataFrames keep their column schema instead of returning a column-less frame.

⚠ removed_reason column gated by include= (#796, #847)#

In v2.5.1 Solution.constraints_df carried a removed_reason column unconditionally. The initial include= gate of that column landed in 3.0.0a2 (#796), and 3.0.0a3 finalizes it into the kind= / include= / removed= dispatch shape documented above (#847): the column is opted in by "removed_reason" in include= (a unit flag that controls both the reason name and removed_reason.{key} parameter columns). Rows whose constraint was not removed before evaluation get NA in those columns.

# Before (2.5.1)
df = solution.constraints_df  # contains a 'removed_reason' column

# After (3.0.0a3 — `*_df` are now methods)
df = solution.constraints_df()  # no removed_reason column
df = solution.constraints_df(include=("metadata", "parameters", "removed_reason"))
# ↳ adds removed_reason / removed_reason.{key} (NA for active rows)

The same kind= / include= shape applies on SampleSet. On Instance and ParametricInstance, removed=True returns active + removed rows in one DataFrame and auto-sets "removed_reason" so removed rows are distinguishable.

⚠ to_bytes / from_bytes removed from non-top-level types (#845)#

Bytes serialization is removed from the following component-level types:

These methods originally existed to ferry values across the Python ↔ Rust boundary back when the Python SDK had its own protobuf-based wrapper layer and had to serialize on every hop. With the v3 transition to direct PyO3 re-exports the boundary disappears, so element-level bytes round-trips no longer serve a purpose, and keeping them aligned with the upcoming metadata-storage redesign would only add maintenance cost. to_bytes / from_bytes remain available on the container types (Instance, ParametricInstance, Solution, SampleSet) and on the cross-evaluate DTOs (State, Samples, Parameters) — use those when you need to persist or exchange data on disk or over the wire.

🆕 Write-through metadata wrappers: AttachedConstraint / AttachedDecisionVariable (#849, #850, #852)#

Instance.add_constraint / instance.constraints[id] and the matching accessors on ParametricInstance now return write-through handles bound to the parent host instead of snapshot copies. Reads pull live data from the host and metadata setters write straight to its SoA metadata store, so two handles pointing at the same id observe the same state.

c = instance.add_constraint(x + y == 0)         # AttachedConstraint
c.set_name("budget")                             # writes through to instance
assert instance.constraints[c.constraint_id].name == "budget"

Five write-through types ship: AttachedConstraint, AttachedIndicatorConstraint, AttachedOneHotConstraint, AttachedSos1Constraint, and AttachedDecisionVariable. Constraint and DecisionVariable are unchanged in shape — they remain the snapshot wrappers used for modeling input (operator overloading, Instance.from_components). Each AttachedX exposes .detach() to obtain an equivalent snapshot when you need to break the back-reference to the host.

As part of the same change, instance.decision_variables now returns list[AttachedDecisionVariable] (previously list[DecisionVariable] snapshots), aligning with instance.constraints and the special-constraint accessors.

🆕 OpenTelemetry-based tracing and profiling (#816, #823, #826, #828, #829)#

The legacy log + pyo3-log → Python logging bridge is replaced by a tracing + pyo3-tracing-opentelemetry pipeline, so the Rust core’s spans can now be consumed through the Python OTel SDK.

Two entry points ship under ommx.tracing:

  • %%ommx_trace — a Jupyter cell magic that renders a per-cell span tree and a Chrome Trace JSON download link

  • capture_trace / @traced — a context manager and decorator for the same workflow from regular Python scripts, tests, and CI

See Tracing and Profiling for the full walkthrough, configuring your own TracerProvider, and troubleshooting.

🆕 Tracing spans in solver/sampler adapters (#833)#

Every OMMX adapter now emits three OpenTelemetry spans per solve/sample call, so the OTel tracing pipeline above can attribute wall-clock time to the three phases an adapter actually spends time in:

  • convert — OMMX Instance → solver-native problem translation

  • solve / sample — the call into the underlying solver / sampler itself

  • decode — decoding the solver’s response back to Solution / SampleSet (Rust-side evaluate spans nest underneath)

Each adapter uses its own tracer name, so runs from different solvers are easy to distinguish in the tree view:

Adapter

Tracer

Spans

ommx-pyscipopt-adapter

ommx.adapter.pyscipopt

convert / solve / decode

ommx-highs-adapter

ommx.adapter.highs

convert / solve / decode

ommx-python-mip-adapter

ommx.adapter.python_mip

convert / solve / decode

ommx-openjij-adapter

ommx.adapter.openjij

convert / sample / decode

 
from ommx.tracing import capture_trace, render_text_tree
from ommx_pyscipopt_adapter import OMMXPySCIPOptAdapter

with capture_trace() as trace:
    solution = OMMXPySCIPOptAdapter.solve(instance)

print(render_text_tree(trace))  # shows convert / solve / decode with durations

Spans are emitted through the standard OpenTelemetry API, so they are a no-op when no TracerProvider is installed — there is no runtime cost for users who do not opt in.

🆕 Function.evaluate_bound is now available from Python (#831)#

Function.evaluate_bound is now exposed on Function. Given per-variable bounds, it returns a Bound that contains the range of the function value — useful when deriving feasibility bounds or doing simple presolve on the Python side.

from ommx.v1 import Function, Linear, Bound

f = Function(Linear(terms={1: 2}, constant=3))  # 2*x1 + 3
b = f.evaluate_bound({1: Bound(0.0, 2.0)})
# b.lower == 3.0, b.upper == 7.0

The bound is computed monomial-wise and summed, so it is a sound over-approximation of the true range but is not guaranteed to be tight when multiple terms share variables (the classic dependency problem in interval arithmetic). Variable IDs missing from bounds are treated as unbounded.

3.0.0 Alpha 2#

Static Badge

See the GitHub Release above for full details. The following summarizes the main changes. This is a pre-release version. APIs may change before the final release.

⚠ Removal of the Constraint.id field (#806)#

The id field (along with the .id getter, set_id(), and id= constructor argument) is removed from Constraint and its variants (IndicatorConstraint / OneHotConstraint / Sos1Constraint / EvaluatedConstraint / SampledConstraint / RemovedConstraint). A constraint’s ID now exists only as the key of the dict[int, Constraint] passed to Instance.from_components.

# Before (2.5.1)
c = Constraint(function=x + y, equality=Constraint.EQUAL_TO_ZERO, id=5)
Instance.from_components(..., constraints=[c], ...)

# After (3.0.0a2)
c = Constraint(function=x + y, equality=Constraint.EQUAL_TO_ZERO)
Instance.from_components(..., constraints={5: c}, ...)

Global ID counters (next_constraint_id and friends) and per-constraint to_bytes / from_bytes are also removed. For full details and migration steps, see the Python SDK v2 to v3 Migration Guide.

🆕 First-class special constraint types (#789, #790, #795, #796, #798)#

In addition to regular constraints, the following three special constraint types are now first-class citizens — they can be passed to Instance.from_components via indicator_constraints= / one_hot_constraints= / sos1_constraints=, and read back through constraints_df() / constraints_df() with kind= selecting the family.

For concrete usage, evaluation-result access, and the Indicator relax / restore workflow, see Special Constraints.

Accordingly, the legacy ConstraintHints / OneHot / Sos1 classes, the Instance.constraint_hints property, and the PySCIPOpt Adapter’s use_sos1 flag are removed.

🆕 Adapter Capability Model (#790, #805, #810, #811, #814)#

Alongside the special constraint types, adapters now declare their own supported capabilities via an ADDITIONAL_CAPABILITIES class attribute. When super().__init__(instance) is called, any undeclared special constraint is automatically converted to regular constraints (Big-M for Indicator / SOS1, linear equality for OneHot) before the instance reaches the solver.

Existing OMMX Adapters must be updated for Python SDK 3.0.0 to call super().__init__(instance). Currently the PySCIPOpt Adapter declares support for Indicator and SOS1.

For details and the manual conversion APIs, see Adapter Capability Model and Conversions.

🔄 numpy scalar support (#794)#

The Function constructor now accepts numpy.integer and numpy.floating values. In v2.5.1, Function(numpy.int64(3)) raised TypeError.

3.0.0 Alpha 1#

Static Badge

See the GitHub Release above for full details. The following summarizes the main changes. This is a pre-release version. APIs may change before the final release.

Complete Rust re-export of ommx.v1 and ommx.artifact types (#770, #771, #774, #775, #782)#

Python SDK 3.0.0 is fully based on Rust/PyO3. In 2.0.0, the core implementation was rewritten in Rust while Python wrapper classes remained for compatibility. In 3.0.0, those Python wrappers are removed entirely — all types in ommx.v1 and ommx.artifact are now direct re-exports from Rust, and the protobuf Python runtime dependency is eliminated. The .raw attribute that previously provided access to the underlying PyO3 implementation has also been removed.

Migration to Sphinx and ReadTheDocs hosting (#780, #785)#

In v2, the Sphinx-based API Reference and Jupyter Book-based documentation were each hosted on GitHub Pages. In v3, documentation has been fully migrated to Sphinx and is now hosted on ReadTheDocs. GitHub Pages will continue to host the documentation as of v2.5.1, but all future updates will be on ReadTheDocs only.

Contents