diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..a1bf6b0 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,47 @@ +# Codex Instructions + +## Scope + +These instructions apply to the whole `bec_docs` workspace. + +## Project Overview + +- This repository contains the BEC documentation site. +- It combines the documentation for three repositories: + - `bec` (the main BEC codebase): https://github.com/bec-project/bec + - `ophyd_devices` (ophyd device components for BEC): https://github.com/bec-project/ophyd_devices + - `bec_widgets` (BEC-specific widgets for the UI): https://github.com/bec-project/bec_widgets +- Main documentation sources live under `docs/`. +- Navigation is configured in `zensical.toml`. +- The site is built with Zensical and follows a diataxis structure. + +## Editing Guidelines + +- Follow the diataxis structure when adding or editing pages: + - Getting Started: high-level tutorials for new users, focused on teaching concepts and workflows without overwhelming details and lots of explanations. + - How-To: focused guides for specific tasks. Should be concise and practical, with just enough explanation to understand the steps. Reference learning pages for deeper dives into concepts. A user coming back to the how-to page should not be bombarded with information that is not directly relevant to the task at hand. + - Learning: in-depth explanations of concepts and features + - Reference: API documentation and technical details, treated as a lookup resource rather than a narrative +- Prefer small, focused documentation edits that preserve the existing structure and tone. +- Match the wording and formatting already used in nearby pages. +- When adding tables that need fixed column widths, prefer inline HTML tables over Markdown tables. +- Keep links relative and consistent with the surrounding docs. +- Do not remove or overwrite user changes outside the requested scope. +- If you need to make a larger structural change, please discuss it first before implementing. + +## Admonitions +- Getting-started / tutorial pages must start with a info admonition "Goal" that clearly states the learning outcomes for the page and end with a success admonition "What you have learned" that summarizes the key takeaways. +- How-to pages must start with an info admonition "Overview" that explains the purpose of the page and end with a success admonition "Congratulations!" that celebrates the completion of the task. +- Learning pages must provide a `!!! info "What to remember"` for end-of-page takeaways. + +## Content Conventions + +- Use sentence case in prose and keep headings consistent with neighboring pages. +- Prefer short examples that reflect real BEC usage. +- When appropriate, add related links to other documentation pages at the top of the page in a `related` section. + +## Validation + +- If you change navigation-relevant docs, check whether `zensical.toml` also needs an update. +- If you delete or merge pages, update internal links so no stale references remain. +- When practical, verify links and references with fast text searches before finishing. diff --git a/docs/how-to/devices/development/use-status-objects-in-a-custom-device.md b/docs/how-to/devices/development/use-status-objects-in-a-custom-device.md new file mode 100644 index 0000000..9be7682 --- /dev/null +++ b/docs/how-to/devices/development/use-status-objects-in-a-custom-device.md @@ -0,0 +1,128 @@ +--- +related: + - title: Custom ophyd devices + url: ../../../learn/devices/custom-ophyd-devices.md + - title: Write a custom ophyd device + url: write-a-custom-device.md + - title: BEC signals for custom devices + url: ../../../learn/devices/bec-signals.md +--- + +# Use status objects in a custom ophyd device + +!!! Info "Overview" + Use `CompareStatus` and `TransitionStatus` in custom ophyd device hooks when the device action starts immediately but completes only after a signal reaches the expected state. + +## Prerequisites + +- You already have a custom device class based on `PSIDeviceBase`. +- Your device exposes signal values that indicate readiness, acquisition, or completion. +- You know whether success is best described as one target value or as a sequence of transitions. + +!!! learn "[Learn when custom devices should return status objects](../../../learn/devices/custom-ophyd-devices.md#status-objects-and-asynchronous-work){ data-preview }" + +## Use `CompareStatus` in `on_pre_scan()` + +`CompareStatus` is a good fit when pre-scan preparation should complete as soon as one signal reaches one target value. + +For example, assume a detector has: + +- an `arm` signal that starts preparation +- a `ready` signal that becomes `1` once arming is complete +- a `state` signal that may go to `"error"` if arming fails + +In that case, `on_pre_scan()` can start arming and return a `CompareStatus`: + +```py +from ophyd_devices.utils.psi_device_base_utils import CompareStatus + + +def on_pre_scan(self): + self.arm.put(1) + status = CompareStatus( + signal=self.ready, + value=1, + operation_success="==", + timeout=5, + ) + self.cancel_on_stop(status) + return status +``` + +If the device has an explicit error state, include that as a failure value: + +```py +def on_pre_scan(self): + self.arm.put(1) + status = CompareStatus( + signal=self.state, + value="armed", + operation_success="==", + failure_value=["error", "fault"], + timeout=5, + ) + self.cancel_on_stop(status) + return status +``` + +Use this pattern when one value is enough to decide whether the device is ready. + +## Use `TransitionStatus` in `on_complete()` + +`TransitionStatus` is a better fit when completion is expressed as a sequence of states. + +A common detector pattern is: + +- `acquire = 1` while acquisition is running +- `acquire = 0` again once acquisition is complete + +If the signal should explicitly transition back to idle, `on_complete()` can return: + +```py +from ophyd_devices.utils.psi_device_base_utils import TransitionStatus + + +def on_complete(self): + status = TransitionStatus( + signal=self.acquire, + transitions=[1, 0], + strict=True, + timeout=10, + ) + self.cancel_on_stop(status) + return status +``` + +This tells BEC to wait until acquisition is first observed as active and then observed as idle again. + +If the controller may pass through an error code, fail early with `failure_states`: + +```py +def on_complete(self): + status = TransitionStatus( + signal=self.acquire, + transitions=[1, 0], + strict=True, + failure_states=[-1], + timeout=10, + ) + self.cancel_on_stop(status) + return status +``` + +Use this pattern when completion depends on a state transition rather than one static value. + +## Choose the right helper + +- Use `CompareStatus` when success means “the signal reached this value”. +- Use `TransitionStatus` when success means “the signal moved through these values in order”. + +## Common pitfalls + +- Returning `None` even though the device is still changing state asynchronously. +- Forgetting `cancel_on_stop(status)`, which can leave BEC waiting on a status after an abort. +- Using `CompareStatus` when the real requirement is to observe a full transition sequence. +- Using `TransitionStatus` with an incomplete transition list, for example waiting only for `0` when the real completion pattern is `1 -> 0`. + +!!! success "Congratulations!" + You have used status objects to describe asynchronous device behavior in a way that BEC can wait for reliably during scans. diff --git a/docs/how-to/devices/development/write-a-custom-device.md b/docs/how-to/devices/development/write-a-custom-device.md new file mode 100644 index 0000000..a732116 --- /dev/null +++ b/docs/how-to/devices/development/write-a-custom-device.md @@ -0,0 +1,175 @@ +--- +related: + - title: Custom ophyd devices in BEC + url: ../../../learn/devices/custom-ophyd-devices.md + - title: Introduction to ophyd + url: ../../../learn/devices/introduction-to-ophyd.md + - title: BEC signals for custom devices + url: ../../../learn/devices/bec-signals.md + - title: Device config in BEC + url: ../../../learn/devices/device-config-in-bec.md +--- + +# Write a custom ophyd device + +!!! Info "Overview" + Create a reusable ophyd device for BEC by subclassing `PSIDeviceBase`, declaring the device signals as ophyd components, implementing the scan hooks you need, and exposing the class in your device config. + +## Prerequisites + +- You have a beamline plugin repository or a local checkout of `ophyd_devices`. +- You are comfortable editing Python classes and reloading the BEC device config. +- You already know the control-system endpoints you need, for example EPICS PV suffixes. + +!!! learn "[Learn how custom ophyd devices fit into BEC](../../../learn/devices/custom-ophyd-devices.md){ data-preview }" + +## 1. Create the device class + +In BEC, custom devices should usually inherit from `ophyd_devices.PSIDeviceBase`. It wraps the normal ophyd `Device` lifecycle with BEC-specific hooks such as `on_connected()`, `on_stage()`, `on_pre_scan()`, `on_trigger()`, and `on_stop()`. + +Create a new module in your plugin, for example `/devices/beam_stop_shutter.py`: + +```py +from ophyd import Component as Cpt +from ophyd import EpicsSignal, EpicsSignalRO, StatusBase + +from ophyd_devices import PSIDeviceBase + + +class BeamStopShutter(PSIDeviceBase): + """Simple shutter device that can be prepared automatically for scans.""" + + open_cmd = Cpt(EpicsSignal, "OPEN", kind="omitted") + close_cmd = Cpt(EpicsSignal, "CLOSE", kind="omitted") + state = Cpt(EpicsSignalRO, "STATE", kind="hinted") + ready = Cpt(EpicsSignalRO, "READY", kind="normal") + config = Cpt(EpicsSignalRO, "CONFIG", kind="config") + + def __init__(self, prefix: str, name: str, ready_timeout: float = 5.0, **kwargs) -> None: + super().__init__(prefix=prefix, name=name, **kwargs) + self._ready_timeout = ready_timeout +``` + +This keeps the device reusable: the class defines the structure once, and each BEC config entry supplies the concrete `prefix` and optional configuration values. + +## 2. Set defaults in `on_connected()` + +`on_connected()` is called by the BEC device manager after the device and its signals are connected. Use it for default setup that depends on live signals. + +```py +class BeamStopShutter(PSIDeviceBase): + ... + + def on_connected(self) -> None: + self.config.set(0).wait(timeout=3) + +``` + +This is a good place to: + +- register callbacks +- apply beamline-specific default values + +## 3. Implement the scan hooks you need + +You do not need to override every hook. Only implement the ones that match the device behavior you want in BEC. + +For a shutter that must be opened automatically before the first trigger, `on_pre_scan()` is often enough: + +```py + +from ophyd_devices import CompareStatus + +class BeamStopShutter(PSIDeviceBase): + ... + + def on_pre_scan(self) -> StatusBase: + status = CompareStatus(self.state, "open", timeout=self._ready_timeout) + self.open_cmd.put(1) + self.cancel_on_stop(status) + return status + + def on_stop(self) -> None: + self.close_cmd.put(1) + + def on_unstage(self) -> None: + self.close_cmd.put(1) +``` + +Important details: + +- Return a status object from `on_pre_scan()` that resolves when the shutter is open and ready. That way, BEC will wait for the shutter to be ready before starting the scan. +- Register long-running statuses with `cancel_on_stop(...)` so BEC can fail them cleanly when a scan is interrupted. + +!!! related "[How to use CompareStatus in a hook](../../../how-to/devices/development/use-status-objects-in-a-custom-device.md){ data-preview }" + +## 4. Use `scan_info` when scan context matters + +`PSIDeviceBase` keeps the current scan metadata on `self.scan_info`. That lets your device adjust its behavior to the active scan. + +For example: + +```py +def on_stage(self): + exp_time = self.scan_info.msg.scan_parameters["exp_time"] + self.exposure_time.put(exp_time) +``` + +This is useful when the device needs information such as exposure time, number of frames, scan name, or other request parameters before acquisition starts. + +## 5. Export the class + +Expose the new class in the `__init__.py` of the devices module, this makes it easier to reference in the BEC config: + +```py +from .beam_stop_shutter import BeamStopShutter +``` + +## 6. Add the device to the BEC config + +Reference the class in your device config and pass constructor arguments through `deviceConfig`: + +```yaml +beamstop: + readoutPriority: baseline + description: Beam stop shutter + deviceClass: .devices.BeamStopShutter + deviceConfig: + prefix: "X01DA-FE-OPEN:" + ready_timeout: 5.0 + deviceTags: + - shutter + enabled: true + readOnly: false + softwareTrigger: false +``` + +The keys inside `deviceConfig` must match the constructor signature of your device class. + +!!! learn "[Learn more about device config in BEC](../../../learn/devices/device-config-in-bec.md){ data-preview }" + +## 7. Reload and verify the device + +Reload the plugin repository and the YAML config and verify that: + +- the device appears in the client as `dev.beamstop` +- `dev.beamstop.read()` returns the signals you expect (`state` and `ready` in this example) +- `dev.beamstop.read_configuration()` returns the config signals you expect (`config` in this example) +- a scan that uses the device reaches `on_pre_scan()` and opens the shutter +- stopping the scan closes the shutter and cancels any outstanding status objects + +## Common pitfalls + +- Initializing live signal defaults in `__init__()` instead of `on_connected()`. +- Returning `None` from an asynchronous hook even though the device still has work to finish. +- Forgetting `cancel_on_stop(status)` for long-running `pre_scan`, `trigger`, or `kickoff` logic. +- Polling without `check_stopped=True`, which can leave the device hanging after a scan stop. +- Forgetting to export the class from the package used by `deviceClass`. + +## Next steps + + +- If the device streams files, previews, or async data, use the BEC-specific signal classes documented in [BEC signals for custom devices](../../../learn/devices/bec-signals.md). + +!!! success "Congratulations!" + You have written a custom ophyd device for BEC, connected it to the BEC lifecycle, and exposed it through the device config so it can participate in scans like any other device. diff --git a/docs/learn/devices/custom-ophyd-devices.md b/docs/learn/devices/custom-ophyd-devices.md new file mode 100644 index 0000000..d0c50fb --- /dev/null +++ b/docs/learn/devices/custom-ophyd-devices.md @@ -0,0 +1,249 @@ +--- +related: + - title: Write a custom ophyd device + url: ../../how-to/devices/development/write-a-custom-device.md + - title: Use status objects in a custom ophyd device + url: ../../how-to/devices/development/use-status-objects-in-a-custom-device.md + - title: Introduction to ophyd + url: introduction-to-ophyd.md + - title: BEC signals for custom devices + url: bec-signals.md + - title: Device config in BEC + url: device-config-in-bec.md +--- + +# Custom Ophyd Devices + +If you are familiar with ophyd and want to integrate a new device, we recommend that you use the `PSIDeviceBase` class from `ophyd_devices`. It essentially serves as a template for how to structure your custom device class and allows you to focus on local device logic. + +Ophyd itself provides a nice abstraction with lifecycle methods such as `stage()`, `unstage()`, `trigger()`, and so on. BEC leverages these methods in its scan execution. The idea is that if your device always needs to follow a certain pattern for a scan, then we will place this logic on the device level. A good example of this is a detector with its own backend that writes files. In that case, the device requires the relevant information about where to write files, how many frames to acquire, and so on. + +## The role of `PSIDeviceBase` + +At its core, `PSIDeviceBase` is still an ophyd `Device`, so you define signals and sub-devices in the usual way with `Component` declarations. The difference is that it wraps a few important ophyd lifecycle methods and forwards them into explicit hook methods that are easier to override during integration work: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
HookPurpose
on_init()Runs at the end of device initialization. Use it for setup logic that does not depend on connected signals.
on_connected()Runs after the device and its signals are connected. Use it for setting default signal values, subscriptions, and logic that depends on connected signals.
on_stage()Runs during stage(). Use it to prepare the device for an upcoming scan.
on_unstage()Runs during unstage(). Use it to reset the state of the device after a scan, must be idempotent.
on_pre_scan()Runs right before scan execution starts. Use it for actions that must happen immediately before acquisition.
on_trigger()Runs when the device is triggered. This is typically called during the at_each_point hook from the scan interface in BEC.
on_complete()Runs when BEC checks whether the device has finished. Use it to report if the acquisition finished successfully.
on_kickoff()Runs when a fly-scan style acquisition is started explicitly. Use it for devices that begin a longer-running acquisition.
on_stop()Runs when the device is stopped. Use it to stop the device. It should be fast and non-blocking.
on_destroy()Runs when the device is destroyed. Use it for final cleanup of threads, sockets, or other resources.
+ +## When the hooks run + +The most useful way to understand the hooks is to think about what kind of device logic belongs into each phase of the lifecycle. + +### Initialization + +`on_init()` runs at the end of `__init__()`. + +At this point the Python object exists, but you should assume that signals are not connected yet. This hook is therefore best for local setup such as: + +- creating helper attributes +- initializing caches or configuration state +- preparing task or file-handling helpers + +### Connection + +`on_connected()` is not called from the constructor. Instead, it is called later by the device manager after the device and its signals are connected. + +This is the right place for: + +- setting default signal values +- any setup that needs signal access +- installing callbacks or subscriptions + +### Stage and unstage + +`on_stage()` is the main entry point for scan preparation logic. It is called during `stage()`, which BEC calls before a scan starts. + +- use relevant scan metadata from the `ScanStatusMessage` accessible through `self.scan_info.msg` to set up the device for the specific scan configuration. +- if stage is called twice, it will raise an exception. Therefore you can assume that staging is a deliberate action that only happens once per scan, and is undone by `unstage()`. + +`on_unstage()` is the place to undo temporary scan setup and return the device to a known state. + +- it should be idempotent because BEC calls it during cleanup after a scan, even if staging failed or was interrupted. +- the right place to reset scan-dependent state such as counters, temporary files, or backend resources that should not persist after the scan is over. + +Typical tasks in this phase are applying scan-dependent settings, allocating temporary resources, and resetting transient state once the scan is over. + +### Pre-scan + +`on_pre_scan()` is a hook that BEC adds, which is not part of the standard ophyd `Device` API. It is useful for setup that must happen immediately before scan execution starts on all participating devices. Typical examples are: + +- arming a detector backend +- opening a shutter +- checking readiness of an external controller + +Because this phase sits close to actual acquisition, it is often where devices change from a configured state into an armed or ready state. It is most often also implemented as an asynchronous hook that returns a status object, so BEC can wait for the device to become ready before starting the scan. + +### Trigger and complete + +If a scan is software triggered, `on_trigger()` is the main entry point for acquisition logic. It is called from the `at_each_point` hook in the scan interface, so it runs once per scan point, and should be implemented as an asynchronous hook that returns a status object. + +- trigger the acquisition on the device or its backend +- return a status object that resolves when the acquisition is done + +At the end of a scan, BEC checks whether the device has finished by calling `on_complete()`. This is where the device should report whether acquisition or backend work has actually completed successfully. It is an asynchronous hook that returns a status object. + +- check that the acquisition is done, all files are successfully written and BEC can move on to for example linking files +- Raise an exception if the acquisition failed +- It is typically helpful to implement a timeout logic in this hook, so that if the device or backend hangs, the scan will fail at some point with a meaningful error instead of hanging indefinitely. + +### Kickoff + +`on_kickoff()` is a separate hook fairly similar to `on_pre_scan()`. It is part of the fly-scan interface from ophyd, and is an asynchronous hook that returns a status object. The status should resolved once the kickoff action is done and the device is actively acquiring. + +- use it for a fly-scan acquisition, for example to start a trajectory motion on a controller +- resolve immediately after the controller starts moving, NOT after the move finished. That way, the kickoff status only represents the time it takes to start the acquisition, and the rest of the scan can proceed while the acquisition is still running. Dependening on the logic of your scan, BEC will check for acquisition completion either in `on_complete()` or in a custom method. + +### Stop and destroy + +`on_stop()` should usually be fast and non-blocking. Any cleanup logic needed to stop the device should go here. + +`on_destroy()` is the final cleanup hook. Use it for resources that should disappear when the device object is torn down, such as threads, sockets, or file handles. It should be safe to call and not raise an exception. + +## Status objects and asynchronous work + +The most important design choice when implementing a hook is whether the action is synchronous or asynchronous. + +A synchronous action finishes before the hook returns. In that case, the hook can simply perform the work and return `None`. + +An asynchronous action starts now but finishes later. This is common for the integration of detectors or other more complex devices. In that case, the hook should return a `StatusBase` or `DeviceStatus` object. + +You can think of these status objects as future-like objects. They represent work that is still in progress and will later resolve either: + +- successfully, when the action is done +- with an exception, when the action fails or times out + +BEC waits on the returned status before moving on to the next scan step or lifecycle phase. + +This is the key rule: + +- return `None` only when the hook has finished synchronously +- return a status object when completion happens later + +In practice, a status object answers questions such as: + +- has the detector really become ready yet? +- has the move really completed yet? +- has the acquisition backend returned to idle? + +`PSIDeviceBase` also helps with interruption handling through `cancel_on_stop(status)`. If a scan is stopped, the base class marks registered status objects as failed with `DeviceStoppedError`. That keeps long-running acquisitions from hanging forever after an abort. + +!!! info + + If you interested in more details about status objects, how to create them, and how to use them in custom devices, check out the [Use status objects in a custom ophyd device](../../how-to/devices/development/use-status-objects-in-a-custom-device.md) guide. + +## How scan metadata reaches the device + +Custom devices often need information about the active scan: exposure time, number of frames, scan type, or user metadata. `PSIDeviceBase` stores that context on `self.scan_info`. + +In current BEC scan implementations, the scan object builds a `ScanInfo` model and updates it with values such as: + +- `scan_name` +- `scan_id` +- `num_points` +- `exp_time` +- `frames_per_trigger` +- `settling_time` +- `request_inputs` +- `user_metadata` + +That is why custom devices can make decisions in hooks such as `on_stage()` or `on_pre_scan()` based on the active scan configuration. + +## The role of the device manager + +The constructor of `PSIDeviceBase` accepts `device_manager`. BEC passes this through during device creation so custom devices can resolve other devices or beamline services when needed. + +Many simple devices never need it. It becomes more important for devices that: + +- depend on other devices +- need cross-device coordination +- create pseudo or composite abstractions + +If your device depends on other devices, prefer making that relationship explicit in the config and in the constructor rather than hard-coding session-specific names deep inside the class. + +## BEC-specific signal types + +A custom device can use ordinary ophyd signals, but BEC also provides specialized signal classes for data that should flow through BEC services in a richer way. + +Examples include: + +- `ProgressSignal` for scan progress updates +- `FileEventSignal` for produced files +- `PreviewSignal` for 1D or 2D preview data +- `AsyncSignal` and `AsyncMultiSignal` for asynchronous streams + +These signals carry BEC-native message structures and are especially useful for detectors, streamers, and devices that publish more than simple scalar readbacks. + +!!! learn "[Learn about BEC-specific signal classes](bec-signals.md){ data-preview }" + +## A useful mental model + +When you write a custom device for BEC, think in layers: + +1. ophyd defines the device structure and the signal hierarchy +2. `PSIDeviceBase` defines how that device joins the BEC lifecycle +3. the device config tells BEC how to instantiate the class in one session + +That separation is what makes custom devices reusable. The Python class captures behavior, while the BEC config supplies beamline-specific names, prefixes, and parameters. + +## What to read next + +- [Write a custom ophyd device](../../how-to/devices/development/write-a-custom-device.md) +- [Use status objects in a custom ophyd device](../../how-to/devices/development/use-status-objects-in-a-custom-device.md) +- [BEC signals for custom devices](bec-signals.md) +- [Device config in BEC](device-config-in-bec.md) + +!!! info "What to remember" + - `PSIDeviceBase` is the main integration point for custom ophyd devices in BEC. + - Hook implementations should distinguish clearly between synchronous and asynchronous work. + - Status objects are future-like objects that let BEC wait for device work to complete. + - `CompareStatus` is useful for one target condition, while `TransitionStatus` is useful for ordered state changes. + - `cancel_on_stop(...)` and `wait_for_condition(..., check_stopped=True)` help custom devices stop cleanly. + - `self.scan_info` gives the device access to the active scan context. diff --git a/zensical.toml b/zensical.toml index 93e940f..0b44dca 100644 --- a/zensical.toml +++ b/zensical.toml @@ -82,6 +82,10 @@ nav = [ { "Select a Signal Kind" = "how-to/devices/how-to-select-an-ophyd-kind.md" }, { "Select a Readout Priority" = "how-to/devices/how-to-select-readout-priority.md" }, ] }, + { "Custom Devices" = [ + { "Write a Custom Ophyd Device" = "how-to/devices/development/write-a-custom-device.md" }, + { "Use Status Objects in a Custom Ophyd Device" = "how-to/devices/development/use-status-objects-in-a-custom-device.md" }, + ] }, { "Simulation" = [ { "Use Simulated Models from the IPython Client" = "how-to/devices/use-simulated-models-from-ipython.md" }, ] }, @@ -120,6 +124,7 @@ nav = [ { "EPICS Signal Variants" = "learn/devices/epics-signals.md" }, ] }, { "Development" = [ + { "Custom Ophyd Devices" = "learn/devices/custom-ophyd-devices.md" }, { "Pseudo Positioners" = "learn/devices/pseudo-positioners.md" }, { "BEC Signals" = "learn/devices/bec-signals.md" }, { "Simulated Devices" = "learn/devices/simulated-devices.md" },