diff --git a/docs/how-to/devices/add-a-file-event-signal.md b/docs/how-to/devices/add-a-file-event-signal.md new file mode 100644 index 0000000..1349d37 --- /dev/null +++ b/docs/how-to/devices/add-a-file-event-signal.md @@ -0,0 +1,100 @@ +--- +related: + - title: BEC Signals for Custom Devices + url: learn/devices/bec-signals.md + - title: File writing + url: learn/file-writer/introduction.md + - title: ReadoutPriority in BEC + url: learn/devices/readout-priority.md +--- + +# Add a File Event Signal to a Custom Device + +!!! Info "Overview" + Add a `FileEventSignal` to a custom ophyd device when the real measurement data is written to an external file and BEC must track the output path and completion state. + +## Prerequisites + +- You already have a custom device class in Python. +- Your device writes scan data to a file such as HDF5. +- Your device knows the final output path before or during acquisition. +- Your acquisition flow exposes a completion state or status callback. + +!!! learn "[Learn about BEC signal classes](../../learn/devices/bec-signals.md){ data-preview }" + +## 1. Declare the signal on the device class + +Add `FileEventSignal` as a component on your device class: + +```python +from ophyd import Component as Cpt +from ophyd_devices import FileEventSignal +from ophyd_devices.interfaces.base_classes.psi_device_base import PSIDeviceBase + + +class MyDetector(PSIDeviceBase): + file_event = Cpt(FileEventSignal, name="file_event") +``` + +## 2. Emit the initial file event before acquisition starts + +As soon as the final output path is known, publish an initial file event with `done=False`. + +This is the same pattern used in `csaxs_bec/devices/jungfraujoch/eiger.py`: + +```python +self.file_event.put( + file_path=self._full_path, + done=False, + successful=False, + hinted_h5_entries={"data": "entry/data/data"}, +) +``` + +In many devices this is a good fit for `on_stage()`. + +## 3. Emit the final file event when acquisition completes + +When the asynchronous acquisition finishes, publish a second file event with the resolved completion status. + +```python +def _file_event_callback(self, status: DeviceStatus) -> None: + self.file_event.put( + file_path=self._full_path, + done=status.done, + successful=status.success, + hinted_h5_entries={"data": "entry/data/data"}, + ) +``` + +Attach this callback to the status object returned by your asynchronous acquisition or completion logic. + +## 4. Provide HDF5 dataset hints when applicable + +For HDF5-based detectors, set `hinted_h5_entries` so downstream BEC components know where the primary data lives inside the file. + +Example: + +```python +hinted_h5_entries={"data": "entry/data/data"} +``` + +This is especially important for downstream file linking and data discovery. + +## 5. Verify the file event flow + +Run a short acquisition and confirm that: + +- the initial file event is emitted with the expected `file_path` +- the final file event is emitted after completion +- `successful` reflects the actual acquisition outcome +- the path points to the final file, not a temporary staging location + +If your file format or downstream consumer needs extra context, include fields such as `file_type` or `metadata`. + +## 6. Use the file event in BEC +With the file event signal in place, BEC can now track the produced file and its completion state alongside the rest of the acquisition. For more information on how to use files in BEC and how to link them, see the [File Writer documentation](../../learn/file-writer/introduction.md). + + +!!! success "Congratulations!" + You have successfully added a `FileEventSignal` to a custom device. BEC can now track the produced file and its completion state alongside the rest of the acquisition. diff --git a/docs/how-to/devices/add-a-preview-signal.md b/docs/how-to/devices/add-a-preview-signal.md new file mode 100644 index 0000000..f4dfe4f --- /dev/null +++ b/docs/how-to/devices/add-a-preview-signal.md @@ -0,0 +1,100 @@ +--- +related: + - title: BEC Signals for Custom Devices + url: learn/devices/bec-signals.md + - title: Introduction to ophyd + url: learn/devices/introduction-to-ophyd.md + - title: ReadoutPriority in BEC + url: learn/devices/readout-priority.md +--- + +# Add a Preview Signal to a Custom Device + +!!! Info "Overview" + Add a `PreviewSignal` to a custom ophyd device when you want BEC to forward live 1D or 2D preview data such as images, spectra, or monitor streams. + +## Prerequisites + +- You already have a custom device class in Python. +- Your device receives preview-like data from a callback, background thread, stream, or external client. +- You know whether the preview data is 1D or 2D. + +!!! learn "[Learn about BEC signal classes](../../learn/devices/bec-signals.md){ data-preview }" + +## 1. Declare the signal on the device class + +Add `PreviewSignal` as a component on your device class. + +Use `ndim=2` for images: + +```python +from ophyd import Component as Cpt +from ophyd_devices import PreviewSignal +from ophyd_devices.interfaces.base_classes.psi_device_base import PSIDeviceBase + + +class MyDetector(PSIDeviceBase): + preview_image = Cpt(PreviewSignal, name="preview_image", ndim=2) +``` + +Use `ndim=1` instead for line-like previews such as spectra. + +## 2. Connect it to your preview callback + +When your device receives new preview data, extract the payload first and then publish it with `put(...)`. + +`PreviewSignal` accepts preview data in multiple forms: + +- a Python `list` +- a NumPy array +- a fully constructed `DevicePreviewMessage` + + +```python +def _preview_callback(self, message: dict) -> None: + if message.get("type", "") != "image": + return + + data = message.get("data", {}).get("default") + if data is None: + return + + self.preview_image.put(data) +``` + +You first receive or assemble the preview payload in your callback or stream handler, and only then set it on the signal with `put(...)`. + +If you pass a list or NumPy array, `PreviewSignal` wraps it into the BEC preview message type for you. If you already have a `DevicePreviewMessage`, you can pass that directly. + +!!! warning "Stay aware of the data rate" + When connecting a `PreviewSignal`, make sure that the incoming preview data rate is not too high. The purpose of a preview is to provide a lightweight stream that can be forwarded to GUIs and clients without overwhelming the network or BEC. If your preview data arrives at a very high rate, consider adding throttling or decimation logic in your callback before sending it to the signal. + +## 3. Adjust orientation if needed + +If your upstream image arrives in a different orientation than you want to show in BEC, configure the signal declaration with `transpose` or `num_rotation_90`. + +Example: + +```python +preview_image = Cpt( + PreviewSignal, + name="preview_image", + ndim=2, + transpose=True, + num_rotation_90=1, +) +``` + +## 4. Verify the preview in BEC + +Start the device, trigger the upstream preview source, and confirm that the preview appears in the relevant BEC client or GUI. + +If nothing appears: + +- verify that your callback is being called +- verify that `data` is not `None` +- verify that `ndim` matches the actual payload shape +- verify that the device is configured with a suitable `readoutPriority` + +!!! success "Congratulations!" + You have successfully added a `PreviewSignal` to a custom device. BEC can now forward your live preview stream independently of the final stored data. diff --git a/docs/how-to/devices/add-a-progress-signal.md b/docs/how-to/devices/add-a-progress-signal.md new file mode 100644 index 0000000..c6eca65 --- /dev/null +++ b/docs/how-to/devices/add-a-progress-signal.md @@ -0,0 +1,100 @@ +--- +related: + - title: BEC Signals for Custom Devices + url: learn/devices/bec-signals.md + - title: ReadoutPriority in BEC + url: learn/devices/readout-priority.md + - title: Introduction to ophyd + url: learn/devices/introduction-to-ophyd.md +--- + +# Add a Progress Signal to a Custom Device + +!!! Info "Overview" + Add a `ProgressSignal` to a custom ophyd device when you want BEC to track scan progress from that device, for example while data is being acquired or processed asynchronously. + +## Prerequisites + +- You already have a custom device class in Python. +- Your device has a natural progress measure such as completed triggers, completed frames, or processed points. +- You know the current value and the expected maximum value during the operation. + +!!! learn "[Learn about BEC signal classes](../../learn/devices/bec-signals.md){ data-preview }" + +## 1. Declare the signal on the device class + +Add `ProgressSignal` as a component on your device class: + +```python +from ophyd import Component as Cpt, Device +from ophyd_devices import ProgressSignal + + +class MyDetector(Device): + progress = Cpt(ProgressSignal, name="progress") +``` + +This creates one signal that emits BEC progress messages. + +## 2. Decide what progress means for the device + +Choose one quantity that moves toward completion in a clear way. + +Typical examples are: + +- completed frames out of total frames +- completed scan points out of total points +- processed events out of total events + +The most common pattern is to send: + +- the current value +- the maximum value +- whether the operation is done + +## 3. Update the signal during runtime + +When the device makes progress, send a progress update with `put(...)`. + +Example: + +```python +self.progress.put(value=25, max_value=100, done=False) +``` + +When the operation is finished: + +```python +self.progress.put(value=100, max_value=100, done=True) +``` + +## 4. Connect it to your callback or worker loop + +`ProgressSignal` is usually updated from a callback, subscription, worker thread, or polling loop. + +Typical pattern: + +```python +def _update_progress(self, completed, total): + self.progress.put( + value=completed, + max_value=total, + done=completed >= total, + ) +``` + +This keeps the progress signal close to the logic that already knows how much work has been completed. + +## 5. Verify the progress updates + +Run a short acquisition and confirm that: + +- the device sends progress updates while work is ongoing +- `value` increases in the expected direction +- `max_value` is stable and meaningful +- `done` becomes `true` when the operation finishes + +If progress does not appear in BEC, first check whether the callback or worker loop that computes progress is actually running. + +!!! success "Congratulations!" + You have successfully added a `ProgressSignal` to a custom device. BEC can now track the device's runtime progress during the scan. diff --git a/docs/how-to/devices/add-an-async-multi-signal.md b/docs/how-to/devices/add-an-async-multi-signal.md new file mode 100644 index 0000000..41550e5 --- /dev/null +++ b/docs/how-to/devices/add-an-async-multi-signal.md @@ -0,0 +1,114 @@ +--- +related: + - title: BEC Signals for Custom Devices + url: learn/devices/bec-signals.md + - title: ReadoutPriority in BEC + url: learn/devices/readout-priority.md + - title: Introduction to ophyd + url: learn/devices/introduction-to-ophyd.md +--- + +# Add an Async Multi Signal to a Custom Device + +!!! Info "Overview" + Add an `AsyncMultiSignal` to a custom ophyd device when the device produces asynchronous data for multiple named channels or fields and you want BEC to forward that stream as one grouped signal. + +## Prerequisites + +- You already have a custom device class in Python. +- Your device produces asynchronous data outside the normal `read()` path. +- You know the fixed set of sub-signal names you want to expose in BEC. +- You can transform each incoming sample into a dictionary keyed by those sub-signal names. + +!!! learn "[Learn about BEC signal classes](../../learn/devices/bec-signals.md){ data-preview }" + +## 1. Declare the signal on the device class + +Declare one `AsyncMultiSignal` component and list all sub-signal names explicitly. + +```python +from ophyd import Component as Cpt, Device +from ophyd_devices import AsyncMultiSignal + + +class MyFlyer(Device): + data = Cpt( + AsyncMultiSignal, + name="data", + signals=["target_x", "target_y"], + ndim=1, + async_update={"type": "add", "max_shape": [None]}, + max_size=1000, + ) +``` + +This defines one grouped async signal with two sub-signals: + +- `target_x` +- `target_y` + +The `async_update` metadata is part of the signal definition. In this example, each new update is appended to the async stream. + +`max_size` controls how much async data BEC keeps in memory for that signal. + +The resulting data output for a device named `my_flyer` will look like: + +```json +{ + "my_flyer_data_target_x": {"value": ...}, + "my_flyer_data_target_y": {"value": ...} +} +``` + +## 2. Receive the async data first + +`AsyncMultiSignal` is meant for data that arrives asynchronously from a callback, subscription, socket, thread, or other background source. + +Typical pattern: + +```python +def _get_async_data(self): + return { + "target_x": {"value": 10}, + "target_y": {"value": 20}, + } +signals = self._get_async_data() +self.data.put(signals) +``` + +The important sequence is: + +1. receive one async sample from the hardware or upstream source +2. convert it into the required signal dictionary +3. call `put` with that dictionary on the `AsyncMultiSignal` + +## 3. Choose the async update mode + +`AsyncMultiSignal` requires async update metadata describing how new data should be handled. + +Example: + +```python +async_update={"type": "add", "max_shape": [None]} +``` + +Define this once on the signal declaration. After that, the device code should normally call `self.data.put(signals)` without repeating `async_update` on every update. + +The main exception is `add_slice`: if the slice `index` changes between updates, pass updated `async_update` metadata with the individual `put(...)` call. + +!!! note "Learn about async update modes" + See [BEC Signals for Custom Devices](../../learn/devices/bec-signals.md#async-update-metadata) for details on available async update types and their behavior. + +## 4. Verify the data stream + +Run a short acquisition and confirm that: + +- your async callback or background reader is receiving samples +- each sample is converted into a dictionary with valid signal names +- all keys match the declared `signals` list +- `self.data.put(signals)` is called after the sample has been assembled + +If no async data appears in BEC, first check for signal-name mismatches and incomplete payload dictionaries. + +!!! success "Congratulations!" + You have successfully added an `AsyncMultiSignal` to a custom device. BEC can now forward a structured asynchronous stream with multiple named sub-signals from your device. diff --git a/docs/how-to/devices/add-an-async-signal.md b/docs/how-to/devices/add-an-async-signal.md new file mode 100644 index 0000000..a13a5c2 --- /dev/null +++ b/docs/how-to/devices/add-an-async-signal.md @@ -0,0 +1,121 @@ +--- +related: + - title: BEC Signals for Custom Devices + url: learn/devices/bec-signals.md + - title: ReadoutPriority in BEC + url: learn/devices/readout-priority.md + - title: Introduction to ophyd + url: learn/devices/introduction-to-ophyd.md +--- + +# Add an Async Signal to a Custom Device + +!!! Info "Overview" + Add an `AsyncSignal` to a custom ophyd device when the device produces one asynchronous data stream and you want BEC to forward it during a scan. + +## Prerequisites + +- You already have a custom device class in Python. +- Your device produces asynchronous data outside the normal `read()` path. +- You know the dimensionality of the async payload. +- You know how new async updates should be written into the dataset. + +!!! learn "[Learn about BEC signal classes](../../learn/devices/bec-signals.md){ data-preview }" + +## 1. Declare the signal on the device class + +Declare one `AsyncSignal` component on the device. + +```python +from ophyd import Component as Cpt, Device +from ophyd_devices import AsyncSignal + + +class MyDetector(Device): + waveform = Cpt( + AsyncSignal, + name="waveform", + ndim=1, + async_update={"type": "add", "max_shape": [None, 1024]}, + max_size=1000, + ) +``` + +This defines one asynchronous signal named `waveform`. + +- `ndim` describes the payload dimensionality +- `async_update` tells BEC how incoming updates should be stored +- `max_size` controls how much async data BEC keeps in memory for that signal + +## 2. Define `async_update` on the signal + +In normal usage, define `async_update` once on the signal declaration. + +Example for a stream of fixed-length waveforms: + +```python +async_update={"type": "add", "max_shape": [None, 1024]} +``` + +This means each new update adds one more waveform of length `1024`. + +!!! note "When to override it" + For `add_slice`, the slice `index` may need to change between updates. In that case, pass updated `async_update` metadata with the individual `put(...)` call. + +!!! note "Learn about update modes" + See [BEC Signals for Custom Devices](../../learn/devices/bec-signals.md#async-update-metadata) for details on `add`, `add_slice`, and `replace`. + +## 3. Receive the async data first + +`AsyncSignal` is meant for data that arrives asynchronously from a callback, subscription, socket, thread, or other background source. + +Typical pattern: + +```python +def _get_waveform(self): + return [1.0, 2.0, 3.0, 4.0] + + +values = self._get_waveform() +self.waveform.put(values) +``` + +The important sequence is: + +1. receive one async sample from the hardware or upstream source +2. convert it into the payload you want to send +3. call `put(...)` on the `AsyncSignal` + +## 4. Handle `add_slice` if needed + +If the signal uses `add_slice` and the slice `index` changes during runtime, send the updated metadata with the individual update. + +Example: + +```python +self.waveform.put( + first_chunk, + async_update={"type": "add_slice", "index": 0, "max_shape": [None, 1024]}, +) + +self.waveform.put( + second_chunk, + async_update={"type": "add_slice", "index": 1, "max_shape": [None, 1024]}, +) +``` + +If your device naturally emits one complete new waveform, row, or image per update, `add` is usually simpler than `add_slice`. + +## 5. Verify the data stream + +Run a short acquisition and confirm that: + +- your callback or background reader is receiving async data +- the payload shape matches the declared signal +- the signal receives updates with `put(...)` +- the chosen `async_update` mode matches the intended dataset layout + +If no async data appears in BEC, first check whether the payload shape and `async_update` configuration match each other. + +!!! success "Congratulations!" + You have successfully added an `AsyncSignal` to a custom device. BEC can now forward one asynchronous data stream from your device during the scan. diff --git a/docs/learn/devices/bec-signals.md b/docs/learn/devices/bec-signals.md index bcdcf9b..2876dc0 100644 --- a/docs/learn/devices/bec-signals.md +++ b/docs/learn/devices/bec-signals.md @@ -2,82 +2,304 @@ related: - title: File writing url: learn/file-writer/introduction.md + - title: Add a Preview Signal + url: how-to/devices/add-a-preview-signal.md + - title: Add a File Event Signal + url: how-to/devices/add-a-file-event-signal.md + - title: Add an Async Multi Signal + url: how-to/devices/add-an-async-multi-signal.md --- # BEC Signals for Custom Devices -`ophyd_devices.utils.bec_signals` provides convenience signal classes that wrap ophyd signals with BEC-native message types. They are designed to forward preview data, file events, progress updates, and asynchronous signal streams into BEC. +Ophyd devices typically use `Signal` and `Component` objects to communicate with hardware. `BECSignal`s are building their counterparts designed to communicate with BEC instead of hardware. This allows devices to send structured messages and inform BEC about e.g. progress updates, file events, and asynchronous data streams. -## Exported signal classes +!!! info "BEC Signals are internal components" + BEC signals are not meant to be read out directly through the command-line interface. They are internal components that allow devices to send structured messages to BEC. -The module exports the following classes: +## Signal classes -- `ProgressSignal` -- `FileEventSignal` -- `PreviewSignal` -- `DynamicSignal` -- `AsyncSignal` -- `AsyncMultiSignal` +BEC provides several signal classes for different use cases. Some of them are designed for broadcasting runtime information such as progress updates and file events, while others are designed for streaming asynchronous data into the scan dataset and are therefore saved to HDF5. -## Core behavior + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Signal classDescriptionSaved to HDF5?
ProgressSignalUsed to emit a device-related progress using a ProgressMessage.No
FileEventSignalUsed to report external file-writing events using a FileMessage.Yes, as file references
PreviewSignalUsed to stream 1D or 2D preview data using a DevicePreviewMessage.No
AsyncSignalUsed to represent a single asynchronous data channel with async-update metadata. Every update must provide the specified signal.Yes
AsyncMultiSignalUsed to represent multiple asynchronous sub-signals with strict signal-name validation. Every update must provide all specified signals.Yes
DynamicSignalSimilar to AsyncMultiSignal but an update does not require all specified signals.Yes
-All BEC signal classes build on `BECMessageSignal`, which adds: +`FileEventSignal` does not store the detector payload itself in the scan dataset. Instead, BEC records file references in the master HDF5 file so externally written data can be linked there. -- BEC message validation (`BECMessage` subclasses) -- standardized `signal_info` metadata via `describe()` -- conversion support from dictionaries to typed BEC messages -- immediate completion semantics for `set(...)` +## Usage and metadata -`SignalInfo` metadata includes fields such as `data_type`, `saved`, `ndim`, `scope`, `role`, `signals`, `signal_metadata`, and `acquisition_group`. +BEC signals are defined as normal ophyd components: -## Signal classes in practice +```python +class MyDevice(Device): + progress = Cpt(ProgressSignal, name="progress") + file_event = Cpt(FileEventSignal, name="file_event") + preview = Cpt(PreviewSignal, name="preview", ndim=2, max_shape=[1024, 1024]) + async_signal = Cpt( + AsyncSignal, + name="async_signal", + ndim=1, + max_size=1000, + async_update={"type": "add", "max_shape": [None, 1000]}, + ) + async_multi_signal = Cpt( + AsyncMultiSignal, + name="async_multi_signal", + signals=["temperature", "pressure"], + async_update={"type": "add", "max_shape": [None, None]}, + ) +``` -### `PreviewSignal` +and then used from within the device methods: -Use this to stream 1D or 2D preview data (for example beam monitor cameras). It emits `DevicePreviewMessage` and supports optional orientation correction: +```python +self.progress.put(value=50, max_value=100, done=False) +``` -- `num_rotation_90`: rotate 2D data before publishing -- `transpose`: transpose 2D data before publishing +The exact signature depends on the signal class but in general, the `put(...)` can be used to pass in either the data values directly or a structured message object. -You can publish arrays directly with `preview.put(array_data)`. +For more details on how to use each signal class, see the following how-to guides: -### `ProgressSignal` +- [Add a Progress Signal](../../how-to/devices/add-a-progress-signal.md) +- [Add a File Event Signal](../../how-to/devices/add-a-file-event-signal.md) +- [Add a Preview Signal](../../how-to/devices/add-a-preview-signal.md) +- [Add an Async Signal](../../how-to/devices/add-an-async-signal.md) +- [Add an Async Multi Signal](../../how-to/devices/add-an-async-multi-signal.md) -Use this for scan-linked progress updates. It emits `ProgressMessage` and can be updated by either: +## Async update metadata -- `progress.put(msg=...)` -- `progress.put(value=..., max_value=..., done=..., metadata=...)` +`AsyncSignal`, `AsyncMultiSignal`, and `DynamicSignal` need `async_update` metadata so BEC knows how each async update should be aggregated into the scan dataset. -### `FileEventSignal` +The async update metadata supports three modes: -Use this when a device writes external files. It emits `FileMessage` and supports: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ModeDescriptionRequired metadata
addAppend each new update along the first axis of the dataset.type, max_shape
add_sliceWrite each new update into a specific slice of a larger dataset.type, index, max_shape
replaceReplace the current dataset with each new async update instead of extending it.type
-- `file_path` -- `done` -- `successful` -- `file_type` -- `hinted_h5_entries` -- `metadata` -`hinted_h5_entries` is important for downstream file linking in BEC file writing. -### `DynamicSignal` +In normal usage, define `async_update` once on the signal declaration and then send only data values at runtime. -General signal group for `DeviceMessage` payloads with named sub-signals and metadata. It validates signal names and async update metadata. +```python +waveform = Cpt( + AsyncSignal, + name="waveform", + ndim=1, + max_size=1000, + async_update={"type": "add", "max_shape": [None, 1024]}, +) -### `AsyncSignal` +self.waveform.put(values) +``` -Specialized dynamic signal for one asynchronous channel. Requires async-update metadata describing how data is appended or replaced. -### `AsyncMultiSignal` +### `add` -Specialized dynamic signal for multiple asynchronous sub-signals, with strict signal-name validation. +Use `add` when every new update should be appended along the first axis. -## How device-server callbacks route these signals +Typical uses: -In `bec_server.device_server.bec_message_handler.BECMessageHandler.emit`, callback dispatch is based on signal type: +- one growing 1D stream +- a sequence of fixed-length waveforms +- a sequence of variable-length 1D datasets +- a sequence of images -- `FileEventSignal` -> `_handle_file_event_signal` -- `ProgressSignal` -> `_handle_progress_signal` -- `PreviewSignal` -> `_handle_preview_signal` -- `DynamicSignal` (therefore also `AsyncSignal` and `AsyncMultiSignal`) -> `_handle_async_signal` +Example definitions: + +```python +# growing 1D stream +async_update={"type": "add", "max_shape": [None]} + +# stream of fixed-length waveforms +async_update={"type": "add", "max_shape": [None, 1024]} + +# stream of variable-length 1D datasets +async_update={"type": "add", "max_shape": [None, None]} + +# stream of fixed-size images +async_update={"type": "add", "max_shape": [None, 512, 512]} +``` + +What is required: + +- `type="add"` +- `max_shape` + +How to read the `max_shape` argument: + +- `[None]` means one unlimited 1D stream +- `[None, 1024]` means an unlimited number of rows, each of length `1024` +- `[None, None]` means an unlimited number of 1D datasets with varying length +- `[None, 512, 512]` means an unlimited number of images, each `512 x 512` + +When to use it: + +- when each update is one new value block, row, waveform, or image +- when you do not need to target a specific slice index + +!!! warning "Prefer fixed inner sizes when possible" + `[None, None]` is supported for a dataset that is an array of 1D datasets with varying length. However, whenever possible, prefer a fixed inner size such as `[None, 1024]`. Variable-length inner data is the most inefficient way of writing the dataset. + +### `add_slice` + +Use `add_slice` when async updates should be written into a specific slice of a larger dataset: Rather than appending an entire new row, each update fills a slice of the specified `index` along the first axis. + +Example definition: + +```python +waveform = Cpt( + AsyncSignal, + name="waveform", + ndim=1, + max_size=1000, + async_update={"type": "add_slice", "index": 0, "max_shape": [None, 20]}, +) +``` + +Example runtime updates for building up a 2D dataset, one slice at a time: + +```python +# fill the first slice (index 0) +self.waveform.put( + [1, 2, 3, 4, 5, 6, 7, 8, 9, 10], + async_update={"type": "add_slice", "index": 0, "max_shape": [None, 20]} +) + +# append to the first slice (index 0) +self.waveform.put( + [11, 12, 13, 14, 15, 16, 17, 18, 19, 20], + async_update={"type": "add_slice", "index": 0, "max_shape": [None, 20]} +) + +# fill in the second slice (index 1) +self.waveform.put( + [21, 22, 23, 24, 25, 26, 27, 28, 29, 30], + async_update={"type": "add_slice", "index": 1, "max_shape": [None, 20]} +) +``` + + +What is required: + +- `type="add_slice"` +- `max_shape` +- `index` + +When to use it: + +- when updates belong to a specific slice index +- when one logical row or slice is filled over multiple updates + +!!! warning "Specify the slice index on each update" + In contrast to `add` and `replace`, `add_slice` updates require the `async_update` metadata to be passed on each `put(...)` call, because the slice `index` may change between updates. + + +### `replace` + +Use `replace` when each new async update should replace the current dataset instead of extending it. + +Example definition: + +```python +result = Cpt( + AsyncSignal, + name="result", + ndim=1, + max_size=10, + async_update={"type": "replace"}, +) +``` + +At runtime: + +```python +self.result.put(latest_result) +``` + +What is required: + +- `type="replace"` + +When to use it: + +- when the device publishes a refreshed full result +- when old async data should be superseded instead of extended + +!!! info "What to remember" + - `BECSignal` classes are ophyd-style components for sending structured messages from devices into BEC instead of talking directly to hardware. + - Choose the signal class based on the kind of information your device publishes: progress updates, file events, preview data, or asynchronous data streams. + - Declare BEC signals on the device class like normal ophyd components, then publish updates with `put(...)` from the device code. + - `AsyncSignal`, `AsyncMultiSignal`, and `DynamicSignal` need `async_update` metadata so BEC knows how incoming async data should be written into the scan dataset. + - For async streams, use `add` by default, use `add_slice` for indexed slice updates, and use `replace` when each update should overwrite the previous result. diff --git a/docs/learn/devices/device-config-in-bec.md b/docs/learn/devices/device-config-in-bec.md index ed68f1b..9b48215 100644 --- a/docs/learn/devices/device-config-in-bec.md +++ b/docs/learn/devices/device-config-in-bec.md @@ -161,3 +161,10 @@ initialization. Defaults to `5.0` seconds. - That effective configuration becomes the basis of the current device session when BEC loads it. !!! learn "[Learn more about device sessions and device-server initialization](device-sessions-in-bec.md){ data-preview }" + +!!! info "What to remember" + - A BEC device configuration entry tells the device server which class to build and how to manage it. + - `enabled`, `deviceClass`, and `readoutPriority` are the required fields. + - `deviceConfig` carries the class-specific constructor arguments for the selected device class. + - Optional fields such as `needs`, `connectionTimeout`, `readOnly`, and `onFailure` control runtime behavior in BEC. + - One device entry becomes part of a larger effective configuration, which BEC then turns into the active device session. diff --git a/docs/learn/devices/device-sessions-in-bec.md b/docs/learn/devices/device-sessions-in-bec.md index 6396969..3b69703 100644 --- a/docs/learn/devices/device-sessions-in-bec.md +++ b/docs/learn/devices/device-sessions-in-bec.md @@ -74,6 +74,13 @@ Any BEC client or service is subscribed to the published device information in R The architecture of BEC allows for multiple clients and services to share access to the same devices. This requires coordination and control, which is achieved through this concept of a shared device session, and *RPC objects* in the clients. Any client has access to all devices, while the actual device object and connection is handled by the device server. The device server thereby provides a layer of abstraction and control, which allows for a controlled and shared access to devices across multiple clients and services. +!!! info "What to remember" + - A device configuration, a YAML configuration file, and a device session are related but different concepts. + - The device session is the currently active shared set of devices in BEC, and it can differ from the YAML file that was originally loaded. + - Building a device session means initializing ophyd objects, connecting their signals, and publishing the device interface to Redis. + - Clients interact with RPC objects that represent the server-side devices, not with the original ophyd objects directly. + - Initialization failures abort the whole session load, while connection failures disable only the affected devices. + ## What to learn next - Continue with [Device Configuration in BEC](../../learn/devices/device-config-in-bec.md) to learn the individual fields in a device entry. diff --git a/docs/learn/devices/epics-motors.md b/docs/learn/devices/epics-motors.md index b2040ab..0428348 100644 --- a/docs/learn/devices/epics-motors.md +++ b/docs/learn/devices/epics-motors.md @@ -22,3 +22,8 @@ The right choice depends on your EPICS motor implementation. - If you have a VME user motor, choose `ophyd_devices.EpicsUserMotorVME` to get VME-specific signals and behavior. - For everything else, start with `ophyd_devices.EpicsMotor` as the normal default. +!!! info "What to remember" + - BEC provides several EPICS motor classes because different motor backends expose different behavior and signals. + - `ophyd_devices.EpicsMotor` is the normal default choice. + - Use `ophyd_devices.EpicsMotorEC` for ECMC-based motors and `ophyd_devices.EpicsUserMotorVME` for VME user motors. + - Choosing the closest matching motor class gives you the right backend-specific interface in BEC. diff --git a/docs/learn/devices/epics-signals.md b/docs/learn/devices/epics-signals.md index e5d4693..123c561 100644 --- a/docs/learn/devices/epics-signals.md +++ b/docs/learn/devices/epics-signals.md @@ -29,3 +29,9 @@ Each class expects a slightly different `deviceConfig` section in the BEC config - `ophyd_devices.EpicsSignal`: `read_pv`, and optionally `write_pv` if the write PV differs from the read PV - `ophyd_devices.EpicsSignalRO`: `read_pv` - `ophyd_devices.EpicsSignalWithRBV`: `prefix` + +!!! info "What to remember" + - Use EPICS signal classes when you want to expose a single EPICS PV in BEC rather than a full motor device. + - Choose `EpicsSignal` for read/write PVs, `EpicsSignalRO` for read-only PVs, and `EpicsSignalWithRBV` when setpoint and readback follow the usual RBV pattern. + - The correct `deviceConfig` fields depend on the chosen class. + - Picking the right EPICS signal variant keeps the BEC device model aligned with the underlying EPICS record behavior. diff --git a/docs/learn/devices/error-handling-during-session-updates.md b/docs/learn/devices/error-handling-during-session-updates.md index 412ade0..8653a16 100644 --- a/docs/learn/devices/error-handling-during-session-updates.md +++ b/docs/learn/devices/error-handling-during-session-updates.md @@ -80,3 +80,10 @@ To reduce the chance of critical failures, validate YAML files before loading th !!! learn "[Learn how to load and save a device session from the BEC IPython client](../../how-to/devices/load-and-save-a-device-session-from-the-bec-ipython-client.md){ data-preview }" !!! learn "[Learn how to validate a YAML configuration file before loading it](../../how-to/devices/validate-a-yaml-config-file.md){ data-preview }" + +!!! info "What to remember" + - BEC treats device creation failures and device connection failures differently during session updates. + - If BEC cannot create the Python object for a device, the session update is aborted. + - If a device object is created but cannot connect in time, that device is disabled while the rest of the session can still load. + - Cancelling a session upload resets the active session to a clean state rather than keeping a partially updated configuration. + - Validating YAML files and setting realistic `connectionTimeout` values helps avoid avoidable update failures. diff --git a/docs/learn/devices/managing-yaml-configs.md b/docs/learn/devices/managing-yaml-configs.md index bbbf046..4f11ddd 100644 --- a/docs/learn/devices/managing-yaml-configs.md +++ b/docs/learn/devices/managing-yaml-configs.md @@ -51,3 +51,9 @@ For more task-focused guides, take a look at the following how-tos: - [Load and save a device session from the BEC IPython client](../../how-to/devices/load-and-save-a-device-session-from-the-bec-ipython-client.md) - [Validate a YAML configuration file for BEC](../../how-to/devices/validate-a-yaml-config-file.md) + +!!! info "What to remember" + - Large BEC device setups can be split across multiple YAML files instead of living in one file. + - BEC supports `!include` so smaller config files can be composed into one effective configuration. + - The effective configuration, not any single source file, is what BEC loads into the active device session. + - Separating configs by subsystem or beamline area makes large device setups easier to maintain. diff --git a/docs/learn/devices/ophyd-kinds.md b/docs/learn/devices/ophyd-kinds.md index 515d255..dbfa110 100644 --- a/docs/learn/devices/ophyd-kinds.md +++ b/docs/learn/devices/ophyd-kinds.md @@ -101,3 +101,9 @@ The `Kind` attribute determines which signals are included in `device.read()` an `device.read_configuration()`. To understand when BEC calls these methods during data acquisition, see [*Readout Priority*](../../learn/devices/readout-priority.md){ data-preview } or the how-to guide on [*Select a readout priority*](../../how-to/devices/how-to-select-readout-priority.md){ data-preview }. + +!!! info "What to remember" + - `Kind` controls whether a signal participates in `read()`, `read_configuration()`, or neither. + - `normal` and `hinted` signals appear in `read()`, while `config` signals appear in `read_configuration()`. + - `omitted` signals are hidden from both standard reads but can still be accessed directly. + - Parent and child `Kind` values combine when you use sub-devices, which can change what is visible from the root device. diff --git a/docs/learn/devices/pseudo-positioners.md b/docs/learn/devices/pseudo-positioners.md index 717f7dc..01f85fa 100644 --- a/docs/learn/devices/pseudo-positioners.md +++ b/docs/learn/devices/pseudo-positioners.md @@ -124,3 +124,9 @@ The underlying real devices must provide: - `setpoint` or `user_setpoint` - `motor_is_moving` - `move()` + +!!! info "What to remember" + - A pseudo positioner exposes a derived coordinate while delegating motion to one or more real motors. + - `PSIPseudoMotorBase` gives you the shared pseudo-motor behavior, but you must provide the coordinate transform logic yourself. + - `forward_calculation()`, `inverse_calculation()`, and `motors_are_moving()` must match the configured `positioners` mapping. + - The pseudo motor depends on real devices that provide readable positions, writable targets, motion state, and `move()`. diff --git a/docs/learn/devices/readout-priority.md b/docs/learn/devices/readout-priority.md index a6b7469..83e4e41 100644 --- a/docs/learn/devices/readout-priority.md +++ b/docs/learn/devices/readout-priority.md @@ -10,10 +10,6 @@ related: `readoutPriority` is a key part of the device configuration that controls when BEC reads a device during a scan. It is independent of the ophyd `Kind` attribute, which controls which signals are included in those `device.read()` or `device.read_configuration()` operations. Together they determine which data will be included in scan data, configuration data or excluded. -!!! info "Readout Priority vs. ophyd Kind" - - `Kind` controls which signals are included in `read()` and `read_configuration()`. - - `readoutPriority` controls if a device's `read()` is requested during a scan. - ## Readout Priority Options BEC supports four readout priorities that can be assigned to devices in the configuration. The @@ -33,11 +29,9 @@ If the user requests to scan a motor that is not configured as `monitored`, the - Choose `on_request` for devices that should only be read when explicitly requested. - Choose `async` for devices like large-area detectors that produce asynchronous data streams. -!!! info "Keep `readoutPriority` separate from ophyd `Kind`" - - `Kind` controls which signals a device exposes through `read()` and - `read_configuration()`. `readoutPriority` controls when BEC asks that device for - those readings during a scan. In other words, `Kind` defines the contents of a - device readout, while `readoutPriority` defines when that readout participates in - acquisition. This separation lets you adjust scan behavior without changing the - device's underlying read interface. +!!! info "What to remember" + - `readoutPriority` controls when BEC reads a device during acquisition. + - Keep `readoutPriority` separate from ophyd `Kind`: `Kind` defines which signals appear in `read()` and `read_configuration()`, while `readoutPriority` defines when BEC asks for those readings during a scan. + - `monitored`, `baseline`, `on_request`, and `async` serve different scan roles and should be chosen intentionally. + - Scan motors may be promoted to `monitored` for a specific scan even if their static config says otherwise. + - `readoutPriority` and ophyd `Kind` solve different problems: timing versus content. diff --git a/docs/learn/devices/simulated-devices.md b/docs/learn/devices/simulated-devices.md index 0e308f4..4537094 100644 --- a/docs/learn/devices/simulated-devices.md +++ b/docs/learn/devices/simulated-devices.md @@ -96,3 +96,9 @@ bpm4i: Use `sim_init` when a simulated device should always start with the same model in a beamline, plugin, or test configuration. + +!!! info "What to remember" + - Simulated devices use a simulation object to generate readback values instead of real hardware. + - Simulation controls are available only for device classes that expose a `sim` interface to the client. + - Runtime changes through `dev..sim` are best for interactive exploration and temporary testing. + - Use `sim_init` in the device configuration when a simulated device should always start with a specific model and parameter set. diff --git a/docs/learn/file-writer/async-and-sync-writers.md b/docs/learn/file-writer/async-and-sync-writers.md index c6c9fb2..efb37f6 100644 --- a/docs/learn/file-writer/async-and-sync-writers.md +++ b/docs/learn/file-writer/async-and-sync-writers.md @@ -36,3 +36,11 @@ The async writer supports multiple async update modes to accommodate different d - `add` - `add_slice` - `replace` + +To learn how to use these modes, see [BEC Signals](../devices/bec-signals.md){data-preview}. + +!!! info "What to remember" + - BEC uses both sync writing and async writing during normal operation. + - Sync writing creates the final master file after the scan from data collected in scan storage. + - Async writing handles device data that arrives continuously and may be too large or irregular to buffer until the end of the scan. + - Async update modes such as `add`, `add_slice`, and `replace` define how incoming async data is aggregated in the file. diff --git a/docs/learn/file-writer/default-format.md b/docs/learn/file-writer/default-format.md index 389e7be..0319c32 100644 --- a/docs/learn/file-writer/default-format.md +++ b/docs/learn/file-writer/default-format.md @@ -59,3 +59,9 @@ Some common operations include: - add datasets with `create_dataset(...)` - add data through links, either soft links to existing groups or datasets with `create_soft_link(...)` or external links to datasets in external files with `create_ext_link(...)`. - use helper methods such as `self.get_entry(...)` to access device values from scan data + +!!! info "What to remember" + - `DefaultFormat` defines the built-in base HDF5 layout that BEC relies on for scan data and metadata. + - BEC creates the core `/entry/collection/...` structure before any custom formatting runs. + - Custom file writers should extend the existing storage tree rather than overwrite BEC-managed groups. + - Readout groups, configuration data, file references, and beamline states are all part of the default structure. diff --git a/docs/learn/file-writer/file-references.md b/docs/learn/file-writer/file-references.md index ce21a77..08d815c 100644 --- a/docs/learn/file-writer/file-references.md +++ b/docs/learn/file-writer/file-references.md @@ -6,6 +6,10 @@ related: url: learn/file-writer/default-format.md - title: Add a custom NeXuS structure for the file writer url: how-to/customize-bec/add-a-custom-nexus-structure.md + - title: BEC Signals for Custom Devices + url: learn/devices/bec-signals.md + - title: Add a File Event Signal + url: how-to/devices/add-a-file-event-signal.md --- # File References from Devices @@ -34,3 +38,9 @@ This mechanism is useful when: - a beamline-specific layout wants links under `entry/instrument/...` or `entry/data` In those cases, the custom writer can use `self.file_references` together with `create_ext_link(...)` to place links in the desired structure. + +!!! info "What to remember" + - File references let devices contribute externally written files to the BEC master file instead of sending all data through BEC directly. + - Devices usually report these files through helper signals such as `FileEventSignal`. + - `DefaultFormat` collects file references under `/entry/collection/file_references` and creates external links for them. + - Custom writers can reuse the collected file references and place additional links in beamline-specific NeXuS locations. diff --git a/docs/learn/file-writer/plugin-repository-integration.md b/docs/learn/file-writer/plugin-repository-integration.md index 75333b9..37ea82f 100644 --- a/docs/learn/file-writer/plugin-repository-integration.md +++ b/docs/learn/file-writer/plugin-repository-integration.md @@ -24,3 +24,9 @@ The selection logic is: - if no file-writer plugin is installed, BEC uses the built-in default format - if exactly one custom writer class is available, BEC uses it automatically - if multiple custom writer classes are available, BEC uses the class name configured in `file_writer.plugin` + +!!! info "What to remember" + - BEC discovers custom file-writer formats through the `bec.file_writer` entry point group. + - The plugin module must expose the writer classes that BEC should consider. + - Without a plugin, BEC falls back to the built-in default format. + - When multiple custom writer classes are available, `file_writer.plugin` selects which one BEC uses. diff --git a/docs/learn/file-writer/where-files-are-written.md b/docs/learn/file-writer/where-files-are-written.md index 9caf908..6d552c8 100644 --- a/docs/learn/file-writer/where-files-are-written.md +++ b/docs/learn/file-writer/where-files-are-written.md @@ -136,3 +136,8 @@ Users cannot escape this base path from the client side. This prevents one user from writing into another user's directory. +!!! info "What to remember" + - The file writer always starts from the server-configured `file_writer.base_path`. + - The active account influences the effective write path, either by replacing `$account` or by being appended as a subdirectory. + - BEC uses a scan-number-based directory and filename structure by default. + - Scan arguments such as `file_suffix` and `file_directory` can customize parts of the layout, but they cannot escape the configured base path. diff --git a/zensical.toml b/zensical.toml index 93e940f..fa203b5 100644 --- a/zensical.toml +++ b/zensical.toml @@ -82,6 +82,13 @@ nav = [ { "Select a Signal Kind" = "how-to/devices/how-to-select-an-ophyd-kind.md" }, { "Select a Readout Priority" = "how-to/devices/how-to-select-readout-priority.md" }, ] }, + { "BEC Signals" = [ + { "Add a Progress Signal" = "how-to/devices/add-a-progress-signal.md" }, + { "Add a Preview Signal" = "how-to/devices/add-a-preview-signal.md" }, + { "Add a File Event Signal" = "how-to/devices/add-a-file-event-signal.md" }, + { "Add an Async Signal" = "how-to/devices/add-an-async-signal.md" }, + { "Add an Async Multi Signal" = "how-to/devices/add-an-async-multi-signal.md" }, + ] }, { "Simulation" = [ { "Use Simulated Models from the IPython Client" = "how-to/devices/use-simulated-models-from-ipython.md" }, ] },