Producer encoding errors: silent data loss and type mismatch

[zmanian]

Summary

Two related issues with how producer encoding errors are handled:

1. Silent data loss in fanout

In src/streams/producer/worker.rs (~line 224), when Datum::encode() fails during fanout, the error is logged and the datum is dropped. The caller of producer.send() has no way to know this happened.

Additionally, the dropped datum may be routed to the undelivered sink (~line 248), which is designed for "no consumers matched" — not "serialization failed." These two failure modes are conflated.

2. Error type hardcodes postcard

In src/streams/producer/error.rs, the Encoding variant uses the concrete EncodeError (postcard):

use crate::primitives::EncodeError;
// ...
Encoding(D, EncodeError),

But the Datum trait allows custom EncodeError types via associated types. The Clone impl reconstructs the error via EncodeError::custom(e.to_string()), which is lossy for non-postcard error types.

This breaks the abstraction that the customizable encoding feature was meant to provide.

Suggestion

• Propagate encoding errors back to the send() caller, or provide an explicit error callback
• Keep the undelivered sink for "no match" only; handle encoding failures separately
• Make Error<D> generic over D::EncodeError (or use a boxed error) instead of hardcoding postcard