Add support for pickling error

[keskitalo]

No description provided.

[AERAdler]

Didn't really seem to work, both for cases where end of file was reached unexpectedly and where an unrecognised character was found.

[keskitalo]

Can you post the stack trace here? There is nothing obvious in the source code that would explain the failure.

[AERAdler]

I think it's a "pickle vs _pickle" problem?

Proc 6: File "/global/homes/a/aeadler/toast_dev_env/lib/python3.12/site-packages/toast/mpi.py", line 518, in exception_guard
yield
Proc 6: File "/global/homes/a/aeadler/toast_dev_env/bin/toast_run", line 198, in
main(comm=world)
Proc 6: File "/global/homes/a/aeadler/toast_dev_env/bin/toast_run", line 173, in main
main.apply(data)
Proc 6: File "/global/homes/a/aeadler/toast_dev_env/lib/python3.12/site-packages/toast/timing.py", line 101, in df
return f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
Proc 6: File "/global/homes/a/aeadler/toast_dev_env/lib/python3.12/site-packages/toast/ops/operator.py", line 125, in apply
self.exec(data, detectors=detectors, **kwargs)
Proc 6: File "/global/homes/a/aeadler/toast_dev_env/lib/python3.12/site-packages/toast/timing.py", line 101, in df
return f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
Proc 6: File "/global/homes/a/aeadler/toast_dev_env/lib/python3.12/site-packages/toast/ops/operator.py", line 58, in exec
self._exec(
Proc 6: File "/global/homes/a/aeadler/toast_dev_env/lib/python3.12/site-packages/toast/timing.py", line 73, in df
result = f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
Proc 6: File "/global/homes/a/aeadler/toast_dev_env/lib/python3.12/site-packages/toast/ops/pipeline.py", line 145, in _exec
self._exec_operator(
Proc 6: File "/global/homes/a/aeadler/toast_dev_env/lib/python3.12/site-packages/toast/timing.py", line 73, in df
result = f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
Proc 6: File "/global/homes/a/aeadler/toast_dev_env/lib/python3.12/site-packages/toast/ops/pipeline.py", line 263, in _exec_operator
op.exec(data, detectors=detectors, use_accel=run_accel)
Proc 6: File "/global/homes/a/aeadler/toast_dev_env/lib/python3.12/site-packages/toast/timing.py", line 101, in df
return f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
Proc 6: File "/global/homes/a/aeadler/toast_dev_env/lib/python3.12/site-packages/toast/ops/operator.py", line 58, in exec
self._exec(
Proc 6: File "/global/homes/a/aeadler/toast_dev_env/lib/python3.12/site-packages/toast/timing.py", line 73, in df
result = f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
Proc 6: File "/global/homes/a/aeadler/toast_dev_env/lib/python3.12/site-packages/toast/ops/pipeline.py", line 145, in _exec
self._exec_operator(
Proc 6: File "/global/homes/a/aeadler/toast_dev_env/lib/python3.12/site-packages/toast/timing.py", line 73, in df
result = f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
Proc 6: File "/global/homes/a/aeadler/toast_dev_env/lib/python3.12/site-packages/toast/ops/pipeline.py", line 263, in _exec_operator
op.exec(data, detectors=detectors, use_accel=run_accel)
Proc 6: File "/global/homes/a/aeadler/toast_dev_env/lib/python3.12/site-packages/toast/timing.py", line 101, in df
return f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
Proc 6: File "/global/homes/a/aeadler/toast_dev_env/lib/python3.12/site-packages/toast/ops/operator.py", line 58, in exec
self._exec(
Proc 6: File "/global/homes/a/aeadler/toast_dev_env/lib/python3.12/site-packages/toast/timing.py", line 73, in df
result = f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
Proc 6: File "/global/homes/a/aeadler/toast_dev_env/lib/python3.12/site-packages/sotodlib/toast/ops/intensity_templates.py", line 299, in _exec
result = persistent_pickle_load(fname_cache)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Proc 6: File "/global/homes/a/aeadler/toast_dev_env/lib/python3.12/site-packages/sotodlib/toast/ops/utils.py", line 92, in persistent_pickle_load
payload = pickle.load(f)
^^^^^^^^^^^^^^
Proc 6: _pickle.UnpicklingError: pickle data was truncated