Thread safe?

[erikogabrielsson]

Hi,

When using the pyisyntax wrapper for this library I noticed segmentation faults when reading with multiple threads. Not sure if this is a Python problem or if libisyntax is not thread safe?

Can you clarify if the library is thread safe?

[Falcury]

Sorry for the late reply!

I haven't tried the pyisyntax wrapper yet, and I'm not 100% sure about whether it 'should' work in the exact use case in the issue you linked to. But, I suspect it is currently not possible to do it in this way.

At least internally, there is some degree of thread-safety to make sure that tiles can be loaded in parallel (also because to load a single tile, there is usually a dependency on other tiles to be loaded first and there is a lot of complex caching stuff and internal synchronization going on). But, this doesn't necessarily translate to thread safety to the external API.

Maybe @alex-virodov knows a more complete answer for how the reader API is implemented right now?

For the streaming implementation in Slidescape I used a separate thread that collects requests for tiles periodically (each frame), and tries to continually satisfy these requests by pulling from a wishlist and reading as many tiles from that list as possible in a somewhat efficient way. I can imagine we could adjust the implementation to work similar to this, to collect tile requests in parallel and them dispatch them in batches periodically (although that would require some more work to be done).

[jonasteuwen]

@Falcury @erikogabrielsson
The library appears to be thread-safe, when I was integrating it in fastslide. I did strip the library though, so I needed to make this change:

  // CRITICAL FIX: Ensure thread-local memory is initialized
  if (!local_thread_memory) {
    init_thread_memory(0, &global_system_info);
  }

in isyntax_openslide_load_tile_coefficients_ll_or_h. I am not sure if this is because of my fork, or because it was not initialized to begin with (it's also probably not the best fix)