test.py::TestString::test_small_list failing on 32-bit architectures

[tti0]

Hi!

I am the maintainer of the Debian packaging of pyroaring (python3-pyroaring), for which version 1.0.4 was accepted into the sid distribution yesterday.

In Debian, we run pytest test.py on all of Debian's supported architectures in CI. We have noticed that one of these tests, TestString::test_small_list, is consistently failing on the 32-bit i386 architecture, but passes on the other 64-bit architectures. The full traceback is:

____________________________________________________ TestString.test_small_list _____________________________________________________

self = <test.TestString object at 0xf6d467c0>

    @given(bitmap_cls, small_list_of_uin32)
>   def test_small_list(self, cls: type[EitherBitMap], collection: list[int]) -> None:
               ^^^^^^^

test.py:1816:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
test.py:1819: in test_small_list
    string_repr = repr(bm)
                  ^^^^^^^^
pyroaring/abstract_bitmap.pxi:328: in pyroaring.AbstractBitMap.__repr__
    return str(self)
pyroaring/abstract_bitmap.pxi:331: in pyroaring.AbstractBitMap.__str__
    return _string_rep(self)
pyroaring/abstract_bitmap.pxi:76: in pyroaring._string_rep
    row_ints = bm[row_idx * num_columns:(row_idx + 1) * num_columns]
pyroaring/abstract_bitmap.pxi:748: in pyroaring.AbstractBitMap.__getitem__
    return self._get_slice(value)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   if abs(step) == 1 and len(values) <= len(self) / 100:  # contiguous and small chunk of the bitmap
E   OverflowError: Python int too large to convert to C ssize_t
E   Falsifying example: test_small_list(
E       self=<test.TestString object at 0xf6d467c0>,
E       cls=pyroaring.BitMap,
E       collection=[0, 2_147_483_647],
E   )

pyroaring/abstract_bitmap.pxi:705: OverflowError

I have been able to reproduce this locally with:

podman run --rm -it --arch=i386 docker.io/i386/debian:sid bash
apt update
apt install python3-pyroaring python3-pytest python3-hypothesis git
git clone https://github.com/Ezibenroc/PyRoaringBitMap
cd PyRoaringBitMap
pytest test.py

I would appreciate any advice on running test.py in Debian CI, as it is very likely I am missing some context from upstream. Alternatively, I would be very happy to test any proposed patches in Debian if it turns out some guarding is needed in _get_slice around large slice indices.

Thank you!

[Ezibenroc]

Hello and thanks for reporting this !

I have a tentative PR here: #147
I wanted to test with the release workflow by re-enabling the tests everywhere, but this worklow is failing at the moment with what looks like an unrelated infra issue. I will retry later.

[tti0]

Hi @Ezibenroc, thank you very much for the prompt investigation!

I will give #147 a try in Debian in the next few days. I will be able to bring the fix into Debian with a new release of pyroaring, so thank you for looking at the release workflow as well.

[Ezibenroc]

Hello @tti0, just to let you know, I’ve released version 1.1.0 with a potential fix. When you have time, could you confirm whether it resolves the issue?

[tti0]

Hi @Ezibenroc! I have tested the 1.1.0 release in Debian today and confirm that we now have all tests passing on i386! Thank you again for the quick fix. I will close this issue now and get the Debian package uploaded.