Skip to content

Conversation

@aelkiss
Copy link
Member

@aelkiss aelkiss commented Dec 1, 2025

  • use grok from apt.lib.umich.edu
  • install jhove from the distributor
  • fix issue related to TIFFs & imagemagick 7

TODO:

  • resolve failing S3 tests

@aelkiss
Copy link
Member Author

aelkiss commented Dec 1, 2025

This worked for building grok locally:

docker run -v ./grok:/tmp/grok -ti --rm debian:trixie /bin/bash

apt update && apt install wget cmake build-essential cmake-curses-gui exiftool libpsl-dev libjpeg62-turbo-dev pkg-config libcurl4-openssl-dev libtiff-dev libpng-dev liblcms2-dev libcli11-dev
wget https://github.com/GrokImageCompression/grok/archive/refs/tags/v20.0.4.zip

tar zxvf v20.0.4.zip
cd grok-20.0.4

cmake -E make_directory $PWD/build
cmake -S $PWD -B $PWD/build  -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=ON -DGRK_ENABLE_LIBCURL=ON -DGRK_BUILD_JPEG=OFF -DGRK_BUILD_LCMS2=OFF -DGRK_BUILD_LIBCURL=OFF -D GRK_BUILD_LIBTIFF=OFF -DBUILD_TESTING=OFF

cmake --build $PWD/build --config Release -j 8

. /etc/os-release
cmake -S $PWD -B $PWD/build -Wno-dev -D CPACK_GENERATOR:STRING=ZIP -D CPACK_PACKAGE_FILE_NAME:STRING=grok-$ID-$VERSION_CODENAME
cmake --build $PWD/build --config RELEASE --target package

this outputs a zip called grok-debian-trixie.zip

this is probably a good starting point for deb packaging

@aelkiss
Copy link
Member Author

aelkiss commented Dec 1, 2025

These Debian build scripts might also be useful in conjunction with above: https://github.com/GrokImageCompression/grok/tree/debian/master/debian

(note these are not in the current main branch)

@aelkiss aelkiss marked this pull request as draft December 1, 2025 19:20
@aelkiss
Copy link
Member Author

aelkiss commented Dec 1, 2025

It looks like the S3-related tests also fail in github, so that's also something we'll need to look at.

niquerio added a commit to mlibrary/divp that referenced this pull request Jan 15, 2026
relevant HT PRs:
* hathitrust/feed#95
* hathitrust/feed#172

grokj2k is available in the mlibrary apt repo
* use grokj2k from apt.lib.umich.edu

* update arguments for grok - SOP and EPH were spelled out and shouldn't
  have been
The default seems to have changed with ImageMagick 7 for bitonal tiffs
to use photometric interpretation BlackIsZero rather than (as we expect)
WhiteIsZero.
@moseshll
Copy link
Contributor

moseshll commented Feb 11, 2026

Regarding the failing tests:

Can't use an undefined value as an ARRAY reference

in S3->list_objects happens for me when awscli returns something like

{
  'RequestCharged' => undef,
  'Prefix' => 'test.bmvblsso.20260211195011.mets.xml'
};

and the Perl code tries to retrieve nonexistent $result->{Contents}. It appears that the bullseye version of awscli (2.9.x) would return essentially nothing with an empty bucket (e.g., it "handles empty bucket" => sub {...) where in the trixie version (2.23.x) we get the above structure without Contents. I think it should be safe for the bailout condition around S3.pm line 125 to be

last unless $result;
last unless $result->{Contents};

There's a similar error in the storage_audit.t tests that displays the same behavior.

=============
There's also a potentially brittle test in storage_object_store.t line 188 that sometimes fails for me, sometimes succeeds. I traced it as far as Storage::encrypt and noticed (line 110)

  my $success   = ! $?;
  $self->safe_system($cmd);
  # many lines of stufff.....
  return $success;
}

Is it checking the wait status before calling the system wrapper?
I don't actually think this is a smoking gun (for the brittleness on my system) but I don't like it. (Guessing $cmd began life inside backticks?)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants