Migrates bioclip classifier imports to use bioclip.predict#192
Conversation
|
Makes me wonder whether we shouldn't hardcode the desired model rather than pulling whatever the default is. @thompsonmj @egrace479 @NetZissou thoughts either way? |
Yes, I think we should provide the examples in these notebooks with BioCLIP 2 explicitly passed and include a note where checking which model is used matters (for this line). |
|
Actually turns out the patch size is a red herring and has not changed between BioCLIP and BioCLIP2. It's the width of the transformer that has changed (now 1024), and probably as a result the spatial grid size in the token tensor. |
The transformer widths of BioCLIP models can differ, and presumably as a result the spatial grid size for the last transformer block (?). For BioCLIP1 it is 14x14, but for BioCLIP2 it is 16x16. Instead of hard-coding, we are inferring this now from the data unless it is specified.
|
Key changes in the GradCAM notebook are these:
(The rest in that notebook is rerunning it.) |
egrace479
left a comment
egrace479
left a comment
There was a problem hiding this comment.
Looks good! I like the reproducibility comment for the BioCLIP 2 model call.
2 participants
This is in follow-up to #187 and #189.
Also fixes the patch size for GradCAM, which changed from BioCLIP1 (14x14) to BioCLIP2 (16x16).