-
Notifications
You must be signed in to change notification settings - Fork 107
GH-926: Shade arrow-compression with Flight JDBC driver #937
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
This comment has been minimized.
This comment has been minimized.
|
@aarshivv are you able to build the jdbc driver and test it against your server? I created a demo flight server compressing IPC stream and used DBeaver to load the driver but I'd like to validate this with someone else if possible |
|
I will do a quick test too. |
|
@xborder , I tested it against my Rust-based Flight SQL server and it is working as expected for ZSTD compression. However, I am getting the following error when the server uses LZ4_FRAME compression: I confirmed that the shaded JAR includes the compression classes: |
|
@aarshivv could that be a server error? The error message seems to hint that |
|
@xborder , My bad, yes the feature flag was missing at the server. |
jbonofre
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This change looks good.
I tested without problem.
| <excludes> | ||
| <exclude>com.sun.**</exclude> | ||
| <!-- zstd-jni uses JNI native methods bound to original class names --> | ||
| <exclude>com.github.luben.**</exclude> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have to check if the LICENSE should not be updated to reflect this exclude.
What's Changed
Flight JDBC driver does not support IPC compressed formats. This PR includes and shades arrow-compression when packing JDBC driver.
For zstd compression, it is excluding zstd-jni relocation since it needs JNI native methods to keep original location.
Closes #926.