WIP: Pin to Adaptive Parquet Predicate Pushdown by alamb · Pull Request #18368 · apache/datafusion

alamb · 2025-10-29T21:25:52Z

This is a test branch for performance profiling for the code from @hhhizzz upstream in arrow-rs:

[Parquet] Adaptive Parquet Predicate Pushdown arrow-rs#8733

alamb · 2025-10-30T08:29:47Z

🤖 ./gh_compare_branch.sh Benchmark Script Running
Linux aal-dev 6.14.0-1017-gcp #18~24.04.1-Ubuntu SMP Tue Sep 23 17:51:44 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/test_adapative_scan (c181597) to 0a650a0 diff using: tpch_mem clickbench_partitioned clickbench_extended
Results will be posted here when complete

alamb · 2025-10-30T09:27:34Z

🤖: Benchmark completed

Details

Comparing HEAD and alamb_test_adapative_scan
--------------------
Benchmark clickbench_extended.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_test_adapative_scan ┃         Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┩
│ QQuery 0     │  2738.16 ms │                2692.39 ms │      no change │
│ QQuery 1     │  1227.30 ms │                1313.44 ms │   1.07x slower │
│ QQuery 2     │  2316.04 ms │                2423.93 ms │      no change │
│ QQuery 3     │  1139.86 ms │                1153.79 ms │      no change │
│ QQuery 4     │  2232.67 ms │                2416.56 ms │   1.08x slower │
│ QQuery 5     │ 28051.03 ms │               27834.75 ms │      no change │
│ QQuery 6     │  4186.76 ms │                 166.77 ms │ +25.11x faster │
│ QQuery 7     │  3574.54 ms │                3695.72 ms │      no change │
└──────────────┴─────────────┴───────────────────────────┴────────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 45466.36ms │
│ Total Time (alamb_test_adapative_scan)   │ 41697.35ms │
│ Average Time (HEAD)                      │  5683.30ms │
│ Average Time (alamb_test_adapative_scan) │  5212.17ms │
│ Queries Faster                           │          1 │
│ Queries Slower                           │          2 │
│ Queries with No Change                   │          5 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_test_adapative_scan ┃         Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┩
│ QQuery 0     │     2.09 ms │                   2.53 ms │   1.21x slower │
│ QQuery 1     │    50.87 ms │                  53.88 ms │   1.06x slower │
│ QQuery 2     │   136.47 ms │                 137.52 ms │      no change │
│ QQuery 3     │   161.11 ms │                 167.04 ms │      no change │
│ QQuery 4     │  1025.66 ms │                1092.05 ms │   1.06x slower │
│ QQuery 5     │  1463.22 ms │                1488.92 ms │      no change │
│ QQuery 6     │     2.15 ms │                   2.20 ms │      no change │
│ QQuery 7     │    53.84 ms │                  71.18 ms │   1.32x slower │
│ QQuery 8     │  1425.84 ms │                1470.39 ms │      no change │
│ QQuery 9     │  1772.91 ms │                1825.44 ms │      no change │
│ QQuery 10    │   386.84 ms │                 485.98 ms │   1.26x slower │
│ QQuery 11    │   446.25 ms │                 538.85 ms │   1.21x slower │
│ QQuery 12    │  1409.22 ms │                1556.39 ms │   1.10x slower │
│ QQuery 13    │  2156.43 ms │                2348.62 ms │   1.09x slower │
│ QQuery 14    │  1268.16 ms │                1449.71 ms │   1.14x slower │
│ QQuery 15    │  1222.16 ms │                1263.90 ms │      no change │
│ QQuery 16    │  2630.85 ms │                2759.71 ms │      no change │
│ QQuery 17    │  2644.54 ms │                2674.41 ms │      no change │
│ QQuery 18    │  5335.54 ms │                5054.80 ms │  +1.06x faster │
│ QQuery 19    │   124.73 ms │                 144.01 ms │   1.15x slower │
│ QQuery 20    │  2062.39 ms │                2061.97 ms │      no change │
│ QQuery 21    │  2329.07 ms │                2538.07 ms │   1.09x slower │
│ QQuery 22    │  4344.11 ms │                4171.74 ms │      no change │
│ QQuery 23    │ 12780.49 ms │                1395.75 ms │  +9.16x faster │
│ QQuery 24    │   216.12 ms │                 317.21 ms │   1.47x slower │
│ QQuery 25    │   515.37 ms │                 634.02 ms │   1.23x slower │
│ QQuery 26    │   223.40 ms │                 330.81 ms │   1.48x slower │
│ QQuery 27    │  2970.05 ms │                3137.36 ms │   1.06x slower │
│ QQuery 28    │ 22884.87 ms │               24904.65 ms │   1.09x slower │
│ QQuery 29    │   966.13 ms │                 960.83 ms │      no change │
│ QQuery 30    │  1354.70 ms │                1422.04 ms │      no change │
│ QQuery 31    │  1364.26 ms │                1418.44 ms │      no change │
│ QQuery 32    │  5188.96 ms │                4419.72 ms │  +1.17x faster │
│ QQuery 33    │  5989.10 ms │                5624.95 ms │  +1.06x faster │
│ QQuery 34    │  6149.11 ms │                6013.20 ms │      no change │
│ QQuery 35    │  2001.90 ms │                1997.83 ms │      no change │
│ QQuery 36    │   122.10 ms │                  20.21 ms │  +6.04x faster │
│ QQuery 37    │    53.18 ms │                  19.88 ms │  +2.67x faster │
│ QQuery 38    │   121.17 ms │                  20.21 ms │  +5.99x faster │
│ QQuery 39    │   197.39 ms │                  19.74 ms │ +10.00x faster │
│ QQuery 40    │    41.01 ms │                  21.47 ms │  +1.91x faster │
│ QQuery 41    │    38.54 ms │                  20.74 ms │  +1.86x faster │
│ QQuery 42    │    33.45 ms │                  20.95 ms │  +1.60x faster │
└──────────────┴─────────────┴───────────────────────────┴────────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 95665.73ms │
│ Total Time (alamb_test_adapative_scan)   │ 86079.33ms │
│ Average Time (HEAD)                      │  2224.78ms │
│ Average Time (alamb_test_adapative_scan) │  2001.84ms │
│ Queries Faster                           │         11 │
│ Queries Slower                           │         16 │
│ Queries with No Change                   │         16 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃      HEAD ┃ alamb_test_adapative_scan ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1     │ 213.40 ms │                 175.23 ms │ +1.22x faster │
│ QQuery 2     │  30.83 ms │                  26.55 ms │ +1.16x faster │
│ QQuery 3     │  46.60 ms │                  40.82 ms │ +1.14x faster │
│ QQuery 4     │  35.00 ms │                  28.36 ms │ +1.23x faster │
│ QQuery 5     │  77.73 ms │                  76.78 ms │     no change │
│ QQuery 6     │  19.33 ms │                  19.25 ms │     no change │
│ QQuery 7     │ 228.06 ms │                 207.04 ms │ +1.10x faster │
│ QQuery 8     │  33.39 ms │                  30.44 ms │ +1.10x faster │
│ QQuery 9     │  98.33 ms │                 100.35 ms │     no change │
│ QQuery 10    │  60.78 ms │                  58.75 ms │     no change │
│ QQuery 11    │  17.41 ms │                  17.02 ms │     no change │
│ QQuery 12    │  51.78 ms │                  49.57 ms │     no change │
│ QQuery 13    │  48.59 ms │                  45.82 ms │ +1.06x faster │
│ QQuery 14    │  14.12 ms │                  12.85 ms │ +1.10x faster │
│ QQuery 15    │  24.95 ms │                  24.12 ms │     no change │
│ QQuery 16    │  25.02 ms │                  24.57 ms │     no change │
│ QQuery 17    │ 156.23 ms │                 145.67 ms │ +1.07x faster │
│ QQuery 18    │ 322.72 ms │                 329.67 ms │     no change │
│ QQuery 19    │  35.83 ms │                  37.15 ms │     no change │
│ QQuery 20    │  50.77 ms │                  48.63 ms │     no change │
│ QQuery 21    │ 344.51 ms │                 305.96 ms │ +1.13x faster │
│ QQuery 22    │  20.72 ms │                  20.63 ms │     no change │
└──────────────┴───────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 1956.11ms │
│ Total Time (alamb_test_adapative_scan)   │ 1825.23ms │
│ Average Time (HEAD)                      │   88.91ms │
│ Average Time (alamb_test_adapative_scan) │   82.96ms │
│ Queries Faster                           │        10 │
│ Queries Slower                           │         0 │
│ Queries with No Change                   │        12 │
│ Queries with Failure                     │         0 │
└──────────────────────────────────────────┴───────────┘

hhhizzz · 2025-10-30T09:38:01Z

Got some impressive improvement in some query, I'll take a look for the slower query like QQuery 26 . 1.48x slower didn't like noise.

zhuqi-lucas · 2025-10-30T09:48:58Z

Thank you @alamb @hhhizzz , we may also need to trigger the clickbench_pushdown which is the enabled filter pushdown, in order to compare this PR and main.

alamb · 2025-10-30T11:02:56Z

Thank you @alamb @hhhizzz , we may also need to trigger the clickbench_pushdown which is the enabled filter pushdown, in order to compare this PR and main.

Good call -- I forgot about that one. I just hard coded the pushdown on -- will rerun

alamb · 2025-10-30T11:33:27Z

🤖 ./gh_compare_branch.sh Benchmark Script Running
Linux aal-dev 6.14.0-1017-gcp #18~24.04.1-Ubuntu SMP Tue Sep 23 17:51:44 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/test_adapative_scan (a0cbf52) to 930620a diff using: clickbench_pushdown
Results will be posted here when complete

alamb · 2025-10-30T12:02:48Z

🤖: Benchmark completed

Details

Comparing HEAD and alamb_test_adapative_scan
--------------------
Benchmark clickbench_pushdown.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_test_adapative_scan ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │     2.60 ms │                   2.33 ms │ +1.12x faster │
│ QQuery 1     │    55.17 ms │                  54.34 ms │     no change │
│ QQuery 2     │   136.84 ms │                 140.47 ms │     no change │
│ QQuery 3     │   164.07 ms │                 163.01 ms │     no change │
│ QQuery 4     │  1203.30 ms │                1078.05 ms │ +1.12x faster │
│ QQuery 5     │  1581.61 ms │                1526.89 ms │     no change │
│ QQuery 6     │     2.13 ms │                   2.22 ms │     no change │
│ QQuery 7     │    72.51 ms │                  73.17 ms │     no change │
│ QQuery 8     │  1518.06 ms │                1472.00 ms │     no change │
│ QQuery 9     │  1948.40 ms │                1898.97 ms │     no change │
│ QQuery 10    │   501.07 ms │                 485.25 ms │     no change │
│ QQuery 11    │   566.69 ms │                 556.84 ms │     no change │
│ QQuery 12    │  1659.60 ms │                1541.78 ms │ +1.08x faster │
│ QQuery 13    │  2594.14 ms │                2278.78 ms │ +1.14x faster │
│ QQuery 14    │  1735.34 ms │                1460.55 ms │ +1.19x faster │
│ QQuery 15    │  1327.28 ms │                1258.79 ms │ +1.05x faster │
│ QQuery 16    │  2729.06 ms │                2683.83 ms │     no change │
│ QQuery 17    │  2696.97 ms │                2683.94 ms │     no change │
│ QQuery 18    │  5157.72 ms │                4965.78 ms │     no change │
│ QQuery 19    │   149.40 ms │                 141.88 ms │ +1.05x faster │
│ QQuery 20    │  2012.43 ms │                1967.18 ms │     no change │
│ QQuery 21    │  2460.10 ms │                2352.27 ms │     no change │
│ QQuery 22    │  4271.47 ms │                4077.51 ms │     no change │
│ QQuery 23    │  1155.60 ms │                1176.57 ms │     no change │
│ QQuery 24    │   261.13 ms │                 248.77 ms │     no change │
│ QQuery 25    │   681.20 ms │                 635.93 ms │ +1.07x faster │
│ QQuery 26    │   362.74 ms │                 338.25 ms │ +1.07x faster │
│ QQuery 27    │  3293.90 ms │                3101.06 ms │ +1.06x faster │
│ QQuery 28    │ 23992.63 ms │               23346.17 ms │     no change │
│ QQuery 29    │   991.31 ms │                 981.48 ms │     no change │
│ QQuery 30    │  2225.58 ms │                1465.19 ms │ +1.52x faster │
│ QQuery 31    │  2079.26 ms │                1437.83 ms │ +1.45x faster │
│ QQuery 32    │  5371.03 ms │                4334.00 ms │ +1.24x faster │
│ QQuery 33    │  6696.14 ms │                5615.78 ms │ +1.19x faster │
│ QQuery 34    │  6088.96 ms │                6177.78 ms │     no change │
│ QQuery 35    │  2132.40 ms │                2024.73 ms │ +1.05x faster │
│ QQuery 36    │    25.43 ms │                  26.36 ms │     no change │
│ QQuery 37    │    25.38 ms │                  25.48 ms │     no change │
│ QQuery 38    │    24.88 ms │                  25.47 ms │     no change │
│ QQuery 39    │    25.14 ms │                  24.59 ms │     no change │
│ QQuery 40    │    26.51 ms │                  26.12 ms │     no change │
│ QQuery 41    │    25.45 ms │                  25.53 ms │     no change │
│ QQuery 42    │    24.82 ms │                  25.16 ms │     no change │
└──────────────┴─────────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 90055.45ms │
│ Total Time (alamb_test_adapative_scan)   │ 83928.07ms │
│ Average Time (HEAD)                      │  2094.31ms │
│ Average Time (alamb_test_adapative_scan) │  1951.82ms │
│ Queries Faster                           │         15 │
│ Queries Slower                           │          0 │
│ Queries with No Change                   │         28 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘

alamb · 2025-10-30T12:02:52Z

🤖 ./gh_compare_branch.sh Benchmark Script Running
Linux aal-dev 6.14.0-1017-gcp #18~24.04.1-Ubuntu SMP Tue Sep 23 17:51:44 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/test_adapative_scan (a0cbf52) to 930620a diff using: tpch_mem clickbench_partitioned clickbench_extended
Results will be posted here when complete

zhuqi-lucas · 2025-10-30T12:19:04Z

🤖: Benchmark completed

Details

Comparing HEAD and alamb_test_adapative_scan
--------------------
Benchmark clickbench_pushdown.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_test_adapative_scan ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │     2.60 ms │                   2.33 ms │ +1.12x faster │
│ QQuery 1     │    55.17 ms │                  54.34 ms │     no change │
│ QQuery 2     │   136.84 ms │                 140.47 ms │     no change │
│ QQuery 3     │   164.07 ms │                 163.01 ms │     no change │
│ QQuery 4     │  1203.30 ms │                1078.05 ms │ +1.12x faster │
│ QQuery 5     │  1581.61 ms │                1526.89 ms │     no change │
│ QQuery 6     │     2.13 ms │                   2.22 ms │     no change │
│ QQuery 7     │    72.51 ms │                  73.17 ms │     no change │
│ QQuery 8     │  1518.06 ms │                1472.00 ms │     no change │
│ QQuery 9     │  1948.40 ms │                1898.97 ms │     no change │
│ QQuery 10    │   501.07 ms │                 485.25 ms │     no change │
│ QQuery 11    │   566.69 ms │                 556.84 ms │     no change │
│ QQuery 12    │  1659.60 ms │                1541.78 ms │ +1.08x faster │
│ QQuery 13    │  2594.14 ms │                2278.78 ms │ +1.14x faster │
│ QQuery 14    │  1735.34 ms │                1460.55 ms │ +1.19x faster │
│ QQuery 15    │  1327.28 ms │                1258.79 ms │ +1.05x faster │
│ QQuery 16    │  2729.06 ms │                2683.83 ms │     no change │
│ QQuery 17    │  2696.97 ms │                2683.94 ms │     no change │
│ QQuery 18    │  5157.72 ms │                4965.78 ms │     no change │
│ QQuery 19    │   149.40 ms │                 141.88 ms │ +1.05x faster │
│ QQuery 20    │  2012.43 ms │                1967.18 ms │     no change │
│ QQuery 21    │  2460.10 ms │                2352.27 ms │     no change │
│ QQuery 22    │  4271.47 ms │                4077.51 ms │     no change │
│ QQuery 23    │  1155.60 ms │                1176.57 ms │     no change │
│ QQuery 24    │   261.13 ms │                 248.77 ms │     no change │
│ QQuery 25    │   681.20 ms │                 635.93 ms │ +1.07x faster │
│ QQuery 26    │   362.74 ms │                 338.25 ms │ +1.07x faster │
│ QQuery 27    │  3293.90 ms │                3101.06 ms │ +1.06x faster │
│ QQuery 28    │ 23992.63 ms │               23346.17 ms │     no change │
│ QQuery 29    │   991.31 ms │                 981.48 ms │     no change │
│ QQuery 30    │  2225.58 ms │                1465.19 ms │ +1.52x faster │
│ QQuery 31    │  2079.26 ms │                1437.83 ms │ +1.45x faster │
│ QQuery 32    │  5371.03 ms │                4334.00 ms │ +1.24x faster │
│ QQuery 33    │  6696.14 ms │                5615.78 ms │ +1.19x faster │
│ QQuery 34    │  6088.96 ms │                6177.78 ms │     no change │
│ QQuery 35    │  2132.40 ms │                2024.73 ms │ +1.05x faster │
│ QQuery 36    │    25.43 ms │                  26.36 ms │     no change │
│ QQuery 37    │    25.38 ms │                  25.48 ms │     no change │
│ QQuery 38    │    24.88 ms │                  25.47 ms │     no change │
│ QQuery 39    │    25.14 ms │                  24.59 ms │     no change │
│ QQuery 40    │    26.51 ms │                  26.12 ms │     no change │
│ QQuery 41    │    25.45 ms │                  25.53 ms │     no change │
│ QQuery 42    │    24.82 ms │                  25.16 ms │     no change │
└──────────────┴─────────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 90055.45ms │
│ Total Time (alamb_test_adapative_scan)   │ 83928.07ms │
│ Average Time (HEAD)                      │  2094.31ms │
│ Average Time (alamb_test_adapative_scan) │  1951.82ms │
│ Queries Faster                           │         15 │
│ Queries Slower                           │          0 │
│ Queries with No Change                   │         28 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘

The result is nice!

alamb · 2025-10-30T12:47:00Z

🤖: Benchmark completed

Details

Comparing HEAD and alamb_test_adapative_scan
--------------------
Benchmark clickbench_extended.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_test_adapative_scan ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │  2761.31 ms │                2612.38 ms │ +1.06x faster │
│ QQuery 1     │  1346.33 ms │                1347.10 ms │     no change │
│ QQuery 2     │  2498.54 ms │                2479.56 ms │     no change │
│ QQuery 3     │  1152.14 ms │                1204.33 ms │     no change │
│ QQuery 4     │  2253.57 ms │                2231.62 ms │     no change │
│ QQuery 5     │ 27531.42 ms │               28355.44 ms │     no change │
│ QQuery 6     │  4210.59 ms │                4209.56 ms │     no change │
│ QQuery 7     │  3375.54 ms │                3643.74 ms │  1.08x slower │
└──────────────┴─────────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 45129.44ms │
│ Total Time (alamb_test_adapative_scan)   │ 46083.73ms │
│ Average Time (HEAD)                      │  5641.18ms │
│ Average Time (alamb_test_adapative_scan) │  5760.47ms │
│ Queries Faster                           │          1 │
│ Queries Slower                           │          1 │
│ Queries with No Change                   │          6 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_test_adapative_scan ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │     2.70 ms │                   2.29 ms │ +1.18x faster │
│ QQuery 1     │    51.06 ms │                  50.60 ms │     no change │
│ QQuery 2     │   138.61 ms │                 135.59 ms │     no change │
│ QQuery 3     │   166.77 ms │                 167.90 ms │     no change │
│ QQuery 4     │  1119.95 ms │                1065.90 ms │     no change │
│ QQuery 5     │  1550.72 ms │                1508.70 ms │     no change │
│ QQuery 6     │     2.25 ms │                   2.27 ms │     no change │
│ QQuery 7     │    55.76 ms │                  55.97 ms │     no change │
│ QQuery 8     │  1480.58 ms │                1436.01 ms │     no change │
│ QQuery 9     │  1860.29 ms │                1809.27 ms │     no change │
│ QQuery 10    │   403.80 ms │                 393.32 ms │     no change │
│ QQuery 11    │   466.42 ms │                 441.63 ms │ +1.06x faster │
│ QQuery 12    │  1417.37 ms │                1371.35 ms │     no change │
│ QQuery 13    │  2157.15 ms │                2131.16 ms │     no change │
│ QQuery 14    │  1301.84 ms │                1274.99 ms │     no change │
│ QQuery 15    │  1281.30 ms │                1202.79 ms │ +1.07x faster │
│ QQuery 16    │  2704.20 ms │                2643.74 ms │     no change │
│ QQuery 17    │  2677.83 ms │                2632.10 ms │     no change │
│ QQuery 18    │  5009.69 ms │                4921.76 ms │     no change │
│ QQuery 19    │   128.81 ms │                 127.11 ms │     no change │
│ QQuery 20    │  2009.43 ms │                2000.36 ms │     no change │
│ QQuery 21    │  2302.65 ms │                2313.71 ms │     no change │
│ QQuery 22    │  4005.96 ms │                3950.86 ms │     no change │
│ QQuery 23    │ 12976.26 ms │               12908.74 ms │     no change │
│ QQuery 24    │   225.06 ms │                 217.97 ms │     no change │
│ QQuery 25    │   522.26 ms │                 501.31 ms │     no change │
│ QQuery 26    │   224.96 ms │                 214.46 ms │     no change │
│ QQuery 27    │  2926.27 ms │                2871.48 ms │     no change │
│ QQuery 28    │ 22735.49 ms │               22368.77 ms │     no change │
│ QQuery 29    │   972.18 ms │                 991.88 ms │     no change │
│ QQuery 30    │  1389.54 ms │                1320.77 ms │     no change │
│ QQuery 31    │  1355.95 ms │                1311.98 ms │     no change │
│ QQuery 32    │  4361.40 ms │                4567.59 ms │     no change │
│ QQuery 33    │  5668.58 ms │                5617.54 ms │     no change │
│ QQuery 34    │  5998.49 ms │                6129.10 ms │     no change │
│ QQuery 35    │  2012.46 ms │                2007.28 ms │     no change │
│ QQuery 36    │   119.65 ms │                 121.07 ms │     no change │
│ QQuery 37    │    52.22 ms │                  54.13 ms │     no change │
│ QQuery 38    │   121.01 ms │                 124.07 ms │     no change │
│ QQuery 39    │   196.61 ms │                 201.45 ms │     no change │
│ QQuery 40    │    41.27 ms │                  45.22 ms │  1.10x slower │
│ QQuery 41    │    39.20 ms │                  41.46 ms │  1.06x slower │
│ QQuery 42    │    33.40 ms │                  33.46 ms │     no change │
└──────────────┴─────────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 94267.38ms │
│ Total Time (alamb_test_adapative_scan)   │ 93289.12ms │
│ Average Time (HEAD)                      │  2192.26ms │
│ Average Time (alamb_test_adapative_scan) │  2169.51ms │
│ Queries Faster                           │          3 │
│ Queries Slower                           │          2 │
│ Queries with No Change                   │         38 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃      HEAD ┃ alamb_test_adapative_scan ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1     │ 171.69 ms │                 165.08 ms │     no change │
│ QQuery 2     │  25.99 ms │                  25.81 ms │     no change │
│ QQuery 3     │  37.22 ms │                  38.23 ms │     no change │
│ QQuery 4     │  28.53 ms │                  28.44 ms │     no change │
│ QQuery 5     │  78.32 ms │                  77.63 ms │     no change │
│ QQuery 6     │  19.76 ms │                  19.67 ms │     no change │
│ QQuery 7     │ 224.45 ms │                 228.98 ms │     no change │
│ QQuery 8     │  34.21 ms │                  36.02 ms │  1.05x slower │
│ QQuery 9     │ 103.08 ms │                 110.50 ms │  1.07x slower │
│ QQuery 10    │  60.16 ms │                  61.94 ms │     no change │
│ QQuery 11    │  17.02 ms │                  18.37 ms │  1.08x slower │
│ QQuery 12    │  51.95 ms │                  52.19 ms │     no change │
│ QQuery 13    │  46.85 ms │                  49.00 ms │     no change │
│ QQuery 14    │  13.83 ms │                  14.09 ms │     no change │
│ QQuery 15    │  24.71 ms │                  24.72 ms │     no change │
│ QQuery 16    │  25.55 ms │                  24.62 ms │     no change │
│ QQuery 17    │ 151.95 ms │                 153.78 ms │     no change │
│ QQuery 18    │ 333.56 ms │                 333.10 ms │     no change │
│ QQuery 19    │  36.55 ms │                  37.85 ms │     no change │
│ QQuery 20    │  50.07 ms │                  49.79 ms │     no change │
│ QQuery 21    │ 359.75 ms │                 311.37 ms │ +1.16x faster │
│ QQuery 22    │  20.34 ms │                  21.12 ms │     no change │
└──────────────┴───────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 1915.54ms │
│ Total Time (alamb_test_adapative_scan)   │ 1882.29ms │
│ Average Time (HEAD)                      │   87.07ms │
│ Average Time (alamb_test_adapative_scan) │   85.56ms │
│ Queries Faster                           │         1 │
│ Queries Slower                           │         3 │
│ Queries with No Change                   │        18 │
│ Queries with Failure                     │         0 │
└──────────────────────────────────────────┴───────────┘

alamb · 2025-10-30T13:15:42Z

The result is nice!

Yes indeed -- nicely done @hhhizzz

…scan

alamb · 2025-11-07T23:50:03Z

🤖 ./gh_compare_branch.sh Benchmark Script Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/test_adapative_scan (b5aab23) to a899ca0 diff using: clickbench_pushdown
Results will be posted here when complete

alamb · 2025-11-08T00:19:56Z

🤖: Benchmark completed

Details

Comparing HEAD and alamb_test_adapative_scan
--------------------
Benchmark clickbench_pushdown.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_test_adapative_scan ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │     2.19 ms │                   2.63 ms │  1.20x slower │
│ QQuery 1     │    52.97 ms │                  54.31 ms │     no change │
│ QQuery 2     │   139.74 ms │                 136.31 ms │     no change │
│ QQuery 3     │   161.13 ms │                 153.35 ms │     no change │
│ QQuery 4     │  1159.87 ms │                1070.02 ms │ +1.08x faster │
│ QQuery 5     │  1528.34 ms │                1483.21 ms │     no change │
│ QQuery 6     │     2.15 ms │                   2.24 ms │     no change │
│ QQuery 7     │    71.13 ms │                  71.11 ms │     no change │
│ QQuery 8     │  1553.29 ms │                1448.39 ms │ +1.07x faster │
│ QQuery 9     │  1918.78 ms │                1823.18 ms │     no change │
│ QQuery 10    │   494.95 ms │                 504.88 ms │     no change │
│ QQuery 11    │   552.07 ms │                 544.18 ms │     no change │
│ QQuery 12    │  1648.76 ms │                1542.81 ms │ +1.07x faster │
│ QQuery 13    │  2604.62 ms │                2319.10 ms │ +1.12x faster │
│ QQuery 14    │  1764.46 ms │                1434.08 ms │ +1.23x faster │
│ QQuery 15    │  1317.28 ms │                1244.12 ms │ +1.06x faster │
│ QQuery 16    │  2714.57 ms │                2675.97 ms │     no change │
│ QQuery 17    │  2715.14 ms │                2655.58 ms │     no change │
│ QQuery 18    │  5372.95 ms │                4970.23 ms │ +1.08x faster │
│ QQuery 19    │   145.18 ms │                 138.74 ms │     no change │
│ QQuery 20    │  2025.19 ms │                1826.55 ms │ +1.11x faster │
│ QQuery 21    │  2432.56 ms │                2265.32 ms │ +1.07x faster │
│ QQuery 22    │  4603.82 ms │                3869.15 ms │ +1.19x faster │
│ QQuery 23    │  1196.48 ms │                1067.82 ms │ +1.12x faster │
│ QQuery 24    │   266.22 ms │                 240.74 ms │ +1.11x faster │
│ QQuery 25    │   669.92 ms │                 647.87 ms │     no change │
│ QQuery 26    │   346.65 ms │                 347.49 ms │     no change │
│ QQuery 27    │  3233.83 ms │                2868.73 ms │ +1.13x faster │
│ QQuery 28    │ 24207.24 ms │               24604.80 ms │     no change │
│ QQuery 29    │  1008.68 ms │                 966.27 ms │     no change │
│ QQuery 30    │  2222.28 ms │                1359.65 ms │ +1.63x faster │
│ QQuery 31    │  2129.34 ms │                1340.92 ms │ +1.59x faster │
│ QQuery 32    │  5072.35 ms │                4541.65 ms │ +1.12x faster │
│ QQuery 33    │  5925.37 ms │                5530.06 ms │ +1.07x faster │
│ QQuery 34    │  6211.66 ms │                5836.20 ms │ +1.06x faster │
│ QQuery 35    │  2095.10 ms │                2023.66 ms │     no change │
│ QQuery 36    │    25.29 ms │                  26.21 ms │     no change │
│ QQuery 37    │    25.98 ms │                  26.08 ms │     no change │
│ QQuery 38    │    25.32 ms │                  25.38 ms │     no change │
│ QQuery 39    │    24.80 ms │                  25.16 ms │     no change │
│ QQuery 40    │    26.62 ms │                  26.73 ms │     no change │
│ QQuery 41    │    25.45 ms │                  25.40 ms │     no change │
│ QQuery 42    │    25.78 ms │                  25.57 ms │     no change │
└──────────────┴─────────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 89745.51ms │
│ Total Time (alamb_test_adapative_scan)   │ 83791.85ms │
│ Average Time (HEAD)                      │  2087.10ms │
│ Average Time (alamb_test_adapative_scan) │  1948.65ms │
│ Queries Faster                           │         18 │
│ Queries Slower                           │          1 │
│ Queries with No Change                   │         24 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘

alamb · 2025-11-08T00:19:59Z

🤖 ./gh_compare_branch.sh Benchmark Script Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/test_adapative_scan (b5aab23) to a899ca0 diff using: tpch_mem clickbench_partitioned clickbench_extended
Results will be posted here when complete

alamb · 2025-11-08T01:04:12Z

🤖: Benchmark completed

Details

Comparing HEAD and alamb_test_adapative_scan
--------------------
Benchmark clickbench_extended.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_test_adapative_scan ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │  2676.67 ms │                2624.89 ms │     no change │
│ QQuery 1     │  1175.51 ms │                1245.27 ms │  1.06x slower │
│ QQuery 2     │  2354.09 ms │                2288.66 ms │     no change │
│ QQuery 3     │  1196.04 ms │                1192.91 ms │     no change │
│ QQuery 4     │  2356.34 ms │                2267.45 ms │     no change │
│ QQuery 5     │ 28033.52 ms │               28594.32 ms │     no change │
│ QQuery 6     │  4220.81 ms │                3863.08 ms │ +1.09x faster │
│ QQuery 7     │  3500.60 ms │                3778.17 ms │  1.08x slower │
└──────────────┴─────────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 45513.58ms │
│ Total Time (alamb_test_adapative_scan)   │ 45854.75ms │
│ Average Time (HEAD)                      │  5689.20ms │
│ Average Time (alamb_test_adapative_scan) │  5731.84ms │
│ Queries Faster                           │          1 │
│ Queries Slower                           │          2 │
│ Queries with No Change                   │          5 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_test_adapative_scan ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │     2.15 ms │                   2.27 ms │  1.06x slower │
│ QQuery 1     │    49.04 ms │                  50.31 ms │     no change │
│ QQuery 2     │   137.12 ms │                 135.59 ms │     no change │
│ QQuery 3     │   163.46 ms │                 155.53 ms │     no change │
│ QQuery 4     │  1156.73 ms │                1075.21 ms │ +1.08x faster │
│ QQuery 5     │  1593.37 ms │                1465.50 ms │ +1.09x faster │
│ QQuery 6     │     2.20 ms │                   2.28 ms │     no change │
│ QQuery 7     │    54.44 ms │                  57.27 ms │  1.05x slower │
│ QQuery 8     │  1576.22 ms │                1467.02 ms │ +1.07x faster │
│ QQuery 9     │  1954.44 ms │                1864.74 ms │     no change │
│ QQuery 10    │   376.39 ms │                 371.21 ms │     no change │
│ QQuery 11    │   434.67 ms │                 422.10 ms │     no change │
│ QQuery 12    │  1474.97 ms │                1347.53 ms │ +1.09x faster │
│ QQuery 13    │  2193.07 ms │                2104.27 ms │     no change │
│ QQuery 14    │  1349.75 ms │                1340.50 ms │     no change │
│ QQuery 15    │  1348.08 ms │                1233.51 ms │ +1.09x faster │
│ QQuery 16    │  2763.49 ms │                2645.50 ms │     no change │
│ QQuery 17    │  2744.81 ms │                2657.64 ms │     no change │
│ QQuery 18    │  5000.81 ms │                4995.25 ms │     no change │
│ QQuery 19    │   128.94 ms │                 121.88 ms │ +1.06x faster │
│ QQuery 20    │  2046.44 ms │                1869.17 ms │ +1.09x faster │
│ QQuery 21    │  2346.43 ms │                2158.65 ms │ +1.09x faster │
│ QQuery 22    │  4001.51 ms │                3708.05 ms │ +1.08x faster │
│ QQuery 23    │ 12886.66 ms │               12209.69 ms │ +1.06x faster │
│ QQuery 24    │   220.73 ms │                 202.73 ms │ +1.09x faster │
│ QQuery 25    │   483.57 ms │                 455.39 ms │ +1.06x faster │
│ QQuery 26    │   222.85 ms │                 208.23 ms │ +1.07x faster │
│ QQuery 27    │  2819.16 ms │                2626.83 ms │ +1.07x faster │
│ QQuery 28    │ 23410.88 ms │               24040.80 ms │     no change │
│ QQuery 29    │  1006.39 ms │                 990.89 ms │     no change │
│ QQuery 30    │  1420.35 ms │                1322.07 ms │ +1.07x faster │
│ QQuery 31    │  1417.40 ms │                1320.07 ms │ +1.07x faster │
│ QQuery 32    │  5313.83 ms │                4760.46 ms │ +1.12x faster │
│ QQuery 33    │  6117.10 ms │                5727.68 ms │ +1.07x faster │
│ QQuery 34    │  6400.49 ms │                5958.18 ms │ +1.07x faster │
│ QQuery 35    │  2160.34 ms │                2036.23 ms │ +1.06x faster │
│ QQuery 36    │   123.40 ms │                 112.33 ms │ +1.10x faster │
│ QQuery 37    │    51.75 ms │                  51.60 ms │     no change │
│ QQuery 38    │   120.07 ms │                 118.57 ms │     no change │
│ QQuery 39    │   202.82 ms │                 190.63 ms │ +1.06x faster │
│ QQuery 40    │    43.33 ms │                  39.80 ms │ +1.09x faster │
│ QQuery 41    │    39.18 ms │                  38.36 ms │     no change │
│ QQuery 42    │    33.22 ms │                  31.52 ms │ +1.05x faster │
└──────────────┴─────────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 97392.05ms │
│ Total Time (alamb_test_adapative_scan)   │ 93693.06ms │
│ Average Time (HEAD)                      │  2264.93ms │
│ Average Time (alamb_test_adapative_scan) │  2178.91ms │
│ Queries Faster                           │         24 │
│ Queries Slower                           │          2 │
│ Queries with No Change                   │         17 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃      HEAD ┃ alamb_test_adapative_scan ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1     │ 133.56 ms │                 137.71 ms │     no change │
│ QQuery 2     │  29.14 ms │                  28.50 ms │     no change │
│ QQuery 3     │  39.02 ms │                  38.92 ms │     no change │
│ QQuery 4     │  28.93 ms │                  29.27 ms │     no change │
│ QQuery 5     │  86.61 ms │                  88.01 ms │     no change │
│ QQuery 6     │  19.76 ms │                  19.49 ms │     no change │
│ QQuery 7     │ 225.98 ms │                 225.86 ms │     no change │
│ QQuery 8     │  34.67 ms │                  34.45 ms │     no change │
│ QQuery 9     │ 110.83 ms │                 107.14 ms │     no change │
│ QQuery 10    │  65.94 ms │                  64.02 ms │     no change │
│ QQuery 11    │  17.66 ms │                  17.95 ms │     no change │
│ QQuery 12    │  51.99 ms │                  50.96 ms │     no change │
│ QQuery 13    │  48.00 ms │                  47.36 ms │     no change │
│ QQuery 14    │  14.58 ms │                  13.73 ms │ +1.06x faster │
│ QQuery 15    │  25.60 ms │                  24.80 ms │     no change │
│ QQuery 16    │  25.12 ms │                  25.52 ms │     no change │
│ QQuery 17    │ 153.31 ms │                 153.38 ms │     no change │
│ QQuery 18    │ 281.60 ms │                 272.01 ms │     no change │
│ QQuery 19    │  37.36 ms │                  39.34 ms │  1.05x slower │
│ QQuery 20    │  49.35 ms │                  49.39 ms │     no change │
│ QQuery 21    │ 335.59 ms │                 336.23 ms │     no change │
│ QQuery 22    │  21.24 ms │                  21.55 ms │     no change │
└──────────────┴───────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 1835.85ms │
│ Total Time (alamb_test_adapative_scan)   │ 1825.61ms │
│ Average Time (HEAD)                      │   83.45ms │
│ Average Time (alamb_test_adapative_scan) │   82.98ms │
│ Queries Faster                           │         1 │
│ Queries Slower                           │         1 │
│ Queries with No Change                   │        20 │
│ Queries with Failure                     │         0 │
└──────────────────────────────────────────┴───────────┘

alamb · 2025-11-08T01:04:17Z

🤖 ./gh_compare_branch.sh Benchmark Script Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/test_adapative_scan (b5aab23) to a899ca0 diff using: clickbench_pushdown
Results will be posted here when complete

alamb · 2025-11-08T01:19:34Z

🤖: Benchmark completed

Details

Comparing HEAD and alamb_test_adapative_scan
--------------------
Benchmark clickbench_pushdown.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_test_adapative_scan ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │     2.73 ms │                   2.65 ms │     no change │
│ QQuery 1     │    53.19 ms │                  56.31 ms │  1.06x slower │
│ QQuery 2     │   142.12 ms │                 132.08 ms │ +1.08x faster │
│ QQuery 3     │   164.32 ms │                 155.39 ms │ +1.06x faster │
│ QQuery 4     │  1097.45 ms │                1056.46 ms │     no change │
│ QQuery 5     │  1493.46 ms │                1482.88 ms │     no change │
│ QQuery 6     │     2.13 ms │                   2.16 ms │     no change │
│ QQuery 7     │    69.80 ms │                  74.18 ms │  1.06x slower │
│ QQuery 8     │  1505.69 ms │                1459.58 ms │     no change │
│ QQuery 9     │  1903.55 ms │                1842.12 ms │     no change │
│ QQuery 10    │   490.85 ms │                 505.19 ms │     no change │
│ QQuery 11    │   559.43 ms │                 552.13 ms │     no change │
│ QQuery 12    │  1607.21 ms │                1546.88 ms │     no change │
│ QQuery 13    │  2580.67 ms │                2340.69 ms │ +1.10x faster │
│ QQuery 14    │  1701.15 ms │                1450.50 ms │ +1.17x faster │
│ QQuery 15    │  1214.19 ms │                1216.36 ms │     no change │
│ QQuery 16    │  2707.73 ms │                2652.63 ms │     no change │
│ QQuery 17    │  2691.21 ms │                2651.63 ms │     no change │
│ QQuery 18    │  5211.58 ms │                4913.84 ms │ +1.06x faster │
│ QQuery 19    │   148.45 ms │                 140.02 ms │ +1.06x faster │
│ QQuery 20    │  2011.01 ms │                1828.23 ms │ +1.10x faster │
│ QQuery 21    │  2479.96 ms │                2283.61 ms │ +1.09x faster │
│ QQuery 22    │  4792.44 ms │                3877.30 ms │ +1.24x faster │
│ QQuery 23    │  1193.60 ms │                1056.24 ms │ +1.13x faster │
│ QQuery 24    │   260.29 ms │                 250.76 ms │     no change │
│ QQuery 25    │   653.94 ms │                 643.31 ms │     no change │
│ QQuery 26    │   355.81 ms │                 346.08 ms │     no change │
│ QQuery 27    │  3203.92 ms │                2884.90 ms │ +1.11x faster │
│ QQuery 28    │ 24167.73 ms │               24493.13 ms │     no change │
│ QQuery 29    │   977.84 ms │                 989.19 ms │     no change │
│ QQuery 30    │  2173.15 ms │                1354.00 ms │ +1.60x faster │
│ QQuery 31    │  2127.75 ms │                1350.21 ms │ +1.58x faster │
│ QQuery 32    │  4904.42 ms │                4513.08 ms │ +1.09x faster │
│ QQuery 33    │  5862.45 ms │                5608.58 ms │     no change │
│ QQuery 34    │  6109.96 ms │                5893.59 ms │     no change │
│ QQuery 35    │  2074.11 ms │                1980.05 ms │     no change │
│ QQuery 36    │    25.37 ms │                  25.76 ms │     no change │
│ QQuery 37    │    26.06 ms │                  25.57 ms │     no change │
│ QQuery 38    │    25.41 ms │                  25.70 ms │     no change │
│ QQuery 39    │    24.96 ms │                  25.81 ms │     no change │
│ QQuery 40    │    27.33 ms │                  26.59 ms │     no change │
│ QQuery 41    │    25.50 ms │                  25.78 ms │     no change │
│ QQuery 42    │    26.02 ms │                  25.16 ms │     no change │
└──────────────┴─────────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 88875.94ms │
│ Total Time (alamb_test_adapative_scan)   │ 83766.34ms │
│ Average Time (HEAD)                      │  2066.88ms │
│ Average Time (alamb_test_adapative_scan) │  1948.05ms │
│ Queries Faster                           │         14 │
│ Queries Slower                           │          2 │
│ Queries with No Change                   │         27 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘

alamb · 2025-11-08T01:19:37Z

🤖 ./gh_compare_branch.sh Benchmark Script Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/test_adapative_scan (b5aab23) to a899ca0 diff using: tpch_mem clickbench_partitioned clickbench_extended
Results will be posted here when complete

alamb · 2025-11-08T01:44:40Z

🤖: Benchmark completed

Details

Comparing HEAD and alamb_test_adapative_scan
--------------------
Benchmark clickbench_extended.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_test_adapative_scan ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │  2677.12 ms │                2614.27 ms │     no change │
│ QQuery 1     │  1210.97 ms │                1243.11 ms │     no change │
│ QQuery 2     │  2307.48 ms │                2364.97 ms │     no change │
│ QQuery 3     │  1157.83 ms │                1208.61 ms │     no change │
│ QQuery 4     │  2308.14 ms │                2232.21 ms │     no change │
│ QQuery 5     │ 28134.36 ms │               28236.58 ms │     no change │
│ QQuery 6     │  4217.19 ms │                3882.47 ms │ +1.09x faster │
│ QQuery 7     │  3552.86 ms │                3420.73 ms │     no change │
└──────────────┴─────────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 45565.95ms │
│ Total Time (alamb_test_adapative_scan)   │ 45202.95ms │
│ Average Time (HEAD)                      │  5695.74ms │
│ Average Time (alamb_test_adapative_scan) │  5650.37ms │
│ Queries Faster                           │          1 │
│ Queries Slower                           │          0 │
│ Queries with No Change                   │          7 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_test_adapative_scan ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │     2.29 ms │                   2.63 ms │  1.15x slower │
│ QQuery 1     │    49.54 ms │                  50.68 ms │     no change │
│ QQuery 2     │   135.50 ms │                 137.35 ms │     no change │
│ QQuery 3     │   166.00 ms │                 153.39 ms │ +1.08x faster │
│ QQuery 4     │  1131.46 ms │                1188.78 ms │  1.05x slower │
│ QQuery 5     │  1523.07 ms │                1564.03 ms │     no change │
│ QQuery 6     │     2.32 ms │                   2.30 ms │     no change │
│ QQuery 7     │    55.58 ms │                  54.67 ms │     no change │
│ QQuery 8     │  1496.49 ms │                1543.36 ms │     no change │
│ QQuery 9     │  1853.55 ms │                1938.08 ms │     no change │
│ QQuery 10    │   375.31 ms │                 374.69 ms │     no change │
│ QQuery 11    │   427.90 ms │                 430.20 ms │     no change │
│ QQuery 12    │  1391.15 ms │                1345.49 ms │     no change │
│ QQuery 13    │  2168.81 ms │                2161.24 ms │     no change │
│ QQuery 14    │  1273.62 ms │                1313.86 ms │     no change │
│ QQuery 15    │  1208.12 ms │                1305.86 ms │  1.08x slower │
│ QQuery 16    │  2710.14 ms │                2687.53 ms │     no change │
│ QQuery 17    │  2661.88 ms │                2667.53 ms │     no change │
│ QQuery 18    │  4953.89 ms │                4905.19 ms │     no change │
│ QQuery 19    │   131.13 ms │                 121.55 ms │ +1.08x faster │
│ QQuery 20    │  1987.13 ms │                1864.22 ms │ +1.07x faster │
│ QQuery 21    │  2303.94 ms │                2136.93 ms │ +1.08x faster │
│ QQuery 22    │  3956.42 ms │                3673.67 ms │ +1.08x faster │
│ QQuery 23    │ 12760.06 ms │               12277.43 ms │     no change │
│ QQuery 24    │   216.46 ms │                 201.24 ms │ +1.08x faster │
│ QQuery 25    │   479.48 ms │                 462.65 ms │     no change │
│ QQuery 26    │   212.92 ms │                 217.15 ms │     no change │
│ QQuery 27    │  2819.26 ms │                2644.28 ms │ +1.07x faster │
│ QQuery 28    │ 23133.47 ms │               23829.71 ms │     no change │
│ QQuery 29    │   997.10 ms │                 981.24 ms │     no change │
│ QQuery 30    │  1343.73 ms │                1367.32 ms │     no change │
│ QQuery 31    │  1394.63 ms │                1297.92 ms │ +1.07x faster │
│ QQuery 32    │  4483.37 ms │                4617.55 ms │     no change │
│ QQuery 33    │  5758.72 ms │                5657.70 ms │     no change │
│ QQuery 34    │  5974.69 ms │                5646.99 ms │ +1.06x faster │
│ QQuery 35    │  2039.41 ms │                2080.87 ms │     no change │
│ QQuery 36    │   121.75 ms │                 115.61 ms │ +1.05x faster │
│ QQuery 37    │    50.54 ms │                  51.06 ms │     no change │
│ QQuery 38    │   121.30 ms │                 114.57 ms │ +1.06x faster │
│ QQuery 39    │   197.01 ms │                 188.62 ms │     no change │
│ QQuery 40    │    42.89 ms │                  42.29 ms │     no change │
│ QQuery 41    │    38.11 ms │                  39.53 ms │     no change │
│ QQuery 42    │    31.98 ms │                  31.93 ms │     no change │
└──────────────┴─────────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 94182.14ms │
│ Total Time (alamb_test_adapative_scan)   │ 93488.85ms │
│ Average Time (HEAD)                      │  2190.28ms │
│ Average Time (alamb_test_adapative_scan) │  2174.16ms │
│ Queries Faster                           │         11 │
│ Queries Slower                           │          3 │
│ Queries with No Change                   │         29 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃      HEAD ┃ alamb_test_adapative_scan ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1     │ 134.53 ms │                 135.61 ms │     no change │
│ QQuery 2     │  29.20 ms │                  29.64 ms │     no change │
│ QQuery 3     │  35.06 ms │                  40.33 ms │  1.15x slower │
│ QQuery 4     │  29.49 ms │                  29.05 ms │     no change │
│ QQuery 5     │  87.27 ms │                  86.92 ms │     no change │
│ QQuery 6     │  19.74 ms │                  19.63 ms │     no change │
│ QQuery 7     │ 236.91 ms │                 230.03 ms │     no change │
│ QQuery 8     │  33.58 ms │                  34.72 ms │     no change │
│ QQuery 9     │ 111.63 ms │                 113.72 ms │     no change │
│ QQuery 10    │  63.87 ms │                  66.21 ms │     no change │
│ QQuery 11    │  18.77 ms │                  17.52 ms │ +1.07x faster │
│ QQuery 12    │  52.20 ms │                  54.12 ms │     no change │
│ QQuery 13    │  47.63 ms │                  46.86 ms │     no change │
│ QQuery 14    │  14.50 ms │                  14.15 ms │     no change │
│ QQuery 15    │  25.09 ms │                  24.97 ms │     no change │
│ QQuery 16    │  25.05 ms │                  25.59 ms │     no change │
│ QQuery 17    │ 151.70 ms │                 157.76 ms │     no change │
│ QQuery 18    │ 279.98 ms │                 275.68 ms │     no change │
│ QQuery 19    │  37.19 ms │                  39.29 ms │  1.06x slower │
│ QQuery 20    │  50.32 ms │                  49.90 ms │     no change │
│ QQuery 21    │ 335.78 ms │                 347.11 ms │     no change │
│ QQuery 22    │  21.23 ms │                  21.71 ms │     no change │
└──────────────┴───────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 1840.72ms │
│ Total Time (alamb_test_adapative_scan)   │ 1860.52ms │
│ Average Time (HEAD)                      │   83.67ms │
│ Average Time (alamb_test_adapative_scan) │   84.57ms │
│ Queries Faster                           │         1 │
│ Queries Slower                           │         2 │
│ Queries with No Change                   │        19 │
│ Queries with Failure                     │         0 │
└──────────────────────────────────────────┴───────────┘

alamb · 2025-11-25T20:12:21Z

Code is shipped -- no need for this PR anymore!

github-actions bot added the common Related to common crate label Oct 29, 2025

WIP: Pin to Adaptive Parquet Predicate Pushdown

a0cbf52

alamb force-pushed the alamb/test_adapative_scan branch from c181597 to a0cbf52 Compare October 30, 2025 11:04

github-actions bot removed the common Related to common crate label Oct 30, 2025

alamb added 3 commits November 7, 2025 16:00

Update pin to latest

63749f6

Update cargo.lock

029baac

Merge remote-tracking branch 'apache/main' into alamb/test_adapative_…

b5aab23

…scan

alamb closed this Nov 25, 2025

alamb deleted the alamb/test_adapative_scan branch November 25, 2025 20:12

Conversation

alamb commented Oct 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alamb commented Oct 30, 2025

Uh oh!

alamb commented Oct 30, 2025

Uh oh!

hhhizzz commented Oct 30, 2025

Uh oh!

zhuqi-lucas commented Oct 30, 2025

Uh oh!

alamb commented Oct 30, 2025

Uh oh!

alamb commented Oct 30, 2025

Uh oh!

alamb commented Oct 30, 2025

Uh oh!

alamb commented Oct 30, 2025

Uh oh!

zhuqi-lucas commented Oct 30, 2025

Uh oh!

alamb commented Oct 30, 2025

Uh oh!

alamb commented Oct 30, 2025

Uh oh!

alamb commented Nov 7, 2025

Uh oh!

alamb commented Nov 8, 2025

Uh oh!

alamb commented Nov 8, 2025

Uh oh!

alamb commented Nov 8, 2025

Uh oh!

alamb commented Nov 8, 2025

Uh oh!

alamb commented Nov 8, 2025

Uh oh!

alamb commented Nov 8, 2025

Uh oh!

alamb commented Nov 8, 2025

Uh oh!

alamb commented Nov 25, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

alamb commented Oct 29, 2025 •

edited

Loading