[SPARK-58080][PYTHON][TESTS] Add ASV microbenchmark for SQL_GROUPED_MAP_PANDAS_UDF_WITH_STATE

[Yicong-Huang]

What changes were proposed in this pull request?

Add an ASV micro-benchmark for SQL_GROUPED_MAP_PANDAS_UDF_WITH_STATE (applyInPandasWithState) to bench_eval_type.py.

Unlike TransformWithState, no state server socket is involved: ApplyInPandasWithStateSerializer reconstructs each GroupState entirely from a metadata column carried inline in the Arrow stream. The benchmark emits the data columns followed by one trailing struct column (__state, matching the JVM ApplyInPandasWithStateWriter.STATE_METADATA_SCHEMA), and faithfully reproduces the JVM writer's bin-packing and cross-batch chunking (isLastChunk set only on a group's final chunk), so a group split across batches reassembles into one GroupState.

Scenarios cover few/many groups, small/large group sizes, wide columns, mixed value types (string/binary/boolean) and a nested struct column. UDFs: identity_udf, sort_udf, count_udf. identity_udf/sort_udf pass values through; count_udf exercises the per-group state read/write path (reads the running count via getOption, writes it back via update) and re-emits the key plus the count.

Why are the changes needed?

Part of SPARK-55724. Establishes a performance baseline before refactoring SQL_GROUPED_MAP_PANDAS_UDF_WITH_STATE.

Does this PR introduce any user-facing change?

No

How was this patch tested?

COLUMNS=120 asv run --python=same --bench "ApplyInPandasWithState" -a "repeat=3" (one of two stable runs):

ApplyInPandasWithStateUDFTimeBench:

================ ============== ============ ============
--                                 udf
---------------- ----------------------------------------
    scenario      identity_udf    sort_udf    count_udf
================ ============== ============ ============
 few_groups_sm      1.02+-0s      1.06+-0.01s   128+-1ms
 few_groups_lg     6.98+-0.05s    7.18+-0.05s   762+-3ms
 many_groups_sm    6.76+-0.04s    7.62+-0.06s  3.14+-0.01s
 many_groups_lg    4.80+-0.1s     4.95+-0.08s   934+-3ms
   wide_cols       5.92+-0.02s    6.17+-0.1s    878+-20ms
   mixed_cols      4.49+-0.01s    4.76+-0.04s   704+-5ms
 nested_struct     9.17+-0.1s     10.8+-0.07s  3.32+-0.02s
================ ============== ============ ============

ApplyInPandasWithStateUDFPeakmemBench:

================ ============== ========== ===========
--                                udf
---------------- -------------------------------------
    scenario      identity_udf   sort_udf   count_udf
================ ============== ========== ===========
 few_groups_sm        118M         119M        106M
 few_groups_lg        272M         280M        221M
 many_groups_sm       172M         173M        144M
 many_groups_lg       172M         178M        140M
   wide_cols          284M         282M        233M
   mixed_cols         187M         188M        182M
 nested_struct        227M         228M        211M
================ ============== ========== ===========

Was this patch authored or co-authored using generative AI tooling?

No