Improve Metal backend with hybrid CPU+GPU+ANE inference for macOS

[ChinChangYang]

Summary

This PR adds a Metal neural network backend for macOS that leverages Apple's full compute stack—CPU, GPU, and Apple Neural Engine (ANE) simultaneously—through a per-thread multiplexer architecture.

Key Features

• Per-Thread GPU/ANE Multiplexer: Each NN server thread is dedicated to either GPU (MPSGraph) or ANE (CoreML). Multiple threads run in parallel, saturating all compute units without intra-batch splitting overhead.
• Native Model Conversion: Uses the katagocoreml C++ library for on-the-fly conversion from KataGo's .bin.gz format to CoreML .mlpackage (no Python dependency).
• Configurable Precision: FP16 (default) enables both GPU and ANE paths; FP32 uses GPU-only MPSGraph, bypassing CoreML conversion entirely.
• Dynamic Batch Size: Supports runtime batch sizes from 1 to maxBatchSize without model recompilation.
• Flexible Dispatch Configuration: Users configure compute unit assignment per thread via metalDeviceToUseThread<N> (0 = GPU, 100 = ANE), allowing GPU-only, ANE-only, or mux mode.

Architecture

The backend uses a multiplexer design where each NN server thread owns exactly one compute handle:

• GPU threads (metalDeviceToUseThread<N>=0): Use MPSGraph for Metal GPU inference
• ANE threads (metalDeviceToUseThread<N>=100): Use CoreML for CPU+ANE inference

Example mux configuration (recommended for best throughput):

numNNServerThreadsPerModel = 4
metalDeviceToUseThread0 = 0    # GPU
metalDeviceToUseThread1 = 0    # GPU
metalDeviceToUseThread2 = 100  # ANE
metalDeviceToUseThread3 = 100  # ANE

Performance

Benchmarked on Apple M3 Max with mux mode (2 GPU + 2 ANE threads), 2048 visits, -half-batch-size:

b18c384nbt (18 blocks, 384 channels):

Threads	visits/s	nnEvals/s	avgBatchSize
8	735.68	570.97	1.46
16	817.86	629.23	2.70
32	810.71	660.98	5.02

b28c512nbt (28 blocks, 512 channels):

Threads	visits/s	nnEvals/s	avgBatchSize
8	325.57	256.51	1.47
16	335.40	263.98	2.61
32	319.61	255.57	4.91

Per-thread workload distribution (b18c384nbt, 32 threads):

• GPU threads: ~24,200 rows processed
• ANE threads: ~24,572 rows processed

Files Changed

File	Description
cpp/neuralnet/metalbackend.cpp	C++ interface: model conversion, batch processing, data marshaling
cpp/neuralnet/metalbackend.h	Backend header with compute handle and layer descriptors
cpp/neuralnet/metalbackend.swift	CoreML model loading and per-thread compute handle orchestration
cpp/neuralnet/metallayers.swift	MPSGraph layer implementations for GPU inference path
cpp/CMakeLists.txt	Build configuration for Metal backend (Ninja + Swift)
cpp/command/benchmark.cpp	-half-batch-size option for benchmarking
cpp/configs/gtp_example.cfg	Metal backend dispatch configuration documentation
cpp/configs/analysis_example.cfg	Metal backend dispatch configuration documentation
.github/workflows/build.yml	CI job for Metal backend on macOS
Compiling.md	Build instructions for Metal backend

Build Requirements

• macOS 13.0+
• Xcode Command Line Tools with Swift 5.9+
• Ninja build system: brew install ninja
• katagocoreml library: brew tap chinchangyang/katagocoreml-cpp && brew install katagocoreml

cd cpp
cmake -G Ninja -DUSE_BACKEND=METAL -DBUILD_DISTRIBUTED=1
ninja

Test Plan

• ./katago runtests passes
• ./katago benchmark verifies performance with mux mode (GPU+ANE)
• GPU-only mode (metalDeviceToUseThread0=0) works correctly
• ANE-only mode (metalDeviceToUseThread0=100) works correctly
• FP16 mode: both GPU and ANE paths active
• FP32 mode: GPU-only MPSGraph execution (CoreML bypassed)
• Batch size 1: single compute handle works correctly
• CI build succeeds on GitHub Actions

[ChinChangYang]

Cross-Validation Test Report for CoreML Backend

Test Configuration

• Model: b18c384nbt-uec-20221121b.bin.gz (18-block, 384-channel model)
• Board Size: 19x19
• Test Positions: 2,247 positions from built-in test dataset
• Reference: Eigen backend (CPU, FP32)
• Test Command: ./katago testgpuerror

Hardware

• Apple M3 Max
• macOS with CoreML + MPSGraph hybrid backend

Results

CoreML FP32 vs Eigen FP32 Reference

Metric	Average	90th %	99th %	Max	Threshold (99th/Max)
winrateError	0.00006%	0.00017%	0.00035%	0.00066%	0.45% / 1.35%
leadError	0.00002	0.00003	0.00011	0.00033	0.225 / 0.90
scoreMeanError	0.00002	0.00004	0.00012	0.00030	0.225 / 0.90
scoreStdevError	0.00001	0.00001	0.00004	0.00009	0.135 / 0.54
topPolicyDelta	0.00007%	0.00017%	0.00036%	0.00068%	0.45% / 1.35%
policyKLDiv	0.000000	0.000000	0.000000	0.000000	0.0006 / 0.0012
ownershipError	0.00003c	0.00006c	0.00018c	0.00174c	—

Status: PASS — CoreML FP32 matches Eigen FP32 with near-zero error.

CoreML FP16 vs Eigen FP32 Reference

Metric	Average	90th %	99th %	Max	Threshold (99th/Max)
winrateError	0.0997%	0.2553%	0.4857%	0.9147%	2.0% / 5.0%
leadError	0.0277	0.0615	0.1754	0.3852	1.00 / 3.00
scoreMeanError	0.0332	0.0718	0.1721	0.3739	1.00 / 3.00
scoreStdevError	0.0129	0.0274	0.0658	0.1646	0.60 / 1.80
topPolicyDelta	0.0944%	0.2124%	0.4547%	0.8814%	2.50% / 6.00%
policyKLDiv	0.000017	0.000035	0.000119	0.000604	0.0020 / 0.0040
ownershipError	0.0443c	0.1000c	0.2420c	4.3359c	—

Status: PASS — CoreML FP16 is well within acceptable error bounds for half-precision inference.

Summary

Configuration	Max Winrate Error	Max Policy KL Div	Result
CoreML FP32	0.00066%	0.000000	PASS
CoreML FP16	0.91%	0.000604	PASS

The CoreML backend passes all validation checks:

• FP32 mode: Numerically equivalent to Eigen reference (errors < 0.001%)
• FP16 mode: Max winrate error of 0.91% is well below the 5% threshold, consistent with expected half-precision behavior

Conclusion

The hybrid CoreML + MPSGraph backend produces numerically correct results across 2,247 test positions when compared against the Eigen CPU reference implementation. Both FP16 and FP32 precision modes meet KataGo's accuracy requirements for neural network inference.

[KvanTTT]

Is it possible to merge macOS build into a single configuration? As I see some build steps are overlapping.

[ChinChangYang]

Absolutely! It can be merged into a single configuration. Would you like to review it? ChinChangYang#9