Update examples

[maleadt]

No description provided.

[AntonOresten]

Any idea why matmul FLOP/S dropped and layer norm got so much better? The explanations behind the relative performance seems to have flipped, comparing memory and compute-bound kernels @maleadt

[maleadt]

I think I did the previous measurements with nsys, so only measuring kernel times as opposed to measuring the whole call here (including launch overhead etc).

Layernorm got much better because the example wasn't storing the fastest iterating dimension as the contiguous one, as became obvious after the IR / code still containing permute calls post #142 (which shouldn't be necessary anymore).

[AntonOresten]

so only measuring kernel times as opposed to measuring the whole call here

So was the previous comparison not apples-to-apples?

Also, the new paragraph in the README states:

Compute-intensive kernels (matmul, batch matmul, FFT) are slower due to conservative token threading in the generated Tile IR, which serializes loads that could otherwise be pipelined.

Is the logic here that it's not bandwidth limited, but spends a less portion of time on compute because it's waiting on serialized loads?

Does the closing of #1 have the anticipated effect?

[maleadt]

so only measuring kernel times as opposed to measuring the whole call here

So was the previous comparison not apples-to-apples?

It was apples-to-apples, but because the times it was using were lower the reported throughput was slightly larger. That said, these benchmarks are not rigorous, just indicative of the expected performance.

Compute-intensive kernels (matmul, batch matmul, FFT) are slower due to conservative token threading in the generated Tile IR, which serializes loads that could otherwise be pipelined.

Is the logic here that it's not bandwidth limited, but spends a less portion of time on compute because it's waiting on serialized loads?

Yeah. However, fixing #1 didn't fix the performance as I expected (as indicated in that paragraph). I've found the actual issue though; PRs incoming.