Add CAGradientLayer, Allow Layer Flattening for proper masking and opacity compositing by ephemer · Pull Request #408 · flowkey/UIKit-cross-platform

ephemer · 2025-12-12T15:49:02Z

Type of change: Feature

Motivation (current vs expected behavior)

Add a simple CPU-only (for now) CAGradientLayer and the beginnings of a GPU implementation.
Add layer flattening
- With this, non-trivial layers with non-zero opacity and/or a mask are rendered as a separate render tree and then composited onto the window (or into each other)
- This enables proper masking, e.g. via a gradient layer (via alpha) or using non-trivial shapes

…dd CAGradientLayer

ephemer · 2025-12-12T16:37:00Z

Sources/AVPlayerLayer+Mac.swift

            contentsScale = 1.0 // this doesn't work on init because we set contentsScale in UIView.init afterwards
-            contents = VideoTexture(width: width, height: height, format: GPU_FORMAT_RGBA)
-        }
-
-        // Swap R and B values to get RGBA pixels instead of BGRA:
-        let bufferSize = CVPixelBufferGetDataSize(pixelBuffer)
-        for i in stride(from: 0, to: bufferSize, by: 16) {
-            swap(&pixelBytes[i], &pixelBytes[i + 2])
-            swap(&pixelBytes[i+4], &pixelBytes[i + 6])
-            swap(&pixelBytes[i+8], &pixelBytes[i + 10])
-            swap(&pixelBytes[i+12], &pixelBytes[i + 14])
+            contents = VideoTexture(width: width, height: height, format: GPU_FORMAT_BGRA)


this change dramatically improves video performance on macOS – esp. in debug builds

ephemer · 2025-12-12T16:37:58Z

Sources/CALayer+ContentsGravity.swift

+            width: (layer.contents.map { CGFloat($0.width) } ?? layer.bounds.width) / layer.contentsScale,
+            height: (layer.contents.map { CGFloat($0.height) } ?? layer.bounds.height) / layer.contentsScale


turns out this is actually unused but it prevents the crash mentioned on line 22 (I could remove that comment now I guess)

ephemer · 2025-12-12T16:39:37Z

Sources/CALayer+SDL.swift

    @MainActor
-    final func sdlRender(parentAbsoluteOpacity: Float = 1) {
-        guard let renderer = UIScreen.main else { return }
+    final func sdlRender(parentAbsoluteOpacity: Float = 1, renderTarget: RenderTarget) {


the concept of the RenderTarget being able to change is the main innovation in this PR: to begin with we start rendering to the window (which is the base-level RenderTarget). If we encounter a non-trivial CALayer with a mask/opacity, we render its entire subtree to a separate RenderTarget (which is literally an image in memory), and then we copy that pre-rendered image onto the window with the correct mask and opacity.

ephemer · 2025-12-12T16:40:44Z

Sources/CGImage.swift

        height = Int(rawPointer.pointee.h)
    }

+    public func savePNG(filename: String) {


debugging API – I should probably prefix this with an underscore

ephemer · 2025-12-12T16:41:02Z

Sources/FontRenderer+singleLineSize.swift

            let newLineCharacterCode = 10
            if characterCode != spaceCharacterCode, characterCode != newLineCharacterCode, glyph.maxx - glyph.minx <= 0 {
-                assertionFailure("Glyph \(characterCode) ('\(Character(UnicodeScalar(characterCode)!))') has no width")
+//                assertionFailure("Glyph \(characterCode) ('\(Character(UnicodeScalar(characterCode)!))') has no width")


this change was unintentional – will revert it

ephemer · 2025-12-12T16:41:48Z

Sources/RenderTarget.swift

+extension RenderTarget {
+    func blit(
+        _ image: CGImage,
+        anchorPoint: CGPoint,
+        contentsScale: CGFloat,
+        contentsGravity: ContentsGravityTransformation,
+        opacity: Float
+    ) throws {
+        GPU_SetAnchor(image.rawPointer, Float(anchorPoint.x), Float(anchorPoint.y))
+        GPU_SetRGBA(image.rawPointer, 255, 255, 255, opacity.normalisedToUInt8())
+
+        GPU_BlitTransform(
+            image.rawPointer,
+            nil,
+            self.rawPointer,
+            Float(contentsGravity.offset.x),
+            Float(contentsGravity.offset.y),
+            0, // rotation in degrees
+            Float(contentsGravity.scale.width / contentsScale),
+            Float(contentsGravity.scale.height / contentsScale)
+        )
+
+        try throwOnErrors(ofType: [GPU_ERROR_USER_ERROR])
+    }
+
+    func blit(renderTarget: RenderTarget, opacity: Float) throws {
+        GPU_SetAnchor(renderTarget.rawPointer.pointee.image, 0, 0)
+        GPU_SetRGBA(renderTarget.rawPointer.pointee.image, 255, 255, 255, opacity.normalisedToUInt8())
+
+        GPU_BlitTransform(
+            renderTarget.rawPointer.pointee.image,
+            nil,
+            self.rawPointer,
+            0, // offset
+            0, // offset
+            0, // rotation in degrees
+            Float(1 / scale),
+            Float(1 / scale)
+        )
+
+        try throwOnErrors(ofType: [GPU_ERROR_USER_ERROR])
+    }
+
+    func setShapeBlending(_ newValue: Bool) {
+        GPU_SetShapeBlending(newValue)
+    }
+
+    func setShapeBlendMode(_ newValue: GPU_BlendPresetEnum) {
+        GPU_SetShapeBlendMode(newValue)
+    }
+
+    func clear() {
+        GPU_Clear(rawPointer)
+    }
+
+    func fill(_ rect: CGRect, with color: UIColor, cornerRadius: CGFloat) {
+        if cornerRadius >= 1 {
+            GPU_RectangleRoundFilled(rawPointer, rect.gpuRect(scale: scale), cornerRadius: Float(cornerRadius), color: color.sdlColor)
+        } else {
+            GPU_RectangleFilled(rawPointer, rect.gpuRect(scale: scale), color: color.sdlColor)
+        }
+    }
+
+    func outline(_ rect: CGRect, lineColor: UIColor, lineThickness: CGFloat) {
+        // we want to render the outline 'inside' the rect rather
+        // than exceeding the bounds when lineThickness is bigger than 1
+        let offset = lineThickness / 2
+        let scaledGpuRect = CGRect(
+            x: rect.origin.x + offset,
+            y: rect.origin.y + offset,
+            width: rect.size.width - offset,
+            height: rect.size.height - offset
+        ).gpuRect(scale: scale)
+
+        GPU_SetLineThickness(Float(lineThickness))
+        GPU_Rectangle(rawPointer, scaledGpuRect, color: lineColor.sdlColor)
+    }
+
+    func outline(_ rect: CGRect, lineColor: UIColor, lineThickness: CGFloat, cornerRadius: CGFloat) {
+        if cornerRadius > 1 {
+            // we want to render the outline 'inside' the rect rather
+            // than exceeding the bounds when lineThickness is bigger than 1
+            let offset = lineThickness / 2
+            let scaledGpuRect = CGRect(
+                x: rect.origin.x + offset,
+                y: rect.origin.y + offset,
+                width: rect.size.width - offset,
+                height: rect.size.height - offset
+            ).gpuRect(scale: scale)
+
+            GPU_SetLineThickness(Float(lineThickness))
+            GPU_RectangleRound(rawPointer, scaledGpuRect, cornerRadius: Float(cornerRadius), color: lineColor.sdlColor)
+        } else {
+            outline(rect, lineColor: lineColor, lineThickness: lineThickness)
+        }
+    }
+
+    func flip() throws {
+        GPU_Flip(rawPointer)
+        try throwOnErrors(ofType: [GPU_ERROR_USER_ERROR, GPU_ERROR_BACKEND_ERROR])
+    }
+
+    func setVirtualResolution(w: UInt16, h: UInt16) {
+        GPU_SetVirtualResolution(rawPointer, w, h)
+    }
+}


almost everything here is copied directly from UIScreen+render.swift -> previously UIScreen was our only render target, but now we can have various targets (see comments and code above)

ephemer · 2025-12-12T16:42:17Z

Sources/ShaderProgram+Gradient.swift

+//
+//  Mask.swift
+//  UIKit
+//
+//  Created by Geordie Jay on 25.10.17.
+//  Copyright © 2017 flowkey. All rights reserved.
+//


should probably remove this..

ephemer · 2025-12-12T16:42:29Z

Sources/ShaderProgram+Gradient.swift

+    private var startPoint: UniformVariable!
+    private var endPoint: UniformVariable!
+
+    // we only need one MaskShaderProgram, which we can / should treat as a singleton


remove comment

ephemer · 2025-12-12T16:43:14Z

Sources/ShaderProgram+Gradient.swift

This entire file is unused in this PR – it's a GPU implementation for CAGradientLayer but integrating it would require some more work, and we don't need the flexibility or the performance of using the GPU for our needs right now

ephemer · 2025-12-12T16:44:01Z

Sources/UIView+SDL.swift

+        if visibleLayer.mask?.needsDisplay() == true {
+            visibleLayer.mask?.display()
+            visibleLayer.mask?._needsDisplay = false
+        }


this can be removed because we (also) display() in the sdlRender function

Flatten Layers to allow for proper masking and opacity compositing. A…

e7e732a

…dd CAGradientLayer

ephemer requested a review from michaelknoch December 12, 2025 15:49

ephemer changed the base branch from master to multipleVideos December 12, 2025 15:49

ephemer added 4 commits December 12, 2025 16:51

Update Xcode version for CircleCI

eff0e98

Remove x64_64 requirement

cb7aa6c

Fix tests

13cd830

Update iPhone

56dcbec

ephemer commented Dec 12, 2025

View reviewed changes

Fix Android build

d8a7f91

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add CAGradientLayer, Allow Layer Flattening for proper masking and opacity compositing#408

Add CAGradientLayer, Allow Layer Flattening for proper masking and opacity compositing#408
ephemer wants to merge 6 commits intomultipleVideosfrom
flattenLayers

ephemer commented Dec 12, 2025

Uh oh!

ephemer Dec 12, 2025

Uh oh!

ephemer Dec 12, 2025

Uh oh!

ephemer Dec 12, 2025

Uh oh!

ephemer Dec 12, 2025

Uh oh!

ephemer Dec 12, 2025

Uh oh!

ephemer Dec 12, 2025

Uh oh!

ephemer Dec 12, 2025

Uh oh!

ephemer Dec 12, 2025

Uh oh!

ephemer Dec 12, 2025

Uh oh!

ephemer Dec 12, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

		width: (layer.contents.map { CGFloat($0.width) } ?? layer.bounds.width) / layer.contentsScale,
		height: (layer.contents.map { CGFloat($0.height) } ?? layer.bounds.height) / layer.contentsScale

Conversation

ephemer commented Dec 12, 2025

Motivation (current vs expected behavior)

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant