MetalCompilerPlugin support #313

gin66 · 2025-12-09T12:19:16Z

Proposed changes

Application MetalCompilerPlugin to allow shader compilation without Xcode.

swift testpasses, if ml-explore/mlx with pull request 2885 is applied.

README has been updated to point out, that default.metallib is located in the Clmx bundle.

Checklist

Put an x in the boxes that apply.

I have read the CONTRIBUTING document
I have run pre-commit run --all-files to format my code / installed pre-commit prior to committing changes
I have added tests that prove my fix is effective or that my feature works
I have updated the necessary documentation (if needed)

…ith patched mlx

davidkoski · 2025-12-12T18:18:40Z

I think the technique is interesting, and perhaps something we to do, but what is the use case -- what problem is this trying to fix?

Application MetalCompilerPlugin to allow shader compilation without Xcode

Is there a case where builds need to occur without Xcode? Linux requires it, but we are working on a separate build path for that and it can't build metal shaders anyway :-)

Looking at the config it looks like you might have the NAX and size issue dealt with, which is great. Likely we will move the NAX shaders into the JIT collection so those wouldn't need to go through this in the next release.

This change in MLX: ml-explore/mlx#2885 -- I think it would be good if the plugin output a valid bundle with resources in the right area. We can already control which file it looks for with METAL_PATH but I worry that if the bundle is malformed that causes issues elsewhere.

davidkoski · 2025-12-12T18:20:50Z

Package.swift

@@ -1,9 +1,12 @@
-// swift-tools-version: 5.12
+// swift-tools-version: 6.2


I am concerned about this -- it locks out developers who use older Xcode. Why do we need this?

I need to check, if this 6.2 is really necessary.

davidkoski · 2025-12-12T18:23:09Z

Package.swift

+import Foundation
 import PackageDescription

+let inXcode = ProcessInfo.processInfo.environment["XCODE_VERSION_ACTUAL"] != nil


I think this won't work correctly: Xcode can't build the shaders with the correct arguments (in particular the NAX shaders). That won't be a problem once those move to JIT builds, but I think it isn't a good idea to have to different build paths where bugs might show up in one and not in the other (I realize we have swiftPM, Xcode and Cmake so this is already an issue, but the swiftpm and Xcode paths do use the same build process).

For Xcode builds, the MetalShaderCompiler is not needed. And for spm builds it is required.
Not a major difference, because the underlying metal compiler is the same. Not sure, what you mean with inability to build with correct arguments.

The plugin allows you to specify arguments when building the shaders. xcodebuild does not -- the metal compile uses some default arguments, which have been fine up to the 0.30.0 mlx release but are no longer sufficient. We either need to move the shaders that require different arguments into JIT compiles (the current plan) or we need a solution like this that can vary them from file to file (for both swiftpm AND Xcode builds).

davidkoski · 2025-12-12T18:23:36Z

README.md


-## Language Models
-
-LLM and VLM implementations are available in [mlx-swift-lm](https://github.com/ml-explore/mlx-swift-lm).


Why are these removed from the README?

was not intentional

davidkoski · 2025-12-12T18:24:01Z

README.md

-
-</details>
-
-</br>


Why is this removed?

same. I think, I have copied over the Readme instead of redoing the modification. Should be an easy fix.

davidkoski · 2025-12-12T18:25:44Z

Source/Cmlx/.metal-compiler-plugin.json

+        "Source/Cmlx/mlx-generated/metal/layer_norm.metal",
+        "Source/Cmlx/mlx-generated/metal/rope.metal"],
+    "output": "default.metallib",
+    "flags": ["-gline-tables-only", "-frecord-sources", "-Wno-c++20-extensions", "-std=metal4.0"],


I am not sure we can use metal4 for all the shaders, we just want it for the NAX shaders. If we do this then it won't run on pre-macOS 26. I think we need to vary this per file (which is why we can't currently support NAX in main).

One way out would be, that swiftpm shader compilation is only supported from e.g. OS 26 onwards.
Otherwise it gets unnecessary complicated.

davidkoski · 2025-12-12T18:26:22Z

Tests/MLXTests/ArrayAtTests.swift

+
+        // Cmlx is not an automatically loaded bundle. 
+        // Let's load Cmlx manually, so that the mlx-lib can find the default.metallib.
+        // This requires patch #2885 in ml-explore/mlx be applied.


Is this a workaround for the fix from the mlx repo?

No. Both are needed. The issue is, that the plugin can only write to its plugin dir and the module dir, which depends on it. There is no way to get this metallib out of the module in the main module. The mlx-repo already supports searching all bundles‘ resource URL, which is a major step. Now the issue is, that I doubt, that the plugin can write a valid resource. At least I was not successful. So the remedy is, that not only the bundles’ resourceURL are searched, but the bundles‘ root directories, too. This is the purpose of the patch in mlx.

Now the tricky thing is, that on swiftpm level modules are not autloaded. The patch in the test performs the manual loading of the bundle and then the mlx-code is (magically) able to find the bundle in the list.

There is a lot of automatisms in the Xcode build process.

OK, I think this will probably be a blocker until we can figure out how to get the resource production working -- applications can't be required to do this same work also.

I thought that producing resources was the only thing a plugin could do. We used to have a plugin that would prepare the include paths in the shaders:

https://github.com/ml-explore/mlx-swift/blob/16c443ee5231e2961418716e148780dc19d767ba/Plugins/PrepareMetalShaders/main.swift

it didn't compile them. I know it can't produce a .o that is linked in to the resulting binary.

gin66 · 2025-12-13T00:41:17Z

Now what is the motivation/use case of all this besides it can be done.

In short: My motivation is automated unit tests using swiftpm.

No need for Xcode cloud. Xcode server is already discontinued and using a regular Xcode as build server is nearly impossible. My setup is a gitea (similar to github) server and with actions I can do all my module/integration/application tests. With MetalCompilerPlugin I am able to test all my shadercode in the same automated fashion.

So why not using plain xcodebuild? Answer is, you need xcode to create schemes. It is hard to automate test plans or make some tests conditional on what changes have been done. Xcode is hard to control and heavy machinery. swiftpm is light weight and way more understandable in what is going on. It would cost/waste days trying to get same level with github actions on macos with xcodebuild.

Sure, that‘s only my view. As I am convinced it makes sense, I put the effort into it to get this working for mlx-swift. Even though I neither fully understand the mlx-build system, nor all the inherent backward capability. At least it works for me and I can once a while update my own fork as long as I need it.

gin66 · 2025-12-13T00:42:51Z

Writing github comments on iPad has sometimes interesting effects like closing an issue without planning to do so…..

gin66 added 3 commits December 9, 2025 10:48

use MetalCompilerPlugin to create default.metallib

e4d2bc2

fix the warning about missing metal-cpp.patch

047706e

add info for default.metallib handling and ensure swift test passes w…

b499477

…ith patched mlx

gin66 mentioned this pull request Dec 9, 2025

PoC: MetalCompilerPlugin support #310

Closed

4 tasks

davidkoski mentioned this pull request Dec 12, 2025

load metallib not only as resource but in root of a bundle ml-explore/mlx#2885

Open

4 tasks

davidkoski reviewed Dec 12, 2025

View reviewed changes

gin66 closed this Dec 13, 2025

gin66 reopened this Dec 13, 2025

		@@ -1,9 +1,12 @@
		// swift-tools-version: 5.12
		// swift-tools-version: 6.2


		## Language Models

		LLM and VLM implementations are available in [mlx-swift-lm](https://github.com/ml-explore/mlx-swift-lm).


		</details>

		</br>

MetalCompilerPlugin support #313

Are you sure you want to change the base?

MetalCompilerPlugin support #313

Uh oh!

Conversation

gin66 commented Dec 9, 2025

Proposed changes

Checklist

Uh oh!

davidkoski commented Dec 12, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gin66 commented Dec 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gin66 commented Dec 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

gin66 commented Dec 13, 2025 •

edited

Loading