feat: adding support for amdama (supernova) gpus. #7697

mipresmsft · 2026-01-21T19:28:38Z

What this PR does / why we need it:

Adding support for AMDAMA (Supernova) GPUs.

r2k1 · 2026-01-21T21:32:01Z

pkg/agent/datamodel/helper.go

+		return true
+	}
+	return false
+}


This method duplicated 3 times. Should be no more than 2. (Ideally 1)

Just following the method used in each file. Presumably the ones we don't need will get removed when the move from scripted to scriptless is done?

Also. would you recommend breaking form the model to make lint happy?

r2k1 · 2026-01-21T22:32:46Z

pkg/agent/variables.go

 		"userAssignedIdentityID":                 config.UserAssignedIdentityClientID,
 		"isVHD":                                  isVHD(profile),
 		"gpuNode":                                strconv.FormatBool(config.EnableNvidia),
+		"amdamaNode":                             strconv.FormatBool(datamodel.IsAmdAmaEnabledSKU(profile.VMSize)),


This looks unused?

Not sure, but there's one for Nvidia, so I figured better include it just in case.

djsly · 2026-01-23T15:37:49Z

e2e/scenario_test.go

 	})
 }

+func Test_AzureLinuxV3_MA35D(t *testing.T) {


AzureLinux is the only OS that support this VMSku correct ?

With AKS, yes.

djsly · 2026-01-23T15:39:55Z

parts/linux/cloud-init/artifacts/cse_config.sh

 }

+setupAmdAma() {
+    if [ "$(isARM64)" -eq 1 ]; then


is this even possible ? to have AMDAMA_MODE == true and with a arm64 arch ? What would be the consequences here if this cluase is true and we expect to have AMDAMA_MODE and we dont setup anymore.

That should never happen.

djsly · 2026-01-23T15:41:51Z

parts/linux/cloud-init/artifacts/cse_config.sh

+        return
+    fi
+
+    if isMarinerOrAzureLinux "$OS"; then


same, could we have ubuntu ? if it's the case what will happen ? should we baill out instead and cause anode bootstrap failure ? I would expect that Ubuntu/Flatcar and ARM64 will all be blocked at the RP level to provide users with early validation failure ?

One more question, does isMarinerOrAzureLinux return true for OSGuard I recall there were extrra check required to ensure it wasn't LinuxOSGuard

Right. If someone did try to create an AKS cluster with a different OS, it just wouldn't install the driver, but AKS cluster creation would succeed. That's fine.

No idea for OSGuard...

parts/linux/cloud-init/artifacts/cse_config.sh

djsly · 2026-01-23T15:43:38Z

parts/linux/cloud-init/artifacts/cse_config.sh

+        sudo tdnf install -y azurelinux-repos-amd
+        KERNEL_VERSION=$(uname -r | sed 's/-/./g')
+        AMD_AMA_DRIVER_PACKAGE=$(dnf repoquery -y --available "amd-ama-driver*" | grep -E "amd-ama-driver-[0-9]+.*_$KERNEL_VERSION" | sort -V | tail -n 1)
+        sudo tdnf install -y $AMD_AMA_DRIVER_PACKAGE


can we combine into a single dnf call ?

parts/linux/cloud-init/artifacts/cse_config.sh

djsly · 2026-01-23T15:47:02Z

parts/linux/cloud-init/artifacts/cse_config.sh

+        # Install core package
+        sudo tdnf install -y libzip
+        sudo tdnf install -y azurelinux-repos-extended
+        sudo RPM_FRONTEND=noninteractive tdnf install -y https://download.microsoft.com/download/16b04fa7-883e-4a94-88c2-801881a47b28/amd-ama-core_1.3.0-2503242033-amd64.rpm


this is very much hardcoded, if it's for testing purposes for now, I'm ok but we need to move the package to components.json and also ensure we can detect new versions automatically with renovate. Let's do that in another PR.

parts/linux/cloud-init/artifacts/cse_main.sh

djsly · 2026-01-23T15:51:59Z

pkg/agent/baker.go

 }

+// IsAmdAmaEnabledSKU determines if an VM SKU has AMD AMA GPU HW support.
+func IsAmdAmaEnabledSKU(vmSize string) bool {


I believe this is the first time we perform a logic in the VHD to mapping SKU to feature... normally this is done in RP.

This is because aks-rp is unaware that this is a GPU SKU so it doesn't interfere with Nvidia/AMD GPUs, and so we don't have to add a whole new mechanism just for this SKU. This was Yi's decision.

awesomenix · 2026-01-27T04:44:57Z

parts/linux/cloud-init/artifacts/cse_main.sh

    fi

+    # Install and configure AMD AMA (Supernova) drivers if this is an AMA node
+    if [ "${AMDAMA_NODE}" = "true" ]; then


you can query instance metadata server to get the sku and install it directly here instead of going through all the hoops if the intention is default enabled for the SKU

# Retrieves the VM SKU (size) from the cached IMDS instance metadata. # Outputs: # The VM size/SKU (e.g., "Standard_NM16ads_MA35D") or empty string if not found. # Returns: # 0 on success, non-zero on failure get_imds_vm_sku() { local vm_sku="" vm_sku=$(jq -r '.compute.vmSize // ""' "$IMDS_INSTANCE_METADATA_CACHE_FILE") echo "$vm_sku" } if [ "$(get_imds_vm_sku)" = "Standard_NM16ads_MA35D" ]; then echo "Exact match!" fi

Copilot

Pull request overview

This pull request adds support for AMD AMA (Supernova) GPU hardware by enabling detection and configuration for the Standard_NM16ads_MA35D VM SKU. The changes introduce a new VM size detection mechanism, install necessary drivers and packages for AMD AMA GPUs, and configure the required system settings for Azure Linux environments.

Changes:

Added VM SKU detection logic for AMD AMA-enabled instances (Standard_NM16ads_MA35D)
Implemented driver installation and configuration for AMD AMA GPUs on Azure Linux
Added E2E tests to validate AMD AMA GPU functionality
Updated firewall rules to allow access to download.microsoft.com for AMD AMA package installation

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
aks-node-controller/helpers/const.go	Added constant for Standard_NM16ads_MA35D VM size
aks-node-controller/parser/helper.go	Added helper function to detect AMD AMA enabled SKUs
aks-node-controller/parser/parser.go	Added AMDAMA_NODE environment variable to CSE configuration
pkg/agent/baker.go	Added IsAmdAmaEnabledSKU function for template rendering
pkg/agent/datamodel/helper.go	Added IsAmdAmaEnabledSKU function for VM SKU detection
pkg/agent/variables.go	Added amdamaNode variable to CSE command variables
parts/linux/cloud-init/artifacts/cse_cmd.sh	Declared AMDAMA_NODE variable for shell scripts
parts/linux/cloud-init/artifacts/cse_config.sh	Implemented setupAmdAma function to install drivers and configure system
parts/linux/cloud-init/artifacts/cse_main.sh	Added call to setupAmdAma during node preparation
e2e/scenario_test.go	Added E2E tests for AMD AMA GPU support (standard and scriptless)
e2e/aks_model.go	Added firewall rule to allow download.microsoft.com access

Copilot · 2026-01-30T21:46:13Z

pkg/agent/baker.go

+// IsAmdAmaEnabledSKU determines if an VM SKU has AMD AMA GPU HW support.
+func IsAmdAmaEnabledSKU(vmSize string) bool {
+	switch vmSize {
+	case "Standard_NM16ads_MA35D":
+		return true
+	}
+	return false
+}
+


The IsAmdAmaEnabledSKU function is duplicated in both pkg/agent/baker.go and pkg/agent/datamodel/helper.go. This creates maintenance overhead as both implementations need to be updated when new AMD AMA SKUs are added. Consider removing this function from baker.go and calling datamodel.IsAmdAmaEnabledSKU instead, following the pattern used by IsSgxEnabledSKU which only exists in datamodel/helper.go.

Suggested change

// IsAmdAmaEnabledSKU determines if an VM SKU has AMD AMA GPU HW support.

func IsAmdAmaEnabledSKU(vmSize string) bool {

switch vmSize {

case "Standard_NM16ads_MA35D":

return true

}

return false

}

@r2k1 @djsly Are these no longer needed?

aks-node-controller/parser/helper.go

parts/linux/cloud-init/artifacts/cse_config.sh

parts/linux/cloud-init/artifacts/cse_cmd.sh

aks-node-controller/parser/parser.go

pkg/agent/datamodel/helper.go

parts/linux/cloud-init/artifacts/cse_config.sh

pkg/agent/datamodel/helper.go

Copilot

Pull request overview

Copilot reviewed 19 out of 70 changed files in this pull request and generated 2 comments.

Copilot · 2026-02-07T00:06:58Z

parts/linux/cloud-init/artifacts/cse_config.sh

+        sh -c "echo 'vm.nr_hugepages=4096' >> /etc/sysctl.conf"
+        sh -c "echo 4096 >> /proc/sys/vm/nr_hugepages"
+        if [ "$(systemctl is-active kubelet)" = "active" ]; then


Hugepages are configured by appending to /etc/sysctl.conf and writing directly to /proc/sys/vm/nr_hugepages. This is not idempotent (re-runs will keep appending) and can fail if the kernel can’t allocate the requested pages, which would currently abort provisioning under set -e. Consider writing a dedicated drop-in under /etc/sysctl.d/ and applying via sysctl with explicit error handling/logging for allocation failures.

e2e/scenario_test.go

Copilot

Pull request overview

Copilot reviewed 19 out of 70 changed files in this pull request and generated 4 comments.

e2e/aks_model.go

e2e/scenario_test.go

Copilot · 2026-02-09T23:00:51Z

parts/linux/cloud-init/artifacts/cse_config.sh

+        return
+    fi
+
+    if isMarinerOrAzureLinux "$OS"; then


setupAmdAma is gated by isMarinerOrAzureLinux "$OS", but the function installs azurelinux-repos-* packages. This will likely fail on Mariner images (and could break provisioning if someone uses this SKU on Mariner). Consider restricting this to Azure Linux only (and optionally specific Azure Linux versions) before attempting installs.

Suggested change

if isMarinerOrAzureLinux "$OS"; then

if isAzureLinux "$OS"; then

Copilot

Pull request overview

Copilot reviewed 19 out of 70 changed files in this pull request and generated 3 comments.

Copilot · 2026-02-09T23:58:37Z

parts/linux/cloud-init/artifacts/cse_config.sh

+        KERNEL_VERSION=$(uname -r | sed 's/-/./g')
+        AMD_AMA_DRIVER_PACKAGE=$(dnf repoquery -y --available "amd-ama-driver-1.3.0*" | grep -E "amd-ama-driver-[0-9]+.*_$KERNEL_VERSION" | sort -V | tail -n 1)
+        if [ -z "$AMD_AMA_DRIVER_PACKAGE" ]; then
+            echo "Unable to find AMD AMA driver package for current kernel version, exiting..."
+            exit $ERR_AMDAMA_DRIVER_NOT_FOUND
+        fi
+        dnf install -y $AMD_AMA_DRIVER_PACKAGE


In setupAmdAma, KERNEL_VERSION/AMD_AMA_DRIVER_PACKAGE are not local, and $AMD_AMA_DRIVER_PACKAGE is unquoted when passed to dnf install, which can lead to word-splitting/globbing issues. Also, the grep -E pattern isn’t anchored (unlike the similar CUDA logic), increasing the chance of matching the wrong repoquery output. Consider making these variables local, quoting expansions, and tightening the match (e.g., anchor to the start of the package name).

Suggested change

KERNEL_VERSION=$(uname -r | sed 's/-/./g')

AMD_AMA_DRIVER_PACKAGE=$(dnf repoquery -y --available "amd-ama-driver-1.3.0*" | grep -E "amd-ama-driver-[0-9]+.*_$KERNEL_VERSION" | sort -V | tail -n 1)

if [ -z "$AMD_AMA_DRIVER_PACKAGE" ]; then

echo "Unable to find AMD AMA driver package for current kernel version, exiting..."

exit $ERR_AMDAMA_DRIVER_NOT_FOUND

fi

dnf install -y $AMD_AMA_DRIVER_PACKAGE

local KERNEL_VERSION

KERNEL_VERSION=$(uname -r | sed 's/-/./g')

local AMD_AMA_DRIVER_PACKAGE

AMD_AMA_DRIVER_PACKAGE=$(dnf repoquery -y --available "amd-ama-driver-1.3.0*" | grep -E "^amd-ama-driver-[0-9]+.*_${KERNEL_VERSION}$" | sort -V | tail -n 1)

if [ -z "$AMD_AMA_DRIVER_PACKAGE" ]; then

echo "Unable to find AMD AMA driver package for current kernel version, exiting..."

exit $ERR_AMDAMA_DRIVER_NOT_FOUND

fi

dnf install -y "$AMD_AMA_DRIVER_PACKAGE"

Copilot · 2026-02-09T23:58:37Z

e2e/scenario_test.go

+		Description: "Tests that a node using a AzureLinuxV3 can support MA35D SKU",
+		Config: Config{


Grammar nit: "a AzureLinuxV3" should be "an AzureLinuxV3" (both in the non-scriptless and scriptless MA35D scenario descriptions).

Copilot · 2026-02-09T23:58:37Z

e2e/scenario_test.go

+		Description: "Tests that a node using a AzureLinuxV3 can support MA35D SKU",
+		Tags: Tags{


Grammar nit: "a AzureLinuxV3" should be "an AzureLinuxV3" in this scenario description as well.

Copilot

Pull request overview

Copilot reviewed 20 out of 71 changed files in this pull request and generated 2 comments.

Copilot · 2026-02-10T02:09:31Z

e2e/scenario_test.go

+		Description: "Tests that a node using a AzureLinuxV3 can support MA35D SKU",
+		Tags: Tags{


Grammar: "a AzureLinuxV3" should be "an AzureLinuxV3".

Copilot · 2026-02-10T02:09:32Z

e2e/config/config.go

+	TestTimeout                            time.Duration `env:"TEST_TIMEOUT" envDefault:"60m"`
+	TestTimeoutCluster                     time.Duration `env:"TEST_TIMEOUT_CLUSTER" envDefault:"60m"`


Increasing the default E2E timeouts to 60m/60m affects the entire test suite and can mask regressions or significantly lengthen CI when tests hang. If this change is only needed for MA35D/driver bring-up, consider using per-scenario timeouts (if supported) or narrowing the scope of the timeout increase; otherwise, please document why the global increase is required.

mipresmsft · 2026-02-11T21:27:12Z

e2e/config/config.go

 	TestGalleryImagePrefix                 string        `env:"TEST_GALLERY_IMAGE_PREFIX" envDefault:"abe2etest"`
 	TestGalleryNamePrefix                  string        `env:"TEST_GALLERY_NAME_PREFIX" envDefault:"abe2etest"`
 	TestPreProvision                       bool          `env:"TEST_PRE_PROVISION" envDefault:"false"`
-	TestTimeout                            time.Duration `env:"TEST_TIMEOUT" envDefault:"35m"`


@r2k1 @djsly Do we want to keep these?

Moved changes to sync with main branch.

85c15ed

mipresmsft requested review from AbelHu, Devinwong, awesomenix, cameronmeissner, djsly, ganeshkumarashok, juan-lee, junjiezhang1997, lilypan26, phealy, r2k1, timmy-wright and zachary-bailey as code owners January 21, 2026 19:28

mipresmsft changed the title ~~Moved changes to sync with main branch.~~ Adding support for AMDAMA (Supernova) GPUs. Jan 21, 2026

mipresmsft temporarily deployed to test January 21, 2026 19:30 — with GitHub Actions Inactive

r2k1 reviewed Jan 21, 2026

View reviewed changes

djsly requested changes Jan 23, 2026

View reviewed changes

awesomenix reviewed Jan 27, 2026

View reviewed changes

Updated driver install, firewall, tests.

14f36cd

Copilot AI review requested due to automatic review settings January 30, 2026 21:41

mipresmsft requested review from mxj220 and pdamianov-dev as code owners January 30, 2026 21:41

Copilot started reviewing on behalf of mipresmsft January 30, 2026 21:41 View session

Remove unnecessary logging.

5753c77

Copilot AI reviewed Jan 30, 2026

View reviewed changes

Fixed spacing.

cb7df67

Copilot AI review requested due to automatic review settings January 30, 2026 21:54

Copilot started reviewing on behalf of mipresmsft January 30, 2026 21:55 View session

mipresmsft requested review from YaoC and yewmsft as code owners February 6, 2026 23:10

mipresmsft temporarily deployed to test February 6, 2026 23:10 — with GitHub Actions Inactive

Change test location.

bdceeff

Copilot AI review requested due to automatic review settings February 7, 2026 00:01

mipresmsft temporarily deployed to test February 7, 2026 00:01 — with GitHub Actions Inactive

Copilot started reviewing on behalf of mipresmsft February 7, 2026 00:02 View session

Copilot AI reviewed Feb 7, 2026

View reviewed changes

Update test location.

fb15ee9

mipresmsft temporarily deployed to test February 7, 2026 19:55 — with GitHub Actions Inactive

Change scenario location.

6167c0c

Copilot AI review requested due to automatic review settings February 9, 2026 22:54

Copilot started reviewing on behalf of mipresmsft February 9, 2026 22:55 View session

Copilot AI reviewed Feb 9, 2026

View reviewed changes

Merge with latest from main, regenerate test data.

5811283

mipresmsft temporarily deployed to test February 9, 2026 23:02 — with GitHub Actions Inactive

Hardcode system pool VM SKU for MA35D tests.

ddbf7cd

Copilot AI review requested due to automatic review settings February 9, 2026 23:53

Copilot started reviewing on behalf of mipresmsft February 9, 2026 23:54 View session

Merge with latest from main, regenerate test data.

090c892

mipresmsft temporarily deployed to test February 9, 2026 23:57 — with GitHub Actions Inactive

Copilot AI reviewed Feb 9, 2026

View reviewed changes

Updated system pool VM SKU.

663d0cc

mipresmsft temporarily deployed to test February 10, 2026 00:46 — with GitHub Actions Inactive

Increase timeouts.

e191a6c

Copilot AI review requested due to automatic review settings February 10, 2026 02:02

mipresmsft temporarily deployed to test February 10, 2026 02:02 — with GitHub Actions Inactive

Copilot started reviewing on behalf of mipresmsft February 10, 2026 02:03 View session

Copilot AI reviewed Feb 10, 2026

View reviewed changes

mipresmsft commented Feb 11, 2026

View reviewed changes

	if isMarinerOrAzureLinux "$OS"; then
	if isAzureLinux "$OS"; then

		Description: "Tests that a node using a AzureLinuxV3 can support MA35D SKU",
		Config: Config{

		Description: "Tests that a node using a AzureLinuxV3 can support MA35D SKU",
		Tags: Tags{

		TestTimeout time.Duration `env:"TEST_TIMEOUT" envDefault:"60m"`
		TestTimeoutCluster time.Duration `env:"TEST_TIMEOUT_CLUSTER" envDefault:"60m"`

feat: adding support for amdama (supernova) gpus. #7697

Are you sure you want to change the base?

feat: adding support for amdama (supernova) gpus. #7697

Uh oh!

Conversation

mipresmsft commented Jan 21, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mipresmsft Jan 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Feb 7, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Feb 9, 2026

Choose a reason for hiding this comment

mipresmsft Jan 30, 2026 •

edited

Loading