From f2914685347595dcdf78d12b6aa227548a5709c3 Mon Sep 17 00:00:00 2001 From: Mike Alhayek Date: Mon, 27 Jul 2026 21:36:39 +0300 Subject: [PATCH 1/2] Fix Copilot CLI acquisition behind npm mirrors and cache it per machine The GitHub.Copilot.SDK build targets download the Copilot CLI tarball from a hardcoded https://registry.npmjs.org, and MSBuild's DownloadFile task cannot read npm configuration. Machines and build agents behind a corporate proxy or an artifact mirror block that host, so consuming projects fail to build with a connectivity error even though npm itself is configured correctly. The SDK also caches the CLI under the intermediate output path, which is per project and per configuration, so a solution with several referencing projects downloads the same ~100 MB tarball once per project, and again for every fresh clone, worktree or CI agent. The buildTransitive bridge now resolves the effective npm registry from NPM_CONFIG_REGISTRY, falling back to `npm config get registry`, and redirects the SDK cache to a machine wide location under the NuGet global packages folder. Both behaviors are opt-out through CopilotResolveNpmRegistry and CopilotUseSharedCliCache, the cache location is configurable through CopilotCliCacheDir so a pre-seeded directory supports offline builds, and an explicitly set CopilotNpmRegistryUrl, CopilotCliBinaryPath or CopilotSkipCliDownload always takes precedence. The bridge records whether the consumer supplied CopilotNpmRegistryUrl before importing the SDK targets, because the SDK assigns its own default during that import and the resolution would otherwise never run. Also escapes the generated targets content before WriteLinesToFile so semicolons and percent signs in the template are written verbatim instead of being split into separate lines. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../docs/changelog/v1.0.0.md | 1 + .../docs/orchestration/copilot.md | 31 ++++++ .../CrestApps.Core.AI.Copilot.csproj | 3 + .../CrestApps.Core.AI.Copilot.targets.in | 103 ++++++++++++++++++ 4 files changed, 138 insertions(+) diff --git a/src/CrestApps.Core.Docs/docs/changelog/v1.0.0.md b/src/CrestApps.Core.Docs/docs/changelog/v1.0.0.md index 8d91fcc0..3e46da86 100644 --- a/src/CrestApps.Core.Docs/docs/changelog/v1.0.0.md +++ b/src/CrestApps.Core.Docs/docs/changelog/v1.0.0.md @@ -116,3 +116,4 @@ description: Initial standalone release notes for the CrestApps.Core repository. - promotes the parameterized AI tool instances into a first-class opt-in feature registered on the AI suite builder with `AddToolInstances(toolInstances => toolInstances.AddSource(...))` (with an `AddHttpApiRequestSource()` convenience for the built-in source), instead of being wired into the core AI services automatically; persistence is registered on the tool-instances builder via `AddYesSqlStores()`/`AddEntityCoreStores()` rather than on the AI suite; drops the redundant per-instance `DisplayText` in favor of the unique `Name`, localizes the source `DisplayName`/`Description`/`Category`, simplifies `IAIToolInstanceSource.CreateTool(AIToolInstance instance)` to take the instance directly, generalizes the completion-context handler to honor tool instances on any resource so both AI profiles and chat interactions can select them, renames the built-in HTTP source's basic/OAuth credentials to `Username`/`Password` and adds the OAuth 2.0 resource-owner password grant, hardens model-provided paths so they cannot redirect a request off the configured host, and updates the MVC and Blazor sample hosts to let users attach instances to both AI profiles and chat interactions - namespaces every tool-instance function name with the `AIToolInstanceExtensions.FunctionNamePrefix` (`tool_instance_`) prefix so a user-chosen instance name can never collide with a tool registered in code via `AddCoreAITool` (which the model sees under its bare registered name); both kinds of tool now coexist safely in the single function namespace exposed to OpenAI, Azure OpenAI, and every other client - adds a source dropdown to the AI tool instance create form in the MVC and Blazor sample hosts that reveals only the selected source's fields (the source is fixed and shown read-only on edit), and relocates the "AI Tool Instances" admin menu item next to "AI Profiles" in both samples +- makes the Copilot CLI acquisition work behind corporate proxies and artifact mirrors, and downloads it only once per machine: `CrestApps.Core.AI.Copilot` now resolves the effective npm registry from `NPM_CONFIG_REGISTRY` or `npm config get registry` before the `GitHub.Copilot.SDK` targets download the CLI tarball (the SDK hardcodes `https://registry.npmjs.org`, and MSBuild's `DownloadFile` task cannot read npm configuration), and redirects the SDK's per-project, per-configuration cache to a shared cache under the NuGet global packages folder so a multi-project solution, a fresh worktree, or a CI agent no longer re-downloads the same large tarball for every project; both behaviors are opt-out through `CopilotResolveNpmRegistry` and `CopilotUseSharedCliCache`, the cache location is configurable through `CopilotCliCacheDir` (point it at a pre-seeded directory to build offline), and an explicitly set `CopilotNpmRegistryUrl`, `CopilotCliBinaryPath`, or `CopilotSkipCliDownload` always takes precedence diff --git a/src/CrestApps.Core.Docs/docs/orchestration/copilot.md b/src/CrestApps.Core.Docs/docs/orchestration/copilot.md index 4fddde0b..3576f081 100644 --- a/src/CrestApps.Core.Docs/docs/orchestration/copilot.md +++ b/src/CrestApps.Core.Docs/docs/orchestration/copilot.md @@ -35,6 +35,37 @@ public class MyController(IOrchestratorResolver resolver) } ``` +## Copilot CLI Acquisition + +The Copilot orchestrator runs on top of the GitHub Copilot CLI. Referencing `CrestApps.Core.AI.Copilot` is all a host needs to do: the package re-imports the `GitHub.Copilot.SDK` build targets, which download the CLI for the current runtime identifier during build and copy it to `runtimes//native/` in the output folder. No manual install step is required. + +The package layers two improvements on top of the SDK's default acquisition. + +**A machine-wide CLI cache.** The SDK caches the CLI under the intermediate output path, which is per project *and* per configuration, so a solution with several referencing projects downloads the same large tarball once per project, and again for every fresh clone, worktree, or CI agent. The package redirects that to a shared cache under the NuGet global packages folder, so the CLI is downloaded once and reused everywhere. That folder is already restored per machine and is commonly cached by CI pipelines, so build agents benefit as well. + +**Automatic npm registry resolution.** The SDK downloads the CLI tarball from `https://registry.npmjs.org`, and MSBuild's `DownloadFile` task cannot read npm configuration. Machines and build agents behind a corporate proxy or an artifact mirror usually block that host, so the build fails with a connectivity error even though npm itself is configured correctly. The package resolves the effective registry from the `NPM_CONFIG_REGISTRY` environment variable, falling back to `npm config get registry`, and uses that instead. The lookup only runs when a download is actually about to happen, and silently falls back to the SDK default when npm is unavailable or reports something that is not a URL. + +Both behaviors are opt-out and never override a value you set yourself. + +| Property | Default | Description | +| --- | --- | --- | +| `CopilotUseSharedCliCache` | `true` | Set to `false` to keep the SDK's per-project cache. | +| `CopilotCliCacheDir` | `/.crestapps/copilot-cli/` | Location of the shared cache. Point this at a pre-seeded directory to build without network access. | +| `CopilotResolveNpmRegistry` | `true` | Set to `false` to always use the SDK default registry. | +| `CopilotNpmRegistryUrl` | resolved from npm | Pin a registry explicitly. An explicit value always wins over resolution. | + +These are ordinary MSBuild properties, so set them in `Directory.Build.props` to apply them across a solution: + +```xml + + + $(MSBuildThisFileDirectory).copilot-cli\ + + +``` + +The SDK's own `CopilotCliBinaryPath` (use an already-downloaded binary) and `CopilotSkipCliDownload` (skip acquisition entirely) properties continue to work and take precedence over everything above. + ## Problem & Solution The [default orchestrator](./index.md) manages the full agentic pipeline — tool calling, progressive scoping, RAG injection, and streaming — using `Microsoft.Extensions.AI`. This works well when you control the provider connection and model selection. diff --git a/src/Primitives/CrestApps.Core.AI.Copilot/CrestApps.Core.AI.Copilot.csproj b/src/Primitives/CrestApps.Core.AI.Copilot/CrestApps.Core.AI.Copilot.csproj index 03b794e6..e3529f13 100644 --- a/src/Primitives/CrestApps.Core.AI.Copilot/CrestApps.Core.AI.Copilot.csproj +++ b/src/Primitives/CrestApps.Core.AI.Copilot/CrestApps.Core.AI.Copilot.csproj @@ -35,6 +35,9 @@ <_CopilotTargetsTemplate>$([System.IO.File]::ReadAllText('$(MSBuildProjectDirectory)\$(CopilotTargetsTemplateFile)')) <_GeneratedCopilotTargets>$([System.String]::Copy('$(_CopilotTargetsTemplate)').Replace('__GitHubCopilotSdkVersion__', '$(GitHubCopilotSdkVersion)')) + + <_GeneratedCopilotTargets>$([MSBuild]::Escape('$(_GeneratedCopilotTargets)')) diff --git a/src/Primitives/CrestApps.Core.AI.Copilot/buildTransitive/CrestApps.Core.AI.Copilot.targets.in b/src/Primitives/CrestApps.Core.AI.Copilot/buildTransitive/CrestApps.Core.AI.Copilot.targets.in index 6172d3d1..14ff368f 100644 --- a/src/Primitives/CrestApps.Core.AI.Copilot/buildTransitive/CrestApps.Core.AI.Copilot.targets.in +++ b/src/Primitives/CrestApps.Core.AI.Copilot/buildTransitive/CrestApps.Core.AI.Copilot.targets.in @@ -18,6 +18,12 @@ <_GitHubCopilotSdkVersion Condition="'$(_GitHubCopilotSdkVersion)' == ''">__GitHubCopilotSdkVersion__ <_GitHubCopilotSdkPackageRoot>$([MSBuild]::EnsureTrailingSlash('$(NuGetPackageRoot)')) <_GitHubCopilotSdkBuildDir>$(_GitHubCopilotSdkPackageRoot)github.copilot.sdk/$(_GitHubCopilotSdkVersion)/build + + <_CopilotNpmRegistryUrlWasSet Condition="'$(CopilotNpmRegistryUrl)' != ''">true + + + true + $(_GitHubCopilotSdkPackageRoot).crestapps/copilot-cli/ + + + + <_CopilotSharedCliDir>$([MSBuild]::EnsureTrailingSlash('$(CopilotCliCacheDir)'))$(CopilotCliVersion)/$(_CopilotPlatform)/ + <_CopilotSharedCliPath>$(_CopilotSharedCliDir)$(_CopilotBinary) + + $(_CopilotSharedCliPath) + <_CopilotCliBinaryPath Condition="Exists('$(_CopilotSharedCliPath)')">$(_CopilotSharedCliPath) + + + + + true + <_CopilotDefaultNpmRegistryUrl>https://registry.npmjs.org + + + + + <_CopilotResolvedNpmRegistry Condition="'$(NPM_CONFIG_REGISTRY)' != ''">$(NPM_CONFIG_REGISTRY) + + + + + + + + + <_CopilotNpmConfigOutput>$(_CopilotNpmConfigOutput.Trim()) + <_CopilotResolvedNpmRegistry Condition="$(_CopilotNpmConfigOutput.StartsWith('http'))">$(_CopilotNpmConfigOutput) + + + + $(_CopilotResolvedNpmRegistry) + + + + + + + + + <_CopilotSharedCliStagingPath>$(_CopilotSharedCliPath).$([System.Guid]::NewGuid().ToString('N')).tmp + + + + + + + + From f2d0c04c35137f19afb5546ade1b8daff457dd53 Mon Sep 17 00:00:00 2001 From: Mike Alhayek Date: Mon, 27 Jul 2026 21:51:30 +0300 Subject: [PATCH 2/2] Clarify that Copilot CLI acquisition needs no consumer configuration The properties documented alongside the shared cache and npm registry resolution are override knobs, not setup steps. The wording and the Directory.Build.props example made them look required. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- src/CrestApps.Core.Docs/docs/orchestration/copilot.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/CrestApps.Core.Docs/docs/orchestration/copilot.md b/src/CrestApps.Core.Docs/docs/orchestration/copilot.md index 3576f081..3be0ef3c 100644 --- a/src/CrestApps.Core.Docs/docs/orchestration/copilot.md +++ b/src/CrestApps.Core.Docs/docs/orchestration/copilot.md @@ -37,15 +37,15 @@ public class MyController(IOrchestratorResolver resolver) ## Copilot CLI Acquisition -The Copilot orchestrator runs on top of the GitHub Copilot CLI. Referencing `CrestApps.Core.AI.Copilot` is all a host needs to do: the package re-imports the `GitHub.Copilot.SDK` build targets, which download the CLI for the current runtime identifier during build and copy it to `runtimes//native/` in the output folder. No manual install step is required. +The Copilot orchestrator runs on top of the GitHub Copilot CLI. Referencing `CrestApps.Core.AI.Copilot` is the only thing a host has to do — there is nothing to install, configure, or add to your project files. The package re-imports the `GitHub.Copilot.SDK` build targets, which download the CLI for the current runtime identifier during build and copy it to `runtimes//native/` in the output folder. -The package layers two improvements on top of the SDK's default acquisition. +The package layers two improvements on top of the SDK's default acquisition, both of which apply automatically. **A machine-wide CLI cache.** The SDK caches the CLI under the intermediate output path, which is per project *and* per configuration, so a solution with several referencing projects downloads the same large tarball once per project, and again for every fresh clone, worktree, or CI agent. The package redirects that to a shared cache under the NuGet global packages folder, so the CLI is downloaded once and reused everywhere. That folder is already restored per machine and is commonly cached by CI pipelines, so build agents benefit as well. **Automatic npm registry resolution.** The SDK downloads the CLI tarball from `https://registry.npmjs.org`, and MSBuild's `DownloadFile` task cannot read npm configuration. Machines and build agents behind a corporate proxy or an artifact mirror usually block that host, so the build fails with a connectivity error even though npm itself is configured correctly. The package resolves the effective registry from the `NPM_CONFIG_REGISTRY` environment variable, falling back to `npm config get registry`, and uses that instead. The lookup only runs when a download is actually about to happen, and silently falls back to the SDK default when npm is unavailable or reports something that is not a URL. -Both behaviors are opt-out and never override a value you set yourself. +Both behaviors are on by default and require no configuration. The properties below exist only for hosts that need to override them, and none of them has to be set for a normal build. | Property | Default | Description | | --- | --- | --- | @@ -54,7 +54,7 @@ Both behaviors are opt-out and never override a value you set yourself. | `CopilotResolveNpmRegistry` | `true` | Set to `false` to always use the SDK default registry. | | `CopilotNpmRegistryUrl` | resolved from npm | Pin a registry explicitly. An explicit value always wins over resolution. | -These are ordinary MSBuild properties, so set them in `Directory.Build.props` to apply them across a solution: +These are ordinary MSBuild properties. When you do want to override one across a solution, set it in `Directory.Build.props` — for example, to keep the cache inside the repository instead of the NuGet global packages folder: ```xml