Basic HTTP auth not working since `v9.14.0`

[valentinoli]

Describe the problem

Generating a template with copier copy or run_copy() does not work with embedded credentials in the source path since v9.14.0 but works fine in prior versions.

Template

Any template compatible with Copier v9

To Reproduce

Run copier copy template_src destination_path where template_src is a remote URL with embedded credentials (basic auth). For example:

copier copy https://x-access-token:${{ MY_TOKEN }}@github.com/valentinoli/my-copier-template.git ./target-path

Logs

Traceback (most recent call last):
  File "/home/runner/.local/share/hatch/env/virtual/repo/9GJwBYEQ/test/bin/copier", line 10, in <module>
    sys.exit(CopierApp.run())
             ~~~~~~~~~~~~~^^
  File "/home/runner/.local/share/hatch/env/virtual/repo/9GJwBYEQ/test/lib/python3.14/site-packages/plumbum/cli/application.py", line 631, in run
    inst, retcode = subapp.run(argv, exit=False)
                    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/runner/.local/share/hatch/env/virtual/repo/9GJwBYEQ/test/lib/python3.14/site-packages/plumbum/cli/application.py", line 626, in run
    retcode = inst.main(*tailargs)
  File "/home/runner/.local/share/hatch/env/virtual/repo/9GJwBYEQ/test/lib/python3.14/site-packages/copier/_cli.py", line 275, in main
    return _handle_exceptions(inner)
  File "/home/runner/.local/share/hatch/env/virtual/repo/9GJwBYEQ/test/lib/python3.14/site-packages/copier/_cli.py", line 80, in _handle_exceptions
    exit_code = method()
  File "/home/runner/.local/share/hatch/env/virtual/repo/9GJwBYEQ/test/lib/python3.14/site-packages/copier/_cli.py", line 257, in inner
    run_copy(
    ~~~~~~~~^
        template_src,
        ^^^^^^^^^^^^^
    ...<13 lines>...
        skip_tasks=self.skip_tasks,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/runner/.local/share/hatch/env/virtual/repo/9GJwBYEQ/test/lib/python3.14/site-packages/copier/_main.py", line 1527, in run_copy
    with Worker(
         ~~~~~~^
        src_path=src_path,
        ^^^^^^^^^^^^^^^^^^
    ...<23 lines>...
        skip_tasks=skip_tasks,
        ^^^^^^^^^^^^^^^^^^^^^^
    ) as worker:
    ^
  File "/home/runner/.local/share/hatch/env/virtual/repo/9GJwBYEQ/test/lib/python3.14/site-packages/copier/_main.py", line 279, in __exit__
    raise value
  File "/home/runner/.local/share/hatch/env/virtual/repo/9GJwBYEQ/test/lib/python3.14/site-packages/copier/_main.py", line 1554, in run_copy
    worker.run_copy()
    ~~~~~~~~~~~~~~~^^
  File "/home/runner/.local/share/hatch/env/virtual/repo/9GJwBYEQ/test/lib/python3.14/site-packages/copier/_main.py", line 101, in _wrapper
    return func(*args, **kwargs)
  File "/home/runner/.local/share/hatch/env/virtual/repo/9GJwBYEQ/test/lib/python3.14/site-packages/copier/_main.py", line 1080, in run_copy
    self._print_message(self.template.message_before_copy)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/.local/share/hatch/env/virtual/.pythons/3.14/python/lib/python3.14/functools.py", line 1127, in __get__
    val = self.func(instance)
  File "/home/runner/.local/share/hatch/env/virtual/repo/9GJwBYEQ/test/lib/python3.14/site-packages/copier/_template.py", line 364, in message_before_copy
    return self.config_data.get("message_before_copy", "")
           ^^^^^^^^^^^^^^^^
  File "/home/runner/.local/share/hatch/env/virtual/.pythons/3.14/python/lib/python3.14/functools.py", line 1127, in __get__
    val = self.func(instance)
  File "/home/runner/.local/share/hatch/env/virtual/repo/9GJwBYEQ/test/lib/python3.14/site-packages/copier/_template.py", line 304, in config_data
    result = filter_config(self._raw_config)[0]
                           ^^^^^^^^^^^^^^^^
  File "/home/runner/.local/share/hatch/env/virtual/.pythons/3.14/python/lib/python3.14/functools.py", line 1127, in __get__
    val = self.func(instance)
  File "/home/runner/.local/share/hatch/env/virtual/repo/9GJwBYEQ/test/lib/python3.14/site-packages/copier/_template.py", line 261, in _raw_config
    for p in self.local_abspath.glob("copier.*")
             ^^^^^^^^^^^^^^^^^^
  File "/home/runner/.local/share/hatch/env/virtual/.pythons/3.14/python/lib/python3.14/functools.py", line 1127, in __get__
    val = self.func(instance)
  File "/home/runner/.local/share/hatch/env/virtual/repo/9GJwBYEQ/test/lib/python3.14/site-packages/copier/_template.py", line 570, in local_abspath
    self.ref or get_latest_tag(self.url_expanded, self.use_prereleases),
                ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/.local/share/hatch/env/virtual/repo/9GJwBYEQ/test/lib/python3.14/site-packages/copier/_vcs.py", line 145, in get_latest_tag
    for tag in git("ls-remote", "--tags", "--refs", url).splitlines()
               ~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/.local/share/hatch/env/virtual/repo/9GJwBYEQ/test/lib/python3.14/site-packages/plumbum/commands/base.py", line 110, in __call__
    return self.run(args, **kwargs)[1]
           ~~~~~~~~^^^^^^^^^^^^^^^^
  File "/home/runner/.local/share/hatch/env/virtual/repo/9GJwBYEQ/test/lib/python3.14/site-packages/plumbum/commands/base.py", line 249, in run
    return p.run()
           ~~~~~^^
  File "/home/runner/.local/share/hatch/env/virtual/repo/9GJwBYEQ/test/lib/python3.14/site-packages/plumbum/commands/base.py", line 212, in runner
    return run_proc(p, retcode, timeout)
  File "/home/runner/.local/share/hatch/env/virtual/repo/9GJwBYEQ/test/lib/python3.14/site-packages/plumbum/commands/processes.py", line 330, in run_proc
    return _check_process(proc, retcode, timeout, stdout, stderr)
  File "/home/runner/.local/share/hatch/env/virtual/repo/9GJwBYEQ/test/lib/python3.14/site-packages/plumbum/commands/processes.py", line 19, in _check_process
    proc.verify(retcode, timeout, stdout, stderr)
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/.local/share/hatch/env/virtual/repo/9GJwBYEQ/test/lib/python3.14/site-packages/plumbum/machines/base.py", line 29, in verify
    raise ProcessExecutionError(
        getattr(self, "argv", None), self.returncode, stdout, stderr
    )
plumbum.commands.processes.ProcessExecutionError: Unexpected exit code: 128
Command line: | /usr/bin/git ls-remote --tags --refs ***github.com/valentinoli/my-copier-template.git
Stderr:       | remote: Repository not found.
              | fatal: Authentication failed for 'https://github.com/valentinoli/my-copier-template.git/'

Expected behavior

copier copy https://x-access-token:${{ MY_TOKEN }}@github.com/valentinoli/my-copier-template.git ./target-path

This command should work with embedded credentials in the URL

Screenshots/screencasts/logs

No response

Operating system

Linux

Operating system distribution and version

Ubuntu 22

Copier version

9.14.0

Python version

3.14

Installation method

pipx+pypi

Additional context

No response

[sisp]

The error occurs when listing Git tags in the remote Git repo:

/usr/bin/git ls-remote --tags --refs ***github.com/valentinoli/my-copier-template.git

I've tried copier copy ... (using Copier v9.14.0) as well as git ls-remote ... (with a range of git versions) with private repos on github.com and gitlab.com using HTTP Basic Auth credentials embedded in the URL, it works for me. 🤔

Can you check whether git clone and git ls-remote work with your repo URL with embedded credentials?

[valentinoli]

I confirm again that the issue persists with copier==9.14.0 and everything works fine with copier==9.13.1.

I ran

/usr/bin/git ls-remote --tags --refs https://x-access-token:<TOKEN>@github.com/valentinoli/my-copier-template.git

and it works fine with git version 2.43.0

[sisp]

I've tried to reproduce the problem again using Copier v9.14.0 and Git v2.43.0 with a private test project on github.com and a fine-grained PAT with access limited to this project and permissions "Read access to code and metadata", it still works for me. 🤔

$ # Only one commit, no tags
$ uvx copier copy https://x-access-token:github_pat_xxxxxx@github.com/sisp/test-copier-http-basic-auth.git .
No git tags found in template; using HEAD as ref
Copying from template version 0.0.0.post1.dev0+ad73ff5

$ # With tag v0.1.0
$ uvx copier copy https://x-access-token:github_pat_xxxxxx@github.com/sisp/test-copier-http-basic-auth.git .
Copying from template version 0.1.0

$ uvx copier --version
copier 9.14.0

$ uvx --from copier python --version
Python 3.14.3

I'm on Ubuntu 22.04 and also tried it in a alpine/git:v2.43.0 container where I installed uv.

Is there any global Git setting that might be relevant? Your stack trace indicates that the problem is occurring in a GitHub CI job. Do you also see the error locally? And do you see the error in a clean environment like a fresh container?

[valentinoli]

I have a GitHub workflow in my Copier template repo that performs integration tests. I'm applying the multiple templates pattern to decouple and separate responsibilities. In my workflow, I need to first apply the "core" template (which is in a different repo) before applying the current template I'm testing. The workflow steps are as follows (simplified and redacted for readability):

steps:
  - uses: actions/checkout@v6
  
  # [...]

  - name: Install Copier
    run: pipx install copier

  - name: Generate GitHub App Token
    id: generate-token
    uses: actions/create-github-app-token@v3
    with:
      app-id: ${{ secrets.MY_APP_ID }}
      private-key: ${{ secrets.MY_PRIVATE_KEY }}
      owner: valentinoli
      repositories: my-copier-template

  - name: Generate project
    run: |
      copier copy --defaults \
        https://x-access-token:${{ steps.generate-token.outputs.token }}@github.com/valentinoli/my-copier-template.git ${{ env.TARGET_DIR }}

      copier copy --vcs-ref HEAD --defaults ${{ github.workspace }} ${{ env.TARGET_DIR }}

  - name: Integration tests
    run: bash run-som-tests.sh

I discovered that for some reason git ls-remote prioritizes the persisted token from the checkout action. I worked around it by providing persist-credentials: false which fixed my issue. Perhaps it is worth adding a note in your Q&A or troubleshooting guide even though this is not directly related to Copier.

  - uses: actions/checkout@v6
    with:
       persist-credentials: false

[valentinoli]

I confirm again that the issue persists with copier==9.14.0 and everything works fine with copier==9.13.1.

I ran

/usr/bin/git ls-remote --tags --refs https://x-access-token:@github.com/valentinoli/my-copier-template.git
and it works fine with git version 2.43.0

Just to clarify, here I meant that the issue persisted in my CI pipeline, while the git ls-remote command works fine locally.