Do not overwrite credentials with the wal-g prefix in the env dir

[daniel-schub]

Problem

configure_spilo.py writes the generated wal-g bucket prefix into a credentials variable in the wal-g env dir, which breaks backups and WAL archiving whenever credentials come from the cloud metadata service instead of a key file.

On GCS, archive_command fails with:

ERROR: configure primary storage: configure storage with prefix "gs://my-bucket/spilo/my-cluster/wal/17":
create Google Cloud storage: create GCS client: dialing: google: error getting credentials using
GOOGLE_APPLICATION_CREDENTIALS environment variable: open gs://my-bucket/spilo/my-cluster/wal/17:
no such file or directory

because the env dir contains:

/run/etc/wal-e.d/env/GOOGLE_APPLICATION_CREDENTIALS = gs://my-bucket/spilo/my-cluster/wal/17

Cause

Before WAL-E was removed the storage name lists started with the WAL-E prefix, so index 1 was the wal-g prefix:

gs_names = ['WALE_GS_PREFIX', 'WALG_GS_PREFIX', 'GOOGLE_APPLICATION_CREDENTIALS']

After #1143 the lists start with the wal-g prefix:

gs_names = ['WALG_GS_PREFIX', 'GOOGLE_APPLICATION_CREDENTIALS']
s3_names = ['WALG_S3_PREFIX', 'AWS_ACCESS_KEY_ID', ...]

but the compatibility block that back-filled the wal-g prefix was kept and still writes to index 1:

# Set WALG_*_PREFIX for future compatibility
if store_type in ('S3', 'GS') and not walg.get(write_envdir_names[1]):
    walg[write_envdir_names[1]] = walg[prefix_env_name]

So index 1 is now GOOGLE_APPLICATION_CREDENTIALS for GCS and AWS_ACCESS_KEY_ID for S3. The block is also redundant, because write_envdir_names[0] is already WALG_*_PREFIX and is resolved a few lines above.

Impact

• GCS: any setup without a service account key file, e.g. GKE Workload Identity or plain GCE metadata credentials, loses WAL archiving and basebackups. wal-g documents that GOOGLE_APPLICATION_CREDENTIALS must be unset to fall back to the metadata service.
• S3: AWS_ACCESS_KEY_ID is set to the s3:// URL even though AWS_INSTANCE_PROFILE=true was just enabled because no static keys were provided.

The variable is rewritten on every pod start, so it cannot be worked around through the environment.

Fix

Remove the obsolete block.

Verification

Calling write_walg_environment() with only WAL_GS_BUCKET / WAL_S3_BUCKET set and dumping the resulting env dir.

Before:

--- GS
GOOGLE_APPLICATION_CREDENTIALS   = gs://my-bucket/spilo/my-cluster/wal/17
WALG_GS_PREFIX                   = gs://my-bucket/spilo/my-cluster/wal/17
--- S3
AWS_ACCESS_KEY_ID                = s3://my-eu-central-1-bucket/spilo/my-cluster/wal/17
AWS_INSTANCE_PROFILE             = true
WALG_S3_PREFIX                   = s3://my-eu-central-1-bucket/spilo/my-cluster/wal/17

After:

--- GS
WALG_GS_PREFIX                   = gs://my-bucket/spilo/my-cluster/wal/17
--- S3
AWS_INSTANCE_PROFILE             = true
WALG_S3_PREFIX                   = s3://my-eu-central-1-bucket/spilo/my-cluster/wal/17

With the variable no longer poisoned, wal-g backup-list and wal-g wal-push succeed again against GCS using metadata-service credentials.

flake8 --max-line-length=120 postgres-appliance/scripts/configure_spilo.py is clean.