Move Config to Writeable Dir

lambda_function/Dockerfile

@@ -34,10 +34,12 @@ COPY src/. ${FUNCTION_DIR}
 RUN chmod -R 755 ${FUNCTION_DIR}
 RUN chown -R 1000:1000 ${FUNCTION_DIR}
 
-# Package config locations (baked under ${FUNCTION_DIR}config/<pkg>/).
-# See src/config/README.md for the convention and the /tmp/<pkg>/ rule for
-# any package that needs writable scratch space at runtime.
-ENV ccsdspy_CONFIGDIR=${FUNCTION_DIR}config/ccsdspy
+# Package config locations. Source-of-truth YAML lives in src/config/<pkg>/
+# (baked into the image at ${FUNCTION_DIR}config/<pkg>/) and is mirrored to
+# the writable /tmp/config/<pkg>/ at runtime by entry_script.sh. Each package
+# env var below points at the /tmp mirror because some packages (e.g.
+# ccsdspy) require their CONFIGDIR to be writable. See src/config/README.md.
+ENV ccsdspy_CONFIGDIR=/tmp/config/ccsdspy
 
 # Copy entry script into function director (Script is used distinguish dev/production mode)
 COPY entry_script.sh ${FUNCTION_DIR}

lambda_function/entry_script.sh

@@ -1,5 +1,17 @@
 #!/bin/sh
 
+# Seed writable runtime config tree from baked package configs.
+# Some packages (e.g. ccsdspy) require their *_CONFIGDIR env var to point at a
+# writable directory, but the Lambda image filesystem is read-only at runtime
+# (only /tmp is writable). We mirror the baked /lambda_function/config/ tree
+# into /tmp/config/ here, before the Python runtime starts. AWS Lambda mounts
+# a fresh /tmp per execution environment, so this runs on every cold start;
+# warm starts re-copy harmlessly (idempotent). See src/config/README.md.
+if [ -d "/lambda_function/config" ]; then
+    mkdir -p /tmp/config
+    cp -R /lambda_function/config/. /tmp/config/
+fi
+
 if [ -z "${AWS_LAMBDA_RUNTIME_API}" ]; then
     exec /usr/local/bin/aws-lambda-rie python3 -m awslambdaric $@
 else

lambda_function/src/config/README.md

@@ -1,9 +1,8 @@
 # Baked package configuration
 
-This directory holds **read-only, build-time configuration files** that ship
-inside the Lambda image. Each immediate subdirectory corresponds to one
-upstream Python package whose import-time behavior we need to control before
-any handler code runs.
+This directory holds **source-of-truth, build-time configuration files** for
+upstream Python packages whose import-time behavior we need to control before
+any handler code runs. Each immediate subdirectory corresponds to one package.
 
 ```
 lambda_function/src/config/
@@ -12,41 +11,54 @@ lambda_function/src/config/
     └── config.yml     <- (or whatever filename the package expects)
 ```
 
-At build time, the existing `COPY src/. ${FUNCTION_DIR}` step in
-`lambda_function/Dockerfile` ships this tree to `/lambda_function/config/`
+## Three-stage path: repo → image → `/tmp`
+
+Config files live in this directory in the repo (versioned, reviewed). At
+build time, the existing `COPY src/. ${FUNCTION_DIR}` step in
+`lambda_function/Dockerfile` ships the tree to `/lambda_function/config/`
 inside the image, and the recursive `chmod -R 755` / `chown -R 1000:1000`
-steps already apply correct permissions.
+steps apply correct permissions. At runtime, `lambda_function/entry_script.sh`
+mirrors `/lambda_function/config/` into the writable `/tmp/config/` *before*
+the Python runtime starts, and the Dockerfile `ENV` lines point each package
+at the `/tmp/config/<pkg>/` mirror.
+
+```
+repo:        lambda_function/src/config/<pkg>/...
+              └─ COPY src/. ${FUNCTION_DIR}  (build time)
+image:       /lambda_function/config/<pkg>/...    (read-only at runtime)
+              └─ cp -R /lambda_function/config/. /tmp/config/  (cold start, entry_script.sh)
+runtime:     /tmp/config/<pkg>/...                 (writable, what the ENV var points at)
+```
 
 ## Why bake config into the image?
 
 Some packages (e.g. `ccsdspy`) run a config loader and logger init at
-**package import time**. If their default config asks for a writable path
-(log files, cache dirs) on a read-only filesystem, the very first
-`import <package>` blows up before our handler can do anything about it.
-
-Pointing the package at a baked-in `config.yml` via an environment variable
-(see "Dockerfile wiring" below) lets us pre-empt the bad default without
-patching the package or shimming the handler.
+**package import time**. If the default config asks for a writable path on a
+read-only filesystem, or simply requires `<pkg>_CONFIGDIR` itself to be
+writable, the very first `import <package>` blows up before our handler can
+do anything about it. Pointing the package at a pre-seeded, writable
+`/tmp/config/<pkg>/` lets us pre-empt the bad default without patching the
+package or shimming the handler.
 
 ## AWS Lambda runtime filesystem constraint
 
 > **The entire image filesystem is read-only at Lambda runtime. Only `/tmp`
 > is writable.**
 
-That means **nothing under `/lambda_function/config/<pkg>/` may be written
-to at runtime**. If a package needs to write log files, caches, or any
-other state, the config you bake here **must** point those writes at a
-path under `/tmp/<pkg>/`. Two corollaries:
-
-1. The caller (handler module, entry script, or — preferred — the package
-   itself, via an upstream fix) is responsible for `mkdir -p /tmp/<pkg>`
-   on cold start. `/tmp` is a fresh empty mount on each execution
-   environment, so the directory will not survive across cold starts.
-2. Lambda's `/tmp` is wiped between execution environments, so any log
-   files written there are ephemeral. If you want durable, queryable logs,
-   prefer disabling file logging entirely and rely on Lambda's
-   stdout/stderr → CloudWatch pipeline (which is what we do for `ccsdspy`
-   below).
+Consequences:
+
+1. **`/lambda_function/config/<pkg>/` cannot be the env-var target** for any
+   package that requires its config dir to be writable (e.g. `ccsdspy`'s
+   `_get_user_configdir()` calls `mkdir` + `os.access(W_OK)` *before* it
+   reads the YAML). That is why `entry_script.sh` mirrors the tree into
+   `/tmp/config/` on every invocation, and why the `ENV` lines point at the
+   `/tmp` mirror.
+2. **`/tmp` is wiped between execution environments**, so the seed step has
+   to re-run on every cold start. The `cp -R` is idempotent for warm starts.
+3. **Log files, caches, and other runtime state** for a baked-config package
+   may live alongside its config under `/tmp/config/<pkg>/` — the tree is
+   writable. They are ephemeral (gone on the next cold start), so for
+   anything durable prefer Lambda's stdout/stderr → CloudWatch pipeline.
 
 ## Dockerfile wiring
 
@@ -56,75 +68,94 @@ For each package with a directory here, add **one** `ENV` line to
 `# Copy entry script into function director`):
 
 ```dockerfile
-# Package config locations (baked under /lambda_function/config/<pkg>/)
-ENV ccsdspy_CONFIGDIR=/lambda_function/config/ccsdspy
-# ENV <pkg>_CONFIGDIR=/lambda_function/config/<pkg>   <- pattern for future packages
+# Package config locations. Source-of-truth YAML lives in src/config/<pkg>/
+# (baked into the image at ${FUNCTION_DIR}config/<pkg>/) and is mirrored to
+# the writable /tmp/config/<pkg>/ at runtime by entry_script.sh.
+ENV ccsdspy_CONFIGDIR=/tmp/config/ccsdspy
+# ENV <pkg>_CONFIGDIR=/tmp/config/<pkg>   <- pattern for future packages
 ```
 
 Notes:
 
+- The env target is `/tmp/config/<pkg>`, **not** `${FUNCTION_DIR}config/<pkg>`,
+  because some packages require the dir to be writable. The seed step in
+  `entry_script.sh` is what makes `/tmp/config/<pkg>` exist.
 - Use `ENV`, **not** `ARG`. `ARG` values are build-time only and are not
-  visible to the Python runtime via `os.environ`, which is what these
-  packages consult at import time.
-- The exact variable name (`ccsdspy_CONFIGDIR`, `SOMETHING_CONFIG_DIR`,
-  etc.) is dictated by the upstream package — match whatever it reads.
-- Place the `ENV` lines together as a single block so they are easy to
-  grep and review.
+  visible to the Python runtime via `os.environ`.
+- The exact variable name (`ccsdspy_CONFIGDIR`, `SOMETHING_CONFIG_DIR`, etc.)
+  is dictated by the upstream package — match whatever it reads.
+- Place the `ENV` lines together as a single block so they are easy to grep.
 
 ## Worked example: `ccsdspy`
 
-**Problem.** Recent `ccsdspy` releases unconditionally open a
-`logging.FileHandler` against a relative `log_file_path` at package
-import time. In the Lambda image the CWD is `/lambda_function/` (which
-is read-only at runtime), so `import ccsdspy` — pulled in transitively
-by `padre_meddea.io.file_tools` — raises:
+**Problem.** Recent `ccsdspy` releases (a) unconditionally open a
+`logging.FileHandler` against a relative `log_file_path` at package import
+time, and (b) require `ccsdspy_CONFIGDIR` itself to be writable
+(`_get_user_configdir()` does `mkdir(...exist_ok=True)` + `os.access(W_OK)`
+before reading `config.yml`). In the Lambda image both fail: CWD
+`/lambda_function/` is read-only at runtime, and so is any baked subdir.
+`import ccsdspy` — pulled in transitively by `padre_meddea.io.file_tools` —
+raises one of:
 
 ```
 OSError: [Errno 30] Read-only file system: '/lambda_function/ccsdspy.log'
+RuntimeError: Could not write to ccsdspy_CONFIGDIR="/lambda_function/config/ccsdspy"
 ```
 
-**Fix.** Bake a minimal `config.yml` that disables file logging, and
-point `ccsdspy_CONFIGDIR` at it:
+**Fix.** Bake a `config.yml` that disables file logging, seed it into a
+writable `/tmp/config/ccsdspy/` at cold start, and point `ccsdspy_CONFIGDIR`
+at the writable mirror:
 
-`lambda_function/src/config/ccsdspy/config.yml`:
+`lambda_function/src/config/ccsdspy/config.yml` (the relevant key):
 
 ```yaml
 logger:
   log_to_file: false
 ```
 
+`lambda_function/entry_script.sh` (added near the top):
+
+```sh
+if [ -d "/lambda_function/config" ]; then
+    mkdir -p /tmp/config
+    cp -R /lambda_function/config/. /tmp/config/
+fi
+```
+
 `lambda_function/Dockerfile` (in the post-COPY env block):
 
 ```dockerfile
-ENV ccsdspy_CONFIGDIR=/lambda_function/config/ccsdspy
+ENV ccsdspy_CONFIGDIR=/tmp/config/ccsdspy
 ```
 
-`ccsdspy.config.load_config()` finds our YAML first and skips the
-packaged default, so `_init_log` never reaches the `FileHandler` branch.
-The `StreamHandler` still attaches, and log records continue to flow to
-CloudWatch via stdout.
+`ccsdspy._get_user_configdir()` now sees a writable `/tmp/config/ccsdspy/`
+seeded with our YAML, `load_config()` reads it instead of the packaged
+default, and `_init_log` never reaches the `FileHandler` branch. The
+`StreamHandler` still attaches, so log records continue to flow to CloudWatch
+via stdout.
 
 If you ever need `ccsdspy` to write a real log file, change the YAML to:
 
 ```yaml
 logger:
   log_to_file: true
-  log_file_path: /tmp/ccsdspy/ccsdspy.log
+  log_file_path: /tmp/config/ccsdspy/ccsdspy.log
 ```
 
-…and ensure `/tmp/ccsdspy` is created on cold start (in
-`entry_script.sh` or at the very top of `lambda.py`, before any
-`import ccsdspy`). Do **not** point `log_file_path` at anything under
-`/lambda_function/`.
+No extra `mkdir` is needed — the seed step already created
+`/tmp/config/ccsdspy/`. The file is ephemeral (wiped per execution
+environment), so for durable logs stick with the stdout/CloudWatch route.
 
 ## Adding a new package
 
 1. `mkdir lambda_function/src/config/<pkg>/` and drop the config file(s)
-   the package expects.
-2. If the package needs writable scratch space, make sure the baked
-   config points it at `/tmp/<pkg>/...`, never inside `config/`.
-3. Add one `ENV <pkg>_CONFIGDIR=/lambda_function/config/<pkg>` line to
-   the post-COPY env block in `lambda_function/Dockerfile`.
-4. If `/tmp/<pkg>/` needs to exist before the package is imported, add a
-   `mkdir -p /tmp/<pkg>` to `entry_script.sh` (preferred over runtime
-   Python so it happens before the Python interpreter starts).
+   the package expects. The existing `COPY src/. ${FUNCTION_DIR}` step ships
+   it; no Dockerfile COPY/chmod edits needed.
+2. If the baked config refers to any writable runtime path (log file, cache),
+   point it at `/tmp/config/<pkg>/...`. The `entry_script.sh` seed step
+   already guarantees that directory exists and is writable.
+3. Add one `ENV <pkg>_CONFIGDIR=/tmp/config/<pkg>` line to the post-COPY env
+   block in `lambda_function/Dockerfile` (match whatever env var name the
+   package actually reads).
+4. No `entry_script.sh` change is needed — the generic seed step already
+   mirrors the entire `/lambda_function/config/` tree.

lambda_function/src/config/ccsdspy/config.yml

@@ -15,13 +15,16 @@ logger:
   log_format: "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
 
   # Whether to always log messages to a log file
-  log_to_file: false
+  log_to_file: true
 
   # Whether the log file should be in JSON format
   log_file_json: false
 
-  # The file to log messages to
-  log_file_path: ccsdspy.log
+  # The file to log messages to.
+  # Must point at a writable location at Lambda runtime. /tmp/config/ccsdspy/
+  # is seeded by entry_script.sh and is the directory ccsdspy_CONFIGDIR points
+  # at, so it is guaranteed to exist and be writable. See src/config/README.md.
+  log_file_path: /tmp/config/ccsdspy/ccsdspy.log
 
   # Threshold for logging messages to log_file_path
   log_file_level: INFO