Make the scheduler Dag cache size and TTL configurable

[jason810496]

• Depends on: Honor the API server Dag cache TTL when no size limit is set #71814 and Report Dag cache metrics under each component's own namespace #71815
• Diff for early review: feat/dagbag-cache-metric-namespace...feat/scheduler-dag-cache-config

Why

The scheduler's cache bound landed in #71704 as a fixed constant. Deployments whose active Dag-version working set exceeds that limit cannot raise it, while memory-constrained deployments cannot lower it or choose time-based eviction.

How

The API server and scheduler read and validate their size and TTL pairs through one helper. Each component constructs CachedDBDagBag for every configuration. When both values are 0, the configured cache uses a plain unbounded dict and performs no eviction.

What

• Adds [scheduler] dag_cache_size, defaulting to the existing LRU limit of 512 versions.
• Adds [scheduler] dag_cache_ttl, defaulting to 0 with no time-based eviction.
• Defines 0 consistently: size 0 disables size-based eviction, TTL 0 disables time-based eviction, and both 0 disable eviction for the process lifetime.
• Documents the scheduler tuning guidance and the corresponding metrics.
• Adds a feature newsfragment for the new scheduler settings.

---

Was generative AI tooling used to co-author this PR?

• Yes, with help from Claude Code (Opus 5) and Codex (GPT-5), following the project's AI-assisted contribution guidelines.