[Zacgoose]

This pull request introduces several important enhancements and fixes to the API response caching and egress tracking system. The main changes include: extending the egress ledger to track per-client IP statistics and maintain a multi-day history in the local file, clarifying cache and retention policy behaviors, and adding more explicit cache invalidation controls. These changes improve observability, enable finer-grained analytics, and align the retention of local and table-based egress data.

Egress Ledger Enhancements:

• The EgressLedger now tracks statistics not only by client/app ID but also by client IP address, allowing for more granular monitoring and reporting of API usage and rate limiting. This includes updates to Record, RecordInteractive, and RecordShed methods and the underlying data structures. [1] [2] [3] [4] [5] [6] [7] [8]
• The ledger now maintains a history of completed days in the local file, enabling retention of multiple days (default 7) of egress data for local inspection and analytics. The retention window is enforced via the new PruneHistory method, and the local file format is updated to version 4 to support this. [1] [2] [3] [4] [5]

Configuration and Policy Clarifications:

• The EgressLimitSettings.RetentionDays documentation is clarified to specify that the retention period applies to both the database table and the local ledger file.
• A new EndpointCaching enum is introduced to allow endpoints to explicitly declare their caching policy (convention-based, always cached, or never cached), and the CraftEndpointAttribute is extended to use this. [1] [2]

Cache and Invalidation Improvements:

• The CacheService now provides a method to explicitly invalidate cache entries via a query parameter, supporting on-demand cache refresh for clients.
• The CraftResult class exposes methods for serializing/deserializing cacheable responses, ensuring only safe (JSON, 2xx/3xx) responses are stored in the cache.

Internal Refactoring and Maintenance:

• Refactored egress ledger file loading, flushing, and snapshotting logic to support the new multi-day history and ensure thread safety. [1] [2] [3] [4]
• Updated table sync logic to aggregate per-IP stats alongside per-endpoint stats for more comprehensive reporting. [1] [2]

These changes collectively improve the robustness, flexibility, and transparency of API usage tracking and caching in the system.