Skip to content

Compress output#175

Draft
djmaxus wants to merge 8 commits intomainfrom
compress-output
Draft

Compress output#175
djmaxus wants to merge 8 commits intomainfrom
compress-output

Conversation

@djmaxus
Copy link
Collaborator

@djmaxus djmaxus commented Mar 14, 2026

Closes #171
Closes #172

djmaxus added 2 commits March 13, 2026 15:20
Capillary pressure computation with J-function values as arguments
@djmaxus djmaxus self-assigned this Mar 14, 2026
Copilot AI review requested due to automatic review settings March 14, 2026 18:38
@djmaxus djmaxus force-pushed the compress-output branch 4 times, most recently from fa1ce7d to b7cfcf9 Compare March 14, 2026 18:44
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new “upscaled tables” representation and a lossy compression pipeline for saturation-function outputs, and updates plotting + OGS/OPM exporters to consume the new representation. Also changes the OPM JFUNC phase keyword from WATER to GAS per the OPM Flow manual guidance.

Changes:

  • Restructure strata_trapper output by organizing per-region/per-direction upscaled tables (UpscaledTables) and adding mapping indirection to enable compression.
  • Implement compression utilities (quantize, PCA/LET feature transforms, de/quantization helpers) and demo-time validation/visualization.
  • Update OPM/OGS exporters and plotting to use the organized tables format (and adjust OPM JFUNC to 'GAS').

Reviewed changes

Copilot reviewed 23 out of 23 changed files in this pull request and generated 12 comments.

Show a summary per file
File Description
src/strata_trapper.m Adds organize() to convert raw arrays into UpscaledTables + mapping and removes old raw-table fields from output.
src/plot_result.m Switches plotting to use tables/dequantize instead of raw per-cell arrays; adds param_id validation call.
src/export/opm_export.m Updates JFUNC phase to 'GAS' and rewires mapping/export flow to use compressed tables and new write_krnum signature.
src/export/ogs_export.m Rewires curve generation/mapping to use compressed tables; updates curves.inc generation to include per-region/per-dir files.
src/export/ogs/generate_sfn.m Writes OGS characteristic curve files based on representative tables + mapping instead of one-per-cell tables.
src/export/normalize_lj.m New helper intended to normalize Leverett-J values consistently with normalized CapPressure multipliers for export.
src/export/common/write_krnum.m Refactors KRNUM generation to use tables(...).mapping directly (instead of cell_to_curve).
src/compress/to_features.m Implements feature extraction/decoding pipeline (optionally LET fit + PCA) for quantization.
src/compress/quantize.m Main compression entry point: dedup + optional k-means quantization and reconstruction into UpscaledTables.
src/compress/fit_LET.m Fits LET parameters for krw/krg using fminsearch.
src/compress/dequantize.m Expands representative-table arrays back to per-cell arrays using .mapping.
src/compress/compare_tables.m Compares two table sets in feature space and returns per-(param,dir) reconstruction error.
src/compress/calc_let_wg.m Computes LET krw/krg curves given parameters (used by decoder/fit residual).
src/compress/UpscaledTables.m Introduces the table container class (leverett_j, krw, krg, mapping).
src/compress/README.md Documents the compression pipeline and options at a high level.
src/compress/QuantizeOptions.m Adds hyperparameter container for quantization and k-means settings.
src/CodeGenMex.m Extends MEX build/clear to include fit_LET and track multiple built artifacts.
src/CapPressure.m Adds from_lj() helper and adjusts func() to reuse it.
demo.m Adds a compression demo path that exercises quantize, compare_tables, and plotting.
buildfile.m Refactors test options construction and adds statement coverage reporting for newer MATLAB releases.
.github/copilot-instructions.md Adds repository-specific Copilot guidance and workflow documentation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

@djmaxus djmaxus force-pushed the compress-output branch 2 times, most recently from e583909 to 713d519 Compare March 15, 2026 14:19
Arrange results into `UpscaledTable`s

Reorganizes upscaled outputs into table groups

Improves data flow by storing upscaled curve results per region and direction, reducing scattered arrays and making downstream processing consistent.

Updates export and plotting paths to consume grouped tables, generate per-region curve files, and build deterministic directional mappings.

Normalizes Leverett J values before export so generated curves stay consistent across regions.
@djmaxus djmaxus marked this pull request as draft March 15, 2026 14:46
djmaxus added 5 commits March 15, 2026 14:49
Add lossy compression pipeline for table quantization

Introduces a quantization system that reduces table data size through feature extraction, deduplication, and k-means clustering. This enables efficient storage and transmission of upscaled relative permeability and capillary pressure tables while maintaining acceptable accuracy.

The implementation supports multiple reduction strategies:
- Parametric fitting via LET model
- Principal component analysis for dimensionality reduction
- Duplicate sample elimination to remove redundant data points
- K-means quantization to discretize feature space

New modules provide options configuration, feature transformation pipelines, and comparison utilities for validating compression trade-offs between fidelity and size reduction.
Establishes a centralized reference guide for AI-assisted development on the StrataTrapper MATLAB toolkit. Covers project architecture, core components, critical conventions (units, function signatures, parallel computing), developer workflows, testing infrastructure, and commit standards.

Enables consistent collaboration between human developers and agentic coding sessions by documenting the expected patterns for units handling, MATLAB Coder compatibility, grid/rock input conventions, and the structured commit format (Conventional Commits + gitmoji with co-author trailers).

Includes agentic session ground rules for minimal, focused edits with clear auditability and establishes a feedback loop for extracting transferrable best practices back into the instructions after feature completion.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(compress): Implement saturation table compression feature OPM exort: try using JFUNC 'GAS' instead of WATER

2 participants