Restructure package as plottle and secure formula evaluation

.github/workflows/docs.yml

@@ -23,13 +23,13 @@ jobs:
     name: Build site
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@v6
 
       - name: Configure Pages
-        uses: actions/configure-pages@v4
+        uses: actions/configure-pages@v6
 
       - name: Upload Pages artifact
-        uses: actions/upload-pages-artifact@v3
+        uses: actions/upload-pages-artifact@v5
         with:
           path: ./docs/
 
@@ -43,4 +43,4 @@ jobs:
     steps:
       - name: Deploy to GitHub Pages
         id: deployment
-        uses: actions/deploy-pages@v4
+        uses: actions/deploy-pages@v5

.github/workflows/tests.yml

@@ -6,8 +6,18 @@ on:
   pull_request:
     branches: [main]
 
+# Read-only is all either job needs.
+permissions:
+  contents: read
+
+# Superseding pushes cancel in-flight runs instead of queueing behind them.
+concurrency:
+  group: tests-${{ github.ref }}
+  cancel-in-progress: true
+
 jobs:
   lint:
+    name: Lint and type-check
     runs-on: ubuntu-latest
 
     steps:
@@ -17,74 +27,96 @@ jobs:
         uses: actions/setup-python@v6
         with:
           python-version: "3.11"
+          cache: pip
+          cache-dependency-path: pyproject.toml
 
-      - name: Install lint tools
-        run: pip install ruff mypy types-PyYAML types-requests
+      # Install the project itself, not just the lint tools. Previously this job
+      # installed only `ruff mypy types-*`, so numpy/pandas/matplotlib/streamlit were
+      # all absent and --ignore-missing-imports collapsed them to `Any` -- mypy was
+      # reporting "Success" while checking almost nothing. ruff and mypy are pinned in
+      # the dev extra so a tool release cannot turn CI red on its own. See audit A-10.
+      - name: Install project and dev tooling
+        run: pip install -e ".[dev,formats,nist]"
 
       - name: ruff check
-        run: ruff check modules/ cli.py __init__.py
+        run: ruff check plottle/
 
       - name: ruff format check
-        run: ruff format --check modules/ cli.py __init__.py
+        run: ruff format --check plottle/
 
       - name: mypy check
         run: |
           mypy \
-            modules/io.py \
-            modules/math.py \
-            modules/plotting.py \
-            modules/signal.py \
-            modules/peaks.py \
-            modules/data_tools.py \
-            modules/annotations.py \
-            modules/batch.py \
-            modules/report.py \
-            modules/plugin_loader.py \
-            modules/spectroscopy.py \
-            modules/nist.py \
-            cli.py \
+            plottle/io.py \
+            plottle/math.py \
+            plottle/plotting.py \
+            plottle/signal.py \
+            plottle/peaks.py \
+            plottle/data_tools.py \
+            plottle/annotations.py \
+            plottle/batch.py \
+            plottle/report.py \
+            plottle/plugin_loader.py \
+            plottle/spectroscopy.py \
+            plottle/nist.py \
+            plottle/cli.py \
             --ignore-missing-imports --no-strict-optional --explicit-package-bases
 
   test:
-    runs-on: ubuntu-latest
+    name: Tests (${{ matrix.os }}, Python ${{ matrix.python-version }})
+    runs-on: ${{ matrix.os }}
+
+    strategy:
+      # One failing cell should not hide the others -- knowing whether a break is
+      # version-specific or universal is most of the diagnosis.
+      fail-fast: false
+      matrix:
+        # pyproject declares requires-python = ">=3.9,!=3.9.7" and classifiers for
+        # 3.9-3.12, so every one of those is tested. Testing only 3.11 is what let
+        # G-010 (st.Page on 3.9) and the tomllib import in the Settings page (3.11+
+        # stdlib) reach users. See audit A-12.
+        os: [ubuntu-latest]
+        python-version: ["3.9", "3.10", "3.11", "3.12"]
+        include:
+          # Non-Linux spot checks: the setup/launch scripts target macOS and Windows.
+          # Note macOS runners are arm64 and actions/setup-python has no 3.9 build for
+          # them, so the exact G-010 combination (macOS + 3.9) needs an Intel runner.
+          # Until then it is covered by ubuntu 3.9 plus the absolute-path assertions in
+          # tests/test_gui_paths.py, which are OS-independent.
+          - os: macos-latest
+            python-version: "3.12"
+          - os: windows-latest
+            python-version: "3.11"
 
     steps:
       - uses: actions/checkout@v6
 
-      - name: Set up Python 3.11
+      - name: Set up Python ${{ matrix.python-version }}
         uses: actions/setup-python@v6
         with:
-          python-version: "3.11"
+          python-version: ${{ matrix.python-version }}
+          cache: pip
+          cache-dependency-path: pyproject.toml
 
-      - name: Install dependencies
-        run: pip install -r requirements.txt
+      # `[test,...]`, not `[dev,...]`: the lint tools pinned in `dev` have higher
+      # Python floors than this matrix spans (mypy 2.3 requires >=3.10), and the test
+      # job does not run them.
+      - name: Install project and test dependencies
+        run: pip install -e ".[test,formats,nist]"
 
-      - name: Run tests with coverage
+      # Run the whole suite. This previously enumerated 15 files by hand and so never
+      # ran tests/test_cli.py (27 tests, the entire CLI surface) or
+      # tests/test_integration.py (14 tests) -- a hand-maintained list gives no signal
+      # when it falls behind. See audit A-11.
+      - name: Run tests
         env:
           MPLBACKEND: Agg
-        run: |
-          pytest \
-            tests/test_io.py \
-            tests/test_math.py \
-            tests/test_plotting.py \
-            tests/test_plotting_advanced.py \
-            tests/test_utils.py \
-            tests/test_signal.py \
-            tests/test_peaks.py \
-            tests/test_data_tools.py \
-            tests/test_annotations.py \
-            tests/test_spectroscopy.py \
-            tests/test_molecular_parsers.py \
-            tests/test_batch.py \
-            tests/test_report.py \
-            tests/test_plugin_loader.py \
-            tests/test_nist.py \
-            --cov=modules \
-            --cov-report=xml \
-            --cov-report=term-missing \
-            -v
+        run: pytest tests/ --cov=plottle --cov-report=xml --cov-report=term-missing -v
 
+      # One cell only: with a matrix, every job uploading the same artifact name is an
+      # error on upload-artifact v4+.
       - name: Upload coverage report
+        if: matrix.os == 'ubuntu-latest' && matrix.python-version == '3.11'
         uses: actions/upload-artifact@v7
         with:
           name: coverage-report

README.md

@@ -48,23 +48,28 @@ python -m venv .venv
 
 # Windows
 .venv\Scripts\activate
-pip install -r requirements.txt
 
 # macOS / Linux
 source .venv/bin/activate
-pip install -r requirements.txt
 
-streamlit run modules/Home.py
+# Install Plottle and its dependencies. The editable install is what puts the
+# `plottle` command on your PATH.
+pip install -e ".[formats,nist]"
+
+streamlit run plottle/Home.py
 ```
 
 #### 3 — Or use the CLI
 
 ```bash
-python cli.py --help
-python cli.py plot data.csv --plot scatter --x-column x --y-column y
-python cli.py stats data.csv
+plottle --help
+plottle plot data.csv --plot scatter --x-column x --y-column y
+plottle stats data.csv
 ```
 
+The `plottle` command comes from the `pip install` above. From a source checkout
+without installing, use `python -m plottle.cli` instead.
+
 ## GUI Pages
 
 | Page | Description |
@@ -97,11 +102,11 @@ python cli.py stats data.csv
 Plottle includes a command-line interface with 5 subcommands:
 
 ```bash
-python cli.py plot    <file> --plot <type> --x-column X --y-column Y [--output out.png]
-python cli.py stats  <file>
-python cli.py batch  <config.json> [--verbose]
-python cli.py compare <file1> <file2> --plot line [--output comparison.png]
-python cli.py convert <input> <output>
+plottle plot    <file> --plot <type> --x-column X --y-column Y [--output out.png]
+plottle stats  <file>
+plottle batch  <config.json> [--verbose]
+plottle compare <file1> <file2> --plot line [--output comparison.png]
+plottle convert <input> <output>
 ```
 
 See the [CLI Tutorial](docs/tutorials/cli_guide.md) for full usage and examples.
@@ -180,12 +185,13 @@ See [`requirements.txt`](requirements.txt) for pinned version ranges.
 ## Project Structure
 
 ```text
-plottle/
-├── modules/
+plottle/                            ← repository root
+├── plottle/                        ← the installable package
 │   ├── Home.py                     ← Streamlit entry point
+│   ├── cli.py                      ← CLI entry point (5 subcommands)
 │   ├── io.py                       ← 18-format data loader/saver
 │   ├── math.py                     ← 25 analysis functions
-│   ├── plotting.py                 ← 26 plot types (Matplotlib, Seaborn, Plotly)
+│   ├── plotting.py                 ← 27 plot types (Matplotlib, Seaborn, Plotly)
 │   ├── signal.py                   ← 16 signal processing functions
 │   ├── peaks.py                    ← 5 peak analysis functions
 │   ├── data_tools.py               ← 12 non-destructive DataFrame operations
@@ -197,20 +203,24 @@ plottle/
 │   ├── plugin_loader.py            ← plugin discovery and loading
 │   ├── molecular/                  ← CPK atom data + vibrational parsers
 │   ├── pages/                      ← 13 Streamlit pages (1_*.py – 14_*.py)
-│   └── utils/                      ← session state, plot config, user settings
-├── cli.py                          ← CLI entry point (5 subcommands)
-├── tests/                          ← 17 test files, 900+ tests
+│   ├── utils/                      ← session state, plot config, user settings
+│   ├── assets/                     ← logo + NCCU branding (shipped in the wheel)
+│   ├── gallery/                    ← pre-rendered gallery PNGs + manifest.json
+│   └── example-data/Artificial/    ← built-in sample datasets
+├── tests/                          ← 17 test files, 950+ tests
 ├── plugins/                        ← plugin_example.py starter template
 ├── examples/                       ← 15+ standalone scripts + batch_config.json
-├── notebooks/                      ← 4 Jupyter tutorials (Binder-ready)
-├── docs/                           ← tutorials, gallery, cheatsheet
-├── example-data/                   ← generated sample datasets
+├── docs/                           ← tutorials, cheatsheet, Sphinx sources
+├── generate_gallery.py             ← regenerates plottle/gallery/
 ├── requirements.txt
 ├── pyproject.toml
 ├── setup.bat / setup.command       ← First-run setup (Windows / macOS — double-click)
 └── launch.bat / launch.command     ← App launcher (Windows / macOS — double-click)
 ```
 
+Runtime assets (`assets/`, `gallery/`, `example-data/`) live **inside** the package
+rather than at the repository root, so they are present after `pip install`.
+
 ## Documentation
 
 - [Getting Started](docs/getting_started.md) — installation walkthrough and API examples

SECURITY.md

@@ -39,11 +39,46 @@ A useful report includes:
 The following behaviors are **intentional** and documented — please do not report
 them as vulnerabilities unless you have found a bypass of the existing mitigations:
 
-- **`eval()` in Analysis Tools and Data Tools:** User-provided function expressions
-  (e.g., custom curve-fit formulas) are evaluated with `eval()`. The execution context
-  is restricted to a minimal namespace. This is acceptable for a local/classroom tool
-  but should not be exposed to untrusted users on a public server without additional
-  sandboxing.
+- **User-provided expressions in Analysis Tools and Data Tools.** Formula and
+  curve-fit expressions are supplied as text and evaluated by Plottle.
+
+  `plottle.data_tools.add_formula_column` evaluates them with an **AST whitelist**
+  (`_safe_eval`): the expression is parsed, and only arithmetic, comparisons,
+  `and`/`or`/`not`, conditional expressions, subscripting, literals, whitelisted
+  names, and direct calls to the documented math functions are executed. Attribute
+  access, imports, lambdas, comprehensions, assignments, and indirect calls are
+  rejected. Exponents and expression length are bounded to prevent trivially
+  constructed hangs.
+
+  Row filtering (`filter_rows`) delegates to `pandas.DataFrame.eval`, which uses
+  pandas' own restricted parser.
+
+  **What this does and does not guarantee.** The whitelist is intended to prevent an
+  expression from reaching the Python runtime — the filesystem, the process, or the
+  import system. It is not a resource sandbox: an expression can still allocate large
+  arrays or run for a long time. Treat it as a robust barrier against code execution,
+  not as a guarantee of availability. If you find an expression that reaches attribute
+  access, an import, or any callable outside the documented list, that **is** a
+  vulnerability — please report it.
+
+  Note that Plottle is designed to run on `localhost` for a single user. Binding the
+  Streamlit GUI to a public interface exposes every input surface described below to
+  anyone who can reach the port, and is not a supported configuration.
+
+- **File uploads execute no code, by design.** Session files
+  (Export → Load session) are decoded without `pickle`: arrays carry an explicit
+  dtype, shape, and raw buffer, and anything not safely representable is refused
+  rather than deserialized. Session files written by Plottle 2.0.1 and earlier used a
+  pickle-based encoding; those entries are **refused on load**, not unpickled, and the
+  loader reports what it skipped.
+
+  `plottle.io.load_pickle` / `save_pickle` remain available in the **Python API**,
+  where the caller has chosen to trust the file. `.pkl` is deliberately *not* accepted
+  by the GUI's file uploader or batch folder import.
+
+- **Local filesystem access from the GUI.** The Data Upload page's batch-folder
+  import reads a user-supplied absolute path from the machine running the server. This
+  is intended for local single-user use; do not expose it on a shared host.
 
 ## Scope
 

binder/requirements.txt

@@ -4,5 +4,14 @@ pandas>=2.0.0
 matplotlib>=3.7.0
 seaborn>=0.12.0
 plotly>=5.14.0
-streamlit>=1.25.0
+streamlit>=1.49.0
 openpyxl>=3.1.0
+
+# Optional format support the GUI advertises (HDF5 / NetCDF) and the
+# NIST WebBook integration -- omitting these made the Binder badge launch an
+# environment that could not open half the documented formats.
+requests>=2.28.0
+h5py>=3.0.0
+xarray>=0.20.0
+netcdf4>=1.6.0
+tomli>=2.0.0; python_version < "3.11"