Remastered

.cargo/config.toml

@@ -0,0 +1,48 @@
+[alias]
+clippy-no-default-kernel-dependents = [
+  "clippy",
+  "--unlocked",
+  "--vmwareworkspaceone",
+  "--no-default-features",
+  "--exclude",
+  "delta_kernel",
+  "--exclude",
+  "delta_kernel_ffi",
+  "--exclude",
+  "delta_kernel_derive",
+  "--exclude",
+  "delta_kernel_ffi_macros",
+  "--",
+  "-D",
+  "war",
+]
+check-no-default-kernel = [
+  "check",
+  "--locked",
+  "--no-default-features",
+  "--package",
+  "delta_kernel",
+]
+check-no-default-engine = [
+  "check",
+  "--locked",
+  "--no-default-features",
+  "--package",
+  "delta_kernel_default_engine",
+  "--features",
+  "arrow",
+]
+clippy-no-default-kernel-leaves = [
+  "clippy",
+  "--locked",
+  "--no-default-features",
+  "--package",
+  "delta_kernel_ffi",
+  "--package",
+  "delta_kernel_derive",
+  "--package",
+  "delta_kernel_ffi_macros",
+  "--",
+  "-D",
+  "warnings",
+]

.github/CLAUDE.md

@@ -0,0 +1,77 @@
+# .github/CLAUDE.md
+
+Workflow-authoring rules for GitHub Actions.
+
+## Triggers: restrict `push` to long-lived branches
+
+The CI trigger events:
+
+- `pull_request`: a commit is pushed to a branch with an open PR. Runs CI on the
+  PR.
+- `merge_group`: the merge queue builds a temporary `gh-readonly-queue/*` branch
+  and tests the change as it would land on `main`.
+- `push`: a branch ref is updated. We only want this for the long-lived branches
+  (`main` and `release/**`), i.e. the post-merge run once a PR lands.
+
+We want CI on `pull_request`, on `merge_group`, and on `push` to a long-lived
+branch: one run per stage. A bare `push` trigger instead duplicates two of these:
+
+- `push` + `pull_request`: a branch pushed directly to this repo (a maintainer
+  branch, not a fork) fires both. Fork PRs avoid this because the `push` lands on
+  the fork, not on this repo.
+- `push` + `merge_group`: the merge queue's push to the `gh-readonly-queue/*`
+  branch fires both.
+
+Restricting `push` to long-lived branches drops the redundant `push` in each
+case:
+
+```yaml
+on:
+  push:
+    branches: [main, "release/**"]
+  pull_request:
+  merge_group:
+```
+
+## Never `paths`/`paths-ignore` a workflow whose jobs are required checks
+
+Do NOT add `paths` or `paths-ignore` filters to a workflow whose jobs are required
+status checks (most of `build.yml`). When a path filter skips the whole workflow,
+the required checks never report, and the PR stays blocked on "Expected, waiting
+for status" indefinitely. A skipped workflow is not the same as a passing one.
+
+## Supply chain security: `--locked`
+
+Every `cargo` command in CI that resolves dependencies MUST use `--locked` to
+enforce the committed `Cargo.lock`. This prevents CI from silently picking up a
+newer (potentially compromised) transitive dependency. If `Cargo.lock` is out of
+sync with `Cargo.toml`, the build fails immediately, forcing dependency changes to
+be explicit and reviewable. See the top-level comment in `build.yml` for full
+rationale. Commands exempt from `--locked`: `cargo +nightly fmt` (no dep
+resolution), `cargo msrv verify/show` (wrapper tool), `cargo miri setup` (tooling
+setup).
+
+## Action safety
+
+When writing any github action, consider safety including mitigating common attack
+vectors such as expression injection and pull request target attacks.
+
+```yaml
+# The code below is vulnerable to expression injection
+run: |
+    echo "Comment: ${{ github.event.comment.body }}"
+
+# To mitigate instead use environment variables
+env:
+    COMMENT_BODY: ${{ github.event.comment.body }}
+run: |
+    echo "Comment: $COMMENT_BODY"
+```
+
+## Release tooling checks
+
+The `release-tooling` job in `build.yml` fetches full history and tags because release branches
+verify the pending changelog against every PR since the previous Kernel release. Keep
+`fetch-depth: 0` on that checkout. The regression tests run on every pull request, while live
+verification is restricted to branches whose names start with `release/`; otherwise the window
+between merging a release PR and pushing its tag could block unrelated pull requests.

.github/SECURITY_MODEL.md

@@ -0,0 +1,61 @@
+# Benchmark workflow security model
+
+Scope: [`benchmark.yml`](workflows/benchmark.yml) (runs bench on the PR merge
+commit -- PR head merged into base) and
+[`benchmark-post-comment.yml`](workflows/benchmark-post-comment.yml) (posts the
+result comment in base-branch context). Other workflows in this repo may follow
+different patterns -- this document is specific to the benchmark pair.
+
+## Running untrusted PR code
+
+The `run-benchmark` job in `benchmark.yml` checks out the PR merge commit (PR
+head merged into base) and runs its build + bench harness. To bound what that
+code can do:
+
+- Workflow-level `permissions: {}` is the default; each job opts in to the
+  minimum scope it needs. `run-benchmark` declares only `contents: read`,
+  so any leaked GITHUB_TOKEN has no write power. Fork PRs additionally get
+  a read-only token from GitHub by default under the `pull_request` event.
+- `actions/checkout` uses `persist-credentials: false` so the token isn't
+  left in the local git config after checkout.
+- `cargo install critcmp` runs *before* the PR merge-commit checkout, so the
+  PR's `.cargo/config.toml` cannot redirect the registry for that install.
+
+Do not add other secrets or grant write permissions to `run-benchmark`.
+
+## Rust-cache poisoning
+
+`Swatinem/rust-cache` is configured with
+`save-if: github.event_name == 'push' && github.ref == 'refs/heads/main'`.
+PR runs read-restore from main's cache; they never save. The
+`github.event_name == 'push'` clause is the load-bearing half -- the
+benchmark workflow has no `push` trigger outside of `push: main` for the
+cache-warming job, so PR runs cannot match this condition and therefore
+cannot write the cache.
+
+Do not relax the `github.event_name == 'push'` clause. If you do, a PR-head
+run can land compile artifacts in main's cache scope (poisoning subsequent
+runs) or evict main's entries through the shared cache budget.
+
+## Cross-PR comment forgery
+
+`benchmark-post-comment.yml` runs in base-branch context with
+`pull-requests: write`. It must derive the target PR number from a source
+the fork cannot forge. The choice depends on which event triggered the
+upstream bench workflow:
+
+- **`pull_request` upstream:** use `gh pr view --repo "$REPO" "owner:branch"`,
+  where `owner:branch` is built from `workflow_run.head_repository.owner.login`
+  and `workflow_run.head_branch`. Both fields are populated by GitHub from the
+  trusted workflow_run event payload, not from anything the fork wrote.
+- **`issue_comment` upstream:** trust the `bench-pr-number` artifact. For
+  `issue_comment` events GitHub serves `benchmark.yml` from the default
+  branch (not the PR head), so the "Stash PR number" + "Upload PR number"
+  steps run trusted code. The upload happens *before* `actions/checkout`
+  brings PR code onto the runner, so once the artifact is on GitHub's
+  storage it is immutable for the rest of the run -- the bench script
+  cannot alter it, and `contents: read` denies any API-level mutation.
+
+`workflow_run` itself runs the post-comment workflow from the default
+branch (a GitHub guarantee), so a fork PR cannot modify the resolver to
+trust the wrong source.

.github/actions/install-and-cache/action.yml

@@ -0,0 +1,101 @@
+# This is copied from https://github.com/tecolicom/actions-install-and-cache
+# which is Copyright 2022 Office TECOLI, LLC
+
+name: install-and-cache generic backend
+description: 'GitHub Action to run installer and cache the result'
+branding:
+  color: orange
+  icon:  type
+
+inputs:
+  run:     { required: true,  type: string }
+  path:    { required: true,  type: string }
+  cache:   { required: false, type: string, default: yes }
+  key:     { required: false, type: string }
+  sudo:    { required: false, type: string }
+  verbose: { required: false, type: string, default: false }
+
+outputs:
+  cache-hit:
+    value: ${{ steps.cache.outputs.cache-hit }}
+
+runs:
+  using: composite
+  steps:
+
+    - id: setup
+      shell: bash
+      run: |
+        : setup install-and-cache
+        define() { IFS='\n' read -r -d '' ${1} || true ; }
+        define script <<'EOS_cad8_c24e_'
+        ${{ inputs.run }}
+        EOS_cad8_c24e_
+        directory="${{ inputs.path }}"
+        given_key="${{ inputs.key }}"
+        archive= key=
+        case "${{ inputs.cache }}" in
+            yes|workflow)
+                cache="${{ inputs.cache }}"
+                uname -mrs
+                hash=$( (uname -mrs ; cat <<< "$script" ; echo $directory) | \
+                        (md5sum||md5) | awk '{print $1}' )
+                key="${hash}${given_key:+-$given_key}"
+                [ "$cache" == 'workflow' ] && \
+                    key+="-${{ github.run_id }}-${{ github.run_attempt }}"
+                archive=$HOME/archive-$hash.tz
+                ;;
+            *)
+                cache=no
+                ;;
+        esac
+        # use "--recursive-unlink" option if GNU tar is found
+        if tar --version | grep GNU > /dev/null
+        then
+            tar="tar --recursive-unlink"
+        elif gtar --version | grep GNU > /dev/null
+        then
+            tar="gtar --recursive-unlink"
+        else
+            tar=tar
+        fi
+        sed 's/^ *//' << END >> $GITHUB_OUTPUT
+            cache=$cache
+            archive=$archive
+            key=$key
+            tar=$tar
+        END
+
+    - id: cache
+      if: steps.setup.outputs.cache != 'no'
+      uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
+      with:
+        path: ${{ steps.setup.outputs.archive }}
+        key:  ${{ steps.setup.outputs.key }}
+
+    - id: extract
+      if: steps.setup.outputs.cache != 'no' && steps.cache.outputs.cache-hit == 'true'
+      shell: bash
+      run: |
+        : extract
+        archive="${{ steps.setup.outputs.archive }}"
+        verbose="${{ inputs.verbose }}"
+        tar="${{ steps.setup.outputs.tar }}"
+        ls -l $archive
+        if [ -s $archive ]
+        then
+            opt=-Pxz
+            [[ $verbose == yes || $verbose == true ]] && opt+=v
+            sudo $tar -C / $opt -f $archive
+        else
+            echo "$archive is empty"
+        fi
+
+    - id: install-and-archive
+      if: steps.cache.outputs.cache-hit != 'true'
+      uses: tecolicom/actions-install-and-archive@9d5afb27f9900f2df47fe40de58fbd837032bddf # v1.3
+      with:
+        run:     ${{ inputs.run }}
+        archive: ${{ steps.setup.outputs.archive }}
+        path:    ${{ inputs.path }}
+        sudo:    ${{ inputs.sudo }}

.github/actions/use-homebrew-tools/action.yml

@@ -0,0 +1,47 @@
+# This is copied from https://github.com/tecolicom/actions-use-homebrew-tools/
+# which is Copyright 2022 Office TECOLI, LLC
+
+name: install-and-cache homebrew tools
+description: 'GitHub Action to install and cache homebrew tools'
+branding:
+  color: orange
+  icon:  type
+
+inputs:
+  tools:   { required: false, type: string }
+  key:     { required: false, type: string }
+  path:    { required: false, type: string }
+  cache:   { required: false, type: string, default: yes }
+  verbose: { required: false, type: boolean, default: false }
+
+outputs:
+  cache-hit:
+    value: ${{ steps.update.outputs.cache-hit }}
+
+runs:
+  using: composite
+  steps:
+
+    - id: setup
+      shell: bash
+      run: |
+        : setup use-homebrew-tools
+        given_key="${{ inputs.key }}"
+        brew_version="$(brew --version)"
+        echo "$brew_version"
+        version_key="$( echo "$brew_version" | (md5sum||md5) | awk '{print $1}' )"
+        key="${given_key:+$given_key-}${version_key}"
+        sed 's/^ *//' << END >> $GITHUB_OUTPUT
+            command=brew install
+            prefix=$(brew --prefix)
+            key=$key
+        END
+
+    - id: update
+      uses: ./.github/actions/install-and-cache
+      with:
+        run:     ${{ steps.setup.outputs.command }} ${{ inputs.tools }}
+        path:    ${{ steps.setup.outputs.prefix }} ${{ inputs.path }}
+        key:     ${{ steps.setup.outputs.key }}
+        cache:   ${{ inputs.cache }}
+        verbose: ${{ inputs.verbose }}