Pre-commit: add ruff and enforce all hooks in CI

.github/ISSUE_TEMPLATE/release.md

@@ -70,6 +70,5 @@ Every step says who does it: script (run the command, read the result), CI (a wo
   uv run --no-project python scripts/bump_engine.py --engine <VERSION> --dev-version <NEXT_DEV> --check
   ```
 
-
 - [ ] 9. CI, then owner: merge the bump PR when every check is green, and delete the branch and the worktree. Then remove the tests and skip-list lines that waited for the new engine: `git grep -n -i "native build links\|native image links"`.
 - [ ] 10. owner: check the GitHub Release (notes, pre-release flag) and, after a final release, that the milestone is closed and what was still open in it moved to the next milestone (the open one due next, or one the workflow created named after the next upstream engine version). If the run failed with a message about the next milestone, create it with a due date and re-run `GitHub Release` (Actions, `workflow_dispatch`, the tag). Close this issue with the two run ids and the two PyPI links.

.github/workflows/pre-commit.yml

@@ -0,0 +1,52 @@
+# Enforces the pre-commit hooks (.pre-commit-config.yaml) on every pull request and push to main,
+# so a contribution made without the local hooks is still checked. Contributors run the same hooks
+# locally with `uvx pre-commit install` (see CONTRIBUTING.md).
+name: Pre-commit
+
+on:
+  pull_request:
+  push:
+    branches: [main]
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+jobs:
+  pre-commit:
+    name: Run pre-commit on all files
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          persist-credentials: false
+
+      - name: Set up Python
+        uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
+        with:
+          python-version: "3.12"
+
+      # pre-commit and its hooks run through uvx (a throwaway environment), so nothing is installed
+      # into the runner's Python. uv is pinned in every install URL of this repo.
+      - name: Setup UV package manager
+        shell: bash
+        run: |
+          # Retry transient network failures (a connection reset failed a whole examples job on #8897); the
+          # script is downloaded first so a retried transfer is never piped half-written into sh.
+          installer="$(mktemp)"
+          curl -LsSf --retry 5 --retry-all-errors --retry-delay 5 -o "$installer" https://astral.sh/uv/0.12.0/install.sh
+          sh "$installer"
+          echo "$HOME/.local/bin" >> "$GITHUB_PATH"
+          export PATH="$HOME/.local/bin:$PATH"
+          uv --version
+
+      - name: Run pre-commit on all files
+        env:
+          # shellcheck and hadolint findings are fixed in the follow-up PR.
+          SKIP: shellcheck,hadolint-docker
+        run: uvx pre-commit run --all-files --show-diff-on-failure

.pre-commit-config.yaml

@@ -1,4 +1,19 @@
 repos:
+  # Basic file hygiene (validate and normalize), matching upstream ArcadeDB's pre-commit-hooks set.
+- repo: https://github.com/pre-commit/pre-commit-hooks
+  rev: v6.0.0
+  hooks:
+  - id: trailing-whitespace
+  - id: end-of-file-fixer
+  - id: check-merge-conflict
+  - id: check-added-large-files
+  - id: detect-private-key
+  - id: check-yaml
+    args: [--allow-multiple-documents]
+  - id: check-toml
+  - id: check-json
+  - id: check-xml
+  # Python formatting (unchanged): black + isort with the black profile.
 - repo: https://github.com/psf/black
   rev: 26.10.1
   hooks:
@@ -9,25 +24,55 @@ repos:
   hooks:
   - id: isort
     args: [--profile, black]
+  # Python linting: ruff with its default rules (E4, E7, E9, F) at line length 88 (pyproject.toml).
+  # Check only - fixes land in their own commit; black and isort remain the formatters.
+- repo: https://github.com/astral-sh/ruff-pre-commit
+  rev: v0.17.4
+  hooks:
+  - id: ruff-check
+  # Java, XML, Markdown and JSON formatting, matching upstream ArcadeDB (printWidth 160 for
+  # Java/XML is set in .prettierrc.json; Markdown prose is not hard-wrapped). types_or (OR) is
+  # required: types (AND) of [java, xml] would match no file.
+- repo: https://github.com/rbubley/mirrors-prettier
+  rev: v3.9.10
+  hooks:
+  - id: prettier
+    types_or: [java, xml, markdown, json]
+    additional_dependencies:
+    - prettier@3.9.10
+    - prettier-plugin-java@2.11.0
+    - prettier-plugin-xml
+    args: [--write, --list-different, --ignore-unknown]
+  # Shell formatting (unchanged).
+- repo: https://github.com/scop/pre-commit-shfmt
+  rev: v3.14.1-1
+  hooks:
+  - id: shfmt
+    args: [-i, '4', -ci, -sr, -w]
+  # YAML formatting (unchanged): workflows are linted by actionlint, not reformatted (as upstream).
 - repo: https://github.com/macisamuele/language-formatters-pre-commit-hooks
   rev: v2.16.0
   hooks:
   - id: pretty-format-yaml
     args:
     - --autofix
     - --indent=2
-        # Workflows are linted by actionlint, not reformatted (as upstream).
     exclude: ^\.github/
-- repo: https://github.com/scop/pre-commit-shfmt
-  rev: v3.14.1-1
+  # Shell linting (.sh files). Runs in Docker; actionlint already runs shellcheck over workflow run: blocks.
+- repo: https://github.com/koalaman/shellcheck-precommit
+  rev: v0.11.0
   hooks:
-  - id: shfmt
-    args: [-i, '4', -ci, -sr, -w]
-  # actionlint runs shellcheck over every `run:` block; per-file ignores live in .github/actionlint.yaml.
+  - id: shellcheck
+  # actionlint runs shellcheck over every run: block; per-file ignores live in .github/actionlint.yaml.
 - repo: https://github.com/rhysd/actionlint
   rev: v1.7.12
   hooks:
   - id: actionlint
+  # Dockerfile linting (scripts/Dockerfile.build). Runs in Docker.
+- repo: https://github.com/hadolint/hadolint
+  rev: v2.15.1
+  hooks:
+  - id: hadolint-docker
 - repo: local
   hooks:
   - id: no-old-package-name

.prettierignore

@@ -0,0 +1,8 @@
+# Files that prettier must not reformat.
+# The notices file and the two JDK notice texts are produced by
+# native/wheel/gen_notices.py; reachability-metadata.json is produced by
+# the GraalVM native-image build. Reformatting them would cause the
+# notices drift check (native wheel build) to fail.
+native/wheel/THIRD_PARTY_NOTICES.md
+native/wheel/notice-texts/
+native/src/main/resources/META-INF/native-image/

.prettierrc.json

@@ -0,0 +1,10 @@
+{
+  "printWidth": 80,
+  "proseWrap": "preserve",
+  "overrides": [
+    {
+      "files": ["**/*.java", "**/*.xml"],
+      "options": { "printWidth": 160 }
+    }
+  ]
+}

CONTRIBUTING.md

@@ -87,20 +87,23 @@ uv run --no-project --python 3.12 python native/wheel/build_wheel.py --lib-dir <
 
 ### Pre-commit
 
-This project uses [pre-commit](https://pre-commit.com/) (black, isort, shfmt, pretty-format-yaml, actionlint, and a local check that the old package name does not come back). Run it on the files you changed before you push, and again if it reformatted something:
+This project uses [pre-commit](https://pre-commit.com/): ruff (lint, default rules E4/E7/E9/F at line length 88), black and isort (Python formatting), prettier (Java, XML, Markdown, JSON), shfmt and shellcheck (shell), hadolint (Dockerfile), pretty-format-yaml and the basic hygiene hooks (trailing whitespace, end-of-file, merge-conflict markers, large files, private keys), and actionlint (GitHub Actions). The settings live in `pyproject.toml` (`[tool.ruff]`) and `.prettierrc.json`.
+
+Install the hooks once so they run on every commit:
 
 ```shell
-uvx pre-commit run --files <changed files>
-uvx pre-commit run --all-files
+uvx pre-commit install
 ```
 
-CI also runs Bandit, which must be clean:
+Run them on the files you changed before you push, and again if something was reformatted:
 
 ```shell
-uvx bandit==1.9.4 -c pyproject.toml -r src tests --severity-level low --confidence-level low
-uvx bandit==1.9.4 -c pyproject.toml -r examples --severity-level medium --confidence-level high
+uvx pre-commit run --files <changed files>
+uvx pre-commit run --all-files
 ```
 
+shellcheck and hadolint run as `docker_image` hooks (Docker pulls the images automatically, both locally and in CI). CI runs the same hooks on every pull request and push.
+
 ### Workflow
 
 1. An [issue](https://github.com/ArcadeData/arcadedb-python/issues) states the problem.

README.md

@@ -9,23 +9,23 @@
 
 Python bindings for [ArcadeDB](https://arcadedb.com), the multi-model database (graph, document, key/value, vector and time series, queried with SQL or openCypher). The engine runs inside your Python process, with no server and no network hop. This repository builds two packages, and both import as `arcadedb`: `arcadedb` (the native build, the default) for scripts, notebooks, tests, CLIs, serverless functions and agents, and `arcadedb-jvm` (the JVM build) for heavy, long-running loads. Status: beta.
 
-| Package | Engine | Pick it for |
-|---|---|---|
-| [`arcadedb`](https://pypi.org/project/arcadedb/) (native, the default) | The engine compiled by GraalVM Community Edition into a shared library, loaded with `ctypes`. No JVM, no JPype | Scripts, notebooks, tests, CLIs, serverless functions, agents |
-| [`arcadedb-jvm`](https://pypi.org/project/arcadedb-jvm/) (JVM) | The Java engine in your process through [JPype](https://jpype.readthedocs.io/), with a bundled Java 25 runtime | Heavy, long-running loads, where the JIT wins over time. Also the Java-object APIs and JVM options |
+| Package                                                                | Engine                                                                                                         | Pick it for                                                                                        |
+| ---------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- |
+| [`arcadedb`](https://pypi.org/project/arcadedb/) (native, the default) | The engine compiled by GraalVM Community Edition into a shared library, loaded with `ctypes`. No JVM, no JPype | Scripts, notebooks, tests, CLIs, serverless functions, agents                                      |
+| [`arcadedb-jvm`](https://pypi.org/project/arcadedb-jvm/) (JVM)         | The Java engine in your process through [JPype](https://jpype.readthedocs.io/), with a bundled Java 25 runtime | Heavy, long-running loads, where the JIT wins over time. Also the Java-object APIs and JVM options |
 
 Both packages `import arcadedb`, and they have the same public API. Install one, not both. Not sure which? Read [Which package?](https://github.com/ArcadeData/arcadedb-python/blob/main/docs/modules/ROOT/pages/which-package.adoc).
 
 ## Embedded bindings or drivers?
 
 This repository is the embedded bindings: the engine runs inside your Python process. The official [drivers](https://github.com/ArcadeData/arcadedb-drivers) are clients of a running ArcadeDB server. Do not mix them up.
 
-| | Embedded bindings (this repository) | Drivers |
-|---|---|---|
-| What it is | The ArcadeDB engine, inside your Python process | Python clients of an ArcadeDB server, HTTP (sync and async) or gRPC. They contain no engine |
-| Needs a server? | No. An in-process HTTP server and Studio are optional | Yes: the Docker image `arcadedata/arcadedb` or any ArcadeDB server. The gRPC driver also needs the server's gRPC plugin |
-| Package and import | `arcadedb` or `arcadedb-jvm`, `import arcadedb` | [`arcadedb-driver`](https://pypi.org/project/arcadedb-driver/), `import arcadedb_driver` (HTTP); [`arcadedb-driver-grpc`](https://pypi.org/project/arcadedb-driver-grpc/), `import arcadedb_driver_grpc` (gRPC) |
-| Use it when | One process, no server: tests, notebooks, scripts, an agent that owns its data | Several clients, an existing server, a service boundary, HA or replication, or clients that are not Python |
+|                    | Embedded bindings (this repository)                                            | Drivers                                                                                                                                                                                                         |
+| ------------------ | ------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| What it is         | The ArcadeDB engine, inside your Python process                                | Python clients of an ArcadeDB server, HTTP (sync and async) or gRPC. They contain no engine                                                                                                                     |
+| Needs a server?    | No. An in-process HTTP server and Studio are optional                          | Yes: the Docker image `arcadedata/arcadedb` or any ArcadeDB server. The gRPC driver also needs the server's gRPC plugin                                                                                         |
+| Package and import | `arcadedb` or `arcadedb-jvm`, `import arcadedb`                                | [`arcadedb-driver`](https://pypi.org/project/arcadedb-driver/), `import arcadedb_driver` (HTTP); [`arcadedb-driver-grpc`](https://pypi.org/project/arcadedb-driver-grpc/), `import arcadedb_driver_grpc` (gRPC) |
+| Use it when        | One process, no server: tests, notebooks, scripts, an agent that owns its data | Several clients, an existing server, a service boundary, HA or replication, or clients that are not Python                                                                                                      |
 
 You can also use both, for example the bindings for a job and the HTTP driver for a shared server. Driver docs: [HTTP](https://docs.arcadedb.com/arcadedb/how-to/connectivity/drivers/python-http), [gRPC](https://docs.arcadedb.com/arcadedb/how-to/connectivity/drivers/python-grpc). More in [Which package?](https://github.com/ArcadeData/arcadedb-python/blob/main/docs/modules/ROOT/pages/which-package.adoc#embedded-or-driver).
 
@@ -37,11 +37,13 @@ uv add arcadedb-jvm   # JVM, for heavy long-running loads
 ```
 
 <!-- REMOVE WHEN: the old arcadedb-embedded* shims are retired and two releases have shipped with the two-package layout allow-old-name -->
+
 > The first release of this layout is 26.11.1, and both packages have a final release on PyPI. A plain `uv add arcadedb` installs the final, and a later pre-release (`A.B.C.devN`) needs `uv add --prerelease allow arcadedb`. The same holds for `arcadedb-jvm`.
 >
 > A `.devN` version (for example `26.12.1.dev0`, when one is published) is a development pre-release. It embeds a snapshot of the upstream engine, the same snapshot in both packages, pinned by image digest and jar checksums in `scripts/dev_engine_pin.json`, and the wheel states it (`arcadedb.ENGINE_SNAPSHOT`, `arcadedb.ENGINE_IMAGE_DIGEST`). Install it with `uv add --prerelease allow arcadedb==26.12.1.dev0` (or `arcadedb-jvm==26.12.1.dev0`).
 
 <!-- REMOVE WHEN: the old arcadedb-embedded* shims are retired and two releases have shipped with the two-package layout allow-old-name -->
+
 > Old names: `arcadedb-jvm` was formerly `arcadedb-embedded`, and `arcadedb` was formerly `arcadedb-embedded-native` (archived on PyPI). The last `arcadedb-embedded` release, 26.11.1, is a deprecation shim: it installs `arcadedb`, the native build, and warns once at import. It does not install `arcadedb-jvm`.
 
 - Python: `arcadedb` (native) is tested on 3.11 to 3.14 and also installs on 3.15 (`requires-python` is `>=3.11` with no upper bound; the CI runs a subset of the tests on 3.15). `arcadedb-jvm` needs 3.11 to 3.14 (`requires-python` is `>=3.11,<3.15`; its dependency JPype has no 3.15 wheel yet). Each release builds and tests both packages on Python 3.11 to 3.14 on all four platforms below. Between releases, `main` and the nightly run test `arcadedb` on 3.12 and 3.14 on every platform (with a short subset on 3.11, 3.13 and 3.15), and `arcadedb-jvm` on 3.11 to 3.14 on Linux x86-64 and on 3.11 and 3.14 on the other three. A pull request runs a smaller selection.
@@ -80,16 +82,18 @@ This prints `Name: Alice` (the engine also logs to standard error). `create_data
 - An optional in-process HTTP server (`create_server()`), off until you call it, for other processes that need the same data.
 
 <!-- REMOVE WHEN: the old arcadedb-embedded* shims are retired and two releases have shipped with the two-package layout allow-old-name -->
+
 > New in 26.11.1: the `arcadedb-jvm` wheels carry the Studio web UI, which the server then serves. Studio used to bundle ApexCharts, which is no longer open source ([ArcadeData/arcadedb#10579](https://github.com/ArcadeData/arcadedb/issues/10579)), and [ArcadeData/arcadedb#10627](https://github.com/ArcadeData/arcadedb/pull/10627) replaced it with Apache ECharts. The earlier 26.10.x wheels on PyPI have no Studio.
 
 ## Packages on PyPI
 
-| Package | What it is | Status |
-|---|---|---|
-| [`arcadedb`](https://pypi.org/project/arcadedb/) | The native build: the engine compiled by GraalVM Community Edition, no JVM, no JPype. The default for embedded use | First release of this layout: 26.11.1 |
+| Package                                                  | What it is                                                                                                                      | Status                                      |
+| -------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- |
+| [`arcadedb`](https://pypi.org/project/arcadedb/)         | The native build: the engine compiled by GraalVM Community Edition, no JVM, no JPype. The default for embedded use              | First release of this layout: 26.11.1       |
 | [`arcadedb-jvm`](https://pypi.org/project/arcadedb-jvm/) | The JVM build: the Java engine on a bundled JVM, through JPype. The package this README described before the two-package layout | First release from this repository: 26.11.1 |
 
 <!-- REMOVE WHEN: the old arcadedb-embedded* shims are retired and two releases have shipped with the two-package layout allow-old-name -->
+
 Formerly `arcadedb-embedded` (now `arcadedb-jvm`, the same build) and formerly `arcadedb-embedded-native` (now `arcadedb`). They imported as `arcadedb_embedded` and `arcadedb_native`, and they are deprecated: `arcadedb-embedded-native` is archived, and the last `arcadedb-embedded` release, 26.11.1, is a shim that installs `arcadedb` and warns once at import.
 
 Releases from this repository start with 26.11.1. The 26.10.x wheels were built and published from [humemai/arcadedb-embedded-python](https://github.com/humemai/arcadedb-embedded-python). <!-- allow-old-name -->

SECURITY.md

@@ -33,11 +33,11 @@ To help us triage quickly, please include:
 
 Security fixes are delivered in a new release of the package. Both packages follow the version of the ArcadeDB engine they ship:
 
-| Version | Meaning |
-| ------- | ------- |
-| `A.B.C` | The release. The wheels ship the `A.B.C` engine. |
+| Version       | Meaning                                                                                     |
+| ------------- | ------------------------------------------------------------------------------------------- |
+| `A.B.C`       | The release. The wheels ship the `A.B.C` engine.                                            |
 | `A.B.C.postN` | A fix in the Python code of the released `A.B.C`. The wheels still ship the `A.B.C` engine. |
-| `A.B.C.devN` | A preview of the engine release `A.B.C`, built from upstream snapshot jars. |
+| `A.B.C.devN`  | A preview of the engine release `A.B.C`, built from upstream snapshot jars.                 |
 
 The numbering rule is on the docs page [Release](docs/modules/ROOT/pages/development/release.adoc).