diff --git a/README.md b/README.md index a4186b9..baaaccf 100644 --- a/README.md +++ b/README.md @@ -29,10 +29,10 @@ uv add arcadedb-jvm # JVM, for heavy long-running loads > Old names: `arcadedb-jvm` was formerly `arcadedb-embedded`, and `arcadedb` was formerly `arcadedb-embedded-native` (archived on PyPI). A last `arcadedb-embedded` release will be a deprecation shim that installs `arcadedb`, the native build. -- Python 3.11 to 3.14. CI builds and tests all four versions on every platform below for `main`, every night and each release. A pull request runs a smaller selection. +- Python: `arcadedb` (native) is tested on 3.11 to 3.14 and also installs on 3.15 (`requires-python` is `>=3.11` with no upper bound; the CI runs a subset of the tests on 3.15). `arcadedb-jvm` needs 3.11 to 3.14 (`requires-python` is `>=3.11,<3.15`; its dependency JPype has no 3.15 wheel yet). Each release builds and tests both packages on Python 3.11 to 3.14 on all four platforms below. Between releases, `main` and the nightly run test `arcadedb` on 3.12 and 3.14 on every platform (with a short subset on 3.11, 3.13 and 3.15), and `arcadedb-jvm` on 3.11 to 3.14 on Linux x86-64 and on 3.11 and 3.14 on the other three. A pull request runs a smaller selection. - Linux x86_64 and ARM64, macOS on Apple Silicon, Windows x86_64, for both packages. The Windows wheel of `arcadedb` is the newest. - Import name: `arcadedb`. Check which package you have with `python -c "import arcadedb; print(arcadedb.__backend__)"`. It prints `native` or `jvm`. -- Optional extras: `arrow`, `pandas`, `vector` for `arcadedb-jvm`; `numpy`, `arrow`, `pandas`, `polars` for `arcadedb`. +- Optional extras: `arrow`, `pandas`, `polars`, `vector` for `arcadedb-jvm`; `numpy`, `arrow`, `pandas`, `polars` for `arcadedb`. - `arcadedb-jvm` bundles its own Java 25 runtime, so you install no Java. `arcadedb` needs no Java at all. ## Quick start @@ -88,7 +88,7 @@ Releases from this repository start with 26.11.1. The 26.10.x wheels were built ## Examples -[`examples/`](https://github.com/ArcadeData/arcadedb-python/tree/main/examples) holds 27 runnable scripts, from a first document store to vector search, time series, graph algorithms and server mode. CI runs them ([`test-python-examples.yml`](https://github.com/ArcadeData/arcadedb-python/blob/main/.github/workflows/test-python-examples.yml)). +[`examples/`](https://github.com/ArcadeData/arcadedb-python/tree/main/examples) holds 29 runnable scripts, from a first document store to vector search, time series, graph algorithms and server mode. CI runs them ([`test-python-examples.yml`](https://github.com/ArcadeData/arcadedb-python/blob/main/.github/workflows/test-python-examples.yml)). ## Build and test from source diff --git a/docs/modules/ROOT/pages/how-to/bulk-loads.adoc b/docs/modules/ROOT/pages/how-to/bulk-loads.adoc index 9febc7f..3ba0d54 100644 --- a/docs/modules/ROOT/pages/how-to/bulk-loads.adoc +++ b/docs/modules/ROOT/pages/how-to/bulk-loads.adoc @@ -1,5 +1,5 @@ = Bulk loads and commit size -:description: The default commit size of insert_many, graph_batch and create_vertices on the arcadedb (native) and arcadedb-jvm packages, what it means for a failed load, what to do after an interrupted graph batch, how to change it, how insert_many treats Python's cyclic garbage collector on arcadedb-jvm, which property shapes a graph batch loads in bulk on the JVM build, how to pass time-series columns to append_samples on arcadedb (native), and a known limitation: a graph batch does not reuse the space of deleted edges. +:description: The default commit size of insert_many, graph_batch and create_vertices on the arcadedb (native) and arcadedb-jvm packages, what it means for a failed load, what to do after an interrupted graph batch, how to change it, how insert_many treats Python's cyclic garbage collector on arcadedb-jvm, which property shapes a graph batch loads in bulk on the JVM build, how to pass time-series columns to append_samples (the address path is native only), and a known limitation: a graph batch does not reuse the space of deleted edges. `insert_many`, `graph_batch` and `GraphBatch.create_vertices` load data in transactions. The commit size is the number of rows in one transaction. It decides how much work the engine holds in memory before it commits, and how many rows stay in the database when a load fails. @@ -111,9 +111,9 @@ The table is one laptop study of the `arcadedb-jvm` package: per-row time of thi |=== -== Time-series samples with `append_samples` (`arcadedb`, native) +== Time-series samples with `append_samples` -`db.async_executor().append_samples(type_name, timestamps, *columns)` loads a columnar batch into a TIMESERIES type: the timestamps, then one column for each tag and field in the order of the type's declaration. Call `wait_completion()` before you read the samples back. How to declare the type is in the main ArcadeDB docs: see {arcadedb-docs-url}/concepts/timeseries[Time series]. +`db.async_executor().append_samples(type_name, timestamps, *columns)` loads a columnar batch into a TIMESERIES type: the timestamps, then one column for each tag and field in the order of the type's declaration. Call `wait_completion()` before you read the samples back. How to declare the type is in the main ArcadeDB docs: see {arcadedb-docs-url}/concepts/timeseries[Time series]. The call exists on both packages. The rest of this section describes the fast path of the `arcadedb` (native) package. On `arcadedb-jvm`, a NumPy array for the timestamps or for a numeric column crosses as one buffer copy, and any other sequence converts value by value. On the `arcadedb` (native) package, a NumPy column crosses to the engine by its address, with no JSON text, when it is one of these: diff --git a/docs/modules/ROOT/pages/how-to/concurrency.adoc b/docs/modules/ROOT/pages/how-to/concurrency.adoc index 6028206..7d1d35a 100644 --- a/docs/modules/ROOT/pages/how-to/concurrency.adoc +++ b/docs/modules/ROOT/pages/how-to/concurrency.adoc @@ -17,7 +17,7 @@ Every number on this page, except the ratios in <> to change it. -* *No Java objects, no JVM options.* `get_java_database`, `get_java_document` and the other `get_java_*` calls raise `NotOnNativeYet`. `start_jvm`, `jvm_kwargs` and the flags in <> do not exist. +* *No Java objects, no JVM options.* `get_java_database`, `get_java_document` and the other `get_java_*` calls raise `NotOnNativeYet`. `start_jvm` and the flags in <> do not exist, and `jvm_kwargs` is ignored with a logged warning. * *No console log on Windows by default.* The log goes to the file only. See xref:how-to/logs.adoc[Where the logs go] for the reason and for `ARCADEDB_LOG_CONSOLE`. * *Engine modules left out.* The gRPC and MongoDB wire protocols, Raft HA, Gremlin and the polyglot languages (JavaScript, Python as a query language) are not in the image. See <>. * *Platforms.* Intel Macs, Alpine Linux (musl) and Windows on ARM have no wheel. See <>. diff --git a/docs/modules/ROOT/pages/quickstart.adoc b/docs/modules/ROOT/pages/quickstart.adoc index e339a5e..d3ce1e1 100644 --- a/docs/modules/ROOT/pages/quickstart.adoc +++ b/docs/modules/ROOT/pages/quickstart.adoc @@ -156,6 +156,27 @@ HTTP port: 2480 The server runs only while your Python process runs. See xref:index.adoc#embedded-vs-http[Embedded bindings or the HTTP driver] for when to run the regular ArcadeDB server (the official Docker image `arcadedata/arcadedb`) instead. +The server takes a `config` dictionary. These keys matter first: + +* `host` defaults to `localhost`, so only clients on the same machine connect. A client in a Docker container or on another machine needs `config={"host": "0.0.0.0"}` (all IPv4 interfaces, or `"::"` for IPv6). `host` applies to the HTTP server and to every wire protocol that you enable. A per-protocol key such as `postgres_host` overrides it for that protocol. +* `http_port` (default 2480) and `mode` (default `development`) are read by the bindings. +* Any other key becomes the engine setting `arcadedb.`. For example, `postgres_port` sets `arcadedb.postgres.port`. + +Both packages carry the Postgres, Redis and Bolt wire protocol plugins, and none starts unless you name it in `server_plugins`: + +[source,python] +---- +config = { + "host": "0.0.0.0", + "server_plugins": "Postgres:com.arcadedb.postgres.PostgresProtocolPlugin", + "postgres_port": 5432, # sets arcadedb.postgres.port +} +---- + +The class names of the other two are `com.arcadedb.redis.RedisProtocolPlugin` (port key `redis_port`) and `com.arcadedb.bolt.BoltProtocolPlugin` (port key `bolt_port`). + +The native server has no Studio web UI. + // REMOVE WHEN: the old arcadedb-embedded* shims are retired and two releases have shipped with the two-package layout allow-old-name In the `{jvm-pkg-name}` wheels from 26.11.1 the server also serves the Studio web UI, and `get_studio_url()` returns its address. The earlier 26.10.x wheels do not contain Studio: see the note in xref:installation.adoc#what-is-included[What is included]. diff --git a/docs/modules/ROOT/pages/which-package.adoc b/docs/modules/ROOT/pages/which-package.adoc index 4688e52..8f6f5cf 100644 --- a/docs/modules/ROOT/pages/which-package.adoc +++ b/docs/modules/ROOT/pages/which-package.adoc @@ -93,7 +93,7 @@ The API is the same. The runtime is not. These are the differences you can plan | Available. | JVM options (`jvm_kwargs`, `start_jvm`, `-X` and `-D` flags) -| Do not exist. There is no JVM. +| `start_jvm` and the `-X` and `-D` flags do not exist, because there is no JVM. `jvm_kwargs` is accepted and ignored, with a logged warning. | Available. See xref:installation.adoc#jvm-options[JVM options]. | A few API calls diff --git a/examples/02_social_network_graph.py b/examples/02_social_network_graph.py index 5db8926..836da7a 100644 --- a/examples/02_social_network_graph.py +++ b/examples/02_social_network_graph.py @@ -8,20 +8,21 @@ 1. Creating vertex and edge types (schema definition) 2. Creating vertices (people) with properties 3. Creating edges (friendships) between vertices -4. Querying the graph using SQL MATCH and OpenCypher dialects +4. Querying the graph with openCypher (recommended) and the older SQL MATCH syntax 5. Finding friends, friends of friends, and mutual connections -6. Comparing SQL vs OpenCypher syntax and performance +6. Comparing openCypher with the older SQL MATCH syntax, in syntax and timing Key Concepts: - Vertices represent entities (Person) - Edges represent relationships (FRIEND_OF) - Properties store data on both vertices and edges - Graph traversal allows complex relationship queries -- SQL MATCH and OpenCypher provide different graph querying capabilities +- openCypher and SQL MATCH are two ways to ask the same graph questions ArcadeDB Query Languages: -- SQL MATCH: ArcadeDB's extended SQL syntax for graph traversal -- OpenCypher: openCypher query language for declarative graph patterns +- OpenCypher: the openCypher language for graph patterns and traversals (use it for new code) +- SQL MATCH: ArcadeDB's older SQL graph syntax, supported but not recommended for new code +- Plain SQL stays the language for document and table queries (inserts, aggregates, IS NULL filters) ⚔ Performance Findings (This Example): - SQL MATCH and OpenCypher performance varies by dataset and runtime configuration @@ -374,15 +375,15 @@ def create_sample_data(db): def demonstrate_graph_queries(db): - """Demonstrate various graph queries using SQL and OpenCypher""" + """Demonstrate various graph queries using OpenCypher and SQL""" print("\nšŸ” Demonstrating graph queries...") - # SQL-based queries - demonstrate_sql_queries(db) - - # OpenCypher-based queries + # OpenCypher-based queries (the recommended language for graph patterns) demonstrate_opencypher_queries(db) + # SQL-based queries (the older SQL MATCH syntax, plus plain SQL on the same data) + demonstrate_sql_queries(db) + def demonstrate_sql_queries(db): """Demonstrate graph queries using ArcadeDB's SQL MATCH syntax""" @@ -793,22 +794,27 @@ def compare_query_languages(db): # Show syntax differences print("\n šŸ” Key Syntax Differences:") print(" " + "-" * 40) - print(" SQL MATCH:") - print(" • Nodes: {type: Person, as: alias, where: (condition)}") - print(" • Edges: -EDGE_TYPE-> or -EDGE_TYPE->{as: alias}") - print(" • Conditions: where: (property = 'value')") - print(" • More verbose but explicit about types") - print() print(" OpenCypher:") print(" • Pattern matching with MATCH and relationships") print(" • Filters: WHERE clauses") print(" • Aggregations: count(), distinct") print(" • Declarative graph patterns") + print() + print(" SQL MATCH:") + print(" • Nodes: {type: Person, as: alias, where: (condition)}") + print(" • Edges: -EDGE_TYPE-> or -EDGE_TYPE->{as: alias}") + print(" • Conditions: where: (property = 'value')") + print(" • More verbose but explicit about types") print("\n šŸ’” When to use each:") print(" " + "-" * 30) - print(" • SQL MATCH: Mixing graph and relational queries") - print(" • OpenCypher: Declarative graph patterns") + print( + " • OpenCypher: graph patterns and traversals (recommended for new code)" + ) + print( + " • SQL MATCH: older graph syntax, supported but not recommended for new code" + ) + print(" • Plain SQL: document and table queries") print(" • Both support the same underlying graph operations") print(f" ā±ļø Comparison section: {time.time() - section_start:.3f}s") diff --git a/examples/README.md b/examples/README.md index eb99c85..46303d3 100644 --- a/examples/README.md +++ b/examples/README.md @@ -35,14 +35,14 @@ Perfect introduction to ArcadeDB basics: --- ### šŸ”— [02_social_network_graph.py](./02_social_network_graph.py) -**Vertex Types | Edge Types | Graph Traversal | SQL MATCH vs OpenCypher | NULL Handling** +**Vertex Types | Edge Types | Graph Traversal | OpenCypher vs SQL MATCH | NULL Handling** Complete social network modeling with graph database: - Creating vertex types (Person) and edge types (FRIEND_OF) with rich properties - NULL value handling for optional fields (email, phone, reputation) - Bidirectional relationships with metadata (since, closeness) - Graph traversal patterns (friends, friends-of-friends, mutual connections) -- Comparing SQL MATCH vs OpenCypher query languages +- Comparing OpenCypher (recommended for graph patterns) with the older SQL MATCH syntax - Variable-length path queries (`*1..3`) and graph aggregations - NULL filtering (IS NULL queries for missing contact info) - Relationship property queries (closeness filtering) diff --git a/src/arcadedb/core.py b/src/arcadedb/core.py index d24a780..c177781 100644 --- a/src/arcadedb/core.py +++ b/src/arcadedb/core.py @@ -1974,7 +1974,7 @@ def register_function( is the one exception to that: the statement is aborted and the caller gets a ``KeyboardInterrupt``, chained from the ``ArcadeDBError``, wherever the signal landed. A numeric argument costs about 30 bytes of Python memory per call that - JPype 1.7.1 does not give back (see the Python functions guide). + JPype 1.7.1 does not give back. The engine registers functions for the whole process, not for one database: every database in this process sees the function, and a function stays registered until diff --git a/src/arcadedb/graph_batch.py b/src/arcadedb/graph_batch.py index 938d179..0349584 100644 --- a/src/arcadedb/graph_batch.py +++ b/src/arcadedb/graph_batch.py @@ -31,8 +31,9 @@ class GraphBatch: buffer fills, manage their own transactions. From engine 26.10.1 they refuse to run inside a transaction you opened (an ArcadeDBError) and leave it untouched; on 26.9.1 and earlier they commit it (ArcadeData/arcadedb#9242). - Call them outside your own transactions either way; see - docs/api/graph_batch.md. + Call them outside your own transactions either way. Commit sizes and what + an interrupted batch leaves behind are in + docs/modules/ROOT/pages/how-to/bulk-loads.adoc. """ _VALID_WAL_FLUSH_MODES = { diff --git a/src/arcadedb/results.py b/src/arcadedb/results.py index ece0e9d..ebb344d 100644 --- a/src/arcadedb/results.py +++ b/src/arcadedb/results.py @@ -419,7 +419,7 @@ def to_list(self, convert_types: bool = True) -> List[Dict[str, Any]]: (RIDs, embedded documents, ``float[]``, ...) over as the engine's own object, so each value has the Python type it always had and no value crosses the JVM boundary on its own. That is within a few percent of - ``to_json_list()`` on a wide scan (see the performance guide). + ``to_json_list()`` on a wide scan. For large results, ``to_columns()``, ``to_dataframe()`` or ``to_arrow()`` move the data as columns and are faster still.