From 20b7f030b8dc8a45dbff70130715732c7118754e Mon Sep 17 00:00:00 2001 From: robfrank Date: Thu, 12 Nov 2026 10:08:18 +0100 Subject: [PATCH 1/3] docs: 26.12.1 remote client retry on proxy 503, PROFILE/Bolt zero-row changes, HA quarantine recheck (issues #11387, #11386, #11518, #11517) --- src/main/asciidoc/how-to/operations/ha.adoc | 5 ++-- .../asciidoc/how-to/operations/upgrade.adoc | 27 +++++++++++++++++++ .../reference/java-api/java-api-remote.adoc | 9 +++++++ 3 files changed, 38 insertions(+), 3 deletions(-) diff --git a/src/main/asciidoc/how-to/operations/ha.adoc b/src/main/asciidoc/how-to/operations/ha.adoc index 7188cbb2..813051d7 100644 --- a/src/main/asciidoc/how-to/operations/ha.adoc +++ b/src/main/asciidoc/how-to/operations/ha.adoc @@ -814,7 +814,7 @@ stands the node: `details.divergedDatabases` and `details.divergenceCauses` name the database and the cause; * refuses reads that need linearizability on that database instead of serving them stale, and does not checkpoint its Raft log; -* keeps the quarantine across restarts: it is recorded in `.raft/applied-index`. +* keeps the quarantine across restarts: it is recorded in `.raft/applied-index`. _(Since v26.12.1)_ The writes of that file that a restart depends on (while a quarantine or a read floor stands, the write that lifts it, a snapshot install, a resync) are flushed to disk with their directory, so a power loss cannot leave the file empty and lose the quarantine. Since v26.11.1 an index compaction on the leader is quarantined too when the cluster cannot say whether it accepted the compaction: the publication timed out and, after waiting up to the quorum timeout, the outcome is still unknown, or the entry @@ -823,13 +823,12 @@ so a leader that crashes in between restarts on the same files as every other se alone and the old leader's index could silently miss entries written by the next leader. [[ha-power-loss]] -*After a power loss.* _(Since v26.11.1)_ In production mode, the crash recovery of a database can abort on a gap in its write-ahead log. Before, the database opened anyway and the node served wrong property values and missing index data. Now a database whose crash recovery did not complete is quarantined under the cause `LOCAL_RECOVERY_INCOMPLETE`, and a database that opens with an unavailable unique index under `UNAVAILABLE_UNIQUE_INDEX`. The node refuses readiness, hands leadership off and resyncs the database from the leader, as for any other quarantine. A node that is the only voter has no leader to resync from and only logs a SEVERE message. +*After a power loss.* _(Since v26.11.1)_ In production mode, the crash recovery of a database can abort on a gap in its write-ahead log. Before, the database opened anyway and the node served wrong property values and missing index data. Now a database whose crash recovery did not complete is quarantined under the cause `LOCAL_RECOVERY_INCOMPLETE`, and a database that opens with an unavailable unique index under `UNAVAILABLE_UNIQUE_INDEX`. The node refuses readiness, hands leadership off and resyncs the database from the leader, as for any other quarantine. A node that is the only voter has no leader to resync from and only logs a SEVERE message. _(Since v26.12.1)_ The unavailable unique index and the emptied dictionary are checked not only when the node starts, but also after a replicated schema change reloads the schema, whenever the database is opened again while the node runs (after an idle close, or a lazy open), and on a copy installed from a snapshot, which is rolled back instead of published if it opens damaged. Known limits, tracked upstream: * a recovery that throws on a torn page (a `SerializationException` while replaying the WAL) stops the node from starting on every restart instead of quarantining the database (#11280); * a quarantined database may still answer reads until its resync completes (#11281, not yet confirmed); * a first or header page lost after fsync, and a torn page with no WAL history, are not covered yet (#11258); the second one needs a per-page checksum (#11157); -* an index that becomes unavailable after a later schema reload is not rechecked. The way out is a resync, which reinstalls the database from a peer and lifts the quarantine. The node starts one itself; to force it, run `POST /api/v1/cluster/resync/{database}` on that node. A leader cannot resync from itself: transfer diff --git a/src/main/asciidoc/how-to/operations/upgrade.adoc b/src/main/asciidoc/how-to/operations/upgrade.adoc index ee98ff51..272d57c2 100644 --- a/src/main/asciidoc/how-to/operations/upgrade.adoc +++ b/src/main/asciidoc/how-to/operations/upgrade.adoc @@ -109,6 +109,33 @@ A failure on a later batch of `iterateType`, `iterateBucket` or `queryStream` ra *Action:* catch `SecurityException` for an authentication or permission refusal, and `TimeoutException` for an expired deadline. See <>. +[discrete] +====== 26.12.1 - The Java remote client does not resend a write after a proxy's `503` page + +When an ingress, a load balancer or a sidecar in front of the server answers an error with a page that is not JSON (a `502`, `503` or `504` page), the Java `RemoteDatabase` client now maps it on its HTTP status, and the exception names the status and quotes a short excerpt of the page (issue #11387). Before, it raised the JSON parser's `JSONException`, the status was lost, a `503` was never retried, and the log said it was retrying. A `503` page from such a hop is a `RemoteIntermediaryRefusalException`, a `NeedRetryException`. + +*Impact:* the hop may have answered `503` after it forwarded the request, so the page does not prove that the server did not run it. A read, a `begin` and a replay-protected command are retried. A command that may write and cannot be replayed safely, a streamed write, a time-series write and a `commit` are not sent again: the exception reaches the caller, and a `commit` fails with a `TransactionException` saying it may have been applied, without a rollback. + +*Action:* when a write or a `commit` fails this way, read back the data before retrying it. See <>. + +[discrete] +====== 26.12.1 - Cypher `PROFILE` of a write with no `RETURN` returns no rows + +`PROFILE` of a write that has no `RETURN` (`PROFILE CREATE (n:Person)`) returned the rows of its last write clause, while the same query without `PROFILE` returned none (issue #11386). It now returns no rows over every protocol. The query still runs in full and writes its changes, and the profile is unchanged. + +*Impact:* a client that read those rows gets none. + +*Action:* add a `RETURN` to the query to get rows back. + +[discrete] +====== 26.12.1 - Bolt announces the column names of a result with no rows + +A Bolt `RUN` whose result has no rows announced `fields = []`. It now announces the names of its `RETURN` items (issue #11386), as a result with rows does. A zero-row `RETURN *` and a `CALL ... YIELD` with no `RETURN` still announce no names (#11543). + +*Impact:* `result.keys()` of an empty result now lists the columns. + +*Action:* none, unless a client relied on an empty key list to detect an empty result; check the records instead. + [discrete] ====== 26.11.1 - MCP writes need `allowSchemaChange` to create a type diff --git a/src/main/asciidoc/reference/java-api/java-api-remote.adoc b/src/main/asciidoc/reference/java-api/java-api-remote.adoc index 8f0192f9..99b14371 100644 --- a/src/main/asciidoc/reference/java-api/java-api-remote.adoc +++ b/src/main/asciidoc/reference/java-api/java-api-remote.adoc @@ -48,6 +48,15 @@ the client cannot rebuild (a class of a newer server, or of a plugin). In 26.10. caller as a `RemoteException` that was not an `ArcadeDBException`, with a message of the form `Error on executing remote operation ... (cause: detail:)`. +_(Since v26.12.1.)_ An error answer whose body is not JSON comes from a hop in front of the server (an ingress, a load +balancer, a sidecar), not from the server. The client maps it on its HTTP status, and the message names the status and +quotes up to 256 characters of the page. A `503` page is a `RemoteIntermediaryRefusalException`, a `NeedRetryException`: +a read, a `begin` and a replay-protected command are retried on it. Because the hop may have forwarded the request +before it answered, a command that may write and cannot be replayed safely, a streamed write, a time-series write and a +`commit` are not sent again. A `commit` refused this way raises a `TransactionException` saying the commit may have been +applied, and the transaction is not rolled back: read the data back before retrying. Before v26.12.1 such a page raised +a bare `JSONException` with the status lost. + _(Since v26.11.1.)_ `getType()` answers the record's type on a mutable remote record too (`newDocument()`, `newVertex()`, `modify()`), as it does on a record read from the server, instead of throwing `UnsupportedOperationException`. `toMap()` and `toJSON()` of a remote edge include `@in` and `@out`, as an embedded edge From 40dd1adf0a00227df57f4e6f9bb3943806163368 Mon Sep 17 00:00:00 2001 From: robfrank Date: Thu, 12 Nov 2026 17:55:36 +0100 Subject: [PATCH 2/3] docs: 26.12.1 remote commit keeps created RIDs unless nothing was applied; Bolt zero-row RETURN * and CALL YIELD columns (issues #11541, #11543) --- src/main/asciidoc/how-to/operations/upgrade.adoc | 11 ++++++++++- .../asciidoc/reference/java-api/java-api-remote.adoc | 9 +++++++++ 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/src/main/asciidoc/how-to/operations/upgrade.adoc b/src/main/asciidoc/how-to/operations/upgrade.adoc index 272d57c2..daa7390b 100644 --- a/src/main/asciidoc/how-to/operations/upgrade.adoc +++ b/src/main/asciidoc/how-to/operations/upgrade.adoc @@ -118,6 +118,15 @@ When an ingress, a load balancer or a sidecar in front of the server answers an *Action:* when a write or a `commit` fails this way, read back the data before retrying it. See <>. +[discrete] +====== 26.12.1 - Java remote client: a failed commit keeps the RIDs of the records it created + +A failed `commit()` on `RemoteDatabase` or `RemoteGrpcDatabase` reset the RIDs of every record created in the transaction, whatever the failure (issue #11541). After a commit that had in fact landed, saving the same objects again inserted them a second time. The RIDs are now reset only when the failure proves nothing was applied: a retryable refusal from the server, a duplicated key, a connection that was never established, or the gRPC server answering that it did not commit. + +*Impact:* after a replication timeout, a proxy's `503` page, a connection lost after the request was sent or a `409` "committed remotely", the records keep their RIDs. So do a few failures that did roll back but are not in the list above (a validation error at commit, a session the server no longer holds, an untyped `500`): saving those objects again now fails with `RecordNotFoundException` instead of inserting a duplicate. + +*Action:* after a failed commit, check whether the write landed before saving the same objects again. See <>. + [discrete] ====== 26.12.1 - Cypher `PROFILE` of a write with no `RETURN` returns no rows @@ -130,7 +139,7 @@ When an ingress, a load balancer or a sidecar in front of the server answers an [discrete] ====== 26.12.1 - Bolt announces the column names of a result with no rows -A Bolt `RUN` whose result has no rows announced `fields = []`. It now announces the names of its `RETURN` items (issue #11386), as a result with rows does. A zero-row `RETURN *` and a `CALL ... YIELD` with no `RETURN` still announce no names (#11543). +A Bolt `RUN` whose result has no rows announced `fields = []`. It now announces the names of its `RETURN` items (issue #11386), as a result with rows does. A zero-row `RETURN *` announces the variables in scope at the `RETURN`, in the order the rows of the same query carry them, and a standalone `CALL ... YIELD` announces its yielded fields, all the procedure's outputs for `YIELD *` or no `YIELD` (issue #11543). A `RETURN *` inside a `CALL { }` subquery and `YIELD *` on a procedure whose outputs are unknown still announce no names (#11571). *Impact:* `result.keys()` of an empty result now lists the columns. diff --git a/src/main/asciidoc/reference/java-api/java-api-remote.adoc b/src/main/asciidoc/reference/java-api/java-api-remote.adoc index 99b14371..58056dd5 100644 --- a/src/main/asciidoc/reference/java-api/java-api-remote.adoc +++ b/src/main/asciidoc/reference/java-api/java-api-remote.adoc @@ -223,6 +223,15 @@ The same happens when the server refuses the transaction before running it: the A commit whose connection drops after the request was sent is different: the server may already have applied it. Over gRPC such a commit (status `UNAVAILABLE` with no error from the server) raises a `TransactionException` and is *not* retried, so the work is never applied twice. Read the data back to find out whether the commit landed. _(Since v26.10.1: before, it was reported as a retryable conflict.)_ +_(Since v26.12.1.)_ When a commit fails, the records created in the transaction keep the RIDs the server assigned to +them, unless the failure proves nothing was applied: a retryable refusal from the server, a duplicated key, a +connection that was never established, or the gRPC server answering that it did not commit. Only then are the RIDs +reset, so saving the same objects again inserts them. After any other failure (a replication timeout, a `503` page +from a proxy, a connection lost after the request was sent, a `409` saying the transaction was committed remotely) +the records keep their RIDs, and saving them again updates those RIDs. If the commit did not land, that save fails +with a `RecordNotFoundException` instead of silently inserting a duplicate. This applies to `RemoteDatabase` and +`RemoteGrpcDatabase` (issue #11541). + _(Since v26.10.2.)_ The gRPC client draws the same line for the other writes whose answer can be lost. A bare `UNAVAILABLE` (no error class from the server, and not a failure to connect) on an admin write of `RemoteGrpcServer` (creating or dropping a database, a user or a token, and the other server writes) or on a streaming write of From 8b06db8e30c83c714b95c5acb22d906bbfdadf75 Mon Sep 17 00:00:00 2001 From: robfrank Date: Fri, 13 Nov 2026 10:03:21 +0100 Subject: [PATCH 3/3] docs: a zero-row RETURN * through a CALL subquery returning * announces its columns (#11571) Co-Authored-By: Claude Sonnet 5.5 --- src/main/asciidoc/how-to/operations/upgrade.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/main/asciidoc/how-to/operations/upgrade.adoc b/src/main/asciidoc/how-to/operations/upgrade.adoc index daa7390b..752a6235 100644 --- a/src/main/asciidoc/how-to/operations/upgrade.adoc +++ b/src/main/asciidoc/how-to/operations/upgrade.adoc @@ -139,7 +139,7 @@ A failed `commit()` on `RemoteDatabase` or `RemoteGrpcDatabase` reset the RIDs o [discrete] ====== 26.12.1 - Bolt announces the column names of a result with no rows -A Bolt `RUN` whose result has no rows announced `fields = []`. It now announces the names of its `RETURN` items (issue #11386), as a result with rows does. A zero-row `RETURN *` announces the variables in scope at the `RETURN`, in the order the rows of the same query carry them, and a standalone `CALL ... YIELD` announces its yielded fields, all the procedure's outputs for `YIELD *` or no `YIELD` (issue #11543). A `RETURN *` inside a `CALL { }` subquery and `YIELD *` on a procedure whose outputs are unknown still announce no names (#11571). +A Bolt `RUN` whose result has no rows announced `fields = []`. It now announces the names of its `RETURN` items (issue #11386), as a result with rows does. A zero-row `RETURN *` announces the variables in scope at the `RETURN`, in the order the rows of the same query carry them, and a standalone `CALL ... YIELD` announces its yielded fields, all the procedure's outputs for `YIELD *` or no `YIELD` (issue #11543). A `RETURN *` reached through a `CALL { ... RETURN * }` subquery announces the subquery's columns too (#11571). A `CALL ... YIELD *` (or no `YIELD`) of a name the procedure registry does not know, such as a user function, still announces no names: its columns depend on the value it returns. *Impact:* `result.keys()` of an empty result now lists the columns.