Update IO support matrix after Beam 2.76.0 release

CHANGES.md

@@ -76,6 +76,9 @@
 * (Java/Python) `Watch` can bound its deduplication state by event time, retiring an output key once the greatest emitted timestamp has moved more than the allowed lateness past it. Java adds `Watch.growthOf(...).withTimestampCursor()`. Python adds `allowed_lateness` for the existing `timestamp_cursor` option ([#18459](https://github.com/apache/beam/issues/18459)).
 * (Java) Spark Structured Streaming runner: stateful ParDo with state, timers, `@RequiresTimeSortedInput` and tagged outputs is now supported in batch mode ([#39779](https://github.com/apache/beam/issues/39779)).
 * (Python) Added support for Vertex AI Model Monitoring V2 in RunInference ([#39738](https://github.com/apache/beam/issues/39738)).
+* [Flink Runner] Added opt-in static round-robin split assignment for small bounded sources via the new `sourceStaticSplitThresholdMb` pipeline option. The default of 0 keeps the existing lazy pull-based assignment ([#39873](https://github.com/apache/beam/issues/39873)).
+* Added automatic caching of bounded, single-pane side-input views for classic Java Flink DataStream execution ([#39866](https://github.com/apache/beam/issues/39866)).
+* (Python) Added `Sample.Any`, the Python equivalent of Java's `Sample.any`, which returns up to n arbitrary elements from a PCollection ([#18552](https://github.com/apache/beam/issues/18552)).
 
 ## Breaking Changes
 
@@ -89,7 +92,6 @@
 
 ## Bugfixes
 
-* (Java) Restored binary compatibility for `CoderTranslatorRegistrar` implementations compiled against Beam 2.76 and earlier ([#38714](https://github.com/apache/beam/issues/38714)).
 * (Java) Fixed the Spark runner firing processing-time timers in reverse timestamp order ([#39824](https://github.com/apache/beam/issues/39824)).
 * (Python) Fixed incorrect profiler options handling on portable runners ([#39613](https://github.com/apache/beam/issues/39613)).
 * (Java) KafkaIO dynamic reads no longer require the obsolete `beam_fn_api` experiment ([#29998](https://github.com/apache/beam/issues/29998)).
@@ -123,8 +125,6 @@
 
 ## New Features / Improvements
 
-* [Flink Runner] Added opt-in static round-robin split assignment for small bounded sources via the new `sourceStaticSplitThresholdMb` pipeline option. The default of 0 keeps the existing lazy pull-based assignment ([#39873](https://github.com/apache/beam/issues/39873)).
-* Added automatic caching of bounded, single-pane side-input views for classic Java Flink DataStream execution ([#39866](https://github.com/apache/beam/issues/39866)).
 * Added `GroupIntoBatches` transform and the standard
   `beam:coder:sharded_key:v1` coder to the Go SDK, along with
   `beam.Coder.IsDeterministic`, `beam.PCollection.WindowingStrategy`,
@@ -145,7 +145,6 @@
 * (Python) Added `Watch`, a transform that polls a growing set of outputs for each input element, deduplicates outputs across poll rounds, and stops per a user-supplied termination condition
   ([#21521](https://github.com/apache/beam/issues/21521)).
 * (Python) Added support to analyze core dumps created after python worker segmentation faults with `pystack` (or `gdb` if installed) using the `--profiler_agent=coredump` pipeline option. ([#39484](https://github.com/apache/beam/issues/39484)).
-* (Python) Added `Sample.Any`, the Python equivalent of Java's `Sample.any`, which returns up to n arbitrary elements from a PCollection ([#18552](https://github.com/apache/beam/issues/18552)).
 
 ## Breaking Changes
 
@@ -160,9 +159,6 @@
   Use pipeline option `--updateCompatibilityVersion=2.75.0` (or any older version) to keep the old behavior ([#39344](https://github.com/apache/beam/issues/39344)).
 * `DoFn.process` returning a `str`, `bytes`, or `dict` (instead of an iterable wrapping one) now raises a `TypeError` rather than silently iterating per-character/byte/key (Python) ([#18712](https://github.com/apache/beam/issues/18712)).
 * (Java) Added `DRAINING` and `DRAINED` states to `PipelineResult`, including runner state mappings and Dataflow update handling ([#39020](https://github.com/apache/beam/issues/39020)).
-* (Python) Typehints of dataclass fields are honored during type inferences. To restore the behavior of fallback-to-any,
-  use pipeline option `--exclude_infer_dataclass_field_type` ([#38797](https://github.com/apache/beam/issues/38797)).
-  However fixing forward is recommended.
 * (Java) IcebergIO and projects that use it must now be built with Java 17 or later as a result of Iceberg 1.11.0 upgrade ([#38925](https://github.com/apache/beam/issues/38925)).
 
 ## Bugfixes

sdks/python/build.gradle

@@ -107,16 +107,24 @@ tasks.register("prepareExpansionServicesForYamlDocs") {
 
   dependsOn ":sdks:java:extensions:schemaio-expansion-service:shadowJar"
   dependsOn ":sdks:java:extensions:sql:expansion-service:shadowJar"
+  dependsOn ":sdks:java:io:amazon-web-services2:expansion-service:shadowJar"
+  dependsOn ":sdks:java:io:debezium:expansion-service:shadowJar"
   dependsOn ":sdks:java:io:expansion-service:shadowJar"
   dependsOn ":sdks:java:io:google-cloud-platform:expansion-service:shadowJar"
+  dependsOn ":sdks:java:io:messaging-expansion-service:shadowJar"
+  dependsOn ":sdks:java:io:snowflake:expansion-service:shadowJar"
 
   doLast {
     // Copy expansion service jar into cache path (.apache_beam/cache/jars/).
     copy {
       from project.tasks.findByPath(":sdks:java:extensions:schemaio-expansion-service:shadowJar")
       from project.tasks.findByPath(":sdks:java:extensions:sql:expansion-service:shadowJar")
+      from project.tasks.findByPath(":sdks:java:io:amazon-web-services2:expansion-service:shadowJar")
+      from project.tasks.findByPath(":sdks:java:io:debezium:expansion-service:shadowJar")
       from project.tasks.findByPath(":sdks:java:io:expansion-service:shadowJar")
       from project.tasks.findByPath(":sdks:java:io:google-cloud-platform:expansion-service:shadowJar")
+      from project.tasks.findByPath(":sdks:java:io:messaging-expansion-service:shadowJar")
+      from project.tasks.findByPath(":sdks:java:io:snowflake:expansion-service:shadowJar")
       into "${System.getProperty('user.home')}/.apache_beam/cache/jars/"
     }
   }

website/www/site/content/en/blog/beam-2.76.0.md

@@ -63,7 +63,6 @@ For more information on changes in 2.76.0, check out the [detailed release notes
 * (Python) Added `Watch`, a transform that polls a growing set of outputs for each input element, deduplicates outputs across poll rounds, and stops per a user-supplied termination condition
   ([#21521](https://github.com/apache/beam/issues/21521)).
 * (Python) Added support to analyze core dumps created after python worker segmentation faults with `pystack` (or `gdb` if installed) using the `--profiler_agent=coredump` pipeline option. ([#39484](https://github.com/apache/beam/issues/39484)).
-* (Python) Added `Sample.Any`, the Python equivalent of Java's `Sample.any`, which returns up to n arbitrary elements from a PCollection ([#18552](https://github.com/apache/beam/issues/18552)).
 
 ### Breaking Changes
 
@@ -78,9 +77,6 @@ For more information on changes in 2.76.0, check out the [detailed release notes
   Use pipeline option `--updateCompatibilityVersion=2.75.0` (or any older version) to keep the old behavior ([#39344](https://github.com/apache/beam/issues/39344)).
 * `DoFn.process` returning a `str`, `bytes`, or `dict` (instead of an iterable wrapping one) now raises a `TypeError` rather than silently iterating per-character/byte/key (Python) ([#18712](https://github.com/apache/beam/issues/18712)).
 * (Java) Added `DRAINING` and `DRAINED` states to `PipelineResult`, including runner state mappings and Dataflow update handling ([#39020](https://github.com/apache/beam/issues/39020)).
-* (Python) Typehints of dataclass fields are honored during type inferences. To restore the behavior of fallback-to-any,
-  use pipeline option `--exclude_infer_dataclass_field_type` ([#38797](https://github.com/apache/beam/issues/38797)).
-  However fixing forward is recommended.
 * (Java) IcebergIO and projects that use it must now be built with Java 17 or later as a result of Iceberg 1.11.0 upgrade ([#38925](https://github.com/apache/beam/issues/38925)).
 
 ### Bugfixes

website/www/site/content/en/documentation/io/connectors.md

@@ -429,7 +429,10 @@ This table provides a consolidated, at-a-glance overview of the available built-
       ✔
       <a href="https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/jms/JmsIO.html">native</a>
     </td>
-    <td>Not available</td>
+    <td class="present">
+      ✔
+      <a href="https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.xlang.io.html#apache_beam.transforms.xlang.io.ReadFromJms">via X-language</a>
+    </td>
     <td>Not available</td>
     <td>Not available</td>
     <td>Not available</td>
@@ -790,7 +793,11 @@ This table provides a consolidated, at-a-glance overview of the available built-
       <a href="https://pkg.go.dev/github.com/apache/beam/sdks/v2/go/pkg/beam/io/xlang/debeziumio">via X-language</a>
     </td>
     <td>Not available</td>
-    <td>Not available</td>
+    <td class="present">
+      ✔
+      <br>
+      <a href="https://beam.apache.org/releases/yamldoc/current/#readfromdebezium">read</a>
+    </td>
     <td class="present">✔</td>
     <td class="present">✔</td>
     <td class="absent">✘</td>
@@ -1322,18 +1329,25 @@ This table provides a consolidated, at-a-glance overview of the available built-
   <tr>
     <td>Delta Lake</td>
     <td class="present">✔</td>
-    <td class="present">✘</td>
+    <td class="absent">✘</td>
     <td class="present">
       ✔
       <a href="https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/managed/Managed.html">native</a>
     </td>
+    <td class="present">
+      ✔
+      <a href="https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.managed.html#module-apache_beam.transforms.managed">via X-language</a>
+    </td>
     <td>Not available</td>
     <td>Not available</td>
-    <td>Not available</td>
-    <td>Not available</td>
+    <td class="present">
+      ✔
+      <br>
+      <a href="https://beam.apache.org/releases/yamldoc/current/#readfromdelta">read</a>
+    </td>
     <td class="present">✔</td>
     <td class="absent">✘</td>
-    <td class="absent">✔</td>
+    <td class="present">✔</td>
   </tr>
 </table>
 </div>
@@ -1346,205 +1360,130 @@ This table provides a consolidated, at-a-glance overview of the available built-
     <th>Connector Name</th>
     <th>Source Supported</th>
     <th>Sink Supported</th>
-    <th>Java</th>
-    <th>Python</th>
-    <th>Go</th>
-    <th>Typescript</th>
-    <th>Yaml</th>
+    <th>Language</th>
     <th>Batch Supported</th>
     <th>Streaming Supported</th>
   </tr>
 
   <tr>
     <td>
-      <a href="https://github.com/google/hana-bq-beam-connector">SAP Hana to Google BigQuery</a>
+      <a href="https://cloud.google.com/bigtable/docs/hbase-dataflow-java">Cloud Bigtable (HBase based)</a>
     </td>
     <td class="present">✔</td>
     <td class="present">✔</td>
-    <td class="present">
-      ✔
-      native
-    </td>
-    <td>Not available</td>
-    <td>Not available</td>
-    <td>Not available</td>
-    <td>Not available</td>
+    <td>Java</td>
     <td class="present">✔</td>
     <td class="absent">✘</td>
   </tr>
   <tr>
     <td>
-      <a href="https://github.com/esakik/beam-mysql-connector">MySQL</a>
+      <a href="https://github.com/cognitedata/cdf-beam-connector-java">Cognite Data Fusion</a>
     </td>
     <td class="present">✔</td>
     <td class="present">✔</td>
-    <td>Not available</td>
-    <td class="present">
-      ✔
-      native
-    </td>
-    <td>Not available</td>
-    <td>Not available</td>
-    <td class="present">
-      ✔
-      <br>
-      <a href="https://beam.apache.org/releases/yamldoc/current/#readfrommysql">read</a> / <a href="https://beam.apache.org/releases/yamldoc/current/#writetomysql">write</a>
-    </td>
+    <td>Java</td>
+    <td class="present">✔</td>
     <td class="present">✔</td>
-    <td class="absent">✘</td>
   </tr>
   <tr>
     <td>
-      <a href="https://github.com/Refinitiv/trep-websockets-beam-io">TrepWsIO</a>
+      <a href="https://github.com/kineticadb/kinetica-connector-beam">KineticaDB</a>
     </td>
     <td class="present">✔</td>
-    <td class="absent">✘</td>
-    <td class="present">
-      ✔
-      native
-    </td>
-    <td>Not available</td>
-    <td>Not available</td>
-    <td>Not available</td>
-    <td>Not available</td>
     <td class="present">✔</td>
+    <td>Java</td>
     <td class="present">✔</td>
+    <td class="absent">✘</td>
   </tr>
   <tr>
     <td>
-      <a href="https://github.com/kineticadb/kinetica-connector-beam">KineticaDB</a>
+      <a href="https://github.com/google/hana-bq-beam-connector">SAP Hana to Google BigQuery</a>
     </td>
     <td class="present">✔</td>
     <td class="present">✔</td>
-    <td class="present">
-      ✔
-      native
-    </td>
-    <td>Not available</td>
-    <td>Not available</td>
-    <td>Not available</td>
-    <td>Not available</td>
+    <td>Java</td>
     <td class="present">✔</td>
     <td class="absent">✘</td>
   </tr>
   <tr>
     <td>
-      <a href="https://github.com/cognitedata/cdf-beam-connector-java">Cognite Data Fusion</a>
+      <a href="https://github.com/Refinitiv/trep-websockets-beam-io">TrepWsIO</a>
     </td>
     <td class="present">✔</td>
-    <td class="present">✔</td>
-    <td class="present">
-      ✔
-      native
-    </td>
-    <td>Not available</td>
-    <td>Not available</td>
-    <td>Not available</td>
-    <td>Not available</td>
+    <td class="absent">✘</td>
+    <td>Java</td>
     <td class="present">✔</td>
     <td class="present">✔</td>
   </tr>
   <tr>
     <td>
-      <a href="https://github.com/314e/beam-pyodbc-connector">Pyodbc</a>
+      <a href="https://github.com/beam-pyio">Beam PyIO (Collection of Python IO connectors)</a>
     </td>
     <td class="present">✔</td>
     <td class="present">✔</td>
-    <td>Not available</td>
-    <td class="present">
-      ✔
-      native
-    </td>
-    <td>Not available</td>
-    <td>Not available</td>
-    <td>Not available</td>
+    <td>Python</td>
+    <td class="present">✔</td>
     <td class="present">✔</td>
-    <td class="absent">✘</td>
   </tr>
   <tr>
     <td>
-      <a href="https://github.com/amient/goconnect">Go Connect</a>
+      <a href="https://github.com/jccatrinck/dataflow-cloud-sql-python">Cloud SQL</a>
     </td>
     <td class="present">✔</td>
-    <td class="present">✔</td>
     <td class="absent">✘</td>
-    <td class="absent">✘</td>
-    <td class="present">
-      ✔
-      native
-    </td>
-    <td>Not available</td>
-    <td>Not available</td>
-    <td class="present">✔</td>
+    <td>Python</td>
     <td class="present">✔</td>
+    <td class="absent">✘</td>
   </tr>
   <tr>
     <td>
-      <a href="https://github.com/tinybirdco/tinybird-beam">Tinybird</a>
+      <a href="https://github.com/esakik/beam-mysql-connector">MySQL</a>
     </td>
-    <td class="absent">✘</td>
     <td class="present">✔</td>
-    <td>Not available</td>
-    <td class="present">
-      ✔
-      native
-    </td>
-    <td>Not available</td>
-    <td>Not available</td>
-    <td>Not available</td>
     <td class="present">✔</td>
+    <td>Python</td>
     <td class="present">✔</td>
+    <td class="absent">✘</td>
   </tr>
   <tr>
     <td>
-      <a href="https://github.com/jccatrinck/dataflow-cloud-sql-python">Cloud SQL</a>
+      <a href="https://github.com/314e/beam-pyodbc-connector">Pyodbc</a>
     </td>
     <td class="present">✔</td>
-    <td class="absent">✘</td>
-    <td>Not available</td>
-    <td class="present">
-      ✔
-      native
-    </td>
-    <td>Not available</td>
-    <td>Not available</td>
-    <td>Not available</td>
+    <td class="present">✔</td>
+    <td>Python</td>
     <td class="present">✔</td>
     <td class="absent">✘</td>
   </tr>
   <tr>
     <td>
-      <a href="https://cloud.google.com/bigtable/docs/hbase-dataflow-java">Cloud Bigtable (HBase based)</a>
+      <a href="https://github.com/tinybirdco/tinybird-beam">Tinybird</a>
     </td>
+    <td class="absent">✘</td>
     <td class="present">✔</td>
+    <td>Python</td>
     <td class="present">✔</td>
-    <td class="present">
-      ✔
-      native
-    </td>
-    <td>Not available</td>
-    <td>Not available</td>
-    <td>Not available</td>
-    <td>Not available</td>
     <td class="present">✔</td>
-    <td class="absent">✘</td>
   </tr>
   <tr>
     <td>
-      <a href="https://github.com/beam-pyio">Beam PyIO (Collection of Python IO connectors)</a>
+      <a href="https://github.com/amient/goconnect">Go Connect</a>
     </td>
     <td class="present">✔</td>
     <td class="present">✔</td>
-    <td>Not available</td>
-    <td class="present">
-      ✔
-      native
+    <td>Go</td>
+    <td class="present">✔</td>
+    <td class="present">✔</td>
+  </tr>
+  <tr>
+    <td>
+      <a href="https://beam.apache.org/releases/yamldoc/current/#readfrommysql">MySQL, PostgreSQL, Oracle, SQL Server</a>
     </td>
-    <td>Not available</td>
-    <td>Not available</td>
-    <td>Not available</td>
     <td class="present">✔</td>
     <td class="present">✔</td>
+    <td>Yaml</td>
+    <td class="present">✔</td>
+    <td class="absent">✘</td>
   </tr>
 </table>
 </div>

website/www/site/data/capability_matrix.yaml

@@ -1736,7 +1736,6 @@ capability-matrix:
               l1: "Unverified"
               l2:
               l3:
-
             - class: kafka-streams
               l1: "No"
               l2: not implemented