Phase 4: Streaming and advanced connectors

[netsirius]

Summary

Expands the connector ecosystem with streaming support and 6 new connectors.

New source connectors

• Kafka: batch + streaming modes, configurable offsets, health check
• MongoDB: collection reads, aggregation pipeline support, TCP health check
• REST API: generic with offset/cursor pagination, bearer/api-key auth
• BigQuery source: table reads + SQL queries (was sink-only)

New sink connectors

• Kafka: batch + streaming with checkpointing and triggers
• Elasticsearch: Spark ES connector with REST bulk API fallback

Totals

• 8 source connectors: SQL, PostgreSQL, File, Kafka, MongoDB, REST, BigQuery, Test
• 6 sink connectors: BigQuery, DeltaLake, File, Kafka, Elasticsearch, Test
• 14 total connectors

All connectors include healthCheck for weaver doctor.

Test plan

• sbt compile — All modules compile
• 47 core tests pass
• CI green on GitHub Actions