Skip to content

Compile the scheduler's event-log writer without the REST API #2745

Closed

Description

@andygrove

Is your feature request related to a problem or challenge?

The scheduler's event-log writer (ballista_history::writer::EventLogWriter, enabled at runtime with SchedulerConfig::event_log_dir) is only compiled with the rest-api feature. SchedulerServer::new creates it under #[cfg(feature = "rest-api")], and ballista-history is an optional dependency that only rest-api enables.

So a project that embeds the scheduler and wants per-query event logs, but has no use for Ballista's HTTP API, also has to compile the REST handlers, their OpenAPI types and axum routes, and then turn them off at runtime with disable_rest_api. Apache DataFusion Comet ran into this when adding COMET_SCHEDULER_EVENT_LOG_DIR to its scheduler binary (apache/datafusion-comet#7420). Enabling rest-api for the event log also put the REST API, which has no authentication, on the scheduler's gRPC port, until the binary set disable_rest_api: true.

Describe the solution you'd like

Add an event-log feature for the writer:

event-log = ["dep:ballista-history", "dep:serde_json"]
rest-api = ["event-log", "dep:ballista-api-types", "dep:utoipa", ...]

The writer and its QueryStageScheduler wiring would use #[cfg(feature = "event-log")], and the history server binary and REST routes would stay behind rest-api. Since rest-api enables event-log, default builds don't change.

Describe alternatives you've considered

Keep one feature and rely on disable_rest_api at runtime. That works, but every embedder has to know to set it, and a binary built with rest-api serves the API whenever it is left at its default of false.

Additional context

Writer setup: ballista/scheduler/src/scheduler_server/mod.rs, SchedulerServer::new. Route setup: ballista/scheduler/src/scheduler_process.rs, start_grpc_service.

Activity

  1. milenkovicm commented on Nov 17, 2026

    @milenkovicm
    Contributor

    looks like fixed in #2747 please @andygrove reopen if not correct

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions