Skip to content

Commit fea00aa

Browse files
committed
[C++][FlightRPC] Add an async gRPC Flight server
Serve Flight RPCs on gRPC's generic callback API so handlers return arrow::Future and never occupy a worker thread while waiting. Motivation: the synchronous transport runs every RPC on a gRPC worker thread that the handler occupies until it returns, so throughput is bounded by the thread pool (which grows at most ~1 thread/second) rather than by the work in flight. Clients already have an async path; servers had none. API (server_async.h, flight_data_decoder.h): - AsyncGenericFlightServerBase: *Async handlers returning arrow::Future; unoverridden handlers answer NotImplemented. - AsyncFlightDataStream / AsyncFlightListing / AsyncResultStream: pull-based iterators, one item requested and written at a time. - AsyncFlightMessageReader / AsyncFlightMessageWriter: DoExchange halves. - AsyncFlightDataListener / AsyncFlightDataWriter: push-based DoPut, with PutResult metadata sendable from any thread. Transport (transport/grpc/async_grpc_*.cc): - One translation unit per part of the transport: service dispatch, DoGet, DoPut, DoExchange (reader, writer and reactor), Handshake, the streaming trio and the unary trio. - AsyncGenericFlightService creates one reactor per RPC from the method name; unknown methods get a fallback reactor answering UNIMPLEMENTED, so the typed sync service is not registered alongside it. - DoGet is served on a write-only bidi reactor: the generic callback API has no server-streaming reactor. - Reactors refcount themselves, since gRPC's callback API has no holds; write buffers are reactor members because StartWrite requires them to outlive the call. - DoPut serializes its PutResult writes: WriteMetadata() runs on the listener's thread while OnReadDone()/OnWriteDone() run on gRPC callback threads, and a half-close arriving mid-write now waits for the slot instead of starting a second write on the same buffer. - AsyncCallContext carries the call context; middleware and auth go through the shared GrpcServerCallContextHelper. Fixes found while building this: - Signal teardown ordering: Serve() published the server as the signal target before its self-pipe and shutdown callback were initialized, and did not restore handlers when it failed partway. - SIGPIPE: waking the signal thread with SelfPipe::Shutdown() closed the read end, so a handler entering just before that killed the process; wake with a plain payload instead. - Shutdown() on an uninitialized server cleared the process-wide signal target, disabling signal shutdown for another serving instance. Serve(), Shutdown() and Wait() now validate before entering the signal helper. - Sync/async lifecycle duplication: Init/Serve/Shutdown/Wait/port/location moved into internal::ServerLifecycle, owned by both server classes. - Location::uri() returns Result<util::Uri>, which needs util::Uri complete (Result embeds AlignedStorage<T>): include arrow/util/uri.h from types.h instead of forward-declaring it. - ToGrpcStatus had no CallbackServerContext overload, needed to attach the Arrow status trailers from a reactor. - Middleware CallCompleted was skipped for unknown methods. - DoPut had no PutResult write-back channel while an upload was in flight. - A PutResult write started while the ending ack was in flight overwrote the write buffer; the ending ack is now deferred until the slot is free. - OnFinish() is application code and may call back into the listener's writer; the transport no longer holds its write lock across it. - gRPC 1.65 is required for the async server only; the async client paths still need 1.40, and ARROW_GRPC_REQUIRED_VERSION returns to 1.30.0. Tests: the async transport tests are split by the transport part each one covers, with the shared fixtures in async_grpc_test_fixtures.h. The synchronous server's suites are unchanged. Known limitations: a blocking ServerAuthHandler is refused (it would run on callback threads) in favour of Handshake/ValidateToken; a handler that abandons its future blocks Shutdown() until a deadline; request arguments are call-scoped. Build fixes across toolchains: Fix the cross-platform build failures of the async transport: - async_grpc_compat.h included grpcpp/version_info.h, which does not exist before gRPC 1.52, so builds against older system gRPC (the ARM64 Ubuntu 22.04 job links 1.30.2) failed to compile grpc_server.cc. Reuse the GRPC_CPP_VERSION_CHECK macro from util_internal.h, which already handles releases that predate the version macros, and treat those as "no async server" instead of failing to build. - server_async.h documented one parameter as `context`, which clang rejects under -Wdocumentation (used with -Werror on macOS and in the sanitizer job). Drop the backticks. - FlightDataSerialize, BytesFromBuffer, ToPayload and AsyncGenericFlightService are used by the Flight tests and the async transport across the shared library boundary, but were not tagged ARROW_FLIGHT_EXPORT. With ARROW_BUILD_SHARED the MinGW job failed to link arrow-flight-test and arrow-flight-async-grpc-test. Export them.
1 parent aae29bf commit fea00aa

48 files changed

Lines changed: 7845 additions & 64 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎cpp/examples/arrow/CMakeLists.txt‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -127,6 +127,20 @@ if(ARROW_FLIGHT)
127127
"${CMAKE_CURRENT_BINARY_DIR}/helloworld.pb.cc"
128128
"${CMAKE_CURRENT_BINARY_DIR}/helloworld.grpc.pb.cc")
129129

130+
add_arrow_example(flight_async_generic_doget_example
131+
EXTRA_LINK_LIBS
132+
${FLIGHT_EXAMPLES_LINK_LIBS}
133+
gRPC::grpc++
134+
${ARROW_PROTOBUF_LIBPROTOBUF}
135+
${GFLAGS_LIBRARIES})
136+
137+
add_arrow_example(flight_async_generic_doput_example
138+
EXTRA_LINK_LIBS
139+
${FLIGHT_EXAMPLES_LINK_LIBS}
140+
gRPC::grpc++
141+
${ARROW_PROTOBUF_LIBPROTOBUF}
142+
${GFLAGS_LIBRARIES})
143+
130144
if(ARROW_FLIGHT_SQL)
131145
if(ARROW_BUILD_SHARED AND ARROW_GRPC_USE_SHARED)
132146
set(FLIGHT_SQL_EXAMPLES_LINK_LIBS arrow_flight_sql_shared)
Lines changed: 362 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,362 @@
1+
// Licensed to the Apache Software Foundation (ASF) under one
2+
// or more contributor license agreements. See the NOTICE file
3+
// distributed with this work for additional information
4+
// regarding copyright ownership. The ASF licenses this file
5+
// to you under the Apache License, Version 2.0 (the
6+
// "License"); you may not use this file except in compliance
7+
// with the License. You may obtain a copy of the License at
8+
//
9+
// http://www.apache.org/licenses/LICENSE-2.0
10+
//
11+
// Unless required by applicable law or agreed to in writing,
12+
// software distributed under the License is distributed on an
13+
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
// KIND, either express or implied. See the License for the
15+
// specific language governing permissions and limitations
16+
// under the License.
17+
18+
#include <unistd.h>
19+
#include <csignal>
20+
#if defined(__linux__)
21+
# include <sys/prctl.h>
22+
#endif
23+
24+
#include <chrono>
25+
#include <condition_variable>
26+
#include <cstdint>
27+
#include <cstdlib>
28+
#include <iostream>
29+
#include <memory>
30+
#include <mutex>
31+
#include <optional>
32+
#include <string>
33+
#include <thread>
34+
#include <utility>
35+
36+
#include <arrow/api.h>
37+
#include <arrow/flight/api.h>
38+
#include <gflags/gflags.h>
39+
40+
// Serve DoGet over the async generic Flight transport.
41+
//
42+
// Deriving from AsyncGenericFlightServerBase (arrow/flight/server_async.h)
43+
// selects the async generic gRPC transport. DoGet is answered by DoGetAsync:
44+
// it returns a future for a stream of payloads (AsyncFlightDataStream) instead
45+
// of the payloads themselves, so the gRPC callback thread serving the RPC
46+
// never waits for data.
47+
//
48+
// DelayedAsyncStream shows what that is for: each batch is produced on the
49+
// stream's own worker thread, --batch_delay_ms after the transport asks for
50+
// it. The transport adds a callback to the future and returns, so the delay
51+
// never blocks a gRPC callback thread (which would stall every RPC multiplexed
52+
// onto it). With --batch_delay_ms=0 the delay is gone and this is a plain
53+
// fast DoGet.
54+
//
55+
// Any ticket streams the same batches. Every other RPC - GetFlightInfo
56+
// included - answers UNIMPLEMENTED, so call DoGet directly instead of
57+
// resolving the flight first.
58+
//
59+
// Usage:
60+
// flight-async-generic-doget-example --port=31337 --batch_delay_ms=200
61+
// flight-async-generic-doget-example --port=0 --demo --batches=5
62+
//
63+
// Run with no arguments to print this message and exit.
64+
65+
DEFINE_int32(port, 0, "Port to listen on (0 picks a free port)");
66+
DEFINE_int32(batches, 5, "Number of batches in the DoGet stream");
67+
DEFINE_int32(batch_delay_ms, 50,
68+
"Delay before each batch, spent on the stream's own worker thread");
69+
DEFINE_bool(demo, false, "Run the DoGet self-check against the server, then exit");
70+
71+
namespace flight = arrow::flight;
72+
73+
/// \brief A reader with `num_batches` batches of three int64 values.
74+
class BatchReader : public arrow::RecordBatchReader {
75+
public:
76+
BatchReader(std::shared_ptr<arrow::Schema> schema, int32_t num_batches)
77+
: schema_(std::move(schema)), num_batches_(num_batches) {}
78+
79+
std::shared_ptr<arrow::Schema> schema() const override { return schema_; }
80+
81+
arrow::Status ReadNext(std::shared_ptr<arrow::RecordBatch>* batch) override {
82+
if (next_batch_ >= num_batches_) {
83+
*batch = nullptr; // end of stream
84+
return arrow::Status::OK();
85+
}
86+
arrow::Int64Builder builder;
87+
for (int64_t i = 0; i < 3; i++) {
88+
ARROW_RETURN_NOT_OK(builder.Append(3 * next_batch_ + i));
89+
}
90+
ARROW_ASSIGN_OR_RAISE(auto array, builder.Finish());
91+
*batch = arrow::RecordBatch::Make(schema_, array->length(), {array});
92+
++next_batch_;
93+
return arrow::Status::OK();
94+
}
95+
96+
private:
97+
std::shared_ptr<arrow::Schema> schema_;
98+
int32_t num_batches_;
99+
int32_t next_batch_ = 0;
100+
};
101+
102+
/// \brief The stream's worker state. Heap-allocated and shared with the worker
103+
/// thread: the transport deletes the stream as soon as a payload completes
104+
/// (the end-of-stream payload is completed *on the worker*), so the worker must
105+
/// not touch the stream object after that. Everything it needs lives here.
106+
struct WorkerState {
107+
std::unique_ptr<flight::FlightDataStream> stream;
108+
std::shared_ptr<arrow::Schema> schema;
109+
int32_t delay_ms = 0;
110+
std::mutex mutex;
111+
std::condition_variable ready;
112+
bool closed = false;
113+
/// The payload the transport is waiting for, if one was requested.
114+
arrow::Future<std::optional<flight::FlightPayload>> pending;
115+
bool has_pending = false;
116+
/// Whether the wrapped stream has been closed, so Close() closes it once.
117+
bool stream_closed = false;
118+
std::thread thread;
119+
};
120+
121+
/// \brief The worker: wait for a payload request, sleep delay_ms, produce it,
122+
/// complete the future the transport holds.
123+
void RunWorker(std::shared_ptr<WorkerState> state) {
124+
#if defined(__linux__)
125+
// Name the thread, so /proc/<pid>/task/*/comm distinguishes this server's own
126+
// per-stream workers (doget_worker) from gRPC's pool threads (event_engine /
127+
// lifeguard / grpc_global_timer). Thread names are inherited, and gRPC names
128+
// its threads, so an anonymous thread created from a gRPC callback would
129+
// otherwise show up as another "event_engine". comm is capped at 15
130+
// characters.
131+
prctl(PR_SET_NAME, "doget_worker", 0, 0, 0);
132+
#endif
133+
std::unique_lock<std::mutex> lock(state->mutex);
134+
// Read the schema on this thread: it is the only one allowed to touch the
135+
// wrapped stream, and the transport asks for the schema first.
136+
state->schema = state->stream->schema();
137+
while (!state->closed) {
138+
state->ready.wait(lock, [&] { return state->has_pending || state->closed; });
139+
if (state->closed) {
140+
break;
141+
}
142+
// Take the request: after this, only this thread resolves the future.
143+
state->has_pending = false;
144+
arrow::Future<std::optional<flight::FlightPayload>> future = state->pending;
145+
lock.unlock();
146+
std::this_thread::sleep_for(std::chrono::milliseconds(state->delay_ms));
147+
lock.lock();
148+
const bool cancelled = state->closed;
149+
lock.unlock();
150+
// Runs the transport's callback inline, on this thread: it serializes the
151+
// payload, and on end of stream it calls Close() (hence the lock dance).
152+
if (cancelled) {
153+
future.MarkFinished(arrow::Status::Cancelled());
154+
} else {
155+
auto payload = state->stream->Next();
156+
if (!payload.ok()) {
157+
future.MarkFinished(payload.status());
158+
} else if (payload->ipc_message.metadata == nullptr) {
159+
// End of stream: the async interface reports it as a nullopt.
160+
future.MarkFinished(std::nullopt);
161+
} else {
162+
future.MarkFinished(std::move(*payload));
163+
}
164+
}
165+
lock.lock();
166+
}
167+
state->ready.notify_all();
168+
}
169+
170+
/// \brief The stream the async transport pulls from, with a sleep per batch.
171+
///
172+
/// NextAsync() hands the transport a future and returns at once, the delay and
173+
/// the serialization of the wrapped synchronous stream happen on this stream's
174+
/// own worker thread. The transport keeps one payload in flight at a time --
175+
/// it asks for the next only once the write completed -- so a single worker is
176+
/// enough.
177+
///
178+
/// One thread per stream. A server with many slow streams may prefer a shared
179+
/// timer or thread pool, swapping WorkerState's thread for a task on that pool.
180+
class DelayedAsyncStream : public flight::AsyncFlightDataStream {
181+
public:
182+
DelayedAsyncStream(std::unique_ptr<flight::FlightDataStream> stream, int32_t delay_ms)
183+
: state_(std::make_shared<WorkerState>()) {
184+
state_->stream = std::move(stream);
185+
state_->delay_ms = delay_ms;
186+
std::shared_ptr<WorkerState> state = state_;
187+
state_->thread = std::thread([state] { RunWorker(state); });
188+
}
189+
190+
~DelayedAsyncStream() override {
191+
Stop();
192+
if (state_->thread.joinable()) {
193+
if (state_->thread.get_id() == std::this_thread::get_id()) {
194+
// Deleted from inside its own worker (the transport releases the
195+
// stream as a payload completes): joining would deadlock.
196+
state_->thread.detach();
197+
} else {
198+
state_->thread.join();
199+
}
200+
}
201+
}
202+
203+
std::shared_ptr<arrow::Schema> schema() override { return state_->schema; }
204+
205+
arrow::Future<flight::FlightPayload> GetSchemaPayloadAsync() override {
206+
// The schema is ready now: no delay, no worker. This runs before the first
207+
// NextAsync(), so it cannot overlap the worker's use of the stream.
208+
return arrow::Future<flight::FlightPayload>::MakeFinished(
209+
state_->stream->GetSchemaPayload());
210+
}
211+
212+
arrow::Future<std::optional<flight::FlightPayload>> NextAsync() override {
213+
arrow::Future<std::optional<flight::FlightPayload>> future =
214+
arrow::Future<std::optional<flight::FlightPayload>>::Make();
215+
{
216+
std::lock_guard<std::mutex> lock(state_->mutex);
217+
if (state_->closed) {
218+
future.MarkFinished(arrow::Status::Cancelled());
219+
return future;
220+
}
221+
state_->pending = future;
222+
state_->has_pending = true;
223+
}
224+
state_->ready.notify_one();
225+
return future;
226+
}
227+
228+
arrow::Status Close() override {
229+
Stop();
230+
{
231+
std::lock_guard<std::mutex> lock(state_->mutex);
232+
if (state_->stream_closed) {
233+
return arrow::Status::OK();
234+
}
235+
state_->stream_closed = true;
236+
}
237+
return state_->stream->Close();
238+
}
239+
240+
private:
241+
/// \brief Stop the worker, resolve a payload it has not taken yet. Idempotent.
242+
/// Does not join: called by the transport, whose threads must not block on
243+
/// the worker's sleep.
244+
void Stop() {
245+
arrow::Future<std::optional<flight::FlightPayload>> pending;
246+
{
247+
std::lock_guard<std::mutex> lock(state_->mutex);
248+
state_->closed = true;
249+
if (state_->has_pending) {
250+
state_->has_pending = false;
251+
pending = state_->pending;
252+
}
253+
}
254+
state_->ready.notify_all();
255+
if (pending.is_valid()) {
256+
// Runs the transport's callback inline, it reacts by calling Close().
257+
pending.MarkFinished(arrow::Status::Cancelled());
258+
}
259+
}
260+
261+
std::shared_ptr<WorkerState> state_;
262+
};
263+
264+
/// \brief The server: DoGetAsync is the only handler overridden, so every other
265+
/// RPC answers UNIMPLEMENTED.
266+
class ExampleServer : public flight::AsyncGenericFlightServerBase {
267+
public:
268+
arrow::Future<std::shared_ptr<flight::AsyncFlightDataStream>> DoGetAsync(
269+
const flight::ServerCallContext& context, const flight::Ticket& request) override {
270+
std::cout << "DoGet: ticket=" << request.ticket << std::endl;
271+
auto schema = arrow::schema({arrow::field("value", arrow::int64())});
272+
auto reader = std::make_shared<BatchReader>(schema, FLAGS_batches);
273+
std::shared_ptr<flight::AsyncFlightDataStream> stream =
274+
std::make_shared<DelayedAsyncStream>(
275+
std::make_unique<flight::RecordBatchStream>(reader), FLAGS_batch_delay_ms);
276+
// The stream is ready here, but its payloads are not: the transport will
277+
// wait on the futures NextAsync() returns.
278+
return arrow::Future<std::shared_ptr<flight::AsyncFlightDataStream>>::MakeFinished(
279+
std::move(stream));
280+
}
281+
};
282+
283+
/// \brief The --demo self-check: drain one DoGet stream and count its batches.
284+
bool RunDemo(int port) {
285+
auto location = flight::Location::ForGrpcTcp("127.0.0.1", port);
286+
if (!location.ok()) {
287+
std::cerr << "Failed to build client location: " << location.status() << std::endl;
288+
return false;
289+
}
290+
auto client = flight::FlightClient::Connect(*location);
291+
if (!client.ok()) {
292+
std::cerr << "Failed to connect: " << client.status() << std::endl;
293+
return false;
294+
}
295+
auto reader = (*client)->DoGet(flight::Ticket("demo"));
296+
if (!reader.ok()) {
297+
std::cerr << "DoGet failed: " << reader.status() << std::endl;
298+
return false;
299+
}
300+
int64_t num_batches = 0;
301+
while (true) {
302+
auto chunk = (*reader)->Next();
303+
if (!chunk.ok()) {
304+
std::cerr << "DoGet drain failed: " << chunk.status() << std::endl;
305+
return false;
306+
}
307+
if (chunk->data == nullptr) break; // end of stream
308+
++num_batches;
309+
}
310+
if (num_batches == FLAGS_batches) {
311+
std::cout << "PASS: DoGet streamed " << num_batches << " batches" << std::endl;
312+
return true;
313+
}
314+
std::cout << "FAIL: DoGet streamed " << num_batches << " batches, expected "
315+
<< FLAGS_batches << std::endl;
316+
return false;
317+
}
318+
319+
int main(int argc, char** argv) {
320+
if (argc == 1) {
321+
// As in the other examples: a bare run (e.g. from ctest) does not start a
322+
// server, it just prints the usage.
323+
std::cout << "Usage: " << argv[0]
324+
<< " [--port=PORT] [--batches=N] [--batch_delay_ms=MS] [--demo]"
325+
<< std::endl;
326+
return EXIT_SUCCESS;
327+
}
328+
gflags::ParseCommandLineFlags(&argc, &argv, true);
329+
330+
ExampleServer server;
331+
auto location = flight::Location::ForGrpcTcp("0.0.0.0", FLAGS_port);
332+
if (!location.ok()) {
333+
std::cerr << location.status() << std::endl;
334+
return EXIT_FAILURE;
335+
}
336+
auto status = server.Init(flight::FlightServerOptions(*location));
337+
if (!status.ok()) {
338+
std::cerr << status.ToString() << std::endl;
339+
return EXIT_FAILURE;
340+
}
341+
std::cout << "Serving Flight on " << server.location().ToString() << std::endl;
342+
std::cout << "Server pid " << getpid() << ", port " << server.port() << std::endl;
343+
344+
if (FLAGS_demo) {
345+
bool ok = RunDemo(server.port());
346+
ARROW_WARN_NOT_OK(server.Shutdown(), "Error shutting down server");
347+
ARROW_WARN_NOT_OK(server.Wait(), "Error waiting for server shutdown");
348+
return ok ? EXIT_SUCCESS : EXIT_FAILURE;
349+
}
350+
351+
status = server.SetShutdownOnSignals({SIGINT, SIGTERM});
352+
if (!status.ok()) {
353+
std::cerr << status.ToString() << std::endl;
354+
return EXIT_FAILURE;
355+
}
356+
status = server.Serve();
357+
if (!status.ok()) {
358+
std::cerr << status.ToString() << std::endl;
359+
return EXIT_FAILURE;
360+
}
361+
return EXIT_SUCCESS;
362+
}

0 commit comments

Comments
 (0)