Let a curl upload's read state own what it points at - #34
Conversation
A read state subscribes to the stream's IO event and keeps a raw pointer to it. Nothing counted that subscription: zend_async_callbacks_push() counts the callback, and the stream's own ref_count rises only around a parked coroutine. So an fclose() between two of curl's reads found ref_count at zero, disposed the io, and curl_async_read_state_free() then removed its subscription through the freed block. The subscription now holds a reference of its own, taken where the callback is added and released where it is removed, which is the shape the reactor already uses for an io it names. Keyed on the callback rather than on the pointer: file.io is assigned outside the LIBCURL_VERSION_NUM guard and file.io_cb inside it, so a build against libcurl below 8.11.1 has the one without the other. A CURLFile part carries a second read state, hung on the mime callback argument, and curl_async_event_stop() knew only the handle's. That state went on naming an event about to be freed. The handle owns the mime arguments, so stop() walks them and severs the back-link. Freeing them stays with curl_async_free_cb(), which libcurl calls when it releases the part: the completion of a read that outlives its event releases the request and the subscription and leaves the state, and each field it releases is cleared, so the free_cb finds nothing to release twice. The add_callback result is checked at both sites. A failure used to leave a callback allocated and named by a state that would never hear from it. tests/curl/071-upload_stream_closed_mid_read.phpt in php-async covers the first half: heap-use-after-free under ASAN before this change, clean after. The second half is not covered by a test. The dangling event pointer was observed directly, with a read in flight, by instrumenting the event's destructor, but no script made a completion land in the window between the destructor and the free_cb. php#295 php#294
CURL_READ_OWNS_FD guarded two close() calls and was set nowhere: the assignment went out with the raw VCWD_OPEN it belonged to, when CURLFile moved to php_stream. Every descriptor a read state holds now comes from php_stream_cast() on a stream somebody else closes, so setting the flag again would close a number another open() may already have taken. php#294
|
Checked against the libcurl versions that actually matter, not only the one on this machine. Built from source and linked into three separate PHP builds:
Identical with and without the change on both older versions, which is what the code One real find from this: the new test failed on 8.5.0. There the upload read is a
|
Fixes true-async/php-async#295 and true-async/php-async#294.
php#295 — the subscription counted nothing
curl_async_readparks on the stream'szend_async_io_tand keeps a raw pointerto it. Neither the subscription nor the state took a reference:
zend_async_callbacks_pushcounts the callback, and the stream's ownref_countin
main/streams/plain_wrapper.crises only around a parked coroutine. So anfclose()between two of curl's reads foundref_countat zero,php_stdiop_closedisposed the io, and
curl_async_read_state_free()then removed its subscriptionthrough the freed block.
Reproduced under ASAN,
USE_ZEND_ALLOC=0(the io is freed through ZendMM, so thesanitizer sees it only with the allocator out of the way):
The subscription now holds a reference of its own, taken where the callback is
added and released where it is removed — the shape the reactor already uses for
an io it names (
libuv_reactor.c:6222and:5285), and safe becauselibuv_io_event_disposeis refcount-aware. It is keyed onfile.io_cbratherthan on
file.io:file.iois assigned outside theLIBCURL_VERSION_NUM >= 0x080B01guard and
file.io_cbinside it, whileconfig.m4admits libcurl from 7.87.0, soa build in that range has the pointer without the subscription and must not
release a reference it never took.
The
add_callbackresult is checked at both subscription sites. A failure used toleave a callback allocated and named by a state that would never hear from it.
php#294 — a CURLFile read state was out of reach from its event
A
CURLFilepart carries a second read state, hung on the mime callback argument,and
curl_async_event_stop()knew onlych->async_read_state. That state went onnaming an event about to be freed.
The handle owns the mime arguments —
ch->to_free->streamis azend_llistofmime_data_cb_arg_t *— sostop()walks them and severs the back-link. Freeingstays with
curl_async_free_cb(), which libcurl calls when it releases the part:the completion of a read that outlives its event releases the request and the
subscription and leaves the state alone, marked by the new
CURL_READ_MIMEflag,and every field it releases is cleared, so the
free_cbfinds nothing to releasetwice.
This half has no test. The dangling pointer itself was observed directly, with
a read in flight, by instrumenting the event's destructor:
evis the event the state names andthisthe event being destroyed. What noscript produced is a completion landing in the window between that destructor and
the
free_cb: ten attempts, including a fifo source (useless forCURLFile— thepart's length comes from
stat, which is 0 for a fifo, so the body is sent empty),a single-thread
UV_THREADPOOL_SIZEwith eight concurrent uploads, and 160 handleremovals at varying points of the upload. The fix is by construction.
The dead flag
CURL_READ_OWNS_FDguarded twoclose()calls and was set nowhere:60fdf5a5fc7deleted its only assignment along with the raw
VCWD_OPENit belonged to. Everydescriptor a read state holds comes from
php_stream_cast()on a stream somebodyelse closes, so setting the flag again would close a number another
open()mayalready have taken. Removed with both branches, in its own commit.
Checks
tests/curl/071-upload_stream_closed_mid_read.phpt(php-async, sent separately):heap-use-after-free under ASAN on a rebuild of this tree with the first commit
reverted, green with it.
ext/async/tests/curlunder ASAN — 68 passed, 0 failed.ext/async/tests— 1194 passed, 0 failed.fuzzy-tests/_generated— 724 passed,0 failed.
ext/curl/testsunder ASAN — 168 passed, 4 failed:curl_writefunction_throws_abort,curl_readfunction_throws_abort,curl_headerfunction_throws_abortandcurl_curlfile_seek. All four fail the same way on a build of this tree withoutthese commits.
detect_leaks=1on either probe.