relay: make subscribers wait for an in-flight local forwarder setup

[afrind]

A subscriber that arrived while another subscriber's upstream subscribe was still in flight joined the local forwarder right away: it got a SUBSCRIBE_OK with no largest object, and it kept its subscription even when the upstream subscribe came back with an error.

---

This change is 

[akash-a-n]

@akash-a-n reviewed 3 files and all commit messages, and made 1 comment.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on afrind).

---

src/MoqxRelay.cpp line 1950 at r1 (raw file):

    co_await folly::coro::co_awaitTry(std::move(pending->ready));
    // Re-resolve: the entry may have been displaced while we were waiting. A failed setup
    // fails this subscriber too.

I don't think I completely understand this part. Is it enough to just use getIfReady? shouldn't there be a co-await or a loop of sorts?

[afrind]

@afrind made 1 comment.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on akash-a-n).

---

src/MoqxRelay.cpp line 1950 at r1 (raw file):

Previously, akash-a-n wrote…

I don't think I completely understand this part. Is it enough to just use getIfReady? shouldn't there be a co-await or a loop of sorts?

We're getting into complex territory here. I think the case is:

1st Subscriber in this thread arrives, gets an entry in LF registry, begins setup
2nd Subscriber (this one) arrives, waits on pending

a) A publisher arrives and does a PUBLISH; this will overwrite the entry. OR
b) The original subscribe setup failed

If it's a) getIfReady returns ready that points to the new publisher's forwarder
If it's b) getIfReady returns !ready, and we fail the waiting subscriber

Of course there's the happy path which is that setup succeeded and the 1st subscriber called markReady. Then ready will be the same one we were waiting on.

There might be a ways to continue to improve this (eg a subscriber that gets a failed setup could try again), but it's easy to get lost here, and I want to keep the changes small.