Add pacing logic to Lighter

[hacheigriega]

Motivation

To prevent hitting Lighter's rate limits

Explanation of Changes

To address the limit of "Max Messages Sent By Client Per Minute" set at 200, we add a pacing logic to the Lighter module WS client by using a single outbound queue parametrized by maxMessagesPerMinute (defaults to 180).

Also adds parsing and logging of error frames sent by Lighter and Binance. (These are separate from transport-level WS failures that end up in the error handler)

Testing

A few unit tests.
Also tested locally that we no longer get "Too Many Websocket Messages" errors upon burst of requests that we get on main.

[gluax]

LGTM, but maybe a second opinion from someone more familiar with TS/DP?

[Thomasvdam]

I think we can express the pacing queue more elegantly with a semaphore.

import { NodeRuntime } from "@effect/platform-node";
import { Duration, Effect, Schedule, Stream } from "effect";

const MAX_MESSAGES_PER_WINDOW = 5;
const WINDOW_SIZE_SECONDS = 5

const task = Effect.fn(function* (a: number) {
	yield* Effect.logInfo();
	yield* Effect.sleep(Duration.seconds(WINDOW_SIZE));
});

const program = Effect.gen(function* () {
	const mutex = yield* Effect.makeSemaphore(MAX_MESSAGES_PER_WINDOW);

	const stream = Stream.iterate(0, (n) => n + 1).pipe(
		Stream.schedule(
			Schedule.spaced(Duration.seconds(.5)).pipe(
				Schedule.jitteredWith({ min: 0.8, max: 1.2 }),
			),
		),
	);

	yield* stream.pipe(
		Stream.mapEffect(
			(n) =>
				mutex
					.withPermits(1)(
						Effect.gen(function* () {
							yield* task(n);
						}),
					),
			{ concurrency: "unbounded" },
		),
		Stream.runDrain,
	);
}).pipe(
	Effect.withSpan("program", {
		attributes: { source: "Playground" },
	}),
);

program.pipe(NodeRuntime.runMain);

This produces the following output:

[11:32:14.533] INFO (#6):
[11:32:15.134] INFO (#9):
[11:32:15.636] INFO (#12):
[11:32:16.078] INFO (#15):
[11:32:16.623] INFO (#18):
[11:32:19.541] INFO (#21):
[11:32:20.137] INFO (#24):
[11:32:20.639] INFO (#27):
[11:32:21.082] INFO (#30):
[11:32:21.626] INFO (#33):
[11:32:24.545] INFO (#36):
[11:32:25.144] INFO (#39):
[11:32:25.643] INFO (#42):
[11:32:26.087] INFO (#45):
[11:32:26.631] INFO (#48):
[11:32:29.550] INFO (#51):
[11:32:30.148] INFO (#54):

Which seems like the behaviour we're looking for? I think this pattern also makes it easier to abstract into a utility and test separately.