Create Subscription Option for ActiveMQ

[adeutscher]

Below is Cursor's opinion on the viability of subscribing using ActiveMQ.

ActiveMQ Artemis. Same dual API as RabbitMQ. The current source uses IMessageConsumer.ReceiveAsync with a short timeout. NMS also has a Listener (MessageListener) that the broker dispatches into. Client-ack, no per-message heartbeat, prefetch on the consumer — this maps almost directly onto RabbitMqSubscribeJobSource.

Testing

Similar to RabbitMQ, testing criteria is:

• The implementation must be able to handle messages (spoiler alert)
• In order for this effort to be worth merging into develop alongside the polling implementation, it should be immediately responsive to new messages
• The implementation must be able to handle messages after a connection is lost/recovered. Restarting the message source should not be a process-breaker for the JobWorker.
• The implementation should respect the JOBS__HALT_ON_FAILURE environment variable being set to true for major problems. Break the exception arbiter in a local test if you have to in order to confirm this. Works in cases, low priority. See comments

[adeutscher]

Progress is going well. Building for local testing now.

[adeutscher]

The implementation works and is snappy, but so far it isn't able to recover the way that RabbitMQ does in reaction to my go-to interruption test (bringing the container down and back up). Investigation continues.

[adeutscher]

Recovery behaviour working better now, just going to add a log message for when the connection is interrupted.

[adeutscher]

Is respecting halt-on-failure when I deliberately set bad credentials in SSM, but the failover settings that I enforce on subscriptions are suppressing a lot of other items. This is acceptable.

[adeutscher]

Updated things to pull new credentials out of the secret manager to address a security error if HaltOnFailure being false gives it the time to do so.

There's an edge case somewhere of "but how does the session get handled if the credentials are changed in ActiveMQ and the session dies and therefore the reconnect fails", but I'm considering that to be too niche for this current branch.

[adeutscher]

Decided to handle that edge case from my previous comment after all.

In this scenario, all of the below points are true:

• ActiveMQ is in use.
• ActiveMQ is configured for subscribe mode
• ActiveMQ credentials for the given account are changed
• ActiveMQ connection is interrupted

These 4 stacking conditions make this pretty niche, but it's still a possibility.

Addressing this by stripping ActiveMQ's regular failover for subscriptions instead of enforcing it. When a connection breaks, it triggers ActiveMQ's ExceptionListener.