/v1/console: prevent concurrent use of the same session by multiple requests

[julianbrost]

If there are such requests, without this change, they would all be allowed and processed, resulting in unsafe concurrent (write) access to these data structures, which can ultimately crash the daemon or lead to other unintended behavior.

This PR is a follow-up to #10675 adding more required locking.

Tests

I was able to somewhat reliably trigger that race condition using the following command (ab is ApacheBench):

ab -n 100000 -c 64 -A root:icinga -m POST -H 'Accept: application/json' 'https://localhost:5665/v1/console/execute-script?command=1&session=always-the-same'

Before (0f44133)

With the current master, I was able to observe Icinga 2 crashing with a SIGSEGV (typically after around ~10k to ~30k):

Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007fcdeaf99752 in std::_Rb_tree_insert_and_rebalance(bool, std::_Rb_tree_node_base*, std::_Rb_tree_node_base*, std::_Rb_tree_node_base&) ()
   from /lib/x86_64-linux-gnu/libstdc++.so.6
[Current thread is 1 (Thread 0x7fcde91576c0 (LWP 186))]
(gdb) set pagination off
(gdb) bt
#0  0x00007fcdeaf99752 in std::_Rb_tree_insert_and_rebalance(bool, std::_Rb_tree_node_base*, std::_Rb_tree_node_base*, std::_Rb_tree_node_base&) ()
   from /lib/x86_64-linux-gnu/libstdc++.so.6
#1  0x000055e6996dc065 in std::_Rb_tree_iterator<std::pair<icinga::String const, icinga::String> > std::_Rb_tree<icinga::String, std::pair<icinga::String const, icinga::String>, std::_Select1st<std::pair<icinga::String const, icinga::String> >, std::less<icinga::String>, std::allocator<std::pair<icinga::String const, icinga::String> > >::_M_emplace_hint_unique<std::piecewise_construct_t const&, std::tuple<icinga::String const&>, std::tuple<> >(std::_Rb_tree_const_iterator<std::pair<icinga::String const, icinga::String> >, std::piecewise_construct_t const&, std::tuple<icinga::String const&>&&, std::tuple<>&&) [clone .isra.0] ()
#2  0x000055e69975093e in std::map<icinga::String, icinga::String, std::less<icinga::String>, std::allocator<std::pair<icinga::String const, icinga::String> > >::operator[](icinga::String const&) ()
#3  0x000055e69970a982 in icinga::ConsoleHandler::ExecuteScriptHelper(icinga::HttpApiRequest const&, icinga::HttpApiResponse&, icinga::String const&, icinga::String const&, bool) ()
#4  0x000055e69970b32a in icinga::ConsoleHandler::HandleRequest(boost::intrusive_ptr<icinga::WaitGroup> const&, icinga::HttpApiRequest const&, icinga::HttpApiResponse&, boost::asio::basic_yield_context<boost::asio::executor>&) ()
#5  0x000055e6996f02cb in icinga::HttpHandler::ProcessRequest(boost::intrusive_ptr<icinga::WaitGroup> const&, icinga::HttpApiRequest&, icinga::HttpApiResponse&, boost::asio::basic_yield_context<boost::asio::executor>&) ()
#6  0x000055e6997163e0 in icinga::HttpServerConnection::ProcessMessages(boost::asio::basic_yield_context<boost::asio::executor>) ()
#7  0x000055e699717dbe in boost::coroutines::detail::pull_coroutine_object<boost::coroutines::push_coroutine<void>, void, boost::asio::detail::spawned_coroutine_thread::entry_point<boost::asio::detail::old_spawn_entry_point<boost::asio::io_context::strand, icinga::IoEngine::SpawnCoroutine<boost::asio::io_context::strand, icinga::HttpServerConnection::Start()::{lambda(boost::asio::basic_yield_context<boost::asio::executor>)#1}>(boost::asio::io_context::strand&, icinga::HttpServerConnection::Start()::{lambda(boost::asio::basic_yield_context<boost::asio::executor>)#1})::{lambda(boost::asio::basic_yield_context<boost::asio::executor>)#1}, void (*)()> >, boost::coroutines::basic_standard_stack_allocator<boost::coroutines::stack_traits> >::run() ()
#8  0x00007fcdeabb718f in make_fcontext () from /lib/x86_64-linux-gnu/libboost_context.so.1.83.0
#9  0x0000000000000000 in ?? ()

However, I also observed instances where Icinga 2 stopped responding to ab.

After (0d376b5)

With this PR, Icinga 2 now survives the ab-torture (even multiple runs, so more than 100k requests).

[jschmidt-icinga]

Makes sense.

[backbot-ci]

Backport failed for support/2.15, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin support/2.15
git worktree add -d .worktree/backport-10681-to-support/2.15 origin/support/2.15
cd .worktree/backport-10681-to-support/2.15
git switch --create backport-10681-to-support/2.15
git cherry-pick -x 0d376b5d5a64599d2f641471ac08c6d1678fdbbf

[backbot-ci]

Successfully created backport PR for support/2.15:

• [Backport support/2.15] /v1/console: prevent concurrent use of the same session by multiple requests #10755