Current implementation of certain commands is not atomic

[bixuanzju]

For example INCR command 1) get the value 2) increment 3) put new value in the store, which would lead to data races when multiple clients perform the same operation concurrently.

The easiest fix would be to add mutex around those commands, not sure if you would agree.

[charlesgardyn]

I've been thinking of 2 options for the future of the system:

1. Make it completely single threaded, use background threads only for async flush, like AOF or RDB.
   a. Replace the current TCP server to use io_uring / kqueue for async IO for that to work well.
   b. Tigerbeetle does that and reaches almost a million transactions per second.
2. Make it multithreaded using a shared nothing architecture, like what DragonflyDB does. Each thread has its key/value storage, so they do not access each other data.
   a. It's not easy to do. Specially in multi keys commands, like KEYS, or SCAN.
   b. Mutex is terrible for performance, and we should avoid it at any cost. I've been reading some papers about state-of-the-art lock managers, like this one, but I'm still learning it.

Let me know what you think! Thanks for engaging with the project!

[barddoo]

Started the work here: #37

[barddoo]

This release solves it. https://github.com/barddoo/zedis/releases/tag/0.1.0

[bixuanzju]

Haven’t looked closely but the release note says you switched to std.Io.Threaded, so we still allow multi-threaded access to the store — how does it solve the issue?

[barddoo]

It uses Io.Threaded because Io.Evented doesn’t have network yet.
The current implementation uses a queue to publish the commands from the Io threads, so the storage itself is single thread.
I’m still experimenting with shared nothing on this: #37. It’s much more efficient, but it’s a huge undertaking, specially for multi key commands.

[bixuanzju]

Understood thx!