so finally something really good happened from the redis criticism thread. at the end of the work day i was reading about redis as ap and merge operations on twitter. at the same time i was having a private email exchange with alexis richa
so finally something really good happened from the redis criticism thread.
at the end of the work day i was reading about redis as ap and merge operations on twitter. at the same time i was having a private email exchange with alexis richardson (from rabbitmq, and, my boss). alexis at some point proposed that perhaps a way to improve safety was to asynchronously ack the client about what commands actually were not received so that the client could retry. this seemed a lot of efforts in the client side, but somewhat totally opened my view on the matter.
so the idea is, we can't go for synchronous replication, but indeed we get acks from the replicas, asynchronous acks, specifically.
what about retaining all the writes not acknowledged into a buffer, and re-play them to the current master when the partition heals?
the window we need to require to take the log is very small if the acks are frequent enough (currently the frequency is 1 per second, but this could be more easily).
if we give up availability after n times the window we can say, ok, no more room, we now start to reply with errors to queries.
the huge difference with this approach is that this works regardless of the size of values. there are also semantical differences since the stream of operations is preserved instead of the value itself, so there is more context. think for example about incr.
of course this would not work for anything, but one could mark in the command table what command to reply and what to discard. sadd is an example of perfect command since the order of operations does not matter. del is likely something to avoid replying. and so forth. in turn if we reply against the wrong (stale) master, it will accumulate the commands and so forth. details may vary, but this is the first thing that really makes a difference.
probably many of you that are into eventually consistent databases know about the log vs merge strategies already, but i had to re-invent the wheel as i was not aware. this is the kind of feedback i expected in the redis thread that i did not received.
another cool thing about this approach is that it's pretty opt-in, it can be just a state in the connection. send a command and the connection is of safe type, so all the commands sent will be retained and replayed if not acknowledged, and so forth.
this is not going to be in the first version of redis cluster as i'm more happy to ship asap the current design, but it is a solid incremental idea that could be applied later, so a little actual result into the evolution of the design.comments