Kafka为什么不像redis和mysql可以支持主从分离呢(Why does Kafka not support master-slave separation like redis and MySQL)


第三、Kafka副本机制使用的是异步消息拉取,因此存在leader和follower之间的不一致性。如果要采用读写分离,必然要处理副本lag引入的一致性问题,比如如何实现read-your-writes、如何保证单调读(monotonic reads)以及处理消息因果顺序颠倒的问题。相反地,如果不采用读写分离,所有客户端读写请求都只在Leader上处理也就没有这些问题了——当然最后全局消息顺序颠倒的问题在Kafka中依然存在,常见的解决办法是使用单分区,其他的方案还有version vector,但是目前Kafka没有提供。


Author: huxihx
Link: https://www.zhihu.com/question/327925275/answer/705690755
Source: Zhihu

First of all, it is clear that there is no absolute advantage or disadvantage whether the master and slave are separated or not. It is only an architecture design, and each has its own applicable scenarios.
Second, as you said, both redis and MySQL support master-slave read-write separation. Personally, I think this is related to their use scenarios. For the load type with many read operations and relatively infrequent write operations, the use of read-write separation is a very good scheme – we can add many followers to expand horizontally to improve the performance of read operations. In contrast, Kafka’s main scenario is to provide external reading services in the message engine rather than in the form of data storage. It usually involves frequent production and consumption of messages, which does not belong to the typical scenario of reading more and writing less. Therefore, the read-write separation scheme is not suitable in this scenario.
Third, Kafka replica mechanism uses asynchronous message pull, so there is inconsistency between leader and follower. If you want to use read-write separation, you must deal with the consistency problems introduced by replica lag, such as how to implement read-your-writes, how to ensure monotonic reads, and how to deal with the reversal of causal order of messages. On the contrary, if the read-write separation is not adopted, all client read-write requests are processed only on the leader, which will eliminate these problems – of course, the problem of reversing the global message order still exists in Kafka. The common solution is to use a single partition. Other solutions include version vector, but Kafka does not provide it at present.