Kafka为什么不像redis和mysql可以支持主从分离呢(Why does Kafka not support master-slave separation like redis and MySQL)

作者:huxihx
链接:https://www.zhihu.com/question/327925275/answer/705690755
来源:知乎

首先明确一下:主从分离与否没有绝对的优劣,它仅仅是一种架构设计,各自有适用的场景。
第二、如你所说,Redis和MySQL都支持主从读写分离,我个人觉得这和它们的使用场景有关。对于那种读操作很多而写操作相对不频繁的负载类型而言,采用读写分离是非常不错的方案——我们可以添加很多follower横向扩展,提升读操作性能。反观Kafka,它的主要场景还是在消息引擎而不是以数据存储的方式对外提供读服务,通常涉及频繁地生产消息和消费消息,这不属于典型的读多写少场景,因此读写分离方案在这个场景下并不太适合。
第三、Kafka副本机制使用的是异步消息拉取,因此存在leader和follower之间的不一致性。如果要采用读写分离,必然要处理副本lag引入的一致性问题,比如如何实现read-your-writes、如何保证单调读(monotonic reads)以及处理消息因果顺序颠倒的问题。相反地,如果不采用读写分离,所有客户端读写请求都只在Leader上处理也就没有这些问题了——当然最后全局消息顺序颠倒的问题在Kafka中依然存在,常见的解决办法是使用单分区,其他的方案还有version vector,但是目前Kafka没有提供。

————————

Author: huxihx
Link: https://www.zhihu.com/question/327925275/answer/705690755
Source: Zhihu

First of all, it is clear that there is no absolute advantage or disadvantage whether the master and slave are separated or not. It is only an architecture design, and each has its own applicable scenarios.
Second, as you said, both redis and MySQL support master-slave read-write separation. Personally, I think this is related to their use scenarios. For the load type with many read operations and relatively infrequent write operations, the use of read-write separation is a very good scheme – we can add many followers to expand horizontally to improve the performance of read operations. In contrast, Kafka’s main scenario is to provide external reading services in the message engine rather than in the form of data storage. It usually involves frequent production and consumption of messages, which does not belong to the typical scenario of reading more and writing less. Therefore, the read-write separation scheme is not suitable in this scenario.
Third, Kafka replica mechanism uses asynchronous message pull, so there is inconsistency between leader and follower. If you want to use read-write separation, you must deal with the consistency problems introduced by replica lag, such as how to implement read-your-writes, how to ensure monotonic reads, and how to deal with the reversal of causal order of messages. On the contrary, if the read-write separation is not adopted, all client read-write requests are processed only on the leader, which will eliminate these problems – of course, the problem of reversing the global message order still exists in Kafka. The common solution is to use a single partition. Other solutions include version vector, but Kafka does not provide it at present.