Introducing Share Consumer Sup... Note
Spring

Introducing Share Consumer Support (Kafka Queues) in Spring for Apache Kafka

Apache Kafka 4.0.0 introduces Share Groups as a new consumption model alongside traditional consumer groups. Share Groups distribute individual records, not entire partitions, providing record-level distribution for increased flexibility. This differs from traditional consumer groups where partitions are assigned to specific consumers, guaranteeing order within those partitions. Share Groups are ideal for high-volume, independent events where throughput is prioritized over sequence. They also offer dynamic scaling benefits for workloads with variable demand. Traditional consumer groups remain essential when processing order is critical or for stateful processing requiring partition affinity. In Share Groups, a broker-level Share Coordinator manages record distribution. Records are acquired by consumers with a time-based lock and automatically returned to the pool if not acknowledged within the timeout. Consumers can acknowledge records as ACCEPT, RELEASE for retry, or REJECT for permanent failure. The broker tracks delivery attempts, automatically archiving messages after a configured limit to protect against poison messages. Spring for Apache Kafka 4.0.0 fully supports Share Groups, offering both programmatic container creation and annotation-driven @KafkaListener setups. Configuring a ShareConsumerFactory and ShareKafkaListenerContainerFactory is necessary. Both implicit and explicit acknowledgment modes are supported. Implicit acknowledgment automatically accepts on successful method completion and rejects on exceptions, while explicit acknowledgment allows fine-grained control using a ShareAcknowledgment parameter. Explicit acknowledgment, however, requires all records to be acknowledged before new records can be polled. Share Groups also enable scaling with concurrency within a single container, allowing multiple consumer threads to process records in parallel.