...
Internally queues are implemented as a linked list (QueueEntryList) of nodes (QueueEntry). The linked list is implemented from first principals. It is uses a thread safe and lock-less algorithm (it uses compare and swap operations).
...
When a message is enqueued (using the AbstractQueue#enqueue() method) . This it adds the message to the tail of the queue and notifies a interested subscriber (consumer) about the new message. The connection that owns the consumer is then awoken and events proceed as described above in the Producing Bytes. This is described by Consumer-Queue-Interactions
...
The diagram below shows point to point queue with three subscribers attached.
Messages
Each queue node QueueEntry refers to a ServerMessage. The server message encapsulates:
- Message meta-data (loosely the message's headers)
- Message payload
- Original routing information,
Many QueueEntries may refer to the same ServerMessage. In the case where a incoming message is routed through an exchange to many queues, the QueueEntry point to the same ServerMessage. This means only one copy of the message exists in the Broker, regardless of however many queues refer to it. This is important for topics where the same message may be sent to hundreds of subscribers.
ServerMessage uses a Reference counting system to control its lifecycle. When the reference reaches zero, it knows no one references it and it can safely delete itself.
The ServerMessage refers to StoredMessage. The StoredMessage the backs the underlying message storage. It provides methods that get the content and the metadata. This might return cached copies, or it might cause store operations to fetch the data from the disk.
StoredMessage can be flowed to disk. The Broker (FlowToDiskCheckingTask) responds to memory pressure by flowing messages that are in-memory only (i.e. transient messages) to disk and freeing the cached copies of persistent messages from memory. This approach frees up memory for messages.
Message and Configuration Store
Messages are written to the MessageStore and configuration to the DurableConfigurationStore. It is possible to back these with the same underlying provider or use a different provider for configuration and messages.
There are several store provider implementations:
- JSON - Configuration Store only
- Berkeley BDB JE - Durable Configuration and/or Message Store
- Derby - Durable Configuration and/or Message Store
- JDBC - Durable Configuration and/or Message Store
These interfaces are pluggable.
Management
The Broker exposes two management layers:
...