MaXaM Concepts

Message Storage

Storage/persistence of Avro encoded messages is in a Key-Value State Store (KVSS). The KVSS is so named because for a given key, which represents a topic (STOMP Destination) it stores a single value (in this usage a MESSAGE) at a time. Another term used for such a "queue" is Last Value Queue for example, ActiveMQ last value queue.

Notifications and Message Routing

MaXaM implements a push model so whenever a message store is updated a notification is generated and in turn this generates an update (pushes the updated value) to any subscribers. The STOMP SEND interface to MaXaM delivers a message to be stored by MaXaM and generates a notification that can trigger the delivery of the updated value as a MESSAGE to one or more SUBSCRIBEd STOMP clients as soon as each client is ready to receive a new value.

Last Value Queue Subscriptions

When a SUBSCRIBEr subscribes to a Destination that resolves to a KVSS and a key, this creates a subscription that will attempt to keep the subscriber up to date with the last (latest) value associated with that key. If the subscriber is unable to keep up with every update (and hence every notification) MaXaM avoids sending additional MESSAGES containing updated values for that Destination to the SUBSCRIBEr until the SUBSCRIBEr acknowledges the previous MESSAGE for that Destination.

Typically a client will be SUBSCRIBEd to receive updates for a large number of distinct keys in the KVSS, so it is expected that while awaiting an acknowledgement for one key, a MESSAGE about an update to another key, which is not already awaiting an acknowledgement can and will be sent to the subscriber. The subscription logic for a given client connection attempts to send values for keys that have notifications active but not sent (to that client) before acting on any acknowledgements from the client, so as to ensure that the values for all subscribed keys are kept equally up-to-date at the client regardless of individual key update frequency.

If a SUBSCRIBEr makes a durable subscription to a KVSS Destination, and disconnects/reconnects, that SUBSCRIBEr will receive the last (latest) MESSAGE for every such Destination it is durably subscribed to. This will include MESSAGEs that have not actually changed since the SUBSCRIBEr disconnected. Typically, SUBSCRIBERs stay connected, and if they are disconnected it is highly likely that many values will have changed. Handling last-value subscriptions in this way allows MaXaM to achieve a higher message throughput while connected to clients, at the cost of some redundant messages on reconnect, which should be rare (connections should be long lived).

It also reduces the risk that any data will be missed/lost across a disconnect that is actually a client restart if the client does not perform fully durable transactional acknowledgement. This can make it easier to implement a high performance client.

Representing (Publishing) Removal of Values

A topic that represents some entity / attribute that has been logically removed will be updated to reflect its final/removed status, rather than simply vanishing. The representation of this status should typically not be a null value but rather the last known status. The way a particular topic represents this status is outside the scope of MaXaM itself as it depends on individual topic semantics. Topic SENDers and SUBSCRIBERs must agree on removal / deletion semantics similarly to any other message semantics. Topic SENDers should include a Tombstone content element in any message that represents such a status so as to provide a general way to recognise such final values. The Tombstone content element value is a timestamp used to manage purging of final values.

Handling actual Deletion (purging) of KVSS Keys

Clients must be aware that the system will delete entries in the KVSS under various circumstances. To a SUBSCRIBER, this will appear as a topic that has stopped being updated. Even when Tombstone element is used (see [Representing (Publishing) Removal of Values](#Representing (Publishing) Removal of Values) above) such entries will eventually be purged (actually deleted). There is a chance such a purge could occur without a given SUBSCRIBER ever seeing the Tombstone state of the topic. SUBSCRIBERs should implement mechanisms of their own to purge zombie entities that will never be seen again from the SENDer. A subscriber can detect zombies by setting eager:True when creating a distinct/new subscription, or simply ending and resuming a durable SUBSCRIPTION. The eager:true subscription will cause the SUBSCRIBER to receive the last value for all subscribed topic(s). A topic for which no data is received can be assumed to have been deleted.