SMQ is an easy to use Message Queues architecture solution that follows the publish subscribe broadcast design pattern. The message queues provide pub/sub features similar to that of historical architectures, however in addition, the newly designed protocol structure includes features that are specifically implemented to simplify today's IoT design.
Clients initiate the IoT communication with the SMQ Broker by use of a standard web URL, which is either HTTP for non secure or HTTPS for secure communication. The device to broker communication is established in a similar fashion to how WebSockets are initiated.
Once a connected session occurs, the SMQ Client may then publish and subscribe to topic names. A topic name can be any string value, but is typically structured in a hierarchical fashion, which is by common comparison equivalent to that of a file system hierarchy found in a UNIX like environment.
Topic names are then translated into numbers by the server, thus enabling fast processing of messages by using lookup tables and values stored on the server. The client subscribes to a topic and the server responds with a topic ID that identifies the topic channel. Each client is also assigned a unique ID by the broker know as the ephemeral topic ID. The ephemeral topic ID is used in a unique way such as "publisher's address" by subscriber, thus enabling each subscriber the ability to send messages directly to a device by using the pehemeral topic ID. The ephemeral topic ID also enables an application to simulate a remote procedure (RPC) call.
The SMQ broker provides easy segmentation of products/customers by enabling an unlimited number of broker instances running in the server. Each Message Queue entry (URL) in the server is associated with one broker instance, thus the number of instances that can be installed in the server at any given time are only limited to any inherent memory limitations.
The WebSocket protocol uses the word "upgrade" when an HTTP(S) connection is morphed into a persistent socket connection. The two C client stacks are also initiating the connection using HTTP and HTTPS.
The difference between the browser and device connections is that a browser HTTP(S) connection is morphed into a WebSocket connection and the device HTTP(S) connection is simply morphed into a standard TCP socket. The WebSocket protocol adds a lot of processing and management specifically designed for browsers, and this management is not required for devices.
Our construction keeps the device code to a minimum. The SMQ Client morphs an HTTP connection into a persistent TCP connection, and SharkMQ morphs an HTTPS connection into a persistent and secure (encrypted) TCP connection.
The SMQ protocol includes the publish/subscribe pattern found in other pub/sub protocols such as MQTT and AMQP. The publish/subscribe pattern (pub/sub) is an alternative to the traditional client-server model, where a client communicates directly with an endpoint. Pub/Sub decouples a client, who is sending a particular message (called publisher) from another client (or more clients), who is receiving the message (called subscriber). Traditional pub/sub is a messaging pattern where senders of messages do not program the messages to be sent directly to specific receivers, but instead characterize published messages into topics without knowledge of which subscribers, if any, there may be. Similarly, subscribers express interest in one or more topics and only receive messages that are of interest, without knowledge of which publishers, if any, there are.
SMQ is designed for IoT communication and extends the traditional pub/sub design pattern to include some additional services such as one-to-one communication and sender's address. One-to-one communication eliminates the need for creating Dynamic Topic Names, such as topic names based on the client's MAC address, when using SMQ for IoT communication.
Messages published by a publisher are sent to the broker, and it is the broker that is responsible for finding the subscriber(s) for the message and sending the message to those subscriber(s). However, we can ignore the broker when we look at the SMQ protocol from a conceptual/abstract perspective. Just remember that messages are always routed via the broker.
Traditional pub/sub protocols use topic names. A client may subscribe to any number of topics and publishers may publish messages to those topics. In SMQ, a topic is a 32 bit number created by the broker. You subscribe to topic names as you would in a traditional pub/sub protocol, however, the topic name (string) is converted, by the broker, into a random 32 bit number. The clients use the number and not the topic name when publishing and subscribing to messages.
A topic name (string) is converted to a number and this number exists for the lifetime of the broker -- in other words, the number will not change and should be considered static. The topic name converted to a number is a unique ID and is referred to as the Topic ID or tid for short.
The broker also creates a unique number for each client connected. This number is referred to as the Ephemeral Topic ID or etid for short. The number is valid as long as the client is connected to the broker. The number will change if the client disconnects and reconnects with the broker.
Sender's address is conceptually similar to caller ID on your phone. Once a person calls you, you acquire the person's phone number and can call the person back at any time.
Sender's Address is a concept that comes from SDL, where SENDER provides the identity of the process instance from which the last consumed signal was sent.
When a client publishes a message, the client's Ephemeral Topic ID (etid) is included in the published message. The etid embedded in a published message is referred to as the Sender's Address or stid for short. A subscriber receiving a message will learn the etid of the sender of the message.
When a subscriber receives a message published to a named topic or to the client's own etid, the client receiving the message learns the etid of the sender. The client may then immediately send a message to this Topic ID, thus simulating a Remote Procedure Call (RPC).
A publisher may publish to the Topic ID of a named topic such as "alarm/sensor1". A publisher may also publish to the etid of any other client as long as the publisher knows the etid of the destination client. We refer to this as one-to-one communication.
A message published to a named topic may have any number of subscribers, but a message published to an etid has only one destination, the owner of the etid. All clients are automatically subscribed to messages sent to their own etid.
A client cannot publish a message, using one-to-one communication, to another client unless the client knows the other client's etid. A client can learn the etid of another client when the client receives a message published to a named topic. There is one exception to this rule; an SMQ client running in the server can learn the topic ID of clients as soon as they connect. An SMQ client, implemented in Lua and running on the server, can for this reason learn the etid of all connected clients without having to receive a message sent to a named topic. An SMQ client running on the server has elevated rights and additional APIs to its disposal.
A client sending messages to an etid should first request the broker to supervise the destination and request the broker to send a "change" notification event to the client should the destination go offline. See supervising subscribers for details.
In object oriented design, it's common to have an object or interface with a number of methods. SMQ provides a similar feature for topics. This optional feature is referred to as a sub-topic and enables fine grained control of messages in IoT design. The sub-topic also enables multiple message types to be sent to an ephemeral tid. The sub-topic is simply a 32 bit number and this number is completely transparent to the broker. The number is included in all published messages and can optionally be used by applications using the SMQ protocol.
Many embedded systems use messages internally. It is common to use #defines for naming message numbers as shown below.
#define MESSAGE_A 100 #define MESSAGE_B 101 #define MESSAGE_C 102
The C code can then simply refer to the defined names instead of the numbers, thus making it easier for the programmer to remember the message types.
Say you are using a pub/sub protocol for designing a temperature alarm system for a nuclear reactor. The temperature sensor publishes data when the temperature is too high. What if the temperature subscriber is offline and there are no subscribers to the temperature topic? What happens then? In a standard pub/sub protocol, the temperature would be discarded and the publisher would have no way of knowing this.
The SMQ protocol includes a feature that enables a publisher to discover how many subscribers are subscribed to a particular topic. The publisher sends an observe request to the broker and the publisher will then get a "change" notification event when the number of subscribers change.
A publisher can get change notification events from named topics and from ephemeral topic IDs. The number of connected subscribers for an ephemeral ID can only be one, which means the client is connected. Receiving a change notification for an ephemeral ID means the client has disconnected/terminated.
The SMQ protocol also includes a feature where an ondrop server side callback function can be installed in the broker. The "ondrop" callback function is called for each message published to a topic with no subscribers. Server side code can then perform various types of actions such as storing the data in a database, send an email to an email address, and so on.
The protocol is provided in source code and can easily be implemented into any computer language when needed.
|SMQ Client||ANSI-C||Non Secure||Device|
|SharkMQ Client||ANSI-C||Secure (TLS)||Device|
|JavaMQ Client||Java||Secure (TLS)||Java & Android|
|smq/broker.lua (*)||Lua||Secure||Lua Broker (server)|
(*) The broker, implemented in Lua, is designed for the cosocket API in the Barracuda Application Server. You can use the broker in any Barracuda Application Server derivative product. In addition, the broker comes pre-integrated with the Mako Server, a Barracuda Application Server derivative product.
The SMQ broker cannot be used on some RTOS ports due to limitations in TCP/IP stack. See limitations for cosockets for details.
|A basic chat client that shows how to use publish and subscribe.|
|Enhanced chat clients that shows how the basic chat can be improved by using one-to-one communication.|
|The SMQ Weather Application illustrates an example of the SMQ Architecture capability as used in web-based control of an IoT Device Thermostat. The example provides cloud-based dynamic control of the client via on-demand content generation and settings manipulation from either a local display or a remote browser-based HMI.|
|documentation||The Christmas Light Controller is a fun project that lets you provide public access to your outdoor lights during the holiday season.|
Download SMQ Source Code and Libraries for the above examples.