Messaging – information sharing – MQTT

MQTT (Message Queuing Telemetry Transport) is an ISO standard messaging protocol, that is designed for connections with remote locations where a “small code footprint” is required or when the network bandwidth is limited. It is a good match for the information exchange between onboard systems, the vehicle gateway, back office solutions and other integrated units in a fleet management system. One of the great benefits with MQTT is the standard, which means that it, compared to proprietary solutions, is easy to connect new units to the system. It is also relatively easy to share information with external systems, thanks to the standardized protocol. Every client just has to have a unique ID and you have to know what information to subscribe to, or publish on.

Developed by IBM

The MQTT protocol was developed by IBM, and the first version was authored in 1999. It works on top of the TCP/IP protocol, which in turn is a set of protocols used on the internet and similar computer networks.

Brokers/servers

The MQTT protocol transmits information through servers called brokers, where clients (publishers and subscribers) can send and receive messages. The subscribers (information receivers) subscribe to topics, which means that they will receive any message from the server (broker) that is posted on certain topics. The publishers (information creators) send information to the server, tagged by topics. The topic structure make it possible for subscribers to receive only the relevant information, while the publisher doesn’t have to know which units that will receive the information. It just sends the information to the broker/server tagged with the right topic. The protocol is known as a publish-subscribe-based messaging protocol, and it always requires a server/broker to receive messages from publishers and deliver the messages to the subscribers. A client can be either a publisher, a subscriber or both.

Several servers/brokers

Clients only interact with a broker, but a system may contain several brokers that exchange data. A server/broker can be a client to another server which means there can be a web of servers transmitting information back and forth between each other, and between their clients and clients of other servers.  

Brokers and the vehicle communication platform

There is an MQTT broker in the Vehicle Gateway, and there are brokers in the back-office system and in other places where data needs to be transmitted. There could for example be a broker located at the office of a customer or in a server hall run by Microsoft or Amazon, connected to a part of the cloud solution. This is one of the benefits of the system, that it is easy to set up another connection, since MQTT is a standardized communication protocol.

What is back-office

It might be in place to explain what the back-office is in this scenario. It is the cloud solution, and it can be hosted at a server at the office of the system supplier, but it can also physically reside in an external server hall, for example in the care of Microsoft or Amazon. The Vehicle Gateway, the back-office and the front end, with the apps and interface solutions towards the user, together form the communication platform.

In the vehicle

The Vehicle Gateway provides the server (broker) function for each of the connected onboard systems, which means that it receives information from connected publisher devices and distributes information to subscribers according to the information topic. One example could be that the GPS device publishes the GPS position, which is then transmitted by the server/broker in the Gateway to any system subscribing to that information topic. The passenger information system receives the information, interprets and translates it into information about, for example the current bus stop for the vehicle, and then publishes the transformed information back to the broker. The broker then transmits the new information to any system or unit subscribing to the connected topic.

Authentication and security

By default the MQTT-brokers are open and any client may publish or subscribe to all topics. It is highly recommended to use some type of authentication (user/password or certificate) to limit the access to an MQTT. Access can be limited to a specific topic -subtree, being read-only or read-write. To secure the communication it is also possible to use SSL to encrypt the communication.

Challenges

One specific challenge is that you need to know exactly what information is transmitted from what units and under what topic, since there is no extra information about the content in itself. This makes it necessary to create subtrees, documenting what should be communicated (CAN data, door opening, odometer data etc.), source device , target device within the topic structure. A subtree is a representation of the communication network and in structure, it resembles a folder catalogue in a computer.

Topic structure

The topic structure is an important part of the MQTT network design. When you want a unit to subscribe to information, you must know what is communicated, in what level of the subtree and under what topic. It is therefore important to know what information is accessible from what level of the structure, so that the unit can subscribe to the right part of the tree with rights to read and/or write. With many clients in a network, one of the big challenges is to create data streams in a logical way.

No defined form of information

Another challenge is that it isn’t defined what you can send over MQTT. This means that you really must know what information to send and receive. This calls for thorough documentation and understanding what type of information each publishing device will create.

Unique ID

It is important that every unit has it’s well defined and unique ID. A client ID clash can easily cause problems, because there is a minimal built-in fault management regarding ID. For example, if a unit enters the network with an ID that is already in use, the current client with that ID will be kicked out with no warning. If the unit that gets kicked out tries to connect back into the network, it will kick the intruder out, and so it will continue. This will cause problems, so you must be absolutely certain that the ID for each unit is unique.

Customer subscription

If another agent, like a customer is to receive information from  a system based on MQTT, for example, they can subscribe to anything under their subtopic. A topic could look something like this: Customer/Greenbus Inc/VehicleId/GPS/json

This way it is easy to connect a customer server, as long as you have a clear understanding of what level the customer should be connected to. It could for example be interesting for a customer to get real time data about GPS or automatic passenger counting.

Information retention

When a publisher unit has a new item of data to distribute, it sends a control message with the data to the connected broker. The broker then distributes the information to any clients that have subscribed to that topic.

If a broker receives a topic for which there are no current subscribers, it will discard the topic unless the publisher indicates that the topic is to be retained. The retention function allows new subscribers to a topic to receive the last published value rather than waiting for the next update from a publisher.When a publishing client first connects to the broker, it can set up a default message to be sent to subscribers if the broker detects that the publishing client has unexpectedly disconnected from the broker. This function is called “last will”.

Example of an MQTT connection (QoS 0) with connect, publish/subscribe, and disconnect. The first message from client B is stored due to the retain flag.

Area of use

MQTT is a good match if there are small amounts of information that are to be transmitted, and not too often. For example, transmitting the engine speed, would create too much information since it changes all the time, but odometer or door opening, are examples of parameters that works well to transmit with MQTT

MQTT and ITxPT

ITxPT is working with standardization and harmonization towards MQTT. The standard isn’t there yet, but it will come. Many solutions move from webservice and non-standard towards MQTT within ITxPT. MQTT is in itself a more standardized way of sharing data.

Good stress results

With the right type of server and setup, MQTT handles information sharing well, and will achieve good stress test results. It is just a matter of using the right server for the right purpose. Some servers are good at handling large amounts of data, and some are built to be small in size, and maybe not with a great capacity in mind.

MQTT origin and development

The original area of use for MQTT has been within the industry to handle communication with sensors within the Internet of Things development. Apart from IBM, which has developed the standard from the start, and maintains development, Microsoft is a player with their Microsoft Azure IoT hub, as well as Amazon Iot, announced in 2015. The development moves fast in this area, since IoT is such a dynamic field of development.

Disclaimer: The content of this blog post is the author’s opinion and doesn’t reflect the opinion of any other person or organisation.