Introduction to MQTT: Chapter 10 Sparkplug

Sparkplug: A Standardization of MQTT

If you’ve read my previous articles, or have research MQTT on your own, you’re already aware that MQTT was developed specifically for use in SCADA (Supervisory Control and Data Acquisition) applications and has been finding unique applications in other markets (like Facebook Messenger, for example). As adoption began to take hold, MQTT-based solutions had an increasing need to be interoperable within the industrial automation markets, and (once again) more specifically in the SCADA space.

One of the great benefits of MQTT lies in the flexibility to create namespaces and transmit various payload datatypes and structures. That flexibility, however, comes at a cost of potentially complicated naming conventions and data formatting inconsistencies, which require the user to expend more effort in creating their subscriber/client applications.

In 2020, the Eclipse Foundation announced the launch of the Sparkplug Working Group, which aims to drive the development and adoption of the Eclipse Sparkplug Specification, which “enables the creation of open, interoperable, Industrial IoT (IIoT) solutions utilizing MQTT.”

Sparkplug Definitions

Sparkplug defines MQTT topics namespace, payload, and session state management, greatly reducing that effort and providing standardization of devices from different vendors, which allows clients to automatically discover available data. Further, the Sparkplug specification defines the infrastructure components as follows:

MQTT Server: The MQTT broker, must be 100% compliant with the MQTT v3.1.1 standard.

MQTT EoN Node: An MQTT “Edge of Network” Node is any device, sensor, or hardware that directly connects to the MQTT server with the payload and topic conventions as defined in the Sparkplug specification.

Device/Sensor: The device/sensor is any physical or logical device connected to an EoN node to provide data, process variables, or metrics.

SCADA/IIOT Host: The SCADA/IIOT Host is any MQTT client application that subscribes to and publishes messages. Typically, only one host node is responsible for a group of EoN nodes, although redundancy or backup hosts are supported.

Of note, there are 2 versions of Sparkplug: Sparkplug A and Sparkplug B. For this overview, Sparkplug B is the “current” version.

Session State Management in Sparkplug

In any industrial application, it is important to know the state of the network and devices. In traditional SCADA applications where poll/response mechanisms and protocols were used, the host would periodically request data from a device or sensor and assume the connection had failed if no response was received before a timeout occurred. Using MQTT, the devices periodically publish data without a request from the host, so new features are required to verify a device is connected.

In other words, SCADA/IIoT host could only rely on data arriving reliably if it could be assured of the state of the MQTT session. The Sparkplug specification outlines the use of the MQTT V3.1.1 “Last Will and Testament” feature to provide MQTT session state information to any other interested MQTT client in the infrastructure. The session state awareness is implemented around a set of defined “Birth” and “Death” Topic Namespace and Payload definitions in conjunction with the MQTT connection “Keep-Alive” timer. 

Sparkplug MQTT Topic Namespace

MQTT uses “topics” to identify the payload and are the mechanism by which the broker knows which clients to send the data to. Topic names can be completely arbitrary, such as “Home/Temp” to identify the temperature of your house. By subscribing to the topic “Home/Temp” an MQTT client can receive the temperature value of your living room, but if other users want to understand the data exchange, the Topic Namespace should be more descriptive and standardized.

Sparkplug therefore defines the Topic Namespace structure:

namespace/group_id/message_type/edge_node_id/[device_id]

namespace

The namespace element defines both the structure of the remaining namespace elements as well as the encoding used for the associated payload data. The current Sparkplug specification defines two namespaces.

For the Sparkplug A version of the payload definition, the UTF-8 string constant for the namespace element is: “spAv1.0”

For the Sparkplug™ B version of the specification, the UTF-8 string constant for the namespace element is: “spBv1.0”

group_id

The group_id element of the Topic Namespace allows the grouping of MQTT nodes. In most use cases to minimize bandwidth, it should be descriptive but as small as possible. Group id could be used to identify nodes in a specific production hall or manufacturing line.

The message type element of the Topic Namespace indicates the payload of the message. The encoding of the payload will vary depending on the Sparkplug implementation as (A or B) indicated by the namespace element.

message_type

The following message type elements are defined for the Sparkplug Topic Namespace:

• NBIRTH – An important feature of the Sparkplug implementation for real-time SCADA applications is to ensure the host knows the state or health of a node. The Birth Certificate “NBIRTH” is the first message that an EoN node must publish after establishing a session with a broker. The EoN Birth Certificate payload contains everything required to build out a data structure for all metrics for this EoN node.

• NDEATH – The EoN Death Certificate “NDEATH” topic and is defined and specified. The Will Topic and Will Message registered in the MQTT CONNECT session establishment collectively make up the Death Certificate. Note that the delivery of the Death Certificate upon any MQTT client going offline unexpectedly is part of the MQTT protocol specification, not part of the Sparkplug specification. Immediately upon reception of an EoN Death Certificate, any MQTT client subscribed to this EoN node should set the data quality of all metrics to STALE and note the time stamp of the NDEATH message.

• DBIRTH – The EoN node is responsible for managing all attached devices or sensors. Once the EoN has published its NBIRTH, any Device connected to the EoN node must provide a Birth certificate “DBIRTH” before consumer applications set metrics to a “GOOD” state.

• DDEATH – It is the responsibility of the EoN node to publish a Death certificate for Devices “DDEATH” for any end device which becomes unavailable. Any client subscribed to this data should set the quality of any metrics associated with this device to “STALE” and note the time stamp of when the DDEATH message was received.

• STATE – In systems where multiple Servers are used for redundancy or scalability, the EoN nodes must be aware of the “STATE” of the SCADA/IIOT host. This is accomplished by the host publishing birth and death certificates when a new connection is established. The Birth Certificate payload is the string “ONLINE” while the Last Will and Testament payload of the Death Certificate is “OFFLINE.”

• NDATA – Once an EoN node is online, it will begin publishing data either in report by exception or time-based report mode. The payload data of NDATA messages contain any metric data or process values that need to be reported to subscribing MQTT clients.

• NCMD – The Node command message topic is used to send commands to any EoN nodes. 

• DDATA – Once the EoN node and the devices are online, the device will begin publishing data in the Device data topic “DDATA”.

• DCMD – The Device command topic “DCMD” is used to send commands to any connected device.

edge_node_id

The edge_node_id element uniquely identifies the MQTT EoN node within the infrastructure. The group_id combined with the edge_node_id element must be unique from any other group_id/edge_node_id assigned in the MQTT infrastructure. The topic element edge_node_id travels with every message published and should be as short as possible.

device_id 

The device_id element identifies a device attached (physically or logically) to the MQTT EoN node. The device_id is an optional element within the Topic Namespace as some messages will be either originating or destined to the edge_node_id and the device_id is therefore not required. The device_id must be unique from other devices connected to the same EoN node but can be duplicated from EoN node to other EoN nodes. The device_id element travels with every message published and should be as short as possible.

Sparkplug’s standardized MQTT functionality offers end users a near plug-and-play experience with their IIOT devices, however, it is up to device and host vendors to adopt it. Today, a surprising number of large cloud service providers have brokers that are not compliant with MQTT v3.1.1, which quickly limits Sparkplug’s potential. Further, the number of Sparkplug compatible client applications, including SCADA hosts, MES systems, historians, and analytics engines is also limited, so users need to research their preferred vendors’ offerings before settling on an approach to IIOT, which could include some custom development.