OPC UA PubSub Sniffing concepts
Introduction
The sniffing mechanism for OPC UA PubSub allows you to observe network traffic that relates to OPC UA PubSub, and to certain degree, inspect and analyze the contents of the messages. Its uses can be e.g.:
- To discover which OPC UA PubSub participants are active on the network, and what kind of information they provide.
- In some phases of automated OPC UA PubSub discovery process.
- To capture the received message data, for troubleshooting, or for processing of non-standard messages.
The OPC UA PubSub Sniffing is not a substitute for a low-level network analysis, such as that provided by WireShark. Its purpose is somewhat different. Specifically, the OPC UA PubSub Sniffing is not meant to be always able to help with issues related to conformance with OPC UA standards. If the messages received are badly malformed, OPC UA PubSub Sniffing may have problems analyzing them, and that is OK. The OPC UA PubSub Sniffing is designed to help with configuration issues and traffic analysis of otherwise correctly written and operating software.
OPC UA PubSub sniffing produces a stream of capture notifications. Each capture notification contains information about the time the capture has been made (expressed in system ticks, and as date/time).
Capture types
Each notification emitted by the OPC UA PubSub sniffer is qualified with a specific capture type. The capture types available correspond to types of OPC UA PubSub messages, and are described further below. When subscribing to notifications from the OPC UA PubSub sniffer, you can specify which capture types you are interested in. By default, all capture types are emitted.
Capture type "None"
A notification with a special capture type None is emitted when the OPC UA PubSub sniffer successfully initializes the subscription. It is the very first notification that you received from the sniffer. Note: This notification does not mean that a successful connection has been made to the messaging infrastructure.
Individual capture types
The OPC UA PubSub Sniffer emits following non-terminal capture types:
| Name | Display Name | Description |
|---|---|---|
| TransportMessage | Transport Message | A non-terminal message. Represents a full original message, as received by the transport mechanism. |
| NetworkMessage | Network Message | A non-terminal message. Represents a data or discovery message defined by OPC UA PubSub. |
If the transport message cannot be parsed as an OPC UA PubSub message, the TransportMessage is the only capture emitted for the message. Otherwise, there is a NetworkMessage capture emitted following the TransportMessage, containing the network message header information specific to OPC UA PubSub. In special cases (such as with non-standard OPC UA PubSub extensions), there can even be more NetworkMessage captures for one TransportMessage.
The most common NetworkMessage capture is followed by one or more terminal captures of DataSetKeyFrame or similar type (data), but the NetworkMessage can also be followed by one or more terminal captures for discovery probes or announcements. With most discovery messages, there is just one terminal capture emitted. However, separate (possibly multiple) UnparsedAnnouncementDataSetMetaData and/or ProbePublisherDataSetWriter captures are emitted, one for each dataset writer requested in the Publisher information probe. Separate (possibly multiple) AnnouncementDataSetWriter captures are emitted, one for each dataset writer reported in the DataSetWriter configuration announcement message.
The OPC UA PubSub Sniffer emits following terminal capture types:
| Name | Display Name | Description |
|---|---|---|
| UnparsedDataSet | Unparsed DataSet | A terminal message. Represents an unparsed dataset message. |
| DataSetKeyFrame | DataSet Key Frame | A terminal message. Represents a dataset key frame. |
| DataSetDeltaFrame | DataSet Delta Frame | A terminal message. Represents a dataset delta frame. |
| DataSetEvent | DataSet Event | A terminal message. Represents a dataset message carrying an event. |
| DataSetKeepAlive | DataSet Keep-Alive | A terminal message. Represents a dataset keep-alive message. |
| UnparsedDiscoveryProbe | Unparsed Discovery Probe | A terminal message. Represents an unparsed discovery probe message. |
| ProbePublisherServerEndpoints | Probe - Publisher Server Endpoints | A terminal message. Represents a publisher information probe for publisher server endpoints. |
| ProbePublisherDataSetMetaData | Probe - Publisher DataSet MetaData | A terminal message. Represents a publisher information probe for dataset metadata. |
| ProbePublisherDataSetWriter | Probe - Publisher DataSet Writer | A terminal message. Represents a publisher information probe for dataset writer configuration. |
| UnparsedDiscoveryAnnouncement | Unparsed Discovery Announcement | A terminal message. Represents an unparsed discovery announcement message. |
| AnnouncementServerEndpoints | Announcement - Server Endpoints | A terminal message. Represents a (parsed) publisher endpoints announcement message. |
| AnnouncementDataSetMetaData | Announcement - DataSet MetaData | A terminal message. Represents a (parsed) dataset metadata announcement message. |
| AnnouncementDataSetWriter | Announcement - DataSet Writer | A terminal message. Represents a (parsed) dataset writer configuration message. |
| ProbePublisherWriterGroup[1] | Probe - Publisher Writer Group | A terminal message. Represents a publisher information probe for writer group configuration. |
| ProbePublisherPubSubConnections[1] | Probe - Publisher PubSub Connections | A terminal message. Represents a publisher information probe for PubSub connections configuration. |
| ProbeFindApplications[1] | Probe - Find Applications | A terminal message. Represents a (parsed) find applications probe message. |
| AnnouncementPubSubConnection[1] | Announcement - PubSub Connection | A terminal message. Represents a (parsed) PubSub connection configuration announcement message. |
| AnnouncementApplicationDescription[1] | Announcement - Application Description | A terminal message. Represents an application description announcement message. |
| AnnouncementStatus[2] | Announcement - Status | A terminal message. Represents the current operational status of a publisher. |
Capture type groupings
When you are specifying to the OPC UA PubSub Sniffer which messages you are interested in, you will often need a combination of several individual capture types. The commonly used combinations are called capture type groupings, and are available under their own names.
| Name | Display Name | Description |
|---|---|---|
| NetworkTraffic | Network Traffic | A grouping of capture types. Contains all network traffic messages. |
| AllPubSubMessages | All PubSub Messages | A grouping of capture types. Contains all OPC UA PubSub (terminal) messages. |
| AllDataSetMessages | All DataSet Messages | A grouping of capture types. Contains all dataset messages (parsed and unparsed). |
| AllDiscoveryMessages | All Discovery Messages | A grouping of capture types. Contains all discovery messages (probes and announcements). |
| AllDiscoveryProbes | All Discovery Probes | A grouping of capture types. Contains all (parsed and unparsed) discovery probe messages. |
| AllDiscoveryAnnouncements | All Discovery Announcements | A grouping of capture types. Contains all (parsed and unparsed) discovery announcement messages. |
| All | All | A grouping of capture types. Contains all message types (non-terminal and terminal). |
- NetworkTraffic
- This grouping is an opposite of AllPubSubMessages. It is useful when you are interested in the basic physical characteristics of the messages, such as which network traffic is actually present, whether it is actually an OPC UA PubSub traffic, and eventually "who is publishing (or discovering)".
- AllPubSubMessages
- This grouping is an opposite of NetworkTraffic. It is useful when you are not interested in the physical characteristics of the messages, but only in their logical meaning with regard to OPC UA PubSub.
Terminal and non-terminal messages
The capture types denote either non-terminal or terminal messages.
- non-terminal message
- The message is further structured and can contain other non-terminal or terminal messages. Consequently, there can be more captures arising out of a single non-terminal message.
- terminal message
- The message is not structured (or cannot be parsed) to further sub-messages. No further capture will be emitted for the terminal message.
Parsed and unparsed messages
Normally, the capture mechanism parses all parts of the message that are necessary to collect the information provided by the message. When the message or its part is encrypted, it is possible that the capture mechanism will not be able to decrypt (and parse) the full message contents, but will still be able to determine the type of the message and some PubSub header information. Such case is indicated by one of the unparsed capture types.
For more information to when the unparsed capture types are used, see Security and sniffing.
Capture header
The capture header contains the main information collected by the OPC UA PubSub Sniffer for the particular capture. It contains various fields which allow the user to analyze the ongoing PubSub communication. Not all fields are always filled-in. The fields that are filled-in depend on the capture type of the given capture notification.
No capture header fields are filled in with the capture type None. Following fields are filled-in with all other capture types and message mappings: Compressed, Origin, Retain.
| CaptureType | MessageMapping-Name | Configura-tionVersion | DataSet-ClassId | DataSet-WriterId | DataSet-Writer-Name | Encrypted | Publisher-Id | Signed | Writer-GroupId | Writer-GroupName |
|---|---|---|---|---|---|---|---|---|---|---|
| TransportMessage | ||||||||||
| NetworkMessage | Json | [✓] | ✓ | ✓ | ✓ | [✓][2] | ||||
| Uadp | [✓] | [✓] | ✓ | [✓] | ✓ | [✓] | ||||
| UnparsedDataSet | Uadp | [✓] | [✓] | ✓ | ✓ | [✓] | ✓ | [✓] | ||
| DataSetKeyFrame | Json | [✓] | [✓] | [✓] | [✓][1] | ✓ | ✓ | ✓ | [✓] | |
| Uadp | [✓] | [✓] | ✓ | ✓ | [✓] | ✓ | [✓] | |||
| DataSetDeltaFrame | Json | [✓] | [✓] | [✓] | [✓][1] | ✓ | ✓ | ✓ | [✓] | |
| Uadp | [✓] | [✓] | ✓ | ✓ | [✓] | ✓ | [✓] | |||
| DataSetEvent | Json | [✓] | [✓] | [✓] | [✓][1] | ✓ | ✓ | ✓ | [✓] | |
| Uadp | [✓] | [✓] | ✓ | ✓ | [✓] | ✓ | [✓] | |||
| DataSetKeepAlive | Json | [✓] | [✓] | [✓] | [✓][1] | ✓ | ✓ | ✓ | [✓] | |
| Uadp | [✓] | [✓] | ✓ | ✓ | [✓] | ✓ | [✓] | |||
| UnparsedDiscoveryProbe | Uadp | ✓ | ✓ | ✓ | ||||||
| ProbePublisherServerEndpoints | Uadp | ✓ | ✓ | ✓ | ||||||
| ProbePublisherDataSetMetaData | Uadp | ✓ | ✓ | ✓ | ✓ | |||||
| ProbePublisherDataSetWriter | Uadp | ✓ | ✓ | ✓ | ✓ | |||||
| UnparsedDiscoveryAnnouncement | Uadp | ✓ | ✓ | ✓ | ||||||
| AnnouncementServerEndpoints | Uadp, Json[2] | ✓ | ✓ | ✓ | ||||||
| AnnouncementDataSetMetaData | Json | ✓ | ✓ | ✓ | ✓[1] | ✓ | ✓ | ✓ | ||
| Uadp | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ||||
| AnnouncementDataSetWriter | Uadp | ✓ | ✓ | ✓ | ✓ | ✓ | ||||
| ProbePublisherWriterGroup[1] | Uadp | ✓ | ✓ | ✓ | ✓ | |||||
| ProbePublisherPubSubConnections[1] | Uadp | ✓ | ✓ | ✓ | ||||||
| ProbeFindApplications[1] | Uadp | ✓ | ✓ | ✓ | ||||||
| AnnouncementPubSubConnection[1] | Uadp, Json[2] | ✓ | ✓ | ✓ | ||||||
| AnnouncementApplicationDescription[1] | Uadp, Json[2] | ✓ | ✓ | ✓ | ||||||
| AnnouncementStatus[2] | Uadp, Json | ✓ | ✓ | ✓ |
[✓], a checkmark in angle brackets, indicates an information that is optional in the given message mapping.
Fields that are not marked should generally be treated as if they were not filled in with a useful information. The information is either absent, or it can be present in theory, but not in a conformant OPC UA PubSub message, and even when present, it may not correctly relate to the capture type.
Note: The fact that some field is marked as filled-in means that it contains the information that could be present in the message (but it does not have to), or can be derived by other means.
- When the field is optional in the message mapping, and is not present in the message, it will contain a default value (such as 0 for DataSetWriterId).
- Some field values may be implied by the transport or message mapping. For example, the Retain flag is only used with the MQTT transport, and will always be false with UDP or Ethernet transports. Similarly, the Encrypted and Signed flags are only used with the UADP message mapping, and will always be false with the JSON message mapping. The fact that the field has JSON listed under Encrypted and Signed does not imply that the JSON message can support encryption or signing.
The default value is also placed into the fields that are not filled in according to the table above, but such fields should not be interpreted by the consumers, because they never have a significance with the particular capture type.
Special features and considerations
Message mapping
For the OPC UA PubSub sniffer to be able to analyze the messages, it needs to know their message mapping (which includes message encoding). Currently, OPC UA PubSub defines two message mappings: UADP (binary), and JSON. The message mapping is specified to the OPC UA PubSub sniffer by the means (as part of) the transport profile URI. If the message mapping specified to the OPC UA PubSub sniffer does not match the actual mapping used in the received messages, the OPC UA PubSub sniffer will only recognize and emit the transport messages (TransportMessage capture type), but no network messages (NetworkMessage capture type) and other, more detailed capture types.
Thankfully, QuickOPC has a feature that can recognize the message mapping of the received messages automatically, on the fly (and is using its own, special transport profile URIs for that). This feature is on by default whenever an MQTT transport is used, but you can turn it off by specifying one of the standard transport profile URIs for the PubSub connection. The auto-recognition works on a message-by-message basis, which means that not only is the proper message mapping recognized once, but it can actually differ with each message. This comes handy with subscriptions to MQTT topic trees (using topic filters), where some topics may contain messages published in UADP, and some in JSON. For more information, see OPC UA PubSub Automatic Message Mapping Recognition.
Security and sniffing
The OPC UA PubSub messages, or their parts, can be digitally signed (to prevent tampering) and/or encrypted (to prevent eavesdropping). OPC UA PubSub specification provides mechanisms for communication parties to obtain the security keys needed to sign and encrypt the messages on the sender side, and to verify the signatures and decrypt the message son the receiver side.
When using OPC UA PubSub Sniffer, you can specify the information needed for participation in the exchange of security keys. In such case, the sniffer will be able to verify the signatures of the messages, and decrypt them. If the messages are encrypted, but you do not provide the necessary security-related information, the OPC UA PubSub sniffer will only be able to parse the unencrypted parts of the message, and in general you will get less information from the messages as a result.
How precisely the OPC UA PubSub Sniffer deals with signed or encrypted messages depends on the message security mode in security settings.
Dealing with signed messages:
- When the message security mode is SecurityNone, the sniffer attempts to further parse the message, WITHOUT VERIFYING THE SIGNATURE (!).
- Otherwise (when the security mode is SecuritySign or SecuritySignAndEncrypt), the sniffer verifies the signature. When the signature cannot be verified or is incorrect, the sniffer reports an error, and does not emit a corresponding capture.
Dealing with encrypted message:
- When the message security mode is SecurityNone or SecuritySign, the sniffer does not attempt to decrypt the data. Instead, it emits an unparsed capture type, corresponding to the unencrypted parts of the message.
- When the message security mode is SecuritySignAndEncrypt, the sniffer decrypts the message. If successful, it emits a corresponding parse capture type. Otherwise, it reports an error, and does not emit a capture.
Notes:
- If you do not specify any security setting for the OPC UA PubSub sniffer, the message security mode defaults to SecurityNone.
- Only one set of security parameters can be specified with a PubSub connection for the OPC UA PubSub Sniffer. If, for example, messages encrypted using two or more SKS (Security Key Services) are used on the network and you will be capturing them, you need to choose which ones will be decrypted.
Data capture with sniffing
The OPC UA PubSub sniffer also provides you with the original message data, as received by the transport mechanism (e.g. a UDP or Ethernet packet payload, or an MQTT message body). This data is only available with the TransportMessage capture type, and it comes in two forms:
- Raw capture data
- This is always an array of bytes, precisely as received from the transport.
- Capture data
- An object that can either be an array of bytes, for binary-encoded message mappings, or it is a string decoded from the raw array of bytes, for textual message mappings such as JSON.
Transport message transformations
Some transports may not provide exactly the transport message received, but may transform it in some way. In such case, the OPC UA PubSub sniffer will not "see" the very original message as received, but will only be provided and see the transformed message data. This can happen, for example, if the transport provides an option to decompress the data.
Specifically: QuickOPC has an option to detect and auto-decompress data received in GZip format. This option is turned on by default in the MQTT transport, and turned off in the other transports (UDP, Ethernet). This means that when you use MQTT, and a GZip-ped message appears on the input, its capture data will already be decompressed in the TransportMessage capture. The fact that the data has been decompressed is indicated by the fact that the Compressed flag (property) is set in the Capture header.
