aspnetcore/specs/HubProtocol.md

33 KiB

SignalR Hub Protocol

The SignalR Protocol is a protocol for two-way RPC over any Message-based transport. Either party in the connection may invoke procedures on the other party, and procedures can return zero or more results or an error.

Terms

  • Caller - The node that is issuing an Invocation message and receiving Completion, StreamItem and StreamCompletion messages (a node can be both Caller and Callee for different invocations simultaneously)
  • Callee - The node that is receiving an Invocation message and issuing Completion, StreamItem and StreamCompletion messages (a node can be both Callee and Caller for different invocations simultaneously)
  • Binder - The component on each node that handles mapping Invocation messages to method calls and return values to Completion, StreamItem and StreamCompletion messages

Transport Requirements

The SignalR Protocol requires the following attributes from the underlying transport.

  • Reliable, in-order, delivery of messages - Specifically, the SignalR protocol provides no facility for retransmission or reordering of messages. If that is important to an application scenario, the application must either use a transport that guarantees it (i.e. TCP) or provide their own system for managing message order.

Overview

This document describes three encodings of the SignalR protocol: JSON, MessagePack and Protocol Buffers. Only one format can be used for the duration of a connection, and the format must be negotiated after opening the connection and before sending any other messages. However, each format shares a similar overall structure.

In the SignalR protocol, the following types of messages can be sent:

  • Negotiation Message - Sent by the client to negotiate the message format.
  • Invocation Message - Indicates a request to invoke a particular method (the Target) with provided Arguments on the remote endpoint.
  • StreamItem Message - Indicates individual items of streamed response data from a previous Invocation message.
  • Completion Message - Indicates a previous Invocation has completed. Contains an error if the invocation concluded with an error, or the result of method invocation. The result will be absent for void methods.
  • StreamCompletion Message - Indicates a previous Invocation has completed, and no further StreamItem messages will be received. Contains an error if the invocation concluded with an error.
  • CancelInvocation Message - Sent by the client to cancel a streaming invocation on the server.

After opening a connection to the server the client must send a Negotiation message to the server as its first message. The negotiation message is always a JSON message and contains the name of the format (protocol) that will be used for the duration of the connection. If the server does not support the protocol requested by the client or the first message received from the client is not a Negotiation message the server must close the connection.

The Negotiation message contains the following properties:

  • protocol - the name of the protocol to be used for messages exchanged between the server and the client

Example:

{
    "protocol": "messagepack"
}

Communication between the Caller and the Callee

There are three kinds of interactions between the Caller and the Callee:

  • Invocations - the Caller sends a message to the Callee and expects a message indicating that the invocation has been completed and optionally a result of the invocation
  • Non-Blocking Invocations - the Caller sends a message to the Callee and does not expect any further messages for this invocation
  • Streaming Invocations - the Caller sends a message to the Callee and expects one or more results returned by the Callee followed by a message indicating the end of invocation

Invocations

In order to perform a single invocation, the Caller follows the following basic flow:

  1. Allocate a unique Invocation ID value (arbitrary string, chosen by the Caller) to represent the invocation
  2. Send an Invocation message containing the Invocation ID, the name of the Target being invoked, and the Arguments to provide to the method.
  3. If the Invocation is marked as non-blocking (see "Non-Blocking Invocations" below), stop here and immediately yield back to the application.
  4. Wait for a StreamItem, Completion or StreamCompletion message with a matching Invocation ID
  5. If a Completion or StreamCompletion message arrives, go to 8
  6. If the StreamItem message has a payload, dispatch the payload to the application (i.e. by yielding a result to an IObservable, or by collecting the result for dispatching in step 8)
  7. Go to 4
  8. Complete the invocation, dispatching the final payload item (if any) or the error (if any) to the application

The Target of an Invocation message must refer to a specific method, overloading is not permitted. In the .NET Binder, the Target value for a method is defined as the simple name of the Method (i.e. without qualifying type name, since a SignalR endpoint is specific to a single Hub class). Target is case-sensitive

NOTE: Invocation IDs are arbitrarily chosen by the Caller and the Callee is expected to use the same string in all response messages. Callees may establish reasonable limits on Invocation ID lengths and terminate the connection when an Invocation ID that is too long is received.

Non-Blocking Invocations

Invocations can be marked as "Non-Blocking" in the Invocation message, which indicates to the Callee that the Caller expects no results. When a Callee receives a "Non-Blocking" Invocation, it should dispatch the message, but send no results or errors back to the Caller. In a Caller application, the invocation will immediately return with no results. There is no tracking of completion for Non-Blocking Invocations.

Streaming

The SignalR protocol allows for multiple StreamItem messages to be transmitted in response to an Invocation message, and allows the receiver to dispatch these results as they arrive, to allow for streaming data from one endpoint to another.

On the Callee side, it is up to the Callee's Binder to determine if a method call will yield multiple results. For example, in .NET certain return types may indicate multiple results, while others may indicate a single result. Even then, applications may wish for multiple results to be buffered and returned in a single Completion frame. It is up to the Binder to decide how to map this. The Callee's Binder must encode each result in separate StreamItem messages, indicating the end of results by sending a StreamCompletion message.

On the Caller side, the user code which performs the invocation indicates how it would like to receive the results and it is up the Caller's Binder to handle the result. If the Caller expects only a single result, but multiple results are returned, or if the caller expects multiple results but only one result is returned, the Caller's Binder should yield an error. If the Caller wants to stop receiving StreamItem messages before the Callee sends a StreamCompletion message, the Caller can send a CancelInvocation message with the same Invocation ID used for the Invocation message that started the stream. It is possible to receive StreamItem messages or a StreamCompletion message after a CancelInvocation message has been sent, these can be ignored.

Completion and results

An Invocation is only considered completed when the Completion or StreamCompletion message is received. Receiving any message using the same Invocation ID after a Completion or StreamCompletion message has been received for that invocation is considered a protocol error and the recipient may immediately terminate the connection.

If a Callee is going to stream results, it MUST send each individual result in a separate StreamItem message, and complete the invocation with a StreamCompletion. If the Callee is going to return a single result, it MUST not send any StreamItem messages, and MUST send the single result in a Completion message. This is to ensure that the Caller can unambiguously determine the intended streaming behavior of the method.

Errors

Errors are indicated by the presence of the error field in a Completion or StreamCompletion message. Errors always indicate the immediate end of the invocation. In the case of streamed responses, the arrival of a StreamCompletion message indicating an error should not stop the dispatching of previously-received results. The error is only yielded after the previously-received results have been dispatched.

If either endpoint commits a Protocol Error (see examples below), the other endpoint may immediately terminate the underlying connection.

  • It is a protocol error for any message to be missing a required field, or to have an unrecognized field.
  • It is a protocol error for a Caller to send a StreamItem, Completion or StreamCompletion message with an Invocation ID that has not been received in an Invocation message from the Callee
  • It is a protocol error for a Caller to send a StreamItem, Completion or StreamCompletion message in response to a Non-Blocking Invocation (see "Non-Blocking Invocations" above)
  • It is a protocol error for a Caller to send a Completion message when a StreamItem message has previously been sent for the same Invocation ID.
  • It is a protocol error for a Caller to send a Completion message carrying both a result and an error.
  • It is a protocol error for an Invocation message to have an Invocation ID that has already been used by that endpoint. However, it is not an error for one endpoint to use an Invocation ID that was previously used by the other endpoint (allowing each endpoint to track it's own IDs).

Examples

Consider the following C# methods

public int Add(int x, int y)
{
    return x + y;
}

public int SingleResultFailure(int x, int y)
{
    throw new Exception("It didn't work!");
}

public IEnumerable<int> Batched(int count)
{
    for(var i = 0; i < count; i++)
    {
        yield return i;
    }
}

[return: Streamed] // This is a made-up attribute that is used to indicate to the .NET Binder that it should stream results
public IEnumerable<int> Stream(int count)
{
    for(var i = 0; i < count; i++)
    {
        yield return i;
    }
}

[return: Streamed] // This is a made-up attribute that is used to indicate to the .NET Binder that it should stream results
public IEnumerable<int> StreamFailure(int count)
{
    for(var i = 0; i < count; i++)
    {
        yield return i;
    }
    throw new Exception("Ran out of data!");
}

private List<string> _callers = new List<string>();
public void NonBlocking(string caller)
{
    _callers.Add(caller);
}

In each of the below examples, lines starting C->S indicate messages sent from the Caller ("Client") to the Callee ("Server"), and lines starting S->C indicate messages sent from the Callee ("Server") back to the Caller ("Client"). Message syntax is just a pseudo-code and is not intended to match any particular encoding.

Single Result (Add example above)

C->S: Invocation { Id = 42, Target = "Add", Arguments = [ 40, 2 ] }
S->C: Completion { Id = 42, Result = 42 }

NOTE: The following is NOT an acceptable encoding of this invocation:

C->S: Invocation { Id = 42, Target = "Add", Arguments = [ 40, 2 ] }
S->C: StreamItem { Id = 42, Item = 42 }
S->C: Completion { Id = 42 }

Single Result with Error (SingleResultFailure example above)

C->S: Invocation { Id = 42, Target = "SingleResultFailure", Arguments = [ 40, 2 ] }
S->C: Completion { Id = 42, Error = "It didn't work!" }

Batched Result (Batched example above)

C->S: Invocation { Id = 42, Target = "Batched", Arguments = [ 5 ] }
S->C: Completion { Id = 42, Result = [ 0, 1, 2, 3, 4 ] }

Streamed Result (Stream example above)

C->S: Invocation { Id = 42, Target = "Stream", Arguments = [ 5 ] }
S->C: StreamItem { Id = 42, Item = 0 }
S->C: StreamItem { Id = 42, Item = 1 }
S->C: StreamItem { Id = 42, Item = 2 }
S->C: StreamItem { Id = 42, Item = 3 }
S->C: StreamItem { Id = 42, Item = 4 }
S->C: StreamCompletion { Id = 42 }

NOTE: The following is NOT an acceptable encoding of this invocation:

C->S: Invocation { Id = 42, Target = "Stream", Arguments = [ 5 ] }
S->C: StreamItem { Id = 42, Item = 0 }
S->C: StreamItem { Id = 42, Item = 1 }
S->C: StreamItem { Id = 42, Item = 2 }
S->C: StreamItem { Id = 42, Item = 3 }
S->C: Completion { Id = 42, Result = 4 }

This is invalid because the Completion is not a valid message to complete a streaming invocation.

Streamed Result with Error (StreamFailure example above)

C->S: Invocation { Id = 42, Target = "Stream", Arguments = [ 5 ] }
S->C: StreamItem { Id = 42, Item = 0 }
S->C: StreamItem { Id = 42, Item = 1 }
S->C: StreamItem { Id = 42, Item = 2 }
S->C: StreamItem { Id = 42, Item = 3 }
S->C: StreamItem { Id = 42, Item = 4 }
S->C: StreamCompletion { Id = 42, Error = "Ran out of data!" }

This should manifest to the Calling code as a sequence which emits 0, 1, 2, 3, 4, but then fails with the error Ran out of data!.

Streamed Result closed early (Stream example above)

C->S: Invocation { Id = 42, Target = "Stream", Arguments = [ 5 ] }
S->C: StreamItem { Id = 42, Item = 0 }
S->C: StreamItem { Id = 42, Item = 1 }
C->S: CancelInvocation { Id = 42 }
S->C: StreamItem { Id = 42, Item = 2} // This can be ignored

Non-Blocking Call (NonBlocking example above)

C->S: Invocation { Id = 42, Target = "NonBlocking", Arguments = [ "foo" ], NonBlocking = true }

JSON Encoding

In the JSON Encoding of the SignalR Protocol, each Message is represented as a single JSON object, which should be the only content of the underlying message from the Transport. All property names are case-sensitive. The underlying protocol is expected to handle encoding and decoding of the text, so the JSON string should be encoded in whatever form is expected by the underlying transport. For example, when using the ASP.NET Sockets transports, UTF-8 encoding is always used for text.

Invocation Message Encoding

An Invocation message is a JSON object with the following properties:

  • type - A Number with the literal value 1, indicating that this message is an Invocation.
  • invocationId - A String encoding the Invocation ID for a message.
  • nonblocking - A Boolean indicating if the invocation is Non-Blocking (see "Non-Blocking Invocations" above). Optional and defaults to false if not present.
  • target - A String encoding the Target name, as expected by the Callee's Binder
  • arguments - An Array containing arguments to apply to the method referred to in Target. This is a sequence of JSON Tokens, encoded as indicated below in the "JSON Payload Encoding" section

Example:

{
    "type": 1,
    "invocationId": "123",
    "target": "Send",
    "arguments": [
        42,
        "Test Message"
    ]
}

Example (Non-Blocking):

{
    "type": 1,
    "invocationId": "123",
    "nonblocking": true,
    "target": "Send",
    "arguments": [
        42,
        "Test Message"
    ]
}

StreamItem Message Encoding

A StreamItem message is a JSON object with the following properties:

  • type - A Number with the literal value 2, indicating that this message is a StreamItem.
  • invocationId - A String encoding the Invocation ID for a message.
  • item - A Token encoding the stream item (see "JSON Payload Encoding" for details).

Example

{
    "type": 2,
    "invocationId": "123",
    "item": 42
}

Completion Message Encoding

A Completion message is a JSON object with the following properties

  • type - A Number with the literal value 3, indicating that this message is a Completion.
  • invocationId - A String encoding the Invocation ID for a message.
  • result - A Token encoding the result value (see "JSON Payload Encoding" for details). This field is ignored if error is present.
  • error - A String encoding the error message.

It is a protocol error to include both a result and an error property in the Completion message. A conforming endpoint may immediately terminate the connection upon receiving such a message.

Example - A Completion message with no result or error

{
    "type": 3,
    "invocationId": "123"
}

Example - A Completion message with a result

{
    "type": 3,
    "invocationId": "123",
    "result": 42
}

Example - A Completion message with an error

{
    "type": 3,
    "invocationId": "123",
    "error": "It didn't work!"
}

Example - The following Completion message is a protocol error because it has both of result and error

{
    "type": 3,
    "invocationId": "123",
    "result": 42,
    "error": "It didn't work!"
}

StreamCompletion Message Encoding

A StreamCompletion message is a JSON object with the following properties

  • type - A Number with the literal value 4, indicating that this message is a StreamCompletion.
  • invocationId - A String encoding the Invocation ID for a message.
  • error - A String encoding the error message.

Example - A StreamCompletion message with no error

{
    "type": 4,
    "invocationId": "123"
}

Example - A StreamCompletion message with an error

{
    "type": 4,
    "invocationId": "123",
    "error": "It didn't work!"
}

CancelInvocation Message Encoding

A CancelInvocation message is a JSON object with the following properties

  • type - A Number with the literal value 5, indicationg that this is a CancelInvocation.
  • invocationId - A String encoding the Invocation ID for a message.

Example

{
    "type": 5,
    "invocationId": "123"
}

JSON Payload Encoding

Items in the arguments array within the Invocation message type, as well as the item value of the StreamItem message and the result value of the Completion message, encode values which have meaning to each particular Binder. A general guideline for encoding/decoding these values is provided in the "Type Mapping" section at the end of this document, but Binders should provide configuration to applications to allow them to customize these mappings. These mappings need not be self-describing, because when decoding the value, the Binder is expected to know the destination type (by looking up the definition of the method indicated by the Target).

JSON payloads are separated with the record separator (0x1e) to ease the parsing.

MessagePack (MsgPack) encoding

In the MsgPack Encoding of the SignalR Protocol, each Message is represented as a single MsgPack array containing items that correspond to properties of the given hub protocol message. The array items may be primitive values, arrays (e.g. method arguments) or objects (e.g. argument value). The first item in the array is the message type.

MessagePack uses different formats to encode values. Refer to the MsgPack format spec for format definitions.

Invocation Message Encoding

Invocation messages have the following structure:

[1, InvocationId, NonBlocking, Target, [Arguments]]
  • 1 - Message Type - 1 indicates this is an Invocation message
  • InvocationId - A String encoding the Invocation ID for the message
  • NonBlocking - A Boolean indicating if the invocation is Non-Blocking (see "Non-Blocking Invocations" above)
  • Target - A String encoding the Target name, as expected by the Callee's Binder
  • Arguments - An Array containing arguments to apply to the method referred to in Target.

Example:

The following payload

0x95 0x01 0xa3 0x78 0x79 0x7a 0xc3 0xa6 0x6d 0x65 0x74 0x68 0x6f 0x64 0x91 0x2a

is decoded as follows:

  • 0x95 - 5-element array
  • 0x01 - 1 (Message Type - Invocation message)
  • 0xa3 - string of length 3 (InvocationId)
  • 0x78 - x
  • 0x79 - y
  • 0x7a - z
  • 0xc3 - true (NonBlocking)
  • 0xa6 - string of length 6 (Target)
  • 0x6d - m
  • 0x65 - e
  • 0x74 - t
  • 0x68 - h
  • 0x6f - o
  • 0x64 - d
  • 0x91 - 1-element array (Arguments)
  • 0x2a - 42 (Argument value)

StreamItem Message Encoding

StreamItem messages have the following structure:

[2, InvocationId, Item]
  • 2 - Message Type - 2 indicates this is a StreamItem message
  • InvocationId - A String encoding the Invocation ID for the message
  • Item - the value of the stream item

Example:

The following payload:

0x93 0x02 0xa3 0x78 0x79 0x7a 0x2a

is decoded as follows:

  • 0x93 - 3-element array
  • 0x02 - 2 (Message Type - StreamItem message)
  • 0xa3 - string of length 3 (InvocationId)
  • 0x78 - x
  • 0x79 - y
  • 0x7a - z
  • 0x2a - 42 (Item)

Completion Message Encoding

Completion messages have the following structure

[3, InvocationId, ResultKind, Result?]
  • 3 - Message Type - 3 indicates this is a Completion message
  • InvocationId - A String encoding the Invocation ID for the message
  • ResultKind - A flag indicating the invocation result kind:
    • 1 - Error result - Result contains a String with the error message
    • 2 - Void result - Result contains the value returned by the server
    • 3 - Non-Void result - Result is absent
  • Result - An optional item containing the result of invocation. Absent if the server did not return any value (void methods)

Examples:

Error Result:

The following payload:

0x94 0x03 0xa3 0x78 0x79 0x7a 0x01 0xa5 0x45 0x72 0x72 0x6f 0x72

is decoded as follows:

  • 0x94 - 4-element array
  • 0x03 - 3 (Message Type - Result message)
  • 0xa3 - string of length 3 (InvocationId)
  • 0x78 - x
  • 0x79 - y
  • 0x7a - z
  • 0x01 - 1 (ResultKind - Error result)
  • 0xa5 - string of length 5
  • 0x45 - E
  • 0x72 - r
  • 0x72 - r
  • 0x6f - o
  • 0x72 - r

Void Result:

The following payload:

0x93 0x03 0xa3 0x78 0x79 0x7a 0x02

is decoded as follows:

  • 0x93 - 3-element array
  • 0x03 - 3 (Message Type - Result message)
  • 0xa3 - string of length 3 (InvocationId)
  • 0x78 - x
  • 0x79 - y
  • 0x7a - z
  • 0x02 - 2 (ResultKind - Void result)

Non-Void Result:

The following payload:

0x94 0x03 0xa3 0x78 0x79 0x7a 0x03 0x2a

is decoded as follows:

  • 0x94 - 4-element array
  • 0x03 - 3 (Message Type - Result message)
  • 0xa3 - string of length 3 (InvocationId)
  • 0x78 - x
  • 0x79 - y
  • 0x7a - z
  • 0x03 - 3 (ResultKind - Non-Void result)
  • 0x2a - 42 (Result)

StreamCompletion Message Encoding

StreamCompletion messages have the following structure

[4, InvocationId, Error?]
  • 4 - Message Type - 4 indicates this is a StreamCompletion message
  • InvocationId - A String encoding the Invocation ID for the message
  • Error - An optional string containing an error message if the invocation failed. Absent if the invocation completed without error.

Examples:

Completion for Successful Invocations

The following payload:

0x92 0x04 0xa3 0x78 0x79 0x7a

is decoded as follows:

  • 0x92 - 2-element array
  • 0x04 - 4 (Message Type - StreamCompletion message)
  • 0xa3 - string of length 3 (InvocationId)
  • 0x78 - x
  • 0x79 - y
  • 0x7a - z

Completion for Failed Invocations

The following payload:

0x93 0x04 0xa3 0x78 0x79 0x7a 0xa5 0x45 0x72 0x72 0x6f 0x72

is decoded as follows:

  • 0x93 - 3-element array
  • 0x04 - 4 (Message Type - StreamCompletion message)
  • 0xa3 - string of length 3 (InvocationId)
  • 0x78 - x
  • 0x79 - y
  • 0x7a - z
  • 0xa5 - string of length 5
  • 0x45 - E
  • 0x72 - r
  • 0x72 - r
  • 0x6f - o
  • 0x72 - r

CancelInvocation Message Encoding

CancelInvocation messages have the following structure

[5, InvocationId]
  • 5 - Message Type - 5 indicates this is a CancelInvocation message
  • InvocationId - A String encoding the Invocation ID for the message

Example:

The following payload:

0x92 0x05 0xa3 0x78 0x79 0x7a

is decoded as follows:

  • 0x92 - 2-element array
  • 0x05 - 5 (Message Type CancelInvocation message)
  • 0xa3 - string of length 3 (InvocationId)
  • 0x78 - x
  • 0x79 - y
  • 0x7a - z

Protocol Buffers (ProtoBuf) Encoding

Protobuf encoding is currently not implemented

In order to support ProtoBuf, an application must provide a ProtoBuf service definition for the Hub. However, implementations may automatically generate these definitions from reflection information, if the underlying platform supports this. For example, the .NET implementation will attempt to generate service definitions for methods that use only simple primitive and enumerated types. The service definition provides a description of how to encode the arguments and return value for the call. For example, consider the following C# method:

public bool SendMessageToUser(string userName, string message) {}

In order to invoke this method, the application must provide a ProtoBuf schema representing the input and output values and defining the message:

syntax = "proto3";

message SendMessageToUserRequest {
    string userName = 1;
    string message = 2;
}

message SendMessageToUserResponse {
    bool result = 1;
}

service ChatService {
    rpc SendMessageToUser (SendMessageToUserRequest) returns (SendMessageToUserResponse);
}

NOTE: the .NET implementation will provide a way to automatically generate these definitions at runtime, to avoid needing to generate them in advance, but applications still have the option of doing so. A general guideline for mapping .NET types to ProtoBuf types is listed in the "Type Mapping" section at the end of this document. In the current plan, custom .NET classes/structs not already listed in the table below will require a complete ProtoBuf mapping to be provided by the application.

SignalR.proto

SignalR provides an outer ProtoBuf schema for encoding the RPC invocation process as a whole, which is defined by the .proto file below. A SignalR frame is encoded as a single message of type SignalRFrame, then transmitted using the underlying transport. Since the underlying transport provides the necessary framing, we can reliably decode a message without having to know the length or format of the arguments.

syntax = "proto3";

message Invocation {
    string target = 1;
    bool nonblocking = 2;
    bytes arguments = 3;
}

message StreamItem {
    bytes item = 1;
}

message Completion {
    oneof payload {
        bytes result = 1;
        string error = 2;
    }
}

message SignalRFrame {
    string invocationId = 1;
    oneof message {
        Invocation invocation = 2;
        StreamItem streamItem = 3;
        Completion completion = 4;
    }
}

Invocation Message

When an invocation is issued by the Caller, we generate the necessary Request message according to the service definition, encode it into the ProtoBuf wire format, and then transmit an Invocation ProtoBuf message with that encoded argument data as the arguments field. The resulting Invocation message is wrapped in a SignalRFrame message and the invocationId is set. The final message is then encoded in the ProtoBuf format and transmitted to the Callee.

StreamItem Message

When a result is emitted by the Callee, it is encoded using the ProtoBuf schema associated with the service and encoded into the item field of a StreamItem ProtoBuf message. If an error is emitted, the message is encoded into the error field of a StreamItem ProtoBuf message. The resulting StreamItem message is wrapped in a SignalRFrame message and the invocationId is set. The final message is then encoded in the ProtoBuf format and transmitted to the Callee.

Completion Message

When a request completes, a Completion ProtoBuf message is constructed. If there is a final payload, it is encoded the same way as in the StreamItem message and stored in the result field of the message. If there is an error, it is encoded in the error field of the message. The resulting Completion message is wrapped in a SignalRFrame message and the invocationId is set. The final message is then encoded in the ProtoBuf format and transmitted to the Callee.

Type Mappings

Below are some sample type mappings between JSON types and the .NET client. This is not an exhaustive or authoritative list, just informative guidance. Official clients will provide ways for users to override the default mapping behavior for a particular method, parameter, or parameter type

.NET Type JSON Type MsgPack format family ProtoBuf Type
System.Byte, System.UInt16, System.UInt32 Number positive fixint, uint uint32
System.SByte, System.Int16, System.Int32 Number fixit, int int32
System.UInt64 Number positive fixint, uint uint64
System.Int64 Number fixint, int int64
System.Single Number float float
System.Double Number float double
System.Boolean true or false true, false bool
System.String String fixstr, str string
System.Byte[] String (Base64-encoded) bin bytes
IEnumerable<T> Array bin repeated
custom enum Number fixint, int uint64
custom struct or class Object fixmap, map Requires an explicit .proto file definition

MessagePack payloads are wrapped in an outer message framing described below.

Binary encoding

([Length][Body])([Length][Body])... continues until end of the connection ...
  • [Length] - A 32-bit unsigned integer encoded as VarInt. Variable size - 1-5 bytes.
  • [Body] - The body of the message, exactly [Length] bytes in length.
VarInt

VarInt encodes the most significant bit as a marker indicating whether the byte is the last byte of the VarInt or if it spans to the next byte. Bytes appear in the reverse order - i.e. the first byte contains the least significant bits of the value.

Examples:

  • VarInt: 0x35 (%00110101) - the most significant bit is 0 so the value is %x0110101 i.e. 0x35 (53)
  • VarInt: 0x80 0x25 (%10000000 %00101001) - the most significant bit of the first byte is 1 so the remaining bits (%x0000000) are the lowest bits of the value. The most significant bit of the second byte is 0 meaning this is last byte of the VarInt. The actual value bits (%x0101001) need to be prepended to the bits we already read so the values is %01010010000000 i.e. 0x1480 (5248)

The biggest supported payloads are 2GB in size so the biggest number we need to support is 0x7fffffff which when encoded as VarInt is 0xFF 0xFF 0xFF 0xFF 0x07 - hence the maximum size of the length prefix is 5 bytes.

For example, when sending the following frames (\n indicates the actual Line Feed character, not an escape sequence):

  • "Hello\nWorld"
  • 0x01 0x02

The encoding will be as follows, as a list of binary digits in hex (text in parentheses () are comments). Whitespace and newlines are irrelevant and for illustration only.

0x0B                                                   (start of frame; VarInt value: 11)
0x68 0x65 0x6C 0x6C 0x6F 0x0A 0x77 0x6F 0x72 0x6C 0x64 (UTF-8 encoding of 'Hello\nWorld')
0x02                                                   (start of frame; VarInt value: 2)
0x01 0x02                                              (body)

Binary encoding over text protocols

In case of sending messages using binary encoding over text transports (e.g. ServerSentEvents transport) messages should be Base64 encoded and use the following format:

([Length]:[Body];)([Length]:[Body];)... continues until end of the connection ...
  • [Length] - Length of the [Body] field in bytes, specified as UTF-8 digits (0-9, terminated by :). indicating the number of Base64-encoded characters (not the number of bytes in the final decoded message).
  • [Body] - Base64-encoded message

For example the following MsgPack payload (note: the payload consists of the length prefix and the MsgPack message):

0x10 0x95 0x01 0xa1 0x31 0xc3 0xa8 0x4d 0x79 0x4d 0x65 0x74 0x68 0x6f 0x64 0x91 0x2a

will look like this:

24:EJUBoTHDqE15TWV0aG9kkSo=;

when sending over a text transport.