Custom Formatters for MessagePack-CSharp

This post describes how to setup and use custom Formatters for MessagePack-CSharp.

Custom formatters are needed when you want to pass third party types over the network, and its needed for types from third party libraries. I needed it as I wanted to transfer Vector3, Quaternion and Colors from the System.Numerics namespace. In this post I detail the approach and how I did it.

Formatters

Formatters are the MessagePack-CSharp way of handling third-party types. A formatter is defined as a interface with a specific type you want to parse:

IMessagePackFormatter<T>

Typically this is implemented with the concrete type you want to format:

public class SystemNumericsVector3Formatter : IMessagePackFormatter<Vector3>

Then the interface requires that we implement a Serialize and a Deserialize. The original docs can be found on the MessagePack-CSharp GitHub.

The Serialize is easy for a Vector3. We need three fields, and to store them in an array. An array should always be used when there are multiple fields in a type. Its a indicator of the package size.

public void Serialize(
    ref MessagePackWriter writer,
    Vector3 value,
    MessagePackSerializerOptions options
)
{
    // TODO: Consider handling null values gracefully (writer.WriteNil())

    writer.WriteArrayHeader(3);
    writer.Write(value.X); // writer.Write has may overloads. I had some problems finding this as it was hidden within all other methods on writer.
    writer.Write(value.Y);
    writer.Write(value.Z);
}

Deserialize is about the same. You need to write and read in the same order:

public Vector3 Deserialize(ref MessagePackReader reader, MessagePackSerializerOptions options)
{
    // Depth step is the first step to parse any file.
    options.Security.DepthStep(ref reader);

    // TODO: Consider handling null values (reader.IsNil())

    var h = reader.ReadArrayHeader();
    if (h != 3) throw new ArgumentException(h.ToString());

    var x = (float)reader.ReadDouble(); // MessagePack does not have floats.
    var y = (float)reader.ReadDouble();
    var z = (float)reader.ReadDouble();

    return new Vector3(x, y, z);
}

Hey! We now have a nice Vector3Formatter!

Resolving Formatters

We got a formatter, but how to we tell MessagePack to use it?

We need a Resolver.

The IFormatterResolver interface helps us on the way.

public class SystemNumericsResolver : IFormatterResolver
{
    public IMessagePackFormatter<T>? GetFormatter<T>()
}

To pick which formatter we want we need to check the typeof(T) and pick a Formatter instance. This is implemented below.

public class SystemNumericsResolver : IFormatterResolver
{
    public static SystemNumericsResolver Instance = new();

    // Remember to not instantiate a new formatter each time, use a shared instance so we reduce allocations.
    SystemNumericsVector3Formatter vec3Formatter = new();

    public IMessagePackFormatter<T>? GetFormatter<T>()
    {
        if (typeof(T) == typeof(Vector3))
            return (IMessagePackFormatter<T>)vec3Formatter; // Casting back to T formatter is needed

        // TODO: Create and use formatters for Quaternions, etc etc

        return null;
    }
}

Now we have a Resolver!

Combining Resolvers

MessagePack-CSharp has a lot of resolvers built in, the way to expand it is through CompositeResolver.Create

var resolver = CompositeResolver.Create(
        SystemNumericsResolver.Instance,
        StandardResolver.Instance
    );

And we can use this resolver globally with:

// This is a static global setting, and will be used for all calls after this
MessagePackSerializer.DefaultOptions = MessagePackSerializer.DefaultOptions.WithResolver(
      resolver // From snippet above
  );

Cool! Now you know how to format any external class to your networking needs.

Finally, a tip on unit testing. A nice helper method to add to your unit testing suite. This serializes and deserializes the input value, and you can run tests to check equality on the result. Nice!

public static T EnsureRoundtrip<T>(IMessagePackFormatter<T> sut, T value)
{
    var arrayBufferWriter = new ArrayBufferWriter<byte>();
    var messagePackWriter = new MessagePackWriter(arrayBufferWriter);

    var options = MessagePackSerializerOptions.Standard;
    sut.Serialize(ref messagePackWriter, value, options);
    var reader = new MessagePackReader(arrayBufferWriter.GetMemory());
    var res = sut.Deserialize(ref reader, options);
    return res;
}

Thats is for this time. This may be a start on a part on implementing MagicOnion, but only time will tell.

// Nils Henrik