MessagePack

It's like JSON.
but fast and small.

MessagePack is an efficient binary serialization format. It lets you exchange data among multiple languages like JSON. But it's faster and smaller. Small integers are encoded into a single byte, and typical short strings require only one extra byte in addition to the strings themselves.

Redis scripting has support for MessagePack because it is a fast and compact serialization format with a simple to implement specification. I liked it so much that I implemented a MessagePack C extension for Lua just to include it into Redis.

Salvatore Sanfilippo, creator of Redis

Fluentd uses MessagePack for all internal data representation. It's crazy fast because of zero-copy optimization of msgpack-ruby. Now MessagePack is an essential component of Fluentd to achieve high performance and flexibility at the same time.

Sadayuki Furuhashi, creator of Fluentd

Treasure Data built a multi-tenant database optimized for analytical queries using MessagePack. The schemaless database is growing by billions of records every month. We also use MessagePack as a glue between components. Actually we just wanted a fast replacement of JSON, and MessagePack is simply useful.

Kazuki Ohta, CTO

MessagePack has been simply invaluable to us. We use MessagePack + Memcache to cache many of our feeds on Pinterest. These feeds are compressed and very quick to unpack thanks to MessagePack while Memcache gives us fast atomic pushes.

Marty Weiner, Software Engineer

Also use MessagePack? Waiting for your testimonial!

API

 

List your implementation here!

msgpack/msgpack-java https://github.com/msgpack/msgpack-java

MessagePack for Java

QuickStart for msgpack-java is available here.

How to install

You can install msgpack via maven:

<dependencies>
  ...
  <dependency>
    <groupId>org.msgpack</groupId>
    <artifactId>msgpack</artifactId>
    <version>${msgpack.version}</version>
  </dependency>
  ...
</dependencies>
Simple Serialization/Deserialization/Duck Typing using Value
// Create serialize objects.
List<String> src = new ArrayList<String>();
src.add("msgpack");
src.add("kumofs");
src.add("viver");

MessagePack msgpack = new MessagePack();
// Serialize
byte[] raw = msgpack.write(src);

// Deserialize directly using a template
List<String> dst1 = msgpack.read(raw, Templates.tList(Templates.TString));
System.out.println(dst1.get(0));
System.out.println(dst1.get(1));
System.out.println(dst1.get(2));

// Or, Deserialze to Value then convert type.
Value dynamic = msgpack.read(raw);
List<String> dst2 = new Converter(dynamic)
    .read(Templates.tList(Templates.TString));
System.out.println(dst2.get(0));
System.out.println(dst2.get(1));
System.out.println(dst2.get(2));

msgpack/msgpack-d https://github.com/msgpack/msgpack-d

Build Status

MessagePack for D

MessagePack is a binary-based JSON-like serialization library.

MessagePack for D is a pure D implementation of MessagePack.

Features
  • Small size and High performance
  • Zero copy serialization / deserialization
  • Stream deserializer / Direct-conversion deserializer
  • Supports D features (Ranges, Tuples, real type)

Note: The real type is only supported in D. Don't use the real type when communicating with other programming languages. Note that Unpacker will raise an exception if a loss of precision occurs.

Current Limitations
  • No circular references support
  • if you use the LDC compiler you need at least version 0.15.2 beta2
Install

Use dub to add it as a dependency:

% dub install msgpack-d
Usage

Example code can be found in the example directory.

The documentation can be found here

pack / unpack

msgpack-d is very simple to use. Use pack for serialization, and unpack for deserialization:

import std.file;
import msgpack;

struct S { int x; float y; string z; }

void main()
{
    S input = S(10, 25.5, "message");

    // serialize data
    ubyte[] inData = pack(input);

    // write data to a file
    write("file.dat", inData);

    // read data from a file
    ubyte[] outData = cast(ubyte[])read("file.dat");

    // unserialize the data
    S target = outData.unpack!S();

    // verify data is the same
    assert(target.x == input.x);
    assert(target.y == input.y);
    assert(target.z == input.z);
}
Feature: Skip serialization/deserialization of a specific field.

Use the @nonPacked attribute:

struct User
{
    string name;
    @nonPacked int level;  // pack / unpack will ignore the 'level' field
}
Feature: Use your own serialization/deserialization routines for custom class and struct types.

msgpack-d provides the functions registerPackHandler / registerUnpackHandler to allow you to use custom routines during the serialization or deserialization of user-defined class and struct types. This feature is especially useful when serializing a derived class object when that object is statically typed as a base class object.

For example:

class Document { }
class XmlDocument : Document
{
    this() { }
    this(string name) { this.name = name; }
    string name;
}

void xmlPackHandler(ref Packer p, ref XmlDocument xml)
{
    p.pack(xml.name);
}

void xmlUnpackHandler(ref Unpacker u, ref XmlDocument xml)
{
    u.unpack(xml.name);
}

void main()
{
    /// Register the 'xmlPackHandler' and 'xmlUnpackHandler' routines for
    /// XmlDocument object instances.
    registerPackHandler!(XmlDocument, xmlPackHandler);
    registerUnpackHandler!(XmlDocument, xmlUnpackHandler);

    /// Now we can serialize/deserialize XmlDocument object instances via a
    /// base class reference.
    Document doc = new XmlDocument("test.xml");
    auto data = pack(doc);
    XmlDocument xml = unpack!XmlDocument(data);
    assert(xml.name == "test.xml");  // xml.name is "test.xml"
}
The PackerImpl / Unpacker / StreamingUnpacker types

These types are used by the pack and unpack functions.

See the documentation of PackerImpl, Unpacker and StreamingUnpacker for more details.

Links
Copyright
Copyright (c) 2010- Masahiro Nakagawa
License

Distributed under the Boost Software License, Version 1.0.

msgpack/msgpack-python https://github.com/msgpack/msgpack-python

MessagePack for Python
Author: INADA Naoki
Version: 0.4.6
Date: 2015-03-13
What's this

MessagePack is a fast, compact binary serialization format, suitable for similar data to JSON. This package provides CPython bindings for reading and writing MessagePack data.

Install
$ pip install msgpack-python
PyPy

msgpack-python provides pure python implementation. PyPy can use this.

Windows

When you can't use binary distribution, you need to install Visual Studio or Windows SDK on Windows. Without extension, using pure python implementation on CPython runs slowly.

For Python 2.7, Microsoft Visual C++ Compiler for Python 2.7 is recommended solution.

For Python 3.5, Microsoft Visual Studio 2015 Community Edition or Express Edition can be used to build extension module.

How to use
One-shot pack & unpack

Use packb for packing and unpackb for unpacking. msgpack provides dumps and loads as alias for compatibility with json and pickle.

pack and dump packs to file-like object. unpack and load unpacks from file-like object.

>>> import msgpack
>>> msgpack.packb([1, 2, 3])
'\x93\x01\x02\x03'
>>> msgpack.unpackb(_)
[1, 2, 3]

unpack unpacks msgpack's array to Python's list, but can unpack to tuple:

>>> msgpack.unpackb(b'\x93\x01\x02\x03', use_list=False)
(1, 2, 3)

You should always pass the use_list keyword argument. See performance issues relating to use_list option below.

Read the docstring for other options.

Streaming unpacking

Unpacker is a "streaming unpacker". It unpacks multiple objects from one stream (or from bytes provided through its feed method).

import msgpack
from io import BytesIO

buf = BytesIO()
for i in range(100):
   buf.write(msgpack.packb(range(i)))

buf.seek(0)

unpacker = msgpack.Unpacker(buf)
for unpacked in unpacker:
    print unpacked
Packing/unpacking of custom data type

It is also possible to pack/unpack custom data types. Here is an example for datetime.datetime.

import datetime

import msgpack

useful_dict = {
    "id": 1,
    "created": datetime.datetime.now(),
}

def decode_datetime(obj):
    if b'__datetime__' in obj:
        obj = datetime.datetime.strptime(obj["as_str"], "%Y%m%dT%H:%M:%S.%f")
    return obj

def encode_datetime(obj):
    if isinstance(obj, datetime.datetime):
        return {'__datetime__': True, 'as_str': obj.strftime("%Y%m%dT%H:%M:%S.%f")}
    return obj


packed_dict = msgpack.packb(useful_dict, default=encode_datetime)
this_dict_again = msgpack.unpackb(packed_dict, object_hook=decode_datetime)

Unpacker's object_hook callback receives a dict; the object_pairs_hook callback may instead be used to receive a list of key-value pairs.

Extended types

It is also possible to pack/unpack custom data types using the ext type.

>>> import msgpack
>>> import array
>>> def default(obj):
...     if isinstance(obj, array.array) and obj.typecode == 'd':
...         return msgpack.ExtType(42, obj.tostring())
...     raise TypeError("Unknown type: %r" % (obj,))
...
>>> def ext_hook(code, data):
...     if code == 42:
...         a = array.array('d')
...         a.fromstring(data)
...         return a
...     return ExtType(code, data)
...
>>> data = array.array('d', [1.2, 3.4])
>>> packed = msgpack.packb(data, default=default)
>>> unpacked = msgpack.unpackb(packed, ext_hook=ext_hook)
>>> data == unpacked
True
Advanced unpacking control

As an alternative to iteration, Unpacker objects provide unpack, skip, read_array_header and read_map_header methods. The former two read an entire message from the stream, respectively deserialising and returning the result, or ignoring it. The latter two methods return the number of elements in the upcoming container, so that each element in an array, or key-value pair in a map, can be unpacked or skipped individually.

Each of these methods may optionally write the packed data it reads to a callback function:

from io import BytesIO

def distribute(unpacker, get_worker):
    nelems = unpacker.read_map_header()
    for i in range(nelems):
        # Select a worker for the given key
        key = unpacker.unpack()
        worker = get_worker(key)

        # Send the value as a packed message to worker
        bytestream = BytesIO()
        unpacker.skip(bytestream.write)
        worker.send(bytestream.getvalue())
Notes
string and binary type

In old days, msgpack doesn't distinguish string and binary types like Python 1. The type for represent string and binary types is named raw.

msgpack can distinguish string and binary type for now. But it is not like Python 2. Python 2 added unicode string. But msgpack renamed raw to str and added bin type. It is because keep compatibility with data created by old libs. raw was used for text more than binary.

Currently, while msgpack-python supports new bin type, default setting doesn't use it and decodes raw as bytes instead of unicode (str in Python 3).

You can change this by using use_bin_type=True option in Packer and encoding="utf-8" option in Unpacker.

>>> import msgpack
>>> packed = msgpack.packb([b'spam', u'egg'], use_bin_type=True)
>>> msgpack.unpackb(packed, encoding='utf-8')
['spam', u'egg']
ext type

To use ext type, pass msgpack.ExtType object to packer.

>>> import msgpack
>>> packed = msgpack.packb(msgpack.ExtType(42, b'xyzzy'))
>>> msgpack.unpackb(packed)
ExtType(code=42, data='xyzzy')

You can use it with default and ext_hook. See below.

Note for msgpack-python 0.2.x users

The msgpack-python 0.3 have some incompatible changes.

The default value of use_list keyword argument is True from 0.3. You should pass the argument explicitly for backward compatibility.

Unpacker.unpack() and some unpack methods now raises OutOfData instead of StopIteration. StopIteration is used for iterator protocol only.

Note about performance
GC

CPython's GC starts when growing allocated object. This means unpacking may cause useless GC. You can use gc.disable() when unpacking large message.

use_list option

List is the default sequence type of Python. But tuple is lighter than list. You can use use_list=False while unpacking when performance is important.

Python's dict can't use list as key and MessagePack allows array for key of mapping. use_list=False allows unpacking such message. Another way to unpacking such object is using object_pairs_hook.

Development
Test

MessagePack uses pytest for testing. Run test with following command:

$ py.test

msgpack/msgpack-erlang https://github.com/msgpack/msgpack-erlang

MessagePack Erlang

Travis Drone.io hex.pm version

Prerequisites for runtime

Erlang/OTP, >= 17.0 Also based on the new msgpack spec 0b8f5a.

edit rebar.config to use in your application
{deps, [
  {msgpack, ".*",
    {git, "git://github.com/msgpack/msgpack-erlang.git", {branch, "master"}}}
]}.

Or as it is now published at hex.pm, just

{deps, [msgpack]}.

might work.

Simple deserialization
Ham = msgpack:pack(Spam),
{ok, Spam} = msgpack:unpack(Ham).
Stream deserialization
{Term0, Rest0} = msgpack:unpack_stream(Binary),
{Term1, Rest1} = msgpack:unpack_stream(Rest0),
...
Options, for packing and unpacking
{spec, new|old}

Both for packing and unpacking. Default is new. Major difference between old and new spec is:

  • raw family (0xa0~0xbf, 0xda, 0xdb) becomes new str family
  • 0xd9 is new as str8
  • new bin space (0xc4, 0xc5, 0xc6 as bin8, bin16, bin32)
  • new ext space (0xc7, 0xc8, 0xc9 as ext8, ext16, ext32)
  • new fixext space (0xd4, 0xd5, 0xd6, 0xd7, 0xd8 as fixext1, fixext2, fixext4, fixext8, fixext16),

The default is new spec. Old spec mode does not handle these new types but returns error. To use old spec mode, this option is explicitly added.

OldHam = msgpack:pack(Spam, [{spec, old}]),
{ok, Spam} = msgpack:unpack(OldHam, [{spec, old}]).
{allow_atom, none|pack}

Only in packing. Atoms are packed as binaries. Default value is pack. Otherwise, any term including atoms throws badarg.

{known_atoms, [atom()]}

Both in packing and unpacking. In packing, if an atom is in this list a binary is encoded as a binary. In unpacking, msgpacked binaries are decoded as atoms with erlang:binary_to_existing_atom/2 with encoding utf8. Default value is an empty list.

Even if allow_atom is none, known atoms are packed.

{unpack_str, as_binary|as_list}

A switch to choose decoded term style of str type when unpacking. Only available at new spec. Default is as_list.

mode        as_binary    as_list
-----------+------------+-------
bin         binary()     binary()
str         binary()     string()
{validate_string, boolean()}

Only in unpacking, UTF-8 validation at unpacking from str type will be enabled. Default value is false.

{pack_str, from_binary|from_list|none}

A switch to choose packing of string() when packing. Only available at new spec. Default is from_list for symmetry with unpack_str option.

mode        from_list    from_binary    none
-----------+------------+--------------+-----------------
binary()    bin          str*/bin       bin
string()    str*/array   array of int   array of int
list()      array        array          array

But the default option pays the cost of performance for symmetry. If the overhead of UTF-8 validation is unacceptable, choosing none as the option would be the best.

  • * Tries to pack as str if it is a valid string().
{map_format, map|jiffy|jsx}

Both at packing and unpacking. Default value is map.

msgpack:pack(#{ <<"key">> => <<"value">> }, []).
msgpack:pack({[{<<"key">>, <<"value">>}]}, [{format, jiffy}]),
msgpack:pack([{<<"key">>, <<"value">>}], [{format, jsx}]).
{ext, {msgpack_ext_packer(), msgpack_ext_unpacker()}|module()}

At both. The default behaviour in case of facing ext data at decoding is to ignore them as its length is known.

Now msgpack-erlang supports ext type. Now you can serialize everything with your original (de)serializer. That will enable us to handle erlang- native types like pid(), ref() contained in tuple(). See test/msgpack_ext_example_tests.erl for example code.

Packer = fun({ref, Ref}, Opt) when is_reference(Ref) -> {ok, {12, term_to_binary(Ref)}} end,
Unpacker = fun(12, Bin) -> {ok, {ref, binary_to_term(Bin)}} end,
Ref = make_ref(),
Opt = [{ext,{Packer,Unpacker}}],
{ok, {ref, Ref}} = msgpack:unpack(msgpack:pack({ref, Ref}, Opt), Opt).
License

Apache License 2.0

Release Notes
0.5.0
  • Renewed optional arguments to pack/unpack interface. This is incompatible change from 0.4 series.
0.4.0
  • Deprecate nil
  • Moved to rebar3
  • Promote default map unpacker as default format when OTP is >= 17
  • Added QuickCheck tests
  • Since this version OTP older than R16B03-1 are no more supported
0.3.5 / 0.3.4
  • 0.3 series will be the last versions that supports R16B or older versions of OTP.
  • OTP 18.0 support
  • Promote default map unpacker as default format when OTP is >= 18
0.3.3
  • Add OTP 17 series to Travis-CI tests
  • Fix wrong numbering for ext types
  • Allow packing maps even when {format,map} is not set
  • Fix Dialyzer invalid contract warning
  • Proper use of null for jiffy-style encoding/decoding
0.3.2
  • set back default style as jiffy
  • fix bugs around nil/null handling
0.3.0
  • supports map new in 17.0
  • jiffy-style maps will be deprecated in near future
  • set default style as map
0.2.8

0.2 series works with OTP 17.0, R16, R15, and with MessagePack's new and old format. But does not support map type introduced in OTP 17.0.

It also supports JSX-compatible mode.

msgpack/msgpack-ruby https://github.com/msgpack/msgpack-ruby

MessagePack for Ruby
require 'msgpack'
msg = [1,2,3].to_msgpack  #=> "\x93\x01\x02\x03"
MessagePack.unpack(msg)   #=> [1,2,3]
Install
gem install msgpack
Use cases
  • Create REST API returing MessagePack using Rails + RABL
  • Store objects efficiently in memcached or Redis
  • Upload data in efficient format from mobile devices. See also MessagePack for Objective-C and Java
Links
Streaming API
# serialize a 2-element array [e1, e2]
pk = MessagePack::Packer.new(io)
pk.write_array_header(2).write(e1).write(e2).flush
# deserialize objects from an IO
u = MessagePack::Unpacker.new(io)
u.each { |obj| ... }
# event-driven deserialization
def on_read(data)
  @u ||= MessagePack::Unpacker.new
  @u.feed_each(data) { |obj| ... }
end

msgpack/msgpack-scala https://github.com/msgpack/msgpack-scala

MessagePack for Scala

Build Status

An implementation of MessagePack for Scala.

Requirement
  • sbt 0.13.5 or later
SBT

Add dependency.

libraryDependencies += "org.msgpack" %% "msgpack-scala" % "0.6.11"
For developer
Make intellij project >= IntelliJ IDEA 13

You can import sbt file.

< IntelliJ IDEA 13

Please type the following command.Then intellij project will be generated.

$ sbt gen-idea
Quick Start
Serialize/Deserialize class
import org.msgpack.annotation.Message
import org.msgpack.ScalaMessagePack

@Message // Don't forget to add Message annotation.
class YourClass{
  var name : String = ""
  var age : Int = 2
}

val obj = new YourClass()
obj.name = "hoge"
obj.age = 22
val serialized : Array[Byte] = ScalaMessagePack.write(obj)

val deserialized : YourClass = ScalaMessagePack.read[YourClass](serialized)
  • Sorry, case classes are not supported now.

msgpack/msgpack-haskell https://github.com/msgpack/msgpack-haskell

mneumann/MessagePack https://github.com/mneumann/MessagePack

MessagePack

An alternative msgpack.org implementation for Ruby and C++

aaulia/msgpack-haxe https://github.com/aaulia/msgpack-haxe

MIT License Haxelib Version

msgpack-haxe

MessagePack (http://msgpack.org) serialization library for Haxe

How to install:

Simply use haxelib git to use this github repo or haxelib install msgpack-haxe to use the one in the haxelib repository.

Supported Type:
  • Null
  • Bool
  • Int
  • Float
  • Object
  • Bytes
  • String
  • Array
  • IntMap/StringMap
Example code:
package;
import org.msgpack.MsgPack;

class Example {
    public static function main() {
        var i = { a: 1, b: 2, c: "Hello World!" };
        var m = MsgPack.encode(i);
        var o = MsgPack.decode(m);

        trace(i);
        trace(m.toHex());
        trace(o);
    }
}

msgpack/msgpack-cli https://github.com/msgpack/msgpack-cli

MessagePack for CLI
What is it?

This is MessagePack serialization/deserialization for CLI (Common Language Infrastructure) implementations such as .NET Framework, Silverlight, Mono (including Moonlight.) This library can be used from ALL CLS compliant languages such as C#, F#, Visual Basic, Iron Python, Iron Ruby, PowerShell, C++/CLI or so.

Usage

You can serialize/deserialize objects as following: 1. Create serializer via MessagePackSerializer.Create generic method. This method creates dependent types serializers as well. 1. Invoke serializer as following: ** Pack method with destination Stream and target object for serialization. ** Unpack method with source Stream.

// Creates serializer.
var serializer = SerializationContext.Default.GetSerializer<T>();
// Pack obj to stream.
serializer.Pack(stream, obj);
// Unpack from stream.
var unpackedObject = serializer.Unpack(stream);
' Creates serializer.
Dim serializer = SerializationContext.Default.GetSerializer(Of T)()
' Pack obj to stream.
serializer.Pack(stream, obj)
' Unpack from stream.
Dim unpackedObject = serializer.Unpack(stream)
Features
  • Fast and interoperable binary format serialization with simple API.
  • Generating pre-compiled assembly for rapid start up.
  • Flexible MessagePackObject which represents MessagePack type system naturally.
Documentation

See wiki

Installation
  • Binary files distributed via the NuGet package MsgPack.Cli.
  • You can extract binary (DLL) file as following:
    1. Download *.zip file from GitHub Release page.
    2. Extract it.
    3. Under the bin directory, binaries are there!
      • For mono, you can use net461 or net35 drops as you run with.
      • For Unity, unity3d drop is suitable.
How to build
For .NET Framework
  1. Install recent Windows SDK (at least, .NET Framework 4 Client Profile and MSBuild is needed.)
    Or install Visual Studio or Visual Studio Express.
    1. If you want to build unit test assemblies, install NuGet and then restore NUnit packages.
  2. Run:

    msbuild MsgPack.sln

    Or (for .NET 3.5 drops and Unity 3D drops):

    msbuild MsgPack.compats.sln

    Or (for Windows Runtime/Phone drops and Silverlight 5 drops):

    msbuild MsgPack.Windows.sln

    Or (for Xamarin drops, you must have Xamarin Business or upper license and Mac machine on the LAN to build on Windows):

    msbuild MsgPack.Xamarin.sln

Or open one of above solution files in your IDE and run build command in it.

For Mono

Open MsgPack.mono.sln with MonoDevelop and then click Build menu item. (Of cource, you can build via xbuild.)

Own Unity 3D Build

First of all, there are binary drops on github release page, you should use it to save your time. Because we will not guarantee source code organization compatibilities, we might add/remove non-public types or members, which should break source code build. If you want to import sources, you must include just only described on MsgPack.Unity3D.csproj. If you want to use ".NET 2.0 Subset" settings, you must use just only described on MsgPack.Unity3D.CorLibOnly.csproj file, and define CORLIB_ONLY compiler constants.

See also

msgpack/msgpack-c https://github.com/msgpack/msgpack-c

msgpack for C/C++

Version 2.0.0 (under development)Build Status Build status

It's like JSON but small and fast.

Overview

MessagePack is an efficient binary serialization format, which lets you exchange data among multiple languages like JSON, except that it's faster and smaller. Small integers are encoded into a single byte while typical short strings require only one extra byte in addition to the strings themselves.

Example

In C:

#include <msgpack.h>
#include <stdio.h>

int main(void)
{
    /* msgpack::sbuffer is a simple buffer implementation. */
    msgpack_sbuffer sbuf;
    msgpack_sbuffer_init(&sbuf);

    /* serialize values into the buffer using msgpack_sbuffer_write callback function. */
    msgpack_packer pk;
    msgpack_packer_init(&pk, &sbuf, msgpack_sbuffer_write);

    msgpack_pack_array(&pk, 3);
    msgpack_pack_int(&pk, 1);
    msgpack_pack_true(&pk);
    msgpack_pack_str(&pk, 7);
    msgpack_pack_str_body(&pk, "example", 7);

    /* deserialize the buffer into msgpack_object instance. */
    /* deserialized object is valid during the msgpack_zone instance alive. */
    msgpack_zone mempool;
    msgpack_zone_init(&mempool, 2048);

    msgpack_object deserialized;
    msgpack_unpack(sbuf.data, sbuf.size, NULL, &mempool, &deserialized);

    /* print the deserialized object. */
    msgpack_object_print(stdout, deserialized);
    puts("");

    msgpack_zone_destroy(&mempool);
    msgpack_sbuffer_destroy(&sbuf);

    return 0;
}

See QUICKSTART-C.md for more details.

In C++:

#include <msgpack.hpp>
#include <string>
#include <iostream>
#include <sstream>

int main(void)
{
    msgpack::type::tuple<int, bool, std::string> src(1, true, "example");

    // serialize the object into the buffer.
    // any classes that implements write(const char*,size_t) can be a buffer.
    std::stringstream buffer;
    msgpack::pack(buffer, src);

    // send the buffer ...
    buffer.seekg(0);

    // deserialize the buffer into msgpack::object instance.
    std::string str(buffer.str());

    msgpack::object_handle oh =
        msgpack::unpack(str.data(), str.size());

    // deserialized object is valid during the msgpack::object_handle instance is alive.
    msgpack::object deserialized = oh.get();

    // msgpack::object supports ostream.
    std::cout << deserialized << std::endl;

    // convert msgpack::object instance into the original type.
    // if the type is mismatched, it throws msgpack::type_error exception.
    msgpack::type::tuple<int, bool, std::string> dst;
    deserialized.convert(dst);

    return 0;
}

See QUICKSTART-CPP.md for more details.

Usage
C++ Header Only Library

When you use msgpack on C++03 and C++11, you can just add msgpack-c/include to your include path:

g++ -I msgpack-c/include your_source_file.cpp

If you want to use C version of msgpack, you need to build it. You can also install the C and C++ versions of msgpack.

Building and Installing Install from git repository
Using autotools

You will need:

  • gcc >= 4.1.0 or clang >= 3.3.0
  • autoconf >= 2.60
  • automake >= 1.10
  • libtool >= 2.2.4

The build steps below are for C and C++03. If compiling for C++11, add -std=c++11 to the environmental variable CXXFLAGS with export CXXFLAGS="$CXXFLAGS -std=c++11" prior to following the directions below.

$ git clone https://github.com/msgpack/msgpack-c
$ cd msgpack-c
$ ./bootstrap
$ ./configure
$ make

You can install the resulting library like this:

$ sudo make install
Using cmake
Using the Terminal (CLI)

You will need:

  • gcc >= 4.1.0
  • cmake >= 2.8.0

C and C++03:

$ git clone https://github.com/msgpack/msgpack-c.git
$ cd msgpack-c
$ cmake .
$ make
$ sudo make install

If you want to setup C++11 version of msgpack instead, execute the following commands:

$ git clone https://github.com/msgpack/msgpack-c.git
$ cd msgpack-c
$ cmake -DMSGPACK_CXX11=ON .
$ sudo make install
GUI on Windows

Clone msgpack-c git repository.

$ git clone https://github.com/msgpack/msgpack-c.git

or using GUI git client.

e.g.) tortoise git https://code.google.com/p/tortoisegit/

  1. Launch cmake GUI client.

  2. Set 'Where is the source code:' text box and 'Where to build the binaries:' text box.

  3. Click 'Configure' button.

  4. Choose your Visual Studio version.

  5. Click 'Generate' button.

  6. Open the created msgpack.sln on Visual Studio.

  7. Build all.

Documentation

You can get addtional information on the wiki.

Contributing

msgpack-c is developed on GitHub at msgpack/msgpack-c. To report an issue or send a pull request, use the issue tracker.

Here's the list of great contributors.

License

msgpack-c is licensed under the Boost Software License, Version 1.0. See the LICENSE_1_0.txt file for details.

msgpack/msgpack-ocaml https://github.com/msgpack/msgpack-ocaml

MsgPack for OCaml
BULID
$ make
$ sudo make install
EXAMPLE
Serialize/Deserialize for Msgpack Object
(* serialize *)
let bytes = 
  Msgpack.Serialize.serialize_string (`FixArray [`PFixnum 1; `PFixnum 2; `PFixnum 3])

(* deserialize *)
let obj =
  Msgpack.Serialize.deserialize_string bytes
Serialize/Deserialize for OCaml types (with meta_conv)
open Msgpack_conv

type t = {
  int : int;
  str : string;
} with conv(msgpack)

(* serialize *)
let bytes = 
  Msgpack.Serialize.serialize_string (msgpack_of_t { int = 42; str = "ans" })

(* deserialize *)
let obj =
  t_of_msgpack (Msgpack.Serialize.deserialize_string bytes)

See also, examlpe/

TEST
$ ocaml setup.ml -configure --enable-tests
$ make test
PROOF

If you want to use msgpack at OCaml, you need not do this section. This section for user intrested in formal verification.

You need Coq 8.4 and omake.

$ cd proof
$ make
$ cp *.ml* ../lib/core

msgpack/msgpack-smalltalk https://github.com/msgpack/msgpack-smalltalk

msgpack-smalltalk

MessagePack serialization library for various Smalltalk dialects.

  • Squeak
  • Pharo
  • VisualWorks
  • VA Smalltalk
  • Dolphin Smalltalk
  • GNU Smalltalk (Beta)
  • Cuis

Sources are put as Cypress for the neutral accesses from various Smalltalk dialects.

How to use
Serialization
MpMessagePack pack: <your object>

or:

<your object> messagePacked
Deserialization
MpMessagePack unpack: msgpackBytes

or:

Object fromMessagePack: msgBytes
Samples
map := Dictionary new.
map at: 'someArray' asByteArray put: #(1 2.2 #[3 4 5]).
packed := map messagePacked.
(Object fromMessagePack: packed) inspect.
writeStream := WriteStream on: ByteArray new.
encoder := MpEncoder on: writeStream.
encoder nextPut: 1.
encoder nextPut: #(2 3).
dic := Dictionary new.
dic at: 4 put: 5.
encoder nextPut: dic.
encoder nextPut: 'four' asByteArray.
bytes := encoder contents.
readStream := ReadStream on: bytes.
decoder := MpDecoder on: readStream.
[decoder atEnd] whileFalse: [
        Transcript cr; show: decoder next printString
]
How to install

Please read HowToInstall.md.

Loading the latest development version Squeak
Installer squeaksource
    project: 'MessagePack';
    install: 'ConfigurationOfMessagePack'. 
(Smalltalk at: #ConfigurationOfMessagePack) project development load
Pharo
Gofer it
    smalltalkhubUser: 'MasashiUmezawa' project: 'MessagePack';
    configuration;
    load.
(Smalltalk at: #ConfigurationOfMessagePack) project development load

You might need MpTypeMapper initializeAll on new encoder/decoder-related updates.

loteixeira/as3-msgpack https://github.com/loteixeira/as3-msgpack

MessagePack for Actionscript3 (Flash, Flex and AIR).

as3-msgpack was designed to work with the interfaces IDataInput and IDataOutput, thus the API might be easily connected with the native classes that handle binary data (such as ByteArray, Socket, FileStream and URLStream).
Moreover, as3-msgpack is capable of decoding data from binary streams.
Get started: http://loteixeira.github.io/lib/2013/08/19/as3-msgpack/

Basic usage (encoding/decoding):

// create messagepack object
var msgpack:MsgPack = new MsgPack();

// encode an array
var bytes:ByteArray = msgpack.write([1, 2, 3, 4, 5]);

// rewind the buffer
bytes.position = 0;

// print the decoded object
trace(msgpack.read(bytes));

For downloads, source code and further information, check the project repository: https://github.com/loteixeira/as3-msgpack.

msgpack/msgpack-php https://github.com/msgpack/msgpack-php

Msgpack for PHP

Build Status

This extension provide API for communicating with MessagePack serialization.

MessagePack is a binary-based efficient object serialization library. It enables to exchange structured objects between many languages like JSON. But unlike JSON, it is very fast and small.

Requirement
  • PHP 5.0 +
Install
Install from PECL

Msgpack is an PECL extension, thus you can simply install it by:

pecl install msgpack
Compile Msgpack from source
$/path/to/phpize
$./configure 
$make && make install
Example
<?php
$data = array(0=>1,1=>2,2=>3);
$msg = msgpack_pack($data);
$data = msgpack_unpack($msg);
?>
Resources

fperrad/lua-MessagePack http://fperrad.github.io/lua-MessagePack/

MessagePack for Lua (spec v5)

Build Status Coverage Status Licence Dependencies

local mp = require 'MessagePack'
mpac = mp.pack(data)
data = mp.unpack(mpac)
Install

You can use LuaRocks to install lua-MessagePack:

$ luarocks install lua-messagepack

or from the source, with:

$ make install

It is a pure Lua implementation, without any dependency.

Links
Copyright and License

Copyright (c) 2012-2015 Francois Perrad rank

This library is licensed under the terms of the MIT/X11 license, like Lua itself.

mneumann/rust-msgpack https://github.com/mneumann/rust-msgpack

rust-msgpack Build Status

Msgpack implementation for Rust language.

Installation

Simply include the rust-msgpack in your Cargo dependencies.

[dependencies.msgpack]

git = "[email protected]:mneumann/rust-msgpack.git"
Quickstart
extern crate msgpack;

fn main() {
  let arr = vec!["str1".to_string(), "str2".to_string()];
  let str = msgpack::Encoder::to_msgpack(&arr).ok().unwrap();
  println!("Encoded: {}", str);

  let dec: Vec<String> = msgpack::from_msgpack(str).ok().unwrap();
  println!("Decoded: {}", dec);
}

To enable your own data structures to be automatically serialized from and to msgpack, derive from Encodable and Decodable as shown in the following example:

extern crate serialize;

#[deriving(Encodable,Decodable)]
struct MyStruct {
  a: Vec<u32>,
  s: String
}
Testing
cargo test
License

This code licensed under the same terms as Rust itself: dual MIT/Apache2 license options.

mururu/msgpack-elixir https://github.com/mururu/msgpack-elixir

MessagePack for Elixir

Build Status

Installation

Add :message_pack as a dependency in your mix.exs file.

defp deps do
  [{:message_pack, "~> 0.2.0"}]
end
Usage
# pack
MessagePack.pack([1,2,3]) #=> { :ok, <<147,1,2,3>> }
MessagePack.pack!([1,2,3]) #=> <<147,1,2,3>>

# unpack
MessagePack.unpack(<<147,1,2,3>>) #=> { :ok, [1,2,3] }
MessagePack.unpack!(<<147,1,2,3>>) #=> [1,2,3]

# unpack_once
MessagePack.unpack_once(<<147,1,2,3,4>>) #=> {:ok, {[1, 2, 3], <<4>>}}
MessagePack.unpack_once!(<<147,1,2,3,4>>) #=> {[1, 2, 3], <<4>>}
Options
  • enable_string

Support string type. This options is false by default.

iex(1)> { :ok, bin } = MessagePack.pack(<<255>>)
{:ok, <<161, 255>>}
iex(3)> MessagePack.unpack(<<161, 255>>)
{:ok, <<255>>}
iex(4)> MessagePack.unpack(<<161, 255>>, enable_string: true)
{:error, {:invalid_string, <<255>>}}
  • ext

Support extention type.

See test/message_pack_ext_test.exs.

License

MIT

ktakashi/r6rs-msgpack https://github.com/ktakashi/r6rs-msgpack

MessagePack for R6RS Scheme

This is an implementation of MessagePack for R6RS Scheme.

API references

Function (pack! bv message)
Function (pack! bv message offset)

Pack message to message pack format bytevector and put it into the bv destructively. Given bv must have enough length to hold the message.

Optional argument offset indicates where to start with, default is 0.

Function (pack message)

The same as pack! but this one creates a new bytevector.

Function (pack-size message)

Calculate the converted message size.

Function (unpack bv)
Function (unpack bv offset)

Unpack the given message format bytevector to Scheme object.

Optional argument offset indicates where to start with, default is 0.

Function (get-unpack in)

Unpack the given binary input port to Scheme object.

Conversion rules

As you already know, Scheme doesn't have static types so the conversion of Scheme objects to message pack data might cause unexpected results. To avoid it, I will describe how conversion works.

Scheme to message pack Integer conversion

The library automatically decides proper size. More specifically, if it can fit to message pack's fixnum then library uses it, so are uint8-64. If the number is too big, then an error is raised. Users must know it tries to use uint as much as possible. If the given number was negative then sint will be used.

Floating point conversion

Unfortunately R6RS doesn't have difference between float and double. So when flonum is given then it always converts to double number.

Collection conversion

Message pack has collections which are map and array. And these are associated with alist (association list) and vector respectively. When you want to convert alist to message pack data, then you need to make sure the cdr part will be the data and if you put (("key" "value))_ then it will be converted to nested map.

The collection size calculation is done automatically. It tries to use the smallest size.

Message pack to Scheme

The other way around is easy, it can simply restore the byte data to Scheme object. Following describes the conversion rules;

Positive fixnum -> integer
Negative fixnum -> integer
uint8, uint16, uint32, uint64 -> integer
sint8, sint16, sint32, sint64 -> integer
Map -> alist
Array -> vector
fixstr, str8, str16, str32 -> string bit8, bit16, bit32 -> bytevector

Tested Scheme implementations

Sagittarius Scheme

Mosh Scheme

Ypsilon

Your contributions are always welcome.

TODO
  • More testing
  • Extended type handling

ugorji/go https://github.com/ugorji/go

MessagePack and Binc Codec for Go Language.

A High Performance, Feature-Rich, Idiomatic encode/decode and rpc library.

To install:

go get github.com/ugorji/go/codec

Source: [http://github.com/ugorji/go]
Online documentation: [http://godoc.org/github.com/ugorji/go/codec]

Typical usage:

    // create and use decoder/encoder
    var (
        v interface{} // value to decode/encode into
        r io.Reader
        w io.Writer
        b []byte
        mh codec.MsgpackHandle
    )

    dec = codec.NewDecoder(r, &mh)
    dec = codec.NewDecoderBytes(b, &mh)
    err = dec.Decode(&v) 

    enc = codec.NewEncoder(w, &mh)
    enc = codec.NewEncoderBytes(&b, &mh)
    err = enc.Encode(v)

    //RPC Server
    go func() {
        for {
            conn, err := listener.Accept()
            rpcCodec := codec.GoRpc.ServerCodec(conn, h)
            //OR rpcCodec := codec.MsgpackSpecRpc.ServerCodec(conn, h)
            rpc.ServeCodec(rpcCodec)
        }
    }()

    //RPC Communication (client side)
    conn, err = net.Dial("tcp", "localhost:5555")
    rpcCodec := codec.GoRpc.ClientCodec(conn, h)
    //OR rpcCodec := codec.MsgpackSpecRpc.ClientCodec(conn, h)
    client := rpc.NewClientWithCodec(rpcCodec)

suzukaze/mruby-msgpack MessagePack for mruby

mruby-msgpack Build Status
Welcome to MessagePack for mruby

MessagePack for mruby is MessagePack implimented in mruby language.

msg = [1, 2, 3].to_msgpack  #=> "\x93\x01\x02\x03"
MessagePack.unpack(msg)     #=> [1, 2, 3]

This is early vesion. Please check the methods that work in test foloder.

Platform

I test MessagePack for mruby in mac OSX 10.8.4. In the future it will work in Windows and Linux OS.

Getting Started
  1. Download MessagePack for mruby at the command prompt:

    git clone https://github.com/suzukaze/mruby-msgpack.git
    
  2. Add config.gem line to build_config.rb

MRuby::Build.new do |conf|

  # ...(snip)...
  conf.gem :git => 'https://github.com/suzukaze/mruby-msgpack.git'
end
  1. Test at the command prompt:

    rake test
    
  2. Build at the command prompt:

    rake
    
msgpack-ruby commit

mruby-msgpack is based on msgpack-ruby(a7c2dc34ef69c9132167e38009baa8420c460c9b)

Contributing

I encourage you to contribute to MessagePack for mruby!

License

Author : Jun Hiroe

Copyrigh : Copyright (c) 2013 Jun Hiroe

License : MIT License

lexmag/msgpax https://github.com/lexmag/msgpax

Msgpax

Build Status Hex Version

This library provides an API for serializing and de-serializing Elixir terms using the MessagePack format.

Documentation is available online.

Features

A detailed table that shows the relationship between Elixir types and MessagePack types can be found in the documentation for the Msgpax module.

Installation

Add :msgpax as a dependency in your mix.exs file:

def deps do
  [{:msgpax, "~> 0.8"}]
end

After you are done, run mix deps.get in your shell to fetch the dependencies.

License

This software is licensed under the ISC license.

jingweno/msgpack_rails https://github.com/jingweno/msgpack_rails

MessagePack for Rails Build Status

The Rails way to serialize/deserialize objects with Message Pack. It implements the ActiveSupport encoder & decoder and the ActiveModel serializer for Message Pack.

Installation

Add this line to your application's Gemfile:

gem 'msgpack_rails'

And then execute:

$ bundle

Or install it yourself as:

$ gem install msgpack_rails
Usage

msgpack_rails converts data type using as_json before feeding it into msgpack. Here are a few examples:

$ ActiveSupport::MessagePack.encode(:a => :b)
=> "\x81\xA1a\xA1b"

$ ActiveSupport::MessagePack.encode(Time.now)
=> "\xB92013-09-11T10:40:39-07:00"

$ Time.now.as_msgpack
=> "2013-09-11T10:48:13-07:00"

$ Time.now.to_msgpack
=> "\xB92013-09-11T10:40:39-07:00"

$ ActiveSupport::MessagePack.decode Time.now.to_msgpack
=> "2013-09-11T11:23:07-07:00"

# After setting ActiveSupport.parse_msgpack_times to true
$ ActiveSupport::MessagePack.decode Time.now.to_msgpack
=> Wed, 11 Sep 2013 11:25:18 -0700

You can also use it as part of ActiveModel, similar to to_json:

class Contact
  include ActiveModel::Serializers::MessagePack

  ...
end

@contact = Contact.new
@contact.name = 'Owen Ou'
@contact.age = 28
@contact.created_at = Time.utc(2006, 8, 1)
@contact.awesome = true
@contact.preferences = { 'shows' => 'anime' }

@contact.to_msgpack                # => msgpack output
@contact.to_msgpack(:root => true) # => include root in msgpack output
Contributing
  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request

vsergeev/u-msgpack-python https://github.com/vsergeev/u-msgpack-python

u-msgpack-python v2.1

u-msgpack-python is a lightweight MessagePack serializer and deserializer module written in pure Python, compatible with both Python 2 and 3, as well CPython and PyPy implementations of Python. u-msgpack-python is fully compliant with the latest MessagePack specification.

u-msgpack-python is currently distributed on PyPI: https://pypi.python.org/pypi/u-msgpack-python and as a single file: umsgpack.py

Installation

With pip:

$ pip install u-msgpack-python

With easy_install:

$ easy_install u-msgpack-python

or simply drop umsgpack.py into your project!

$ wget https://raw.github.com/vsergeev/u-msgpack-python/master/umsgpack.py
Examples

Basic Example:

>>> import umsgpack
>>> umsgpack.packb({u"compact": True, u"schema": 0})
'\x82\xa7compact\xc3\xa6schema\x00'
>>> umsgpack.unpackb(_)
{u'compact': True, u'schema': 0}
>>> 

A more complicated example:

>>> umsgpack.packb(
        [1, True, False, 0xffffffff, {u"foo": b"\x80\x01\x02",
         u"bar": [1,2,3, {u"a": [1,2,3,{}]}]}, -1, 2.12345] )
'\x97\x01\xc3\xc2\xce\xff\xff\xff\xff\x82\xa3foo\xc4\x03\x80\x01'
'\x02\xa3bar\x94\x01\x02\x03\x81\xa1a\x94\x01\x02\x03\x80\xff\xcb'
'@\x00\xfc\xd3Z\x85\x87\x94'
>>> umsgpack.unpackb(_)
[1, True, False, 4294967295, {u'foo': '\x80\x01\x02',
 u'bar': [1, 2, 3, {u'a': [1, 2, 3, {}]}]}, -1, 2.12345]
>>> 

Streaming serialization with file-like objects:

>>> f = open('test.bin', 'wb')
>>> umsgpack.pack({u"compact": True, u"schema": 0}, f)
>>> umsgpack.pack([1,2,3], f)
>>> f.close()
>>> 
>>> f = open('test.bin', 'rb')
>>> umsgpack.unpack(f)
{u'compact': True, u'schema': 0}
>>> umsgpack.unpack(f)
[1, 2, 3]
>>> f.close()
>>> 

Encoding and decoding an application-defined ext type:

>>> # Create an Ext object with type 0x05 and data b"\x01\x02\x03"
... foo = umsgpack.Ext(0x05, b"\x01\x02\x03")
>>> umsgpack.packb({u"special stuff": foo, u"awesome": True})
b'\x82\xadspecial stuff\xc7\x03\x05\x01\x02\x03\xa7awesome\xc3'
>>> bar = umsgpack.unpackb(_)
>>> print(bar["special stuff"])
Ext Object (Type: 0x05, Data: 01 02 03)
>>> bar["special stuff"].type
5
>>> bar["special stuff"].data
b'\x01\x02\x03'
>>> 

Python standard library style names dump, dumps, load, loads are also available:

>>> import umsgpack
>>> umsgpack.dumps({u"compact": True, u"schema": 0})
'\x82\xa7compact\xc3\xa6schema\x00'
>>> umsgpack.loads(_)
{u'compact': True, u'schema': 0}
>>> 
>>> f = open('test.bin', 'wb')
>>> umsgpack.dump({u"compact": True, u"schema": 0}, f)
>>> f.close()
>>> 
>>> f = open('test.bin', 'rb')
>>> umsgpack.load(f)
{u'compact': True, u'schema': 0}
>>> 
More Information

See the project page for more information on old specification compatibility mode, exceptions, behavior, and testing.

License

u-msgpack-python is MIT licensed. See the included LICENSE file for more details.

kmsquire/MsgPack.jl https://github.com/kmsquire/MsgPack.jl

MsgPack

Build Status

Provides basic support for the msgpack format.

julia> import MsgPack

julia> MsgPack.pack("hi")
3-element Array{Uint8,1}:
 0xa2
 0x68
 0x69

julia> a = MsgPack.pack([1,2,"hi"])
6-element Array{Uint8,1}:
 0x93
 0x01
 0x02
 0xa2
 0x68
 0x69

julia> MsgPack.unpack(MsgPack.pack(4.5))
4.5

julia> f = open("in.mp")
julia> MsgPack.unpack(f)
"hello"

julia> f2 = open("out.mp", "w")
julia> MsgPack.pack(f2, [1,2,"hi"])



NOTE: The standard method for encoding integers in msgpack is to use the most compact representation possible, and to encode negative integers as signed ints and non-negative numbers as unsigned ints.

For compatibility with other implementations, I'm following this convention. On the unpacking side, every integer type becomes an Int64 in Julia, unless it doesn't fit (ie. values greater than 2^63 are unpacked as Uint64).

I might change this at some point, and/or provide a way to control the unpacked types.

The Extension Type

The MsgPack spec defines the extension type to be a tuple of (typecode, bytearray) where typecode is an application-specific identifier for the data in bytearray. MsgPack.jl provides support for the extension type through the Ext immutable.

It is defined like so

immutable Ext
    typecode::Int8
    data::Vector{Uint8}
end

and used like this

julia> a = [0x34, 0xff, 0x76, 0x22, 0xd3, 0xab]
6-element Array{UInt8,1}:
 0x34
 0xff
 0x76
 0x22
 0xd3
 0xab

julia> b = Ext(22, a)
MsgPack.Ext(22,UInt8[0x34,0xff,0x76,0x22,0xd3,0xab])

julia> p = pack(b)
9-element Array{UInt8,1}:
 0xc7
 0x06
 0x16
 0x34
 0xff
 0x76
 0x22
 0xd3
 0xab

julia> c = unpack(p)
MsgPack.Ext(22,UInt8[0x34,0xff,0x76,0x22,0xd3,0xab])

julia> c == b
true

MsgPack reserves typecodes in the range [-128, -1] for future types specified by the MsgPack spec. MsgPack.jl enforces this when creating an Ext but if you are packing an implementation defined extension type (currently there are none) you can pass impltype=true.

julia> Ext(-43, Uint8[1, 5, 3, 9])
ERROR: MsgPack Ext typecode -128 through -1 reserved by implementation
 in call at /Users/sean/.julia/v0.4/MsgPack/src/MsgPack.jl:48 

julia> Ext(-43, Uint8[1, 5, 3, 9], impltype=true)
MsgPack.Ext(-43,UInt8[0x01,0x05,0x03,0x09])
Serialization

MsgPack.jl also defines the extserialize and extdeserialize convenience functions. These functions can turn an arbitrary object into an Ext and vice-versa.

julia> type Point{T}
        x::T
        y::T
       end

julia> r = Point(2.5, 7.8)
Point{Float64}(2.5,7.8)

julia> e = MsgPack.extserialize(123, r)
MsgPack.Ext(123,UInt8[0x11,0x01,0x02,0x05,0x50,0x6f,0x69,0x6e,0x74,0x230x40,0x0e,0x33,0x33,0x33,0x33,0x33,0x33,0x1f,0x40])

julia> s = MsgPack.extdeserialize(e)
(123,Point{Float64}(2.5,7.8))

julia> s[2]
Point{Float64}(2.5,7.8)

julia> r
Point{Float64}(2.5,7.8)

Since these functions use serialize under the hood they are subject to the following caveat.

In general, this process will not work if the reading and writing are done by different versions of Julia, or an instance of Julia with a different system image.

edma2/clojure-msgpack https://github.com/edma2/clojure-msgpack

clojure-msgpack

clojure-msgpack is a lightweight and simple library for converting between native Clojure data structures and MessagePack byte formats. clojure-msgpack only depends on Clojure itself; it has no third-party dependencies.

Installation

Clojars Project Build Status

Usage
Basic
  • pack: Serialize object as a sequence of java.lang.Bytes.
  • unpack Deserialize bytes as a Clojure object.
(require '[msgpack.core :as msg])
(require 'msgpack.clojure-extensions)

(msg/pack {:compact true :schema 0})
; => #<byte[] [[email protected]>

(msg/unpack (msg/pack {:compact true :schema 0}))
; => {:schema 0, :compact true}
Streaming

clojure-msgpack provides a streaming API for situations where it is more convenient or efficient to work with byte streams instead of fixed byte arrays (e.g. size of object is not known ahead of time).

The streaming counterpart to msgpack.core/pack is msgpack.core/pack-stream which returns nil and accepts either java.io.OutputStream or java.io.DataOutput as an additional argument.

msgpack.core/unpack is in "streaming mode" when the argument is of type java.io.DataInput or java.io.InputStream.

(use 'clojure.java.io)

(with-open [s (output-stream "test.dat")]
  (msg/pack-stream {:compact true :schema 0} s))

(with-open [s (input-stream "test.dat")] (msg/unpack s))
; => {:schema 0, :compact true}
Core types
Clojure MessagePack
nil Nil
java.lang.Boolean Boolean
java.lang.Byte Integer
java.lang.Short Integer
java.lang.Integer Integer
java.lang.Long Integer
java.lang.BigInteger Integer
clojure.lang.BigInt Integer
java.lang.Float Float
java.lang.Double Float
java.math.BigDecimal Float
java.lang.String String
clojure.lang.Sequential Array
clojure.lang.IPersistentMap Map
msgpack.core.Ext Extended

Serializing a value of unrecognized type will fail with IllegalArgumentException. See Application types if you want to register your own types.

Clojure types

Some native Clojure types don't have an obvious MessagePack counterpart. We can serialize them as Extended types. To enable automatic conversion of these types, load the clojure-extensions library.

Clojure MessagePack
clojure.lang.Keyword Extended (type = 3)
clojure.lang.Symbol Extended (type = 4)
java.lang.Character Extended (type = 5)
clojure.lang.Ratio Extended (type = 6)
clojure.lang.IPersistentSet Extended (type = 7)

With msgpack.clojure-extensions:

(require 'msgpack.clojure-extensions)
(msg/pack :hello)
; => #<byte[] [[email protected]>

Without msgpack.clojure-extensions:

(msg/pack :hello)
; => IllegalArgumentException No implementation of method: :pack-stream of
; protocol: #'msgpack.core/Packable found for class: clojure.lang.Keyword
; clojure.core/-cache-protocol-fn (core _deftype.clj:544)
Application types

You can also define your own Extended types with extend-msgpack.

(require '[msgpack.macros :refer [extend-msgpack]])

(defrecord Person [name])

(extend-msgpack
  Person
  100
  [p] (.getBytes (:name p))
  [bytes] (->Person (String. bytes)))

(msg/unpack (msg/pack [(->Person "bob") 5 "test"]))
; => (#user.Person{:name "bob"} 5 "test")
Options

All pack and unpack functions take an optional map of options:

  • :compatibility-mode Serialize/deserialize strings and bytes using the raw-type defined here: https://github.com/msgpack/msgpack/blob/master/spec-old.md

    Note: No error is thrown if an unpacked value is reserved under the old spec but defined under the new spec. We always deserialize something if we can regardless of compatibility-mode.

(msg/pack (byte-array (byte 9)) {:compatibility-mode true})
License

clojure-msgpack is MIT licensed. See the included LICENSE file for more details.

tkob/mlmsgpack https://github.com/tkob/mlmsgpack

ML-MessagePack

MessagePack implementation for Standard ML (SML)

Features
  • Portable: Depends only on the required components of the SML Basis Library specification.
  • Composable: Composable combinators for encoding and decoding.
Usage
MLton and MLKit

Include mlmsgpack.mlb in your MLB file.

Poly/ML

From the interactive shell, use .sml files in the following order.

  • mlmsgpack-aux.sml
  • realprinter-default.sml
  • mlmsgpack.sml
SML/NJ

Use mlmsgpack.cm.

Moscow ML

From the interactive shell, use .sml files in the following order.

  • large.sml
  • mlmsgpack-aux.sml
  • realprinter-fail.sml
  • mlmsgpack.sml

Makefile.mosml is also provided.

HaMLet

From the interactive shell, use .sml files in the following order.

  • mlmsgpack-aux.sml
  • realprinter-fail.sml
  • mlmsgpack.sml
Alice ML

Makefile.alice is provided.

make -f Makefile.alice
alicerun mlmsgpack-test
SML#

For separate compilation, .smi files are provided. Require mlmsgpack.smi from your .smi file.

From the interactive shell, use .sml files in the following order.

  • mlmsgpack-aux.sml
  • realprinter-default.sml
  • mlmsgpack.sml
Tutorial

See TUTORIAL.md.

Known Problems

Our recommendation is MLton, MLKit, Poly/ML and SML#(>=2.0.0) as all tests passed on them. SML/NJ and Moscow ML are fine if you don't use real values.

SML/NJ

Packing real values fail or produces imprecise results in some cases.

Moscow ML

Packing real values is not supported, since some components of the SML Basis Library are not provided.

HaMLet

Packing real values is not supported, since some components of the SML Basis Library are not provided. Some functions are very slow, although they work properly. (We tested HaMLet compiled with MLton.)

Alice ML

Packing real values is not supported, since some components of the SML Basis Library are not provided. Also, some unit tests fail.

SML#

Most functions do not work properly because of bugs of SML# prior to version 2.0.0.

See Also

There already exists another MessagePack implemenatation for SML, called MsgPack-SML, which is targeted for MLton.

https://msgpacksml.codeplex.com/

ML-MessagePack is written from scratch and not a fork of MsgPack-SML.

For information on MessagePack, see:

http://msgpack.org/

camgunz/cmp https://github.com/camgunz/cmp

cmp

CMP is a C implementation of the MessagePack serialization format. It currently implements version 5 of the MessagePack Spec.

CMP's goal is to be lightweight and straightforward, forcing nothing on the programmer.

License

While I'm a big believer in the GPL, I license CMP under the MIT license.

Example Usage

The following examples use a file as the backend, and are modeled after the examples included with the msgpack-c project.

#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>

#include "cmp.h"

static bool read_bytes(void *data, size_t sz, FILE *fh) {
    return fread(data, sizeof(uint8_t), sz, fh) == (sz * sizeof(uint8_t));
}

static bool file_reader(cmp_ctx_t *ctx, void *data, size_t limit) {
    return read_bytes(data, limit, (FILE *)ctx->buf);
}

static size_t file_writer(cmp_ctx_t *ctx, const void *data, size_t count) {
    return fwrite(data, sizeof(uint8_t), count, (FILE *)ctx->buf);
}

void error_and_exit(const char *msg) {
    fprintf(stderr, "%s\n\n", msg);
    exit(EXIT_FAILURE);
}

int main(void) {
    FILE *fh = NULL;
    cmp_ctx_t cmp;
    uint32_t array_size = 0;
    uint32_t str_size = 0;
    char hello[6] = {0, 0, 0, 0, 0, 0};
    char message_pack[12] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};

    fh = fopen("cmp_data.dat", "w+b");

    if (fh == NULL)
        error_and_exit("Error opening data.dat");

    cmp_init(&cmp, fh, file_reader, file_writer);

    if (!cmp_write_array(&cmp, 2))
        error_and_exit(cmp_strerror(&cmp));

    if (!cmp_write_str(&cmp, "Hello", 5))
        error_and_exit(cmp_strerror(&cmp));

    if (!cmp_write_str(&cmp, "MessagePack", 11))
        error_and_exit(cmp_strerror(&cmp));

    rewind(fh);

    if (!cmp_read_array(&cmp, &array_size))
        error_and_exit(cmp_strerror(&cmp));

    /* You can read the str byte size and then read str bytes... */

    if (!cmp_read_str_size(&cmp, &str_size))
        error_and_exit(cmp_strerror(&cmp));

    if (str_size > (sizeof(hello) - 1))
        error_and_exit("Packed 'hello' length too long\n");

    if (!read_bytes(hello, str_size, fh))
        error_and_exit(cmp_strerror(&cmp));

    /*
     * ...or you can set the maximum number of bytes to read and do it all in
     * one call
     */

    str_size = sizeof(message_pack);
    if (!cmp_read_str(&cmp, message_pack, &str_size))
        error_and_exit(cmp_strerror(&cmp));

    printf("Array Length: %zu.\n", array_size);
    printf("[\"%s\", \"%s\"]\n", hello, message_pack);

    fclose(fh);

    return EXIT_SUCCESS;
}
Advanced Usage

See the examples folder.

Fast, Lightweight, Flexible, and Robust

CMP uses no internal buffers; conversions, encoding and decoding are done on the fly.

CMP's source and header file together are ~3,300 LOC.

CMP makes no heap allocations.

CMP uses standardized types rather than declaring its own, and it depends only on stdbool.h, stdint.h and string.h. It has no link-time dependencies, not even the C Standard Library.

CMP is written using C89 (ANSI C), aside, of course, from its use of fixed-width integer types and bool.

On the other hand, CMP's test suite depends upon the C Standard Library and requires C99.

CMP only requires the programmer supply a read function and a write function. In this way, the programmer can use CMP on memory, files, sockets, etc.

CMP is portable. It uses fixed-width integer types, and checks the endianness of the machine at runtime before swapping bytes (MessagePack is big-endian).

CMP provides a fairly comprehensive error reporting mechanism modeled after errno and strerror.

CMP is threadsafe; while contexts cannot be shared between threads, each thread may use its own context freely.

CMP is tested using the MessagePack test suite as well as a large set of custom test cases. Its small test program is compiled with clang using -Wall -Werror -Wextra ... along with several other flags, and generates no compilation errors.

CMP's source is written as readably as possible, using explicit, descriptive variable names and a consistent, clear style.

CMP's source is written to be as secure as possible. Its testing suite checks for invalid values, and data is always treated as suspect before it passes validation.

CMP's API is designed to be clear, convenient and unsurprising. Strings are null-terminated, binary data is not, error codes are clear, and so on.

CMP provides optional backwards compatibility for use with other MessagePack implementations that only implement version 4 of the spec.

Building

There is no build system for CMP. The programmer can drop cmp.c and cmp.h in their source tree and modify as necessary. No special compiler settings are required to build it, and it generates no compilation errors in either clang or gcc.

Backwards Compatibility

Version 4 of the MessagePack spec has no BIN type, and provides no STR8 marker. In order to remain backwards compatible with version 4 of MessagePack, do the following:

Avoid these functions:

  • cmp_write_bin
  • cmp_write_bin_marker
  • cmp_write_str8_marker
  • cmp_write_str8
  • cmp_write_bin8_marker
  • cmp_write_bin8
  • cmp_write_bin16_marker
  • cmp_write_bin16
  • cmp_write_bin32_marker
  • cmp_write_bin32

Use these functions in lieu of their v5 counterparts:

  • cmp_write_str_marker_v4 instead of cmp_write_str_marker
  • cmp_write_str_v4 instead of cmp_write_str
  • cmp_write_object_v4 instead of cmp_write_object

danellis/dart-msgpack https://github.com/danellis/dart-msgpack

dart-msgpack

This is a very early release of my MessagePack library for Dart. Currently, message classes must be written by hand. For example:

class NotificationFrame extends Message {
    String kind;
    Map<String, Object> data;

    NotificationFrame(this.kind, this.data);

    static NotificationFrame fromList(List f) => new NotificationFrame(f[0], f[1]);
    List toList() => [kind, data];
}

For each class you need to define the fromList and toList methods, which convert from and to a list of fields respectively.

For example usage, see the unit tests.

reeze/msgpack-hhvm https://github.com/reeze/msgpack-hhvm

msgpack-hhvm

Build Status: Build Status

Msgpack for HHVM, It is a msgpack binding for HHVM

API
  • msgpack_pack(mixed $input) : string; pack a input to msgpack, object and resource are not supported, array and other types supported, false on failure.
  • msgpack_unpack(string $pac) : mixed; unpack a msgpack.
Installation
$ git clone https://github.com/reeze/msgpack-hhvm --depth=1
$ cd msgpack-hhvm
$ hphpize && cmake . && make
$ cp msgpack.so /path/to/your/hhvm/ext/dir

If you don't have hphpize program, please intall package hhvm-dev

$ sudo apt-get install hhvm-dev
Contribution and Issues

Feel free to send Pull Requests for bug report at: http://github.com/reeze/msgpack-hhvm/issues

Authors

Gab-km/msgpack-fsharp https://github.com/Gab-km/msgpack-fsharp

MessagePack for F#
Build status
What is this?

MessagePack is a fast and compact binary serialization library.

MessagePack for F# is a MessagePack implementation of F#, by F#, for F#.

Usage
open MsgPack

[| 1uy; 2uy; 3uy |]
|> Array.map (Value.UInt8)
|> Value.Array
|> Packer.packOne
//=> val it : byte [] = [|147uy; 1uy; 2uy; 3uy|]

Unpacker.unpack [|147uy; 1uy; 2uy; 3uy|]
//=> [|Value.Array [|Value.UInt8 1uy; Value.UInt8 2uy; Value.UInt8 3uy|]|]
Copyright

Copyright (c) 2014- Kazuhiro Matsushima

License

Distributed under the Apache License, Version 2.0 .

komamitsu/jackson-dataformat-msgpack https://github.com/komamitsu/jackson-dataformat-msgpack

jackson-dataformat-msgpack

This project is merged to msgpack-java!! Yay!

See msgpack-java/msgpack-jackson for the updated documents

Overview

Build Status

This Jackson extension library handles reading and writing of data encoded in MessagePack data format. It extends standard Jackson streaming API (JsonFactory, JsonParser, JsonGenerator), and as such works seamlessly with all the higher level data abstractions (data binding, tree model, and pluggable extensions).

Maven dependency

To use this module on Maven-based projects, use following dependency:

<dependency>
  <groupId>org.komamitsu</groupId>
  <artifactId>jackson-dataformat-msgpack</artifactId>
  <version>0.0.3</version>
</dependency>
Usage

Only thing you need to do is to instantiate MessagePackFactory and pass it to the constructor of ObjectMapper.

  ObjectMapper objectMapper = new ObjectMapper(new MessagePackFactory());
  ExamplePojo orig = new ExamplePojo("komamitsu");
  byte[] bytes = objectMapper.writeValueAsBytes(orig);
  ExamplePojo value = objectMapper.readValue(bytes, ExamplePojo.class);
  System.out.println(value.getName()); // => komamitsu

Also, you can exchange data among multiple languages.

Java

  // Serialize
  Map<String, Object> obj = new HashMap<String, Object>();
  obj.put("foo", "hello");
  obj.put("bar", "world");
  byte[] bs = objectMapper.writeValueAsBytes(obj);
  // bs => [-126, -93, 102, 111, 111, -91, 104, 101, 108, 108, 111,
  //        -93, 98, 97, 114, -91, 119, 111, 114, 108, 100]

Ruby

  require 'msgpack'

  # Deserialize
  xs = [-126, -93, 102, 111, 111, -91, 104, 101, 108, 108, 111,
        -93, 98, 97, 114, -91, 119, 111, 114, 108, 100]
  MessagePack.unpack(xs.pack("C*"))
  # => {"foo"=>"hello", "bar"=>"world"}

  # Serialize
  ["zero", 1, 2.0, nil].to_msgpack.unpack('C*')
  # => [148, 164, 122, 101, 114, 111, 1, 203, 64, 0, 0, 0, 0, 0, 0, 0, 192]

Java

  // Deserialize
  bs = new byte[] {(byte) 148, (byte) 164, 122, 101, 114, 111, 1,
                   (byte) 203, 64, 0, 0, 0, 0, 0, 0, 0, (byte) 192};
  TypeReference<List<Object>> typeReference = new TypeReference<List<Object>>(){};
  List<Object> xs = objectMapper.readValue(bs, typeReference);
  // xs => [zero, 1, 2.0, null]

briandw/SwiftPack https://github.com/briandw/SwiftPack

SwiftPack

SwiftPack is MessagePack packer and unpacker written almost entirely in Swift.

Issues
  • There are few Objective-C dependencies, mostly NSData. I hope to remove these dependencies as Swift matures.
  • Maps currently only support String types for keys.
Copyright and License

Copyright (c) 2014 Brian Williams

This software is licensed under the terms of the MIT license.

gabriel/MPMessagePack https://github.com/gabriel/MPMessagePack

Install
pod "MPMessagePack"
Writing
#import <MPMessagePack/MPMessagePack.h>

NSDictionary *dict =
@{
@"n": @(32134123),
@"bool": @(YES),
@"array": @[@(1.1f), @(2.1)],
@"body": [NSData data],
};

NSData *data = [dict mp_messagePack];

Or via MPMessagePackWriter.

NSError *error = nil;
NSData *data = [MPMessagePackWriter writeObject:dict error:&error];

If you need to use an ordered dictionary.

MPOrderedDictionary *dict = [[MPOrderedDictionary alloc] init];
[dict addEntriesFromDictionary:@{@"c": @(1), @"b": @(2), @"a": @(3)}];
[dict sortKeysUsingSelector:@selector(localizedCaseInsensitiveCompare:)];
[dict mp_messagePack];
Reading
id obj = [MPMessagePackReader readData:data error:&error];
MPMessagePackReader *reader = [[MPMessagePackReader alloc] initWithData:data];
id obj1 = [reader read:&error]; // Read an object
id obj2 = [reader read:&error]; // Read another object

mcollina/msgpack5 https://github.com/mcollina/msgpack5

msgpack5  Build Status

A msgpack v5 implementation for node.js and the browser, with extension point support.

Install
npm install msgpack5 --save
Usage
var msgpack = require('msgpack5')() // namespace our extensions
  , a       = new MyType(2, 'a')
  , encode  = msgpack.encode
  , decode  = msgpack.decode

msgpack.register(0x42, MyType, mytipeEncode, mytipeDecode)

console.log(encode({ 'hello': 'world' }).toString('hex'))
// 81a568656c6c6fa5776f726c64
console.log(decode(encode({ 'hello': 'world' })))
// { hello: 'world' }
console.log(encode(a).toString('hex'))
// d5426161
console.log(decode(encode(a)) instanceof MyType)
// true
console.log(decode(encode(a)))
// { value: 'a', size: 2 }

function MyType(size, value) {
  this.value = value
  this.size  = size
}

function mytipeEncode(obj) {
  var buf = new Buffer(obj.size)
  buf.fill(obj.value)
  return buf
}

function mytipeDecode(data) {
  var result = new MyType(data.length, data.toString('utf8', 0, 1))
    , i

  for (i = 0; i < data.length; i++) {
    if (data.readUInt8(0) != data.readUInt8(i)) {
      throw new Error('should all be the same')
    }
  }

  return result
}
In the Browser

This library is compatible with Browserify.

If you want to use standalone, grab the file in the dist folder of this repo, and use in your own HTML page, the module will expose a msgpack5 global.

<script type="text/javascript"
        src="./msgpack5.min.js">
</script>
To build
    npm run build
API

API

msgpack(options(obj))

Creates a new instance on which you can register new types for being encoded.

options:

  • forceFloat64, a boolean to that forces all floats to be encoded as 64-bits floats. Defaults to false.
  • compatibilityMode, a boolean that enables "compatibility mode" which doesn't use str 8 format. Defaults to false.

encode(object)

Encodes object in msgpack, returns a bl.


decode(buf)

Decodes buf from in msgpack. buf can be a Buffer or a bl instance.

In order to support a stream interface, a user must pass in a bl instance.


registerEncoder(check(obj), encode(obj))

Register a new custom object type for being automatically encoded. The arguments are:

  • check, a function that will be called to check if the passed object should be encoded with the encode function
  • encode, a function that will be called to encode an object in binary form; this function must return a Buffer which include the same type for registerDecoder.

registerDecoder(type, decode(buf))

Register a new custom object type for being automatically decoded. The arguments are:

  • type, is a greater than zero integer identificating the type once serialized
  • decode, a function that will be called to decode the object from the passed Buffer

register(type, constructor, encode(obj), decode(buf))

Register a new custom object type for being automatically encoded and decoded. The arguments are:

  • type, is a greater than zero integer identificating the type once serialized
  • constructor, the function that will be used to match the objects with instanceof
  • encode, a function that will be called to encode an object in binary form; this function must return a Buffer that can be deserialized by the decode function
  • decode, a function that will be called to decode the object from the passed Buffer

This is just a commodity that calls registerEncoder and registerDecoder internally.


encoder()

Builds a stream in object mode that encodes msgpack.


decoder()

Builds a stream in object mode that decodes msgpack.

LevelUp Support

msgpack5 can be used as a LevelUp valueEncoding straight away:

var level = require('level')
  , pack  = msgpack()
  , db    = level('foo', {
      valueEncoding: pack
    })
  , obj   = { my: 'obj' }

db.put('hello', obj, function(err) {
  db.get('hello', function(err, result) {
    console.log(result)
    db.close()
  })
})
Related projects
Disclaimer

This library is built fully on JS and on bl to simplify the code. Every improvement that keeps the same API is welcome.

Acknowledgements

This project was kindly sponsored by nearForm.

This library was originally built as the data format for JSChan.

License

MIT

rodrigosetti/messagepack http://hackage.haskell.org/package/messagepack

MessagePack for Haskell

This implementation defines an messagepack Object type, which is an instance of Serialize (from cereal ):

data Object = ObjectNil
            | ObjectUInt   Word64
            | ObjectInt    Int64
            | ObjectBool   Bool
            | ObjectFloat  Float
            | ObjectDouble Double
            | ObjectString ByteString
            | ObjectBinary ByteString
            | ObjectArray  [Object]
            | ObjectMap    (M.Map Object Object )
            | ObjectExt    !Int8 BS.ByteString
    deriving (Eq, Ord, Show)

instance Serialize Object where
    -- ...

Thus, you can use cereal's encode and decode to pack and unpack objects.

ymofen/msgpack-delphi https://github.com/ymofen/msgpack-delphi

Msgpack for Delphi

It's like JSON but small and fast.

unit Owner: D10.Mofen, qdac.swish
contact:
       qq:185511468, 
    email:[email protected]
welcome to report bug
Works with
  • Delphi 7 (tested)
  • Delphi 2007 (tested)
  • XE5, XE6, XE7, FMX (tested)
changes:
  • first release 2014-08-15 13:05:13

  • add array support 2014-08-19 12:18:47

  • add andriod support 2014-09-08 00:45:27

  • fixed int32, int64 parse bug< integer, int64 parse zero> 2014-11-09 22:35:27

  • add EncodeToFile/DecodeFromFile 2014-11-13 12:30:58

  • fix asVariant = null (thanks for cyw(26890954)) 2014-11-14 09:05:52

  • fix AsInteger = -1 bug (thanks for cyw(26890954)) 2014-11-14 12:15:52

  • fix AsInteger = -127 bug check int64/integer/cardinal/word/shortint/smallint/byte assign, encode,decode, read 2014-11-14 12:30:38

  • fix AsFloat = 2.507182 bug thanks fo [珠海]-芒果 1939331207 2014-11-21 12:37:04

  • add AddArrayChild func 2015-03-25 17:47:28

  • add remove/removeFromParent/Delete function 2015-08-29 22:37:48

Code Example
var
  lvMsg, lvMsg2:TSimpleMsgPack;
  lvBytes:TBytes;
  s:string;
begin
  lvMsg := TSimpleMsgPack.Create;
  lvMsg.S['key.obj'] := '汉字,ascii';
  if dlgOpen.Execute then
  begin
    lvMsg.S['key.image.name'] := ExtractFileName(dlgOpen.FileName);

    // file binary data
    lvMsg.ForcePathObject('key.image.data').LoadBinaryFromFile(dlgOpen.FileName);
  end;

  //
  lvBytes := lvMsg.EncodeToBytes;

  lvMsg2 := TSimpleMsgPack.Create;
  lvMsg2.DecodeFromBytes(lvBytes);
  //
  Memo1.Lines.Add(lvMsg2.S['key.obj']);
  if lvMsg2.S['key.image.name'] <> '' then
  begin
    s := ExtractFilePath(ParamStr(0)) + lvMsg2.S['key.image.name'];
    Memo1.Lines.Add('file saved');
    Memo1.Lines.Add(s);
    lvMsg2.ForcePathObject('key.image.data').SaveBinaryToFile(s);    
  end;

chinawsb/qmsgpack-delphi http://www.qdac.cc

QMsgPack-Messagepack for Delphi/C++ Builder

QMsgPack is a simple and powerful Delphi & C++ Builder implementation for messagepack protocol. QMsgPack is a part of QDAC 3.0,Source code hosted in Sourceforge(http://sourceforge.net/p/qdac3).

Feathers

· Full types support,include messagepack extension type

· Full open source,free for used in ANY PURPOSE

· Quick and simple interface

· RTTI support include

Install

QMsgPack is not a desgin time package.So just place QMsgPack files into search path and add to your project.

Support

· Topic in Website (http://www.qdac.cc/?cat=44) ,CHINESE only

· Mail to author ([email protected])

· Post in forum (http://tieba.baidu.com/f?kw=qdac)

· QQ Group No:250530692 (http://jq.qq.com/?_wv=1027&k=ZH6mZR)

Source check out

· HTTP (http://svn.code.sf.net/p/qdac3/code/)

· SVN (svn://svn.code.sf.net/p/qdac3/code/)

Example
var
  lvMsg, lvMsg2:TQMsgPack;
  lvBytes:TBytes;
  s:string;
begin
  lvMsg := TQMsgPack.Create;
  lvMsg.ForcePath('key.obj').AsString := '汉字,ascii';

  //
  lvBytes := lvMsg.Encode;

  lvMsg2 := TQMsgPack.Create;
  lvMsg2.Parse(lvBytes);
  //
  showMessage(lvMsg.ForcePath('key.obj').AsString);
  ....

romixlab/qmsgpack https://github.com/romixlab/qmsgpack

MessagePack for Qt

Full documentation is here mp.marsw.ru

Installation

Clone repository:

git clone https://github.com/romixlab/qmsgpack.git
cd qmsgpack
mkdir build
cd build
cmake ..
make install
Sample usage

Packing

QVariantList list;
list << 1 << 2 << 3;
QByteArray array = MsgPack::pack(list);

Unpacking:

QVariantList unpacked = MsgPack::unpack(array).toList();

Streaming API:

// packing
MsgPackStream stream(&ba, QIODevice::WriteOnly);
stream << 1 << 2.3 << "some string";

// unpacking
MsgPackStream stream(ba);
int a;
double b;
QSting s;
stream >> a >> b >> s;
Qt types and User types

There is packers and unpackers for QColor, QTime, QDate, QDateTime, QPoint, QSize, QRect. Also you can create your own packer/unpacker methods for Qt or your own types. See docs for details.

tinylib/msgp https://github.com/tinylib/msgp

MessagePack Code Generator Build Status

forthebadge forthebadge

This is a code generation tool and serialization library for MessagePack. It is targeted at the go generate tool. You can read more about MessagePack in the wiki, or at msgpack.org.

Why? Quickstart

Note: you need at least go 1.3 to compile this package, and at least go 1.4 to use go generate.

In a source file, include the following directive:

//go:generate msgp

The msgp command will generate serialization methods for all exported type declarations in the file.

You can read more about the code generation options here.

Use

Field names can be set in much the same way as the encoding/json package. For example:

type Person struct {
    Name       string `msg:"name"`
    Address    string `msg:"address"`
    Age        int    `msg:"age"`
    Hidden     string `msg:"-"` // this field is ignored
    unexported bool             // this field is also ignored
}

By default, the code generator will satisfy msgp.Sizer, msgp.Encodable, msgp.Decodable, msgp.Marshaler, and msgp.Unmarshaler. Carefully-designed applications can use these methods to do marshalling/unmarshalling with zero allocations.

While msgp.Marshaler and msgp.Unmarshaler are quite similar to the standard library's json.Marshaler and json.Unmarshaler, msgp.Encodable and msgp.Decodable are useful for stream serialization. (*msgp.Writer and *msgp.Reader are essentially protocol-aware versions of *bufio.Writer and *bufio.Reader, respectively.)

Features
  • Extremely fast generated code
  • Test and benchmark generation
  • JSON interoperability (see msgp.CopyToJSON() and msgp.UnmarshalAsJSON())
  • Support for complex type declarations
  • Native support for Go's time.Time, complex64, and complex128 types
  • Generation of both []byte-oriented and io.Reader/io.Writer-oriented methods
  • Support for arbitrary type system extensions
  • Preprocessor directives

Consider the following:

const Eight = 8
type MyInt int
type Data []byte

type Struct struct {
    Which  map[string]*MyInt `msg:"which"`
    Other  Data              `msg:"other"`
    Nums   [Eight]float64    `msg:"nums"`
}

As long as the declarations of MyInt and Data are in the same file as Struct, the parser will determine that the type information for MyInt and Data can be passed into the definition of Struct before its methods are generated.

Extensions

MessagePack supports defining your own types through "extensions," which are just a tuple of the data "type" (int8) and the raw binary. You can see a worked example in the wiki.

Status

Alpha. I will break stuff. There is an open milestone for Beta stability (targeted for January.) Only the /msgp sub-directory will have a stability guarantee.

You can read more about how msgp maps MessagePack types onto Go types in the wiki.

Here some of the known limitations/restrictions:

  • Identifiers from outside the processed source file are assumed (optimistically) to satisfy the generator's interfaces. If this isn't the case, your code will fail to compile.
  • Like most serializers, chan and func fields are ignored, as well as non-exported fields.
  • Encoding of interface{} is limited to built-ins or types that have explicit encoding methods.
  • Maps must have string keys. This is intentional (as it preserves JSON interop.) Although non-string map keys are not forbidden by the MessagePack standard, many serializers impose this restriction. (It also means any well-formed struct can be de-serialized into a map[string]interface{}.) The only exception to this rule is that the deserializers will allow you to read map keys encoded as bin types, due to the fact that some legacy encodings permitted this. (However, those values will still be cast to Go strings, and they will be converted to str types when re-encoded. It is the responsibility of the user to ensure that map keys are UTF-8 safe in this case.) The same rules hold true for JSON translation.

If the output compiles, then there's a pretty good chance things are fine. (Plus, we generate tests for you.) Please, please, please file an issue if you think the generator is writing broken code.

Performance

If you like benchmarks, see here.

As one might expect, the generated methods that deal with []byte are faster, but the io.Reader/Writer methods are generally more memory-efficient for large (> 2KB) objects.

jakm/msgpack-cli https://github.com/jakm/msgpack-cli

msgpack-cli

msgpack-cli is command line tool that converts data from JSON to Msgpack and vice versa. Also allows calling RPC methods via msgpack-rpc.

Installation
% go get github.com/jakm/msgpack-cli

Debian packages and Windows binaries are available on project's Releases page.

Usage
msgpack-cli

Usage:
    msgpack-cli encode <input-file> [--out=<output-file>] [--disable-int64-conv]
    msgpack-cli decode <input-file> [--out=<output-file>] [--pp]
    msgpack-cli rpc <host> <port> <method> [<params>|--file=<input-file>] [--pp]
        [--timeout=<timeout>][--disable-int64-conv]
    msgpack-cli -h | --help
    msgpack-cli --version

Commands:
    encode                Encode data from input file to STDOUT
    decode                Decode data from input file to STDOUT
    rpc                   Call RPC method and write result to STDOUT

Options:
    -h --help             Show this help message and exit
    --version             Show version
    --out=<output-file>   Write output data to file instead of STDOUT
    --file=<input-file>   File where parameters or RPC method are read from
    --pp                  Pretty-print - indent output JSON data
    --timeout=<timeout>   Timeout of RPC call [default: 30]
    --disable-int64-conv  Disable the default behaviour such that JSON numbers
                          are converted to float64 or int64 numbers by their
                          meaning, all result numbers will have float64 type


Arguments:
    <input-file>          File where data are read from
    <host>                Server hostname
    <port>                Server port
    <method>              Name of RPC method
    <params>              Parameters of RPC method in JSON format
Examples

Encoding/decoding:

$ cat test.json
{
  "firstName": "John",
  "lastName": "Smith",
  "isAlive": true,
  "age": 25,
  "height_cm": 167.6,
  "address": {
    "streetAddress": "21 2nd Street",
    "city": "New York",
    "state": "NY",
    "postalCode": "10021-3100"
  },
  "phoneNumbers": [
    {
      "type": "home",
      "number": "212 555-1234"
    },
    {
      "type": "office",
      "number": "646 555-4567"
    }
  ],
  "children": [],
  "spouse": null
}
$
$ msgpack-cli encode test.json --out test.bin
$
$ ls -l test.* | awk '{print $9, $5}'
test.bin 242
test.json 429
$
$ msgpack-cli decode test.bin --pp  # pretty-print
{
  "address": {
    "city": "New York",
    "postalCode": "10021-3100",
    "state": "NY",
    "streetAddress": "21 2nd Street"
  },
  "age": 25,
  "children": [],
  "firstName": "John",
  "height_cm": 167.6,
  "isAlive": true,
  "lastName": "Smith",
  "phoneNumbers": [
    {
      "number": "212 555-1234",
      "type": "home"
    },
    {
      "number": "646 555-4567",
      "type": "office"
    }
  ],
  "spouse": null
}

RPC calling:

$ # zero params
$ msgpack-cli rpc localhost 8000 echo
[]
$
$ # single param
$ msgpack-cli rpc localhost 8000 echo 3.14159
[3.14159]
$
$ # multiple params (as json array)
$ msgpack-cli rpc localhost 8000 echo '["abc", "def", "ghi", {"A": 65, "B": 66, "C": 67}]'
["abc","def","ghi",{"A":65,"B":66,"C":67}]

ymofen/SimpleMsgPack.Net https://github.com/ymofen/SimpleMsgPack.Net

SimpleMsgPack.Net

MessagePack implementation for C# / msgpack.org[C#]

Binary files distributed via the NuGet package SimpleMsgPack.

It's like JSON but small and fast.

unit Owner: D10.Mofen
contact:
       qq:185511468, 
    email:[email protected]
    homepage:www.diocp.org
if you find any bug, please contact me!
Works with

.NET Framework 4.x

Code Example
    MsgPack msgpack = new MsgPack();
    msgpack.ForcePathObject("p.name").AsString = "张三";
    msgpack.ForcePathObject("p.age").AsInteger = 25;
    msgpack.ForcePathObject("p.datas").AsArray.Add(90);
    msgpack.ForcePathObject("p.datas").AsArray.Add(80);
    msgpack.ForcePathObject("p.datas").AsArray.Add("李四");
    msgpack.ForcePathObject("p.datas").AsArray.Add(3.1415926);

    // pack file
    msgpack.ForcePathObject("p.filedata").LoadFileAsBytes("C:\\a.png");

    // pack msgPack binary
    byte[] packData = msgpack.Encode2Bytes();

    MsgPack unpack_msgpack = new MsgPack();

    // unpack msgpack
    unpack_msgpack.DecodeFromBytes(packData);

    System.Console.WriteLine("name:{0}, age:{1}",
          unpack_msgpack.ForcePathObject("p.name").AsString,
          unpack_msgpack.ForcePathObject("p.age").AsInteger);

    Console.WriteLine("==================================");
    System.Console.WriteLine("use index property, Length{0}:{1}",
          unpack_msgpack.ForcePathObject("p.datas").AsArray.Length,
          unpack_msgpack.ForcePathObject("p.datas").AsArray[0].AsString
          );

    Console.WriteLine("==================================");
    Console.WriteLine("use foreach statement:");
    foreach (MsgPack item in unpack_msgpack.ForcePathObject("p.datas"))
    {
        Console.WriteLine(item.AsString);
    }

    // unpack filedata 
    unpack_msgpack.ForcePathObject("p.filedata").SaveBytesToFile("C:\\b.png");
    Console.Read();

ymofen/thealchemist https://github.com/ymofen/thealchemist

The Alchemist

Build Status

Elixir implementation bundle for MessagePack

The Alchemist

Elixir implementation relies on mururu/msgpack-elixir GitHub version

Pure erlang implementation used with msgpack/msgpack-erlang GitHub version

Usage
 In pure elixir:
# pack
TheAlchemist.pack([1,2,3]) #=> { :ok, <<147,1,2,3>> }
TheAlchemist.pack!([1,2,3]) #=> <<147,1,2,3>>

# unpack
TheAlchemist.unpack(<<147,1,2,3>>) #=> { :ok, [1,2,3] }
TheAlchemist.unpack!(<<147,1,2,3>>) #=> [1,2,3]

# unpack_once
TheAlchemist.unpack_once(<<147,1,2,3,4>>) #=> {:ok, {[1, 2, 3], <<4>>}}
TheAlchemist.unpack_once!(<<147,1,2,3,4>>) #=> {[1, 2, 3], <<4>>}
 With underlying erlang: Deserialization
msgpacked = TheAlchemistErl.pack('{"test":false}')
TheAlchemistErl.unpack(msgpacked)
Stream deserialization
mat = TheAlchemistErl.unpack_stream(data)
TheAlchemistErl.unpack_stream(elem(mat, 1)),
String type

Now this supports string type!

opt = [{:enable_str, true}]
TheAlchemistErl.unpack(TheAlchemistErl.pack("埼玉kanji", opt), opt)
 => {:ok, "埼玉kanji"}
Map Style

Since Erlang/OTP 17.0

msgpack:pack(#{ <<"key">> => <<"value">> }, [{format, map}]).

Or use old jiffy/jsx style

msgpack:pack({[{<<"key">>, <<"value">>}]}, [{format, jiffy}]),
msgpack:pack([{<<"key">>, <<"value">>}], [{format, jsx}]).
Running interactively

Firstly make all to ensure all source files are compiled after that you can bring the ERL shell with

make run

If you want to bring the IEX you can type

make runex

msgpack4z/msgpack4z-core https://github.com/msgpack4z/msgpack4z-core

msgpack4z-core

Build Status Maven Central scaladoc Reference Status Scala.js

example latest stable version
libraryDependencies += "com.github.xuwei-k" %% "msgpack4z-core" % "0.3.3"
snapshot version
libraryDependencies += "com.github.xuwei-k" %% "msgpack4z-core" % "0.3.4-SNAPSHOT"

resolvers += Opts.resolver.sonatypeSnapshots
dependency graph

graph

for scalaz 7.1.x

https://github.com/msgpack4z/msgpack4z-core/tree/0.2.x

jakm/txmsgpackrpc https://github.com/jakm/txmsgpackrpc

txmsgpackrpc

For the latest source code, see http://github.com/jakm/txmsgpackrpc

txmsgpackrpc is a library for writing asynchronous msgpack-rpc servers and clients in Python, using Twisted framework. Library is based on txMsgpack, but some improvements and fixes were made.

Features
  • user friendly API
  • modular object model
  • working timeouts and reconnecting
  • connection pool support
  • TCP, SSL, UDP and UNIX sockets
Python 3 note

To use UNIX sockets with Python 3 please use Twisted framework 15.3.0 and above.

Dependencies
Installation
% pip install txmsgpackrpc

Debian packages are available on project's Releases page.

TCP example

Computation of PI using Chudnovsky algorithm in subprocess. For details, see http://www.craig-wood.com/nick/articles/pi-chudnovsky/.

Results
Computation of PI with 5 places finished in 0.022390 seconds

Computation of PI with 100 places finished in 0.037856 seconds

Computation of PI with 1000 places finished in 0.038070 seconds

Computation of PI with 10000 places finished in 0.073907 seconds

Computation of PI with 100000 places finished in 6.741683 seconds

Computation of PI with 5 places finished in 0.001142 seconds

Computation of PI with 100 places finished in 0.001182 seconds

Computation of PI with 1000 places finished in 0.001206 seconds

Computation of PI with 10000 places finished in 0.001230 seconds

Computation of PI with 100000 places finished in 0.001255 seconds

Computation of PI with 1000000 places finished in 432.574457 seconds

Computation of PI with 1000000 places finished in 402.551226 seconds

DONE
Server
from __future__ import print_function

from collections import defaultdict
from twisted.internet import defer, reactor, utils
from twisted.python import failure
from txmsgpackrpc.server import MsgpackRPCServer


pi_chudovsky_bs = '''
"""
Python3 program to calculate Pi using python long integers, binary
splitting and the Chudnovsky algorithm

See: http://www.craig-wood.com/nick/articles/pi-chudnovsky/ for more
info

Nick Craig-Wood <[email protected]>
"""

import math
from time import time

def sqrt(n, one):
    """
    Return the square root of n as a fixed point number with the one
    passed in.  It uses a second order Newton-Raphson convgence.  This
    doubles the number of significant figures on each iteration.
    """
    # Use floating point arithmetic to make an initial guess
    floating_point_precision = 10**16
    n_float = float((n * floating_point_precision) // one) / floating_point_precision
    x = (int(floating_point_precision * math.sqrt(n_float)) * one) // floating_point_precision
    n_one = n * one
    while 1:
        x_old = x
        x = (x + n_one // x) // 2
        if x == x_old:
            break
    return x

def pi_chudnovsky_bs(digits):
    """
    Compute int(pi * 10**digits)

    This is done using Chudnovsky's series with binary splitting
    """
    C = 640320
    C3_OVER_24 = C**3 // 24
    def bs(a, b):
        """
        Computes the terms for binary splitting the Chudnovsky infinite series

        a(a) = +/- (13591409 + 545140134*a)
        p(a) = (6*a-5)*(2*a-1)*(6*a-1)
        b(a) = 1
        q(a) = a*a*a*C3_OVER_24

        returns P(a,b), Q(a,b) and T(a,b)
        """
        if b - a == 1:
            # Directly compute P(a,a+1), Q(a,a+1) and T(a,a+1)
            if a == 0:
                Pab = Qab = 1
            else:
                Pab = (6*a-5)*(2*a-1)*(6*a-1)
                Qab = a*a*a*C3_OVER_24
            Tab = Pab * (13591409 + 545140134*a) # a(a) * p(a)
            if a & 1:
                Tab = -Tab
        else:
            # Recursively compute P(a,b), Q(a,b) and T(a,b)
            # m is the midpoint of a and b
            m = (a + b) // 2
            # Recursively calculate P(a,m), Q(a,m) and T(a,m)
            Pam, Qam, Tam = bs(a, m)
            # Recursively calculate P(m,b), Q(m,b) and T(m,b)
            Pmb, Qmb, Tmb = bs(m, b)
            # Now combine
            Pab = Pam * Pmb
            Qab = Qam * Qmb
            Tab = Qmb * Tam + Pam * Tmb
        return Pab, Qab, Tab
    # how many terms to compute
    DIGITS_PER_TERM = math.log10(C3_OVER_24/6/2/6)
    N = int(digits/DIGITS_PER_TERM + 1)
    # Calclate P(0,N) and Q(0,N)
    P, Q, T = bs(0, N)
    one = 10**digits
    sqrtC = sqrt(10005*one, one)
    return (Q*426880*sqrtC) // T

if __name__ == "__main__":
    import sys
    digits = int(sys.argv[1])
    pi = pi_chudnovsky_bs(digits)
    print(pi)
'''


def set_timeout(deferred, timeout=30):
    def callback(value):
        if not watchdog.called:
            watchdog.cancel()
        return value

    deferred.addBoth(callback)

    watchdog = reactor.callLater(timeout, defer.timeout, deferred)


class ComputePI(MsgpackRPCServer):

    def __init__(self):
        self.waiting = defaultdict(list)
        self.results = {}

    def remote_PI(self, digits, timeout=None):
        if digits in self.results:
            return defer.succeed(self.results[digits])

        d = defer.Deferred()

        if digits not in self.waiting:
            subprocessDeferred = self.computePI(digits, timeout)

            def callWaiting(res):
                waiting = self.waiting[digits]
                del self.waiting[digits]

                if isinstance(res, failure.Failure):
                    func = lambda d: d.errback(res)
                else:
                    func = lambda d: d.callback(res)

                for d in waiting:
                    func(d)

            subprocessDeferred.addBoth(callWaiting)

        self.waiting[digits].append(d)

        return d

    def computePI(self, digits, timeout):
        d = utils.getProcessOutputAndValue('/usr/bin/python', args=('-c', pi_chudovsky_bs, str(digits)))

        def callback((out, err, code)):
            if code == 0:
                pi = int(out)
                self.results[digits] = pi
                return pi
            else:
                return failure.Failure(RuntimeError('Computation failed: ' + err))

        if timeout is not None:
            set_timeout(d, timeout)

        d.addCallback(callback)

        return d


def main():
    server = ComputePI()
    reactor.listenTCP(8000, server.getStreamFactory())

if __name__ == '__main__':
    reactor.callWhenRunning(main)
    reactor.run()
Client
from __future__ import print_function

import sys
import time
from twisted.internet import defer, reactor, task
from twisted.python import failure

@defer.inlineCallbacks
def main():
    try:

        from txmsgpackrpc.client import connect

        c = yield connect('localhost', 8000, waitTimeout=900)

        def callback(res, digits, start_time):
            if isinstance(res, failure.Failure):
                print('Computation of PI with %d places failed: %s' %
                      (digits, res.getErrorMessage()), end='\n\n')
            else:
                print('Computation of PI with %d places finished in %f seconds' %
                      (digits, time.time() - start_time), end='\n\n')
            sys.stdout.flush()

        defers = []
        for _ in range(2):
            for digits in (5, 100, 1000, 10000, 100000, 1000000):
                d = c.createRequest('PI', digits, 600)
                d.addBoth(callback, digits, time.time())
                defers.append(d)
            # wait for 30 seconds
            yield task.deferLater(reactor, 30, lambda: None)

        yield defer.DeferredList(defers)

        print('DONE')

    except Exception:
        import traceback
        traceback.print_exc()
    finally:
        reactor.stop()

if __name__ == '__main__':
    reactor.callWhenRunning(main)
    reactor.run()
Multicast UDP example

Example servers join to group 224.0.0.5 and listen on port 8000. Their only method echo returns its parameter.

Client joins group to 224.0.0.5, sends multicast request to group on port 8000 and waits for 5 seconds for responses. If some responses are received, protocol callbacks with tuple of results and individual parts are checked for errors. If no responses are received, protocol errbacks with TimeoutError.

Because there is no common way to determine number of peers in group, MsgpackMulticastDatagramProtocol always wait for responses until waitTimeout expires.

$ # setup multicast routing
$ ip route add 224.0.0.0/4 dev eth0
$ echo 1 > /proc/sys/net/ipv4/ip_forward
$
$ # start servers listening on port 8000
$ python examples/tx_rpc_server_udp_multicast.py &
[1] 3584
$ python examples/tx_rpc_server_udp_multicast.py &
[2] 3585
$ python examples/tx_rpc_server_udp_multicast.py &
[3] 3586
$ python examples/tx_rpc_server_udp_multicast.py &
[4] 3587
$ python examples/tx_rpc_server_udp_multicast.py &
[5] 3588
$
$ # execute client
$ python examples/tx_rpc_client_udp_multicast.py
Received results from 5 peers
$
Server
from twisted.internet import defer, reactor, task
from txmsgpackrpc.server import MsgpackRPCServer


class EchoRPC(MsgpackRPCServer):

    @defer.inlineCallbacks
    def remote_echo(self, value, delay=None, msgid=None):
        if delay is not None:
            yield task.deferLater(reactor, delay, lambda: None)
        defer.returnValue(value)


def main():
    server = EchoRPC()
    reactor.listenMulticast(8000, server.getMulticastProtocol('228.0.0.5', ttl=5),
                            listenMultiple=True)

if __name__ == '__main__':
    reactor.callWhenRunning(main)
    reactor.run()
Client
from __future__ import print_function

from twisted.internet import defer, reactor

@defer.inlineCallbacks
def main():
    try:

        from txmsgpackrpc.client import connect_multicast

        c = yield connect_multicast('228.0.0.5', 8000, ttl=5, waitTimeout=5)

        data = {
                    'firstName': 'John',
                    'lastName': 'Smith',
                    'isAlive': True,
                    'age': 25,
                    'height_cm': 167.6,
                    'address': {
                      'streetAddress': "21 2nd Street",
                      "city": 'New York',
                      "state": 'NY',
                      'postalCode': '10021-3100'
                    },
                    'phoneNumbers': [
                      {
                        'type': 'home',
                        'number': '212 555-1234'
                      },
                      {
                        'type': 'office',
                        'number': '646 555-4567'
                      }
                    ],
                    'children': [],
                    'spouse': None
                  }

        results = yield c.createRequest('echo', data)

        assert isinstance(results, tuple)

        print('Received results from %d peers' % len(results))

        for i, result in enumerate(results):
            if result != data:
                print('Result %d mismatch' % i)
                print(result)

    except Exception:
        import traceback
        traceback.print_exc()
    finally:
        reactor.stop()

if __name__ == '__main__':
    reactor.callWhenRunning(main)
    reactor.run()

jonghough/msgpack-j https://github.com/jonghough/msgpack-j

msgpack-j

Implementation of the Message Pack serialization format in J.

Usage

import into projects and open in jqt or other. It's very simple.

Examples

pack packs J types into a byte string. unpack reverses this.

  `pack 'hello world'`

     `�hello world`

  `unpack �hello world`

     `'hello world'`

packObj packs J types into a hex string (string literal). unpackObj reverses this.

e.g.

   `packObj 'Hello World'`

     `ab48656c6c6f20576f726c64`

  `unpackObj 'ab48656c6c6f20576f726c64'`

     `Hello World`
More Usage

Example:

  `pack 2;4.67;'hello, world'`

     `���@┐�┼z�G��hello, world`

  `packObj 2;4.67;'hello, world'`

     `9302cb4012ae147ae147aeac68656c6c6f2c20776f726c64`

JSON representation: [2,4.67,"hello, world"]

Example:

  `unpackObj '81a46461746183a2696401a673636f72657394cb400999999999999acb4016cccccccccccdcb40091eb851eb851fcb4007333333333333a56f7468657283a4736f6d65d1f2b8a46d6f7265ccc8a4646174610c'`

returns a nested dictionary.

JSON representation:

  `{"data":{"id":1,"scores":[3.2,5.7,3.14,2.9],"other":{"some":_3400,"more":200,"data":12}}}`
Handling dictionary / hashmap datatypes

Since J has no native Dictionary / Hashmap type, one has been implemented for the purposes of MsgPack serialization.

Construction:

  `HM =: '' conew 'HashMap'`

This will instantiate a new HashMap object.

  `set__HM 'key';'value'`

This will add a key value pair to the dicitonary. Note the length of the boxed array argument must be two. i.e. if the value is an array itself, then it must be boxed together before appending to the key value.

  `get__HM 'key'`

This will return the value for the given key, if one exists.

To pack a HashMap:

  `packObj s: HM`

Here HM is the HashMap reference name. It must be symbolized first, before packing. Furthermore, to add a HashMap as a value of another HashMap:

  `set__HM 'hashmapkey';s:HM2`

The inner HashMap reference (HM2) must be symbolized before adding to the dictionary. If you are adding a list of HashMaps to the parent HashMap:

  `set__HM 'key'; <(s:HM2;s:HM3;s:HM4)`

Note the HashMap array is boxed so that the argument for set is of length two. Since the HashMap HM stores the reference to the child HashMaps as symbols, they must be desymbolized if retrieved. e.g.

  `ChildHM =: getHashmapFromValue_HashMap_ get__HM 'mychildHashMapkey'`

Here, getHashmapFromValue_HashMap_ ensures that the retrieved object is a reference to a hashmap, as is wanted.

When unpacking data, assuming the root object is a dictionary / hashmap:

  `HM =: 5 s: unpackObj 'some serialized data'`

5 s: must be called to desymbolize the reference to the HashMap. Furthermore, all child HashMaps of HM must also be desymbolized too.

`

a2/MessagePack.swift https://github.com/a2/MessagePack.swift

MessagePack.swift
License

MessagePack.swift is released under the MIT license, which is reproduced below in its entirety:

Copyright (c) 2015 Alexsander Akers

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Availability

MessagePack.swift is intended for use with Swift 2.0. Compatibility with past and future versions is not guaranteed.

akiradeveloper/msgpack-nim https://github.com/akiradeveloper/msgpack-nim

I will start this project once Nim compiler reaches 1.0

msgpack-nim

A MessagePack binding for Nim

Build Status

API: https://rawgit.com/akiradeveloper/msgpack-nim/master/msgpack.html

msgpack-nim currently provides only the basic functionality. Please see what's listed in Todo section. Compared to other language bindings, it's well-tested by 1000 auto-generated test cases by Haskell QuickCheck, which always runs on every commit to Github repository. Please try make quickcheck on your local machine to see what happens (It will take a bit while. Be patient). Have a nice packing!

Overview

Install
$ nimble update
$ nimble install msgpack
Example
import msgpack
import streams

# You can use any stream subclasses to serialize/deserialize
# messages. e.g. FileStream
let st: Stream = newStringStream()

assert(st.getPosition == 0)

# Type checking protects you from making trivial mistakes.
# Now we pack {"a":[5,-3], "b":[1,2,3]} but more complex
# combination of any Msg types is allowed.
#
# In xs we can mix specific conversion (PFixNum) and generic
# conversion (unwrap).
let xs: Msg = wrap(@[PFixNum(5), (-3).wrap])
let ys: Msg = wrap(@[("a".wrap, xs.wrap), ("b".wrap, @[1, 2, 3].wrap)])
st.pack(ys.wrap) # Serialize!

# We need to reset the cursor to the beginning of the target
# byte sequence.
st.setPosition(0)

let msg = st.unpack # Deserialize!

# output:
# a
# 5
# -3
# b
# 1
# 2
# 3
for e in msg.unwrapMap:
  echo e.key.unwrapStr
  for e in e.val.unwrapArray:
    echo e.unwrapInt
Todo
  • Implement unwrapInto to convert Msg object to Nim object handily
  • Evaluate performance and scalability
  • Talk with offical Ruby implementation
  • Don't repeat yourself: The code now has too much duplications. Using templates?
Author

Akira Hayakawa ([email protected])

ludocode/mpack https://github.com/ludocode/mpack

Introduction

MPack is a C implementation of an encoder and decoder for the MessagePack serialization format. It is intended to be:

The core of MPack contains a buffered reader and writer, and a tree-style parser that decodes into a tree of dynamically typed nodes. Helper functions can be enabled to read values of expected type, to work with files, to allocate strings automatically, to check UTF-8 encoding, and more. The MPack featureset can be configured at compile-time to set which features, components and debug checks are compiled, and what dependencies are available.

The MPack code is small enough to be embedded directly into your codebase. The easiest way to use it is to download the amalgamation package and insert the source files directly into your project. Copy mpack.h and mpack.c into to your codebase, and copy mpack-config.h.sample as mpack-config.h. You can use the defaults or edit it if you'd like to customize the MPack featureset.

Build Status

MPack is beta software under development.

Travis-CI AppVeyor Coveralls.io
Build Status Build status Coverage Status
The Node Reader API

The Node API parses a chunk of MessagePack data into an immutable tree of dynamically-typed nodes. A series of helper functions can be used to extract data of specific types from each node.

// parse a file into a node tree
mpack_tree_t tree;
mpack_tree_init_file(&tree, "homepage-example.mp", 0);
mpack_node_t root = mpack_tree_root(&tree);

// extract the example data on the msgpack homepage
bool compact = mpack_node_bool(mpack_node_map_cstr(root, "compact"));
int schema = mpack_node_i32(mpack_node_map_cstr(root, "schema"));

// clean up and check for errors
if (mpack_tree_destroy(tree) != mpack_ok) {
    fprintf(stderr, "An error occurred decoding the data!\n");
    return;
}

Note that no additional error handling is needed in the above code. If the file is missing or corrupt, if map keys are missing or if nodes are not in the expected types, special "nil" nodes and false/zero values are returned and the tree is placed in an error state. An error check is only needed before using the data.

The above example decodes into allocated pages of nodes. A fixed node pool can be provided to the parser instead in memory-constrained environments. For maximum performance and minimal memory usage, the Expect API can be used to parse data of a predefined schema.

The Write API

The MPack Write API encodes structured data to MessagePack.

// encode to memory buffer
char* data;
size_t size;
mpack_writer_t writer;
mpack_writer_init_growable(&writer, &data, &size);

// write the example on the msgpack homepage
mpack_start_map(&writer, 2);
mpack_write_cstr(&writer, "compact");
mpack_write_bool(&writer, true);
mpack_write_cstr(&writer, "schema");
mpack_write_uint(&writer, 0);
mpack_finish_map(&writer);

// finish writing
if (mpack_writer_destroy(&writer) != mpack_ok) {
    fprintf(stderr, "An error occurred encoding the data!\n");
    return;
}

// use the data
do_something_with_data(data, size);
free(data);

In the above example, we encode to a growable memory buffer. The writer can instead write to a pre-allocated or stack-allocated buffer, avoiding the need for memory allocation. The writer can also be provided with a flush function (such as a file or socket write function) to call when the buffer is full or when writing is done.

If any error occurs, the writer is placed in an error state. The writer will flag an error if too much data is written, if the wrong number of elements are written, if the data could not be flushed, etc. No additional error handling is needed in the above code; any subsequent writes are ignored when the writer is in an error state, so you don't need to check every write for errors.

Note in particular that in debug mode, the mpack_finish_map() call above ensures that two key/value pairs were actually written as claimed, something that other MessagePack C/C++ libraries may not do.

Comparison With Other Parsers

MPack is rich in features while maintaining very high performance and a small code footprint. Here's a short feature table comparing it to other C parsers:

MPack
(v0.8)
msgpack-c
(v1.3.0)
CMP
(v14)
No libc requirement
Growable memory writer
File I/O helpers
Tree parser
Propagating errors
Compound size tracking
Incremental parser
Incremental range/match helpers
Tree stream parser
UTF-8 verification

A larger feature comparison table is available here which includes descriptions of the various entries in the table.

This benchmarking suite compares the performance of MPack to other implementations of schemaless serialization formats. MPack outperforms all JSON and MessagePack libraries, and in some tests MPack is several times faster than RapidJSON for equivalent data.

Why Not Just Use JSON?

Conceptually, MessagePack stores data similarly to JSON: they are both composed of simple values such as numbers and strings, stored hierarchically in maps and arrays. So why not just use JSON instead? The main reason is that JSON is designed to be human-readable, so it is not as efficient as a binary serialization format:

  • Compound types such as strings, maps and arrays are delimited, so appropriate storage cannot be allocated upfront. The whole object must be parsed to determine its size.

  • Strings are not stored in their native encoding. Special characters such as quotes and backslashes must be escaped when written and converted back when read.

  • Numbers are particularly inefficient (especially when parsing back floats), making JSON inappropriate as a base format for structured data that contains lots of numbers.

  • Binary data is not supported by JSON at all. Small binary blobs such as icons and thumbnails need to be Base64 encoded or passed out-of-band.

The above issues greatly increase the complexity of the decoder. Full-featured JSON decoders are quite large, and minimal decoders tend to leave out such features as string unescaping and float parsing, instead leaving these up to the user or platform. This can lead to hard-to-find platform-specific and locale-specific bugs, as well as a greater potential for security vulnerabilites. This also significantly decreases performance, making JSON unattractive for use in applications such as mobile games.

While the space inefficiencies of JSON can be partially mitigated through minification and compression, the performance inefficiencies cannot. More importantly, if you are minifying and compressing the data, then why use a human-readable format in the first place?

Running the Unit Tests

The MPack build process does not build MPack into a library; it is used to build and run the unit tests. You do not need to build MPack or the unit testing suite to use MPack.

On Linux, the test suite uses SCons and requires Valgrind, and can be run in the repository or in the amalgamation package. Run scons to build and run the test suite in full debug configuration.

On Windows, there is a Visual Studio solution, and on OS X, there is an Xcode project for building and running the test suite.

You can also build and run the test suite in all supported configurations, which is what the continuous integration server will build and run. If you are on 64-bit, you will need support for cross-compiling to 32-bit, and running 32-bit binaries with 64-bit Valgrind. On Ubuntu, you'll need libc6-dbg:i386. On Arch you'll need gcc-multilib or lib32-clang, and valgrind-multilib. Use scons all=1 -j16 (or some appropriate thread count) to build and run all tests.

3Hren/msgpack-rust https://github.com/3Hren/msgpack-rust

RMP - Rust MessagePack

RMP is a pure Rust MessagePack implementation.

Build Status Coverage Status

This repository consists of three separate crates: the RMP core and two implementations to ease serializing and deserializing Rust structs.

Crates.io API Documentation
rmp RMP
rmp-serialize RMP Serialize
rmp-serde RMP Serde
Usage

To use rmp, first add this to your Cargo.toml:

[dependencies.rmp]
rmp = "^0.7"

Then, add this to your crate root:

extern crate rmp as msgpack; // Or just `rmp`.
Features
  • Convenient API

    RMP is designed to be lightweight and straightforward. There are low-level API, which gives you full control on data encoding/decoding process and makes no heap allocations. On the other hand there are high-level API, which provides you convenient interface using Rust standard library and compiler reflection, allowing to encode/decode structures using derive attribute.

  • Zero-copy value decoding

    RMP allows to decode bytes from a buffer in a zero-copy manner easily and blazingly fast, while Rust static checks guarantees that the data will be valid until buffer lives.

  • Clear error handling

    RMP's error system guarantees that you never receive an error enum with unreachable variant.

  • Robust and tested

    This project is developed using TDD and CI, so any found bugs will be fixed without breaking existing functionality.

Examples

Let's try to encode a tuple of int and string.

extern crate rmp_serialize as msgpack;
extern crate rustc_serialize;

use rustc_serialize::Encodable;
use msgpack::Encoder;

fn main() {
    let val = (42u8, "the Answer");

    // The encoder borrows the bytearray buffer.
    let mut buf = [0u8; 13];

    val.encode(&mut Encoder::new(&mut &mut buf[..]));

    assert_eq!([0x92, 0x2a, 0xaa, 0x74, 0x68, 0x65, 0x20, 0x41, 0x6e, 0x73, 0x77, 0x65, 0x72], buf);
}

Now we have an encoded buffer, which we can decode the same way:

extern crate rmp_serialize as msgpack;
extern crate rustc_serialize;

use rustc_serialize::Decodable;
use msgpack::Decoder;

fn main() {
    let buf = [0x92, 0x2a, 0xaa, 0x74, 0x68, 0x65, 0x20, 0x41, 0x6e, 0x73, 0x77, 0x65, 0x72];

    let mut decoder = Decoder::new(&buf[..]);

    let res: (u8, String) = Decodable::decode(&mut decoder).unwrap();

    assert_eq!((42u8, "the Answer".to_string()), res);
}

RMP also allows to automatically serialize/deserialize custom structures using rustc_serialize reflection. To enable this feature, derive RustcEncodable and RustcDecodable attributes as shown in the following example:

extern crate rmp_serialize as msgpack;
extern crate rustc_serialize;

use rustc_serialize::{Encodable, Decodable};
use msgpack::{Encoder, Decoder};

#[derive(RustcEncodable, RustcDecodable, PartialEq, Debug)]
struct Custom {
    id: u32,
    key: String,
}

fn main() {
    let val = Custom { id: 42u32, key: "the Answer".to_string() };

    let mut buf = [0u8; 13];

    val.encode(&mut Encoder::new(&mut &mut buf[..]));

    assert_eq!([0x92, 0x2a, 0xaa, 0x74, 0x68, 0x65, 0x20, 0x41, 0x6e, 0x73, 0x77, 0x65, 0x72], buf);

    // Now try to unpack the buffer into the initial struct.
    let mut decoder = Decoder::new(&buf[..]);
    let res: Custom = Decodable::decode(&mut decoder).ok().unwrap();

    assert_eq!(val, res);
}
Versioning

This project adheres to Semantic Versioning. However until 1.0.0 comes there will be the following rules:

  • Any API/ABI breaking changes will be notified in the changelog explicitly and results in minor version bumping.
  • API extending features results in minor version bumping.
  • Non-breaking bug fixes and performance improving results in patch version bumping.

jangko/msgpack4nim https://github.com/jangko/msgpack4nim

msgpack4nim

MessagePack implementation written in pure nim

why another implementation?

I am fully aware of another msgpack implementation written in nim. But I want something easier to use. Another motivation come from the nim language itself. The current version of nim compiler offer many improvements, including 'generics ' specialization. I found out nim compiler is smart enough to make serialization/deserialization to/from msgpack easy and convenient.

requirement: nim ver 0.11.2 or later

Example
import msgpack, streams

type
  #lets try with a rather complex object
  CustomType = object
    count: int
    content: seq[int]
    name: string
    ratio: float
    attr: array[0..5, int]
    ok: bool

proc initCustomType(): CustomType =
  result.count = -1
  result.content = @[1,2,3]
  result.name = "custom"
  result.ratio = 1.0
  for i in 0..5: result.attr[i] = i
  result.ok = false

var x = initCustomType()
#you can use another stream compatible
#class here e.g. FileStream
var s = newStringStream()
s.pack(x) #here the magic happened

s.setPosition(0)
var xx: CustomType
s.unpack(xx) #and here too

assert xx == x
echo "OK ", xx.name

see? you only need to call 'pack' and 'unpack', and the compiler do the hard work for you. Very easy, convenient, and works well

if you think setting up a StringStream too much for you, you can simply call pack(yourobject) and it will return a string containing msgpack data.

  var a = @[1,2,3,4,5,6,7,8,9,0]
  var buf = pack(a)
  var aa: seq[int]
  unpack(buf, aa)
  assert a == aa

in case the compiler cannot decide how to serialize or deserialize your very very complex object, you can help it in easy way by defining your own handler pack_type/unpack_type

type
  #not really complex, just for example
  mycomplexobject = object
    a: someSimpleType
    b: someSimpleType

#help the compiler to decide
proc pack_type*(s: Stream, x: mycomplexobject) =
  s.pack(x.a) # let the compiler decide
  s.pack(x.b) # let the compiler decide

#help the compiler to decide
proc unpack_type*(s: Stream, x: var complexobject) =
  s.unpack(x.a)
  s.unpack(x.b)

var s: newStringStream()
var x: mycomplexobject

s.pack(x) #pack as usual

s.setPosition(0)
s.unpack(x) #unpack as usual
Data Conversion
nim msgpack
int8/16/32/64 int8/16/32/64
uint8/16/32/64 uint8/16/32/64
true/false/nil true/false/nil
procedural type ignored
cstring ignored
pointer ignored
ptr see ref-types
ref see ref-types
circular ref see ref-types
distinct types converted to base type
float32/64 float32/64
string string8/16/32
array/seq array
set array
range/subrange int8/16/32/64
enum int8/16/32/64
IntSet,Doubly/SinglyLinkedList array
Doubly/SinglyLinkedRing array
Queue,HashSet,OrderedSet array
Table,TableRef map
OrderedTable,OrderedTableRef map
StringTableRef map
CritBitTree[T] map
CritBitTree[void] array
object/tuple array/map
object and tuple

object and tuple by default converted to msgpack array, however you can tell the compiler to convert it to map by supplying --define:msgpack_obj_to_map

nim c --define:msgpack_obj_to_map yourfile.nim

or --define:msgpack_obj_to_stream to convert object/tuple fields value into stream of msgpack objects

nim c --define:msgpack_obj_to_stream yourfile.nim
ref-types:

ref something :

  • if ref value is nil, it will be packed into msgpack nil, and when unpacked, usually will do nothing except seq[T] will be @[]
  • if ref value not nil, it will be dereferenced e.g. pack(val[]) or unpack(val[])
  • ref subject to some restriction. see restriction below
  • ptr will be treated like ref during pack
  • unpacking ptr will invoke alloc, so you must dealloc it

circular reference: altough detecting circular reference is not too difficult(using set of pointers), the current implementation does not provide circular reference detection. If you pack something contains circular reference, you know something bad will happened

Restriction: For objects their type is not serialized. This means essentially that it does not work if the object has some other runtime type than its compiletime type:

import streams, msgpack

type
  TA = object of RootObj
  TB = object of TA
    f: int

var
  a: ref TA
  b: ref TB

new(b)
a = b

echo stringify(pack(a))
#produces "[ ]" or "{ }"
#not "[ 0 ]" or '{ "f" : 0 }'
limitation:

these types will be ignored:

  • procedural type
  • cstring(it is not safe to assume it always terminated by null)
  • pointer

these types cannot be automatically pack/unpacked:

  • void (will cause compile time error)

however, you can provide your own handler for cstring and pointer

Gotchas: because data conversion did not preserve original data types, the following code is perfectly valid and will raise no exception

import msgpack, streams, tables, sets, strtabs

type
  Horse = object
    legs: int
    foals: seq[string]
    attr: Table[string, string]

  Cat = object
    legs: uint8
    kittens: HashSet[string]
    traits: StringTableRef

proc initHorse(): Horse =
  result.legs = 4
  result.foals = @["jilly", "colt"]
  result.attr = initTable[string, string]()
  result.attr["color"] = "black"
  result.attr["speed"] = "120mph"

var stallion = initHorse()
var tom: Cat

var buf = pack(stallion) #pack a Horse here
unpack(buf, tom)
#abracadabra, it will unpack into a Cat

echo "legs: ", $tom.legs
echo "kittens: ", $tom.kittens
echo "traits: ", $tom.traits

another gotcha:

  type
    KAB = object of RootObj
      aaa: int
      bbb: int

    KCD = object of KAB
      ccc: int
      ddd: int

    KEF = object of KCD
      eee: int
      fff: int

  var kk = KEF()
  echo stringify(pack(kk))
  # will produce "{ "eee" : 0, "fff" : 0, "ccc" : 0, "ddd" : 0, "aaa" : 0, "bbb" : 0 }"
  # not "{ "aaa" : 0, "bbb" : 0, "ccc" : 0, "ddd" : 0, "eee" : 0, "fff" : 0 }"
bin and ext format

this implementation provide function to encode/decode msgpack bin/ext format header, but for the body, you must write it yourself to the StringStream

  • proc pack_bin*(s: Stream, len: int)
  • proc pack_ext*(s: Stream, len: int, exttype: int8)
  • proc unpack_bin*(s: Stream): int
  • proc unpack_ext*(s: Stream): tuple[exttype:uint8, len: int]
import streams, msgpack

const exttype0 = 0

var s = newStringStream()
var body = "this is the body"

s.pack_ext(body.len, exttype0)
s.write(body)

#the same goes to bin format
s.pack_bin(body.len)
s.write(body)

s.setPosition(0)
#unpack_ext return tuple[exttype:uint8, len: int]
let (extype, extlen) = s.unpack_ext()
var extbody = s.readStr(extlen)

assert extbody == body

let binlen = s.unpack_bin()
var binbody = s.readStr(binlen)

assert binbody == body
stringify

you can convert msgpack data to readable string using stringify function

  type
    Horse = object
      legs: int
      speed: int
      color: string
      name: string

  var cc = Horse(legs:4, speed:150, color:"black", name:"stallion")
  var zz = pack(cc)
  echo stringify(zz)

the result will be:

default:
[ 4, 150, "black", "stallion" ]

msgpack_obj_to_map defined:
{ "legs" : 4, "speed" : 150, "color" : "black", "name" : "stallion" }

msgpack_obj_to_stream defined:
4 150 "black" "stallion"
toAny

toAny takes a string of msgpack data or a stream, then it will produce msgAny which you can interrogate of it's type and value during runtime by accessing it's member msgType

toAny recognize all valid msgpack message and translate it into a group of types:

msgMap, msgArray, msgString, msgBool,
msgBin, msgExt, msgFloat32, msgFloat64,
msgInt, msgUint, msgNull

for example, msg is a msgpack data with content [1, "hello", {"a": "b"}], you can interrogate it like this:

var a = msg.toAny()
assert a.msgType == msgArray
assert a.arrayVal[0].msgType == msgInt
assert a.arrayVal[0].intVal == 1
assert a.arrayVal[1].msgType == msgString
assert a.arrayVal[1].stringVal == "hello"
assert a.arrayVal[2].msgType == msgMap
assert a.arrayVal[2].mapVal[0].key.msgType == msgString
assert a.arrayVal[2].mapVal[0].key.stringVal == "a"
assert a.arrayVal[2].mapVal[0].val.msgType == msgString
assert a.arrayVal[2].mapVal[0].val.stringVal == "b"

enjoy it, happy nim-ing

benoist/msgpack-crystal https://github.com/benoist/msgpack-crystal

MessagePack

Build Status

MessagePack implementation in Crystal. It supports Packing and Unpacking from a string or an IO, Mappings, to_msgpack-from_msgpack methods.

Installation

Add this to your application's shard.yml:

dependencies:
  msgpack:
    github: benoist/msgpack-crystal
Usage
require "msgpack"

class Location
  MessagePack.mapping({
    lat: Float64,
    lng: Float64,
  })
end

class House
  MessagePack.mapping({
    address:  String,
    location: {type: Location, nilable: true},
  })
end

house = House.from_msgpack({"address" => "Road12", "location" => {"lat" => 12.3, "lng" => 34.5}}.to_msgpack)
p house # <House:0x1b06de0 @address="Road12", @location=#<Location:0x1b06dc0 @lat=12.3, @lng=34.5>>

house.address = "Something"
house = House.from_msgpack(house.to_msgpack)
p house # #<House:0x13f0d80 @address="Something", @location=#<Location:0x13f0d60 @lat=12.3, @lng=34.5>>

house = House.from_msgpack({"address" => "Crystal Road 1234"}.to_msgpack)
p house # <House:0x1b06d80 @address="Crystal Road 1234", @location=nil>
More Examples

(examples)

Copyright

Copyright 2015 Benoist Claassen

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

kawanet/msgpack-lite https://www.npmjs.com/package/msgpack-lite

msgpack-lite npm version Build Status

Fast Pure JavaScript MessagePack Encoder and Decoder

Sauce Test Status

Online demo: http://kawanet.github.io/msgpack-lite/

Features
  • Pure JavaScript only (No node-gyp nor gcc required)
  • Faster than any other pure JavaScript libraries on node.js v4
  • Even faster than node-gyp C++ based msgpack library (90% faster on encoding)
  • Streaming encoding and decoding interface is also available. It's more faster.
  • Ready for Web browsers including Chrome, Firefox, Safari and even IE8
  • Tested on Node.js v0.10, v0.12, v4 and v5 as well as Web browsers
Encoding and Decoding MessagePack
var msgpack = require("msgpack-lite");

// encode from JS Object to MessagePack (Buffer)
var buffer = msgpack.encode({"foo": "bar"});

// decode from MessagePack (Buffer) to JS Object
var data = msgpack.decode(buffer); // => {"foo": "bar"}

// if encode/decode receives an invalid argument an error is thrown
Writing to MessagePack Stream
var fs = require("fs");
var msgpack = require("msgpack-lite");

var writeStream = fs.createWriteStream("test.msp");
var encodeStream = msgpack.createEncodeStream();
encodeStream.pipe(writeStream);

// send multiple objects to stream
encodeStream.write({foo: "bar"});
encodeStream.write({baz: "qux"});

// call this once you're done writing to the stream.
encodeStream.end();
Reading from MessagePack Stream
var fs = require("fs");
var msgpack = require("msgpack-lite");

var readStream = fs.createReadStream("test.msp");
var decodeStream = msgpack.createDecodeStream();

// show multiple objects decoded from stream
readStream.pipe(decodeStream).on("data", console.warn);
Decoding MessagePack Bytes Array
var msgpack = require("msgpack-lite");

// decode() accepts Buffer instance per default
msgpack.decode(Buffer([0x81, 0xA3, 0x66, 0x6F, 0x6F, 0xA3, 0x62, 0x61, 0x72]));

// decode() also accepts Array instance
msgpack.decode([0x81, 0xA3, 0x66, 0x6F, 0x6F, 0xA3, 0x62, 0x61, 0x72]);

// decode() accepts raw Uint8Array instance as well
msgpack.decode(new Uint8Array([0x81, 0xA3, 0x66, 0x6F, 0x6F, 0xA3, 0x62, 0x61, 0x72]));
Command Line Interface

A CLI tool bin/msgpack converts data stream from JSON to MessagePack and vice versa.

$ echo '{"foo": "bar"}' | ./bin/msgpack -Jm | od -tx1
0000000    81  a3  66  6f  6f  a3  62  61  72

$ echo '{"foo": "bar"}' | ./bin/msgpack -Jm | ./bin/msgpack -Mj
{"foo":"bar"}
Installation
$ npm install --save msgpack-lite
Tests

Run tests on node.js:

$ make test

Run tests on browsers:

$ make test-browser-local
open the following url in a browser:
http://localhost:4000/__zuul
Browser Build

Browser version msgpack.min.js is also available. 44KB minified, 13KB gziped.

<!--[if lte IE 9]>
<script src="https://cdnjs.cloudflare.com/ajax/libs/es5-shim/4.1.10/es5-shim.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/json3/3.3.2/json3.min.js"></script>
<![endif]-->
<script src="https://rawgit.com/kawanet/msgpack-lite/master/dist/msgpack.min.js"></script>
<script>
// encode from JS Object to MessagePack (Uint8Array)
var buffer = msgpack.encode({foo: "bar"});

// decode from MessagePack (Uint8Array) to JS Object
var array = new Uint8Array([0x81, 0xA3, 0x66, 0x6F, 0x6F, 0xA3, 0x62, 0x61, 0x72]);
var data = msgpack.decode(array);
</script>
MessagePack With Browserify

Step #1: write some code at first.

var msgpack = require("msgpack-lite");
var buffer = msgpack.encode({"foo": "bar"});
var data = msgpack.decode(buffer);
console.warn(data); // => {"foo": "bar"}

Proceed to the next steps if you prefer faster browserify compilation time.

Step #2: add browser property on package.json in your project. This refers the global msgpack object instead of including whole of msgpack-lite source code.

{
  "dependencies": {
    "msgpack-lite": "*"
  },
  "browser": {
    "msgpack-lite": "msgpack-lite/global"
  }
}

Step #3: compile it with browserify and uglifyjs.

browserify src/main.js -o tmp/main.browserify.js -s main
uglifyjs tmp/main.browserify.js -m -c -o js/main.min.js
cp node_modules/msgpack-lite/dist/msgpack.min.js js/msgpack.min.js

Step #4: load msgpack.min.js before your code.

<script src="js/msgpack.min.js"></script>
<script src="js/main.min.js"></script>
Interoperability

It is tested to have basic compatibility with other Node.js MessagePack modules below:

Benchmarks

A benchmark tool lib/benchmark.js is available to compare encoding/decoding speed (operation per second) with other MessagePack modules. It counts operations of 1KB JSON document in 10 seconds.

$ npm install msgpack msgpack-js msgpack-js-v5 msgpack-unpack msgpack5 notepack
$ npm run benchmark 10
operation op ms op/s
buf = Buffer(JSON.stringify(obj)); 1055200 10000 105520
obj = JSON.parse(buf); 863800 10000 86380
buf = require("msgpack-lite").encode(obj); 969100 10000 96910
obj = require("msgpack-lite").decode(buf); 600300 10000 60030
buf = require("msgpack").pack(obj); 503500 10001 50344
obj = require("msgpack").unpack(buf); 560200 10001 56014
buf = Buffer(require("msgpack.codec").msgpack.pack(obj)); 653500 10000 65349
obj = require("msgpack.codec").msgpack.unpack(buf); 367500 10001 36746
buf = require("msgpack-js-v5").encode(obj); 189500 10002 18946
obj = require("msgpack-js-v5").decode(buf); 408900 10000 40890
buf = require("msgpack-js").encode(obj); 189200 10000 18920
obj = require("msgpack-js").decode(buf); 375600 10002 37552
buf = require("msgpack5")().encode(obj); 110500 10009 11040
obj = require("msgpack5")().decode(buf); 165500 10000 16550
buf = require("notepack")().encode(obj); 847800 10000 84780
obj = require("notepack")().decode(buf); 599800 10000 59980
obj = require("msgpack-unpack").decode(buf); 48100 10002 4809

Streaming benchmark tool lib/benchmark-stream.js is also available. It counts milliseconds for 1,000,000 operations of 30 bytes fluentd msgpack fragment. This shows streaming encoding and decoding are super faster.

$ npm run benchmark-stream 2
operation (1000000 x 2) op ms op/s
stream.write(msgpack.encode(obj)); 1000000 3027 330360
stream.write(notepack.encode(obj)); 1000000 2012 497017
msgpack.Encoder().on("data",ondata).encode(obj); 1000000 2956 338294
msgpack.createEncodeStream().write(obj); 1000000 1888 529661
stream.write(msgpack.decode(buf)); 1000000 2020 495049
stream.write(notepack.decode(buf)); 1000000 1794 557413
msgpack.Decoder().on("data",ondata).decode(buf); 1000000 2744 364431
msgpack.createDecodeStream().write(buf); 1000000 1341 745712

Test environment: msgpack-lite 0.1.14, Node v4.2.3, Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz

MessagePack Mapping Table

The following table shows how JavaScript objects (value) will be mapped to MessagePack formats and vice versa.

Source Value MessagePack Format Value Decoded
null, undefined nil format family null
Boolean (true, false) bool format family Boolean (true, false)
Number (32bit int) int format family Number (int or double)
Number (64bit double) float format family Number (double)
String str format family String
Buffer bin format family Buffer
Array array format family Array
Object (plain object) map format family Object
Object (see below) ext format family Object (see below)

Note that both null and undefined are mapped to nil 0xC1 type. This means undefined value will be upgraded to null in other words.

Extension Types

The MessagePack specification allows 128 application-specific extension types. The library uses the following types to make round-trip conversion possible for JavaScript native objects.

Type Object Type Object
0x00 0x10
0x01 EvalError 0x11 Int8Array
0x02 RangeError 0x12 Uint8Array
0x03 ReferenceError 0x13 Int16Array
0x04 SyntaxError 0x14 Uint16Array
0x05 TypeError 0x15 Int32Array
0x06 URIError 0x16 Uint32Array
0x07 0x17 Float32Array
0x08 0x18 Float64Array
0x09 0x19 Uint8ClampedArray
0x0A RegExp 0x1A ArrayBuffer
0x0B Boolean 0x1B Buffer
0x0C String 0x1C
0x0D Date 0x1D DataView
0x0E Error 0x1E
0x0F Number 0x1F

Other extension types are mapped to built-in ExtBuffer object.

Custom Extension Types (Codecs)

Register a custom extension type number to serialize/deserialize your own class instances.

var msgpack = require("msgpack-lite");

var codec = msgpack.createCodec();
codec.addExtPacker(0x3F, MyVector, myVectorPacker);
codec.addExtUnpacker(0x3F, myVectorUnpacker);

var data = new MyVector(1, 2);
var encoded = msgpack.encode(data, {codec: codec});
var decoded = msgpack.decode(encoded, {codec: codec});

function MyVector(x, y) {
  this.x = x;
  this.y = y;
}

function myVectorPacker(vector) {
  var array = [vector.x, vector.y];
  return msgpack.encode(array); // return Buffer serialized
}

function myVectorUnpacker(buffer) {
  var array = msgpack.decode(buffer);
  return new MyVector(array[0], array[1]); // return Object deserialized
}

The first argument of addExtPacker and addExtUnpacker should be an integer within the range of 0 and 127 (0x0 and 0x7F). myClassPacker is a function that accepts an instance of MyClass, and should return a buffer representing that instance. myClassUnpacker is the opposite: it accepts a buffer and should return an instance of MyClass.

If you pass an array of functions to addExtPacker or addExtUnpacker, the value to be encoded/decoded will pass through each one in order. This allows you to do things like this:

codec.addExtPacker(0x00, Date, [Number, msgpack.encode]);

You can also pass the codec option to msgpack.Decoder(options), msgpack.Encoder(options), msgpack.createEncodeStream(options), and msgpack.createDecodeStream(options).

If you wish to modify the default built-in codec, you can access it at msgpack.codec.preset.

Custom Codec Options

msgpack.createCodec() function accepts some options.

It does NOT have the preset extension types defined when no options given.

var codec = msgpack.createCodec();

preset: It has the preset extension types described above.

var codec = msgpack.createCodec({preset: true});

safe: It runs a validation of the value before writing it into buffer. This is the default behavior for some old browsers which do not support ArrayBuffer object.

var codec = msgpack.createCodec({safe: true});

useraw: It uses raw formats instead of bin and str.

var codec = msgpack.createCodec({useraw: true});

int64: It decodes msgpack's int64/uint64 formats with int64-buffer object.

var codec = msgpack.createCodec({int64: true});

binarraybuffer: It ties msgpack's bin format with ArrayBuffer object, instead of Buffer object.

var codec = msgpack.createCodec({binarraybuffer: true, preset: true});
Compatibility Mode

The compatibility mode respects for msgpack's old spec. Set true to useraw.

// default mode handles both str and bin formats individually
msgpack.encode("Aa"); // => <Buffer a2 41 61> (str format)
msgpack.encode(new Buffer([0x41, 0x61])); // => <Buffer c4 02 41 61> (bin format)

msgpack.decode(new Buffer([0xa2, 0x41, 0x61])); // => 'Aa' (String)
msgpack.decode(new Buffer([0xc4, 0x02, 0x41, 0x61])); // => <Buffer 41 61> (Buffer)

// compatibility mode handles only raw format both for String and Buffer
var options = {codec: msgpack.createCodec({useraw: true})};
msgpack.encode("Aa", options); // => <Buffer a2 41 61> (raw format)
msgpack.encode(new Buffer([0x41, 0x61]), options); // => <Buffer a2 41 61> (raw format)

msgpack.decode(new Buffer([0xa2, 0x41, 0x61]), options); // => <Buffer 41 61> (Buffer)
msgpack.decode(new Buffer([0xa2, 0x41, 0x61]), options).toString(); // => 'Aa' (String)
Repository See Also License

The MIT License (MIT)

Copyright (c) 2015-2016 Yusuke Kawasaki

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

ludocode/msgpack-tools https://github.com/ludocode/msgpack-tools

Introduction

msgpack-tools contains simple command-line utilities for converting from MessagePack to JSON and vice-versa. They support options for lax parsing, lossy conversions, pretty-printing, and base64 encoding.

  • msgpack2json -- Convert MessagePack to JSON
  • json2msgpack -- Convert JSON to MessagePack

They can be used for dumping MessagePack from a file or web API to a human-readable format, or for converting hand-written or generated JSON to MessagePack. The lax parsing mode supports comments and trailing commas in JSON, making it possible to hand-write your app or game data in JSON and convert it at build-time to MessagePack.

Build Status
Travis-CI
Build Status
Examples

To view a MessagePack file in a human-readable format for debugging purposes:

msgpack2json -di file.mp

To convert a hand-written JSON file to a MessagePack file, ignoring comments and trailing commas, and allowing embedded base64 with a base64: prefix:

json2msgpack -bli file.json -o file.mp

To fetch MessagePack from a web API and view it in a human-readable format:

curl 'http://example/api/url' | msgpack2json -d

To view the MessagePack-equivalent encoding of a JSON string:

$ echo '{"compact": true, "schema": 0}' | json2msgpack | hexdump -C
00000000  82 a7 63 6f 6d 70 61 63  74 c3 a6 73 63 68 65 6d  |..compact..schem|
00000010  61 00                                             |a.|
00000012

To test a MessagePack-RPC server via netcat:

$ echo '[0,0,"sum",[1,2]]' | json2msgpack | nc -q1 localhost 18800 | msgpack2json -d
[
    1,
    0,
    null,
    3
]
Installation
Arch Linux

msgpack-tools is available in the AUR. It can be installed with makepkg or with any AUR helper:

yaourt -S msgpack-tools
OS X

A Homebrew formula for msgpack-tools is included in the repository. This requires Homebrew.

brew install https://raw.githubusercontent.com/ludocode/msgpack-tools/master/tools/msgpack-tools.rb
Other

For other platforms, msgpack-tools currently must be built from source. The latest version of the msgpack-tools source archive should be downloaded from the releases page. This includes the library dependencies and has pre-generated man pages:

https://github.com/ludocode/msgpack-tools/releases

msgpack-tools uses CMake. A configure wrapper is provided that calls CMake, so on Linux and Mac OS X, you can simply run the usual:

./configure && make && sudo make install

If you are building from a clone of the repository, you will need to fetch the dependencies and generate the man pages. These are done with the scripts under tools/fetch.sh and tools/man.sh. md2man is required to generate the man pages.

Differences between MessagePack and JSON

MessagePack is intended to be very close to JSON in supported features, so they can usually be transparently converted from one to the other. There are some differences, however, which can complicate conversions.

These are the differences in what objects are representable in each format:

  • JSON keys must be strings. MessagePack keys can be any type, including maps and arrays.

  • JSON supports "bignums", i.e. integers of any size. MessagePack integers must fit within a 64-bit signed or unsigned integer.

  • JSON real numbers are specified in decimal scientific notation and can have arbitrary precision. MessagePack real numbers are in IEEE 754 standard 32-bit or 64-bit binary.

  • MessagePack supports binary and extension type objects. JSON does not support binary data. Binary data is often encoded into a base64 string to be embedded into a JSON document.

  • The top-level object in a JSON document must be either an array or object (map). The top-level object in MessagePack can be any type.

  • A JSON document can be encoded in UTF-8, UTF-16 or UTF-32, and the entire document must be in the same encoding. MessagePack strings are required to be UTF-8, although this is not enforced by many encoding/decoding libraries.

By default, msgpack2json and json2msgpack convert in strict mode. If an object in the source format is not representable in the destination format, the converter aborts with an error. A lax mode is available which performs a "lossy" conversion, and base64 conversion modes are available to support binary data in JSON.

rybakit/msgpack.php https://github.com/rybakit/msgpack.php

msgpack.php

A pure PHP implementation of the MessagePack serialization format.

Build Status Scrutinizer Code Quality Code Coverage

Features
Table of contents
Installation

The recommended way to install the library is through Composer:

$ composer require rybakit/msgpack
Usage
Packing

To pack values use the Packer class:

use MessagePack\Packer;

$packer = new Packer();

...

$packed = $packer->pack($value);

In the example above, the method pack automatically pack a value depending on its type. But not all PHP types can be uniquely translated to MessagePack types. For example, MessagePack format defines map and array types, which are represented by a single array type in PHP. By default, the packer will pack a PHP array as a MessagePack array if it has sequential numeric keys, starting from 0 and as a MessagePack map otherwise:

$mpArr1 = $packer->pack([1, 2]);                   // MP array [1, 2]
$mpArr2 = $packer->pack([0 => 1, 1 => 2]);         // MP array [1, 2]
$mpMap1 = $packer->pack([0 => 1, 2 => 3]);         // MP map {0: 1, 2: 3}
$mpMap2 = $packer->pack([1 => 2, 2 => 3]);         // MP map {1: 2, 2: 3}
$mpMap3 = $packer->pack(['foo' => 1, 'bar' => 2]); // MP map {foo: 1, bar: 2}

However, sometimes you need to pack a sequential array as a MessagePack map. To do this, use the packMap method:

$mpMap = $packer->packMap([1, 2]); // {0: 1, 1: 2}

Here is a list of all low-level packer methods:

$packer->packNil();                   // MP nil
$packer->packBool(true);              // MP bool
$packer->packArray([1, 2]);           // MP array
$packer->packMap([1, 2]);             // MP map
$packer->packExt(new Ext(1, "\xaa")); // MP ext
$packer->packDouble(4.2);             // MP float
$packer->packInt(42);                 // MP int
$packer->packStr('foo');              // MP str
$packer->packBin("\x80");             // MP bin

Check "Custom Types" section below on how to pack arbitrary PHP objects.

Unpacking

To unpack data use the BufferUnpacker class:

use MessagePack\BufferUnpacker;

$unpacker = new BufferUnpacker();

...

$unpacker->reset($data);
$unpacked = $unpacker->unpack();

If the packed data is received in chunks (e.g. when reading from a stream), use the tryUnpack method, which will try unpack data and return an array of unpacked data instead of throwing a InsufficientDataException:

$unpacker->append($chunk1);
$unpackedBlocks = $unpacker->tryUnpack();

$unpacker->append($chunk2);
$unpackedBlocks = $unpacker->tryUnpack();

To save some keystrokes, the library ships with a syntax sugar class Unpacker, which is no more than a tiny wrapper around BufferUnpacker with a single method unpack:

use MessagePack\Unpacker;

...

$unpacked = (new Unpacker())->unpack($data);
Unsigned 64-bit Integers

The binary MessagePack format has unsigned 64-bit as its largest integer data type, but PHP does not support such integers. By default, while unpacking uint64 value the library will throw a IntegerOverflowException.

You can change this default behavior to unpack uint64 integer to a string:

$unpacker->setIntOverflowMode(BufferUnpacker::INT_AS_STRING);

Or to a Gmp number (make sure that gmp extension is enabled):

$unpacker->setIntOverflowMode(BufferUnpacker::INT_AS_GMP);
Extensions

To define application-specific types use the Ext class:

use MessagePack\Ext;
use MessagePack\Packer;
use MessagePack\Unpacker;

$packerd = (new Packer())->pack(new Ext(42, "\xaa"));
$ext = (new Unpacker())->unpack($packed);

$extType = $ext->getType(); // 42
$extData = $ext->getData(); // "\xaa"
Custom Types

In addition to the basic types, the library provides the functionality to serialize and deserialize arbitrary types. To do this, you need to create a transformer, that converts your type to a type, which can be handled by MessagePack.

For example, the code below shows how to add DateTime object support:

use MessagePack\TypeTransformer\TypeTransformer;

class DateTimeTransformer implements TypeTransformer
{
    private $id;

    public function __construct($id)
    {
        $this->id = $id;
    }

    public function getId()
    {
        return $this->id;
    }

    public function supports($value)
    {
        return $value instanceof \DateTime;
    }

    public function transform($value)
    {
        return $value->format(\DateTime::RFC3339);
    }

    public function reverseTransform($data)
    {
        return new \DateTime($data);
    }
}
use MessagePack\BufferUnpacker;
use MessagePack\Packer;
use MessagePack\TypeTransformer\Collection;

$packer = new Packer();
$unpacker = new BufferUnpacker();

$coll = new Collection([new DateTimeTransformer(5)]);
// $coll->add(new AnotherTypeTransformer(42));

$packer->setTransformers($coll);
$unpacker->setTransformers($coll);

$packed = $packer->pack(['foo' => new DateTime(), 'bar' => 'baz']);
$raw = $unpacker->reset($packed)->unpack();
Exceptions

If an error occurs during packing/unpacking, a PackingFailedException or UnpackingFailedException will be thrown, respectively.

In addition, there are two more exceptions that can be thrown during unpacking:

  • InsufficientDataException
  • IntegerOverflowException
Tests

Run tests as follows:

$ phpunit

Also, if you already have Docker installed, you can run the tests in a docker container. First, create a container:

$ ./dockerfile.sh | docker build -t msgpack -

The command above will create a container named msgpack with PHP 5.6 runtime. You may change the default runtime by defining the PHP_RUNTIME environment variable:

$ PHP_RUNTIME='php:7.0-cli' ./dockerfile.sh | docker build -t msgpack -

See a list of various runtimes here.

Then run the unit tests:

$ docker run --rm --name msgpack -v $(pwd):/msgpack -w /msgpack msgpack
Performance

To check the performance run:

$ php tests/bench.php

This command will output something like:

Filter: MessagePack\Tests\Perf\Filter\NameFilter
Rounds: 3
Iterations: 100000

====================================================================
Test/Target           MessagePack\Packer  MessagePack\BufferUnpacker
--------------------------------------------------------------------
nil ............................. 0.0135 .................... 0.0242
false ........................... 0.0151 .................... 0.0276
true ............................ 0.0151 .................... 0.0256
7-bit uint #1 ................... 0.0169 .................... 0.0199
7-bit uint #2 ................... 0.0186 .................... 0.0188
7-bit uint #3 ................... 0.0185 .................... 0.0232
5-bit sint #1 ................... 0.0208 .................... 0.0252
5-bit sint #2 ................... 0.0178 .................... 0.0242
5-bit sint #3 ................... 0.0191 .................... 0.0242
8-bit uint #1 ................... 0.0299 .................... 0.0479
8-bit uint #2 ................... 0.0280 .................... 0.0449
8-bit uint #3 ................... 0.0275 .................... 0.0486
16-bit uint #1 .................. 0.0321 .................... 0.0537
16-bit uint #2 .................. 0.0297 .................... 0.0544
16-bit uint #3 .................. 0.0309 .................... 0.0550
32-bit uint #1 .................. 0.0289 .................... 0.0720
32-bit uint #2 .................. 0.0296 .................... 0.0750
32-bit uint #3 .................. 0.0293 .................... 0.0751
64-bit uint #1 .................. 0.0450 .................... 0.0870
64-bit uint #2 .................. 0.0426 .................... 0.0914
8-bit int #1 .................... 0.0274 .................... 0.0509
8-bit int #2 .................... 0.0299 .................... 0.0511
8-bit int #3 .................... 0.0294 .................... 0.0518
16-bit int #1 ................... 0.0317 .................... 0.0618
16-bit int #2 ................... 0.0284 .................... 0.0613
16-bit int #3 ................... 0.0281 .................... 0.0567
32-bit int #1 ................... 0.0316 .................... 0.0811
32-bit int #2 ................... 0.0295 .................... 0.0815
32-bit int #3 ................... 0.0314 .................... 0.0767
64-bit int #1 ................... 0.0415 .................... 0.0886
64-bit int #2 ................... 0.0422 .................... 0.0884
64-bit int #3 ................... 0.0414 .................... 0.0933
64-bit float #1 ................. 0.0354 .................... 0.0748
64-bit float #2 ................. 0.0354 .................... 0.0744
64-bit float #3 ................. 0.0374 .................... 0.0748
fix string #1 ................... 0.0325 .................... 0.0254
fix string #2 ................... 0.0326 .................... 0.0411
fix string #3 ................... 0.0320 .................... 0.0403
fix string #4 ................... 0.0361 .................... 0.0427
8-bit string #1 ................. 0.0456 .................... 0.0684
8-bit string #2 ................. 0.0534 .................... 0.0682
8-bit string #3 ................. 0.0598 .................... 0.0739
16-bit string #1 ................ 0.0581 .................... 0.0833
16-bit string #2 ................ 3.2417 .................... 0.3393
32-bit string ................... 3.2793 .................... 0.3664
wide char string #1 ............. 0.0414 .................... 0.0404
wide char string #2 ............. 0.1068 .................... 0.0701
8-bit binary #1 ................. 0.0438 .................... 0.0641
8-bit binary #2 ................. 0.0451 .................... 0.0624
8-bit binary #3 ................. 0.0443 .................... 0.0610
16-bit binary ................... 0.0458 .................... 0.0772
32-bit binary ................... 0.3977 .................... 0.3500
fixext 1 ........................ 0.0423 .................... 0.0868
fixext 2 ........................ 0.0429 .................... 0.0899
fixext 4 ........................ 0.0443 .................... 0.0918
fixext 8 ........................ 0.0453 .................... 0.0955
fixext 16 ....................... 0.0475 .................... 0.0908
8-bit ext ....................... 0.0493 .................... 0.1097
16-bit ext ...................... 0.0510 .................... 0.1194
32-bit ext ...................... 0.3958 .................... 0.4024
fix array #1 .................... 0.0306 .................... 0.0272
fix array #2 .................... 0.1184 .................... 0.1153
16-bit array #1 ................. 0.4040 .................... 0.3510
16-bit array #2 ...................... S ......................... S
32-bit array ......................... S ......................... S
complex array ................... 0.5205 .................... 0.5579
fix map #1 ...................... 0.2467 .................... 0.2446
fix map #2 ...................... 0.0910 .................... 0.0914
fix map #3 ...................... 0.1193 .................... 0.1440
fix map #4 ...................... 0.1214 .................... 0.1128
16-bit map #1 ................... 0.7065 .................... 0.6257
16-bit map #2 ........................ S ......................... S
32-bit map ........................... S ......................... S
complex map ..................... 0.6907 .................... 0.6785
====================================================================
Total                            12.3730                      7.8941
Skipped                                4                           4
Failed                                 0                           0
Ignored                                0                           0

You may change default benchmark settings by defining the following environment variables:

  • MP_BENCH_TARGETS (pure_p, pure_u, pecl_p, pecl_u)
  • MP_BENCH_ITERATIONS/MP_BENCH_DURATION
  • MP_BENCH_ROUNDS
  • MP_BENCH_TESTS

For example:

$ export MP_BENCH_TARGETS=pure_p
$ export MP_BENCH_ITERATIONS=1000000
$ export MP_BENCH_ROUNDS=5
$ export MP_BENCH_TESTS='complex array, complex map'
$ php tests/bench.php

Another example, benchmarking both the library and msgpack pecl extension:

$ MP_BENCH_TARGETS=pure_p,pure_u,pecl_p,pecl_u php tests/bench.php

Filter: MessagePack\Tests\Perf\Filter\NameFilter
Rounds: 3
Iterations: 100000

==================================================================================================
Test/Target           MessagePack\Packer  MessagePack\BufferUnpacker  msgpack_pack  msgpack_unpack
--------------------------------------------------------------------------------------------------
nil ............................. 0.0146 .................... 0.0282 ...... 0.0066 ........ 0.0064
false ........................... 0.0171 .................... 0.0252 ...... 0.0092 ........ 0.0073
true ............................ 0.0163 .................... 0.0304 ...... 0.0086 ........ 0.0068
7-bit uint #1 ................... 0.0209 .................... 0.0214 ...... 0.0078 ........ 0.0066
7-bit uint #2 ................... 0.0166 .................... 0.0229 ...... 0.0073 ........ 0.0061
7-bit uint #3 ................... 0.0188 .................... 0.0216 ...... 0.0080 ........ 0.0058
5-bit sint #1 ................... 0.0209 .................... 0.0270 ...... 0.0077 ........ 0.0061
5-bit sint #2 ................... 0.0193 .................... 0.0252 ...... 0.0078 ........ 0.0074
5-bit sint #3 ................... 0.0184 .................... 0.0244 ...... 0.0077 ........ 0.0061
8-bit uint #1 ................... 0.0268 .................... 0.0469 ...... 0.0107 ........ 0.0066
8-bit uint #2 ................... 0.0289 .................... 0.0447 ...... 0.0090 ........ 0.0065
8-bit uint #3 ................... 0.0292 .................... 0.0458 ...... 0.0071 ........ 0.0064
16-bit uint #1 .................. 0.0298 .................... 0.0512 ...... 0.0088 ........ 0.0079
16-bit uint #2 .................. 0.0296 .................... 0.0542 ...... 0.0080 ........ 0.0066
16-bit uint #3 .................. 0.0309 .................... 0.0523 ...... 0.0093 ........ 0.0074
32-bit uint #1 .................. 0.0289 .................... 0.0723 ...... 0.0087 ........ 0.0067
32-bit uint #2 .................. 0.0333 .................... 0.0706 ...... 0.0078 ........ 0.0064
32-bit uint #3 .................. 0.0288 .................... 0.0743 ...... 0.0079 ........ 0.0066
64-bit uint #1 .................. 0.0423 .................... 0.0851 ...... 0.0089 ........ 0.0067
64-bit uint #2 .................. 0.0408 .................... 0.0882 ...... 0.0091 ........ 0.0076
8-bit int #1 .................... 0.0292 .................... 0.0521 ...... 0.0075 ........ 0.0072
8-bit int #2 .................... 0.0274 .................... 0.0507 ...... 0.0079 ........ 0.0067
8-bit int #3 .................... 0.0279 .................... 0.0526 ...... 0.0071 ........ 0.0064
16-bit int #1 ................... 0.0280 .................... 0.0586 ...... 0.0085 ........ 0.0081
16-bit int #2 ................... 0.0317 .................... 0.0590 ...... 0.0084 ........ 0.0081
16-bit int #3 ................... 0.0286 .................... 0.0621 ...... 0.0075 ........ 0.0066
32-bit int #1 ................... 0.0288 .................... 0.0830 ...... 0.0081 ........ 0.0077
32-bit int #2 ................... 0.0292 .................... 0.0806 ...... 0.0094 ........ 0.0066
32-bit int #3 ................... 0.0304 .................... 0.0859 ...... 0.0080 ........ 0.0065
64-bit int #1 ................... 0.0418 .................... 0.0876 ...... 0.0082 ........ 0.0068
64-bit int #2 ................... 0.0425 .................... 0.0889 ...... 0.0085 ........ 0.0070
64-bit int #3 ................... 0.0447 .................... 0.0917 ...... 0.0084 ........ 0.0066
64-bit float #1 ................. 0.0392 .................... 0.0832 ...... 0.0090 ........ 0.0062
64-bit float #2 ................. 0.0353 .................... 0.0754 ...... 0.0074 ........ 0.0092
64-bit float #3 ................. 0.0368 .................... 0.0768 ...... 0.0073 ........ 0.0064
fix string #1 ................... 0.0310 .................... 0.0278 ...... 0.0076 ........ 0.0075
fix string #2 ................... 0.0351 .................... 0.0413 ...... 0.0079 ........ 0.0080
fix string #3 ................... 0.0316 .................... 0.0435 ...... 0.0083 ........ 0.0085
fix string #4 ................... 0.0366 .................... 0.0401 ...... 0.0082 ........ 0.0079
8-bit string #1 ................. 0.0452 .................... 0.0722 ...... 0.0083 ........ 0.0078
8-bit string #2 ................. 0.0503 .................... 0.0673 ...... 0.0083 ........ 0.0074
8-bit string #3 ................. 0.0575 .................... 0.0683 ...... 0.0133 ........ 0.0095
16-bit string #1 ................ 0.0563 .................... 0.0779 ...... 0.0123 ........ 0.0087
16-bit string #2 ................ 3.2773 .................... 0.3390 ...... 0.3482 ........ 0.2674
32-bit string ................... 3.2505 .................... 0.3546 ...... 0.3409 ........ 0.2704
wide char string #1 ............. 0.0423 .................... 0.0434 ...... 0.0082 ........ 0.0069
wide char string #2 ............. 0.1040 .................... 0.0689 ...... 0.0085 ........ 0.0079
8-bit binary #1 ...................... I ......................... I ........... F ............. I
8-bit binary #2 ...................... I ......................... I ........... F ............. I
8-bit binary #3 ...................... I ......................... I ........... F ............. I
16-bit binary ........................ I ......................... I ........... F ............. I
32-bit binary ........................ I ......................... I ........... F ............. I
fixext 1 ............................. I ......................... I ........... F ............. F
fixext 2 ............................. I ......................... I ........... F ............. F
fixext 4 ............................. I ......................... I ........... F ............. F
fixext 8 ............................. I ......................... I ........... F ............. F
fixext 16 ............................ I ......................... I ........... F ............. F
8-bit ext ............................ I ......................... I ........... F ............. F
16-bit ext ........................... I ......................... I ........... F ............. F
32-bit ext ........................... I ......................... I ........... F ............. F
fix array #1 .................... 0.0318 .................... 0.0273 ...... 0.0172 ........ 0.0082
fix array #2 .................... 0.1204 .................... 0.1175 ...... 0.0226 ........ 0.0216
16-bit array #1 ................. 0.3913 .................... 0.3492 ...... 0.0374 ........ 0.0490
16-bit array #2 ...................... S ......................... S ........... S ............. S
32-bit array ......................... S ......................... S ........... S ............. S
complex array ........................ I ......................... I ........... F ............. F
fix map #1 ........................... I ......................... I ........... F ............. I
fix map #2 ...................... 0.0940 .................... 0.0859 ...... 0.0183 ........ 0.0225
fix map #3 ........................... I ......................... I ........... F ............. I
fix map #4 ........................... I ......................... I ........... F ............. I
16-bit map #1 ................... 0.7112 .................... 0.6340 ...... 0.0404 ........ 0.0708
16-bit map #2 ........................ S ......................... S ........... S ............. S
32-bit map ........................... S ......................... S ........... S ............. S
complex map ..................... 0.6881 .................... 0.6817 ...... 0.0692 ........ 0.0760
==================================================================================================
Total                            10.0680                      5.1397        1.2722          1.1059
Skipped                                4                           4             4               4
Failed                                 0                           0            17               9
Ignored                               17                          17             0               8

Note, that this is not a fair comparison as the msgpack extension (0.5.2+, 2.0) doesn't support ext, bin and utf-8 str types.

License

The library is released under the MIT License. See the bundled LICENSE file for details.

msgpack4z/msgpack4z-circe https://github.com/msgpack4z/msgpack4z-circe

michael-yuji/YSMessagePack https://github.com/michael-yuji/YSMessagePack

YSMessagePack

PyPI platforms

YSMessagePack is a messagePack packer/unpacker written in swift include following features:

  • No classes
  • Pack custom structs and classes / unpack objects by groups and apply handler to each group
  • Asynchronous unpacking
  • Pack and unpack multiple message-packed data regardless of types with only one line of code
  • Specify how many items to unpack
  • Get remaining bytes that were not message-packed ; start packing from some index -- so you can mix messagepack with other protocol!!!
  • Helper methods to cast NSData to desired types
  • Operator +^ and +^= to join NSData
Version

1.6

Installation
  • (recommanded) Simply add files under Sources/ to your project,
  • use cocoapod, add "pod 'YSMessagePack', '~> 1.6.0' to your podfile

since YSMessagePack has such small codebase. It is recommanded to simply add these files to your xcode project so it can link statically. linking any kind of frameworks (dyld) will create an overhead and slow down your app when it launch.

Usage
Pack:
let exampleInt: Int = 1
let exampleStr: String = "Hello World"
let exampleArray: [Int] = [1, 2, 3, 4, 5, 6]
let bool: Bool = true

// To pack items, just put all of them in a single array
// and call the `pack(items:)` function

//this will be the packed data
let msgPackedBytes: NSData = pack(items: [true, foo, exampleInt, exampleStr, exampleArray]) 

// Now your payload is ready to send!!!

But what if we have some custom data structure to send?

//To make your struct / class packable
struct MyStruct: Packable {  //Confirm to this protocol
    var name: String
    var index: Int
    func packFormat() -> [Packable] { //protocol function
        return [name, index] //pack order
    }

    func msgtype() -> MsgPackTypes {
        return .Custom
    }
}

let exampleInt: Int = 1
let exampleStr: String = "Hello World"
let exampleArray: [Int] = [1, 2, 3, 4, 5]
let bool: Bool = true

let foo = MyStruct(name: "foo", index: 626)

let msgPackedBytes = pack(items: [bool, foo, exampleInt, exampleStr, exampleArray])

Or you can pack them individually and add them to a byte array manually (Which is also less expensive)

let exampleInt: Int = 1
let exampleStr: String = "Hello World"
let exampleArray: [Int] = [1, 2, 3, 4, 5, 6]

//Now pack them individually
let packedInt = exampleInt.packed()

//if you didn't specific encoding, the default encoding will be ASCII
#if swift(>=3)
let packedStr = exampleStr.packed(withEncoding: NSASCIIStringEncoding) 
#else
let packedStr = exampleStr.packed(withEncoding: .ascii)
#endif
let packedArray = exampleArray.packed()
//You can use this operator +^ the join the data on rhs to the end of data on lhs
let msgPackedBytes: NSData = packedInt +^ packedStr +^ packedArray
Unpack

YSMessagePack offer a number of different ways and options to unpack include unpack asynchronously, see the example project for detail.

To unpack a messagepacked bytearray is pretty easy:

do {
    //The unpack method will return an array of NSData which each element is an unpacked object
    let unpackedItems = try msgPackedBytes.itemsUnpacked()
    //instead of casting the NSData to the type you want, you can call these `.castTo..` methods to do the job for you
    let int: Int = unpackedItems[2].castToInt()

    //Same as packing, you can also specify the encoding you want to use, default is ASCII
    let str: String = unpackedItem[3].castToString() 
    let array: NSArray = unpackedItems[4].castToArray() 
} catch let error as NSError{
    NSLog("Error occurs during unpacking: %@", error)
}

//Remember how to pack your struct? Here is a better way to unpack a stream of bytes formatted in specific format
 let testObj1 = MyStruct(name: "TestObject1", index: 1)
 let testObj2 = MyStruct(name: "TestObject2", index: 2)
 let testObj3 = MyStruct(name: "TestObject3", index: 3)

 let packed = packCustomObjects(testObj1, testObj2, testObj3) //This is an other method that can pack your own struct easier

 let nobjsInOneGroup = 2

 try! packed.unpackByGroupsWith(nobjsInOneGroup) {
     (unpackedData, isLast) -> Bool

     //you can also involve additional args like number of groups to unpack
     guard let name = unpackedData[0].castToString() else {return false} //abort unpacking hen something wrong
     let index = unpackedData[1]
     let testObj = MyStruct(name: name, index: index) // assembly      
     return true //proceed unpacking, or return false to abort
 } 

If you don't want to unpack every single thing included in the message-pack byte array, you can also specify an amount to unpack, if you want to keep the remaining bytes, you can put true in the returnRemainingBytes argument, the remaining bytes will stored in the end of the NSData array.

do {
    //Unpack only 2 objects, and we are not interested in remaining bytes
    let unpackedItems = try msgPackedBytes.itemsUnpacked(specific_amount: 2, returnRemainingBytes: false)
    print(unpackedItems.count) //will print 2
} catch let error as NSError{
    NSLog("Error occurs during unpacking: %@", error)
}

caesay/MPack https://github.com/caesay/MPack

MPack

This library is a lightweight implementation of the MessagePack binary serialization format. MessagePack is a 1-to-1 binary representation of JSON, and the official specification can be found here: https://github.com/msgpack....

Build status (https://ci.appveyor.com/project/caesay/mpack)

Notes
  • This library is designed to be super light weight.
  • Its easiest to understand how this library works if you think in terms of json. The type MPackMap represents a dictionary, and the type MPackArray represents an array.
  • Create MPack instances with the static method MPack.From(object);. You can pass any simple type (such as string, integer, etc), or any Array composed of a simple type. MPack also has implicit conversions from most of the basic types built in.
  • Transform an MPack object back into a CLR type with the static method MPack.To<T>(); or MPack.To(type);. MPack also has explicit converions going back to most basic types, you can do string str = (string)mpack; for instance.
  • MPack now supports native asynchrounous reading and cancellation tokens. It will not block a thread to wait on a stream.
NuGet

MPack is available as a NuGet package!

PM> Install-Package MPack
Usage

Create a object model that can be represented as MsgPack. Here we are creating a dictionary, but really it can be anything:

MPackMap dictionary = new MPackMap
{
    {
        "array1", MPack.From(new[]
        {
            "array1_value1",  // implicitly converted string
            MPack.From("array1_value2"),
            MPack.FromString("array1_value3"),
        })
    },
    {"bool1", MPack.From(true)}, //boolean
    {"double1", MPack.From(50.5)}, //single-precision float
    {"double2", MPack.From(15.2)},
    {"int1", 50505}, // implicitly converted integer
    {"int2", MPack.From(50)} // integer
};

Serialize the data to a byte array or to a stream to be saved, transmitted, etc:

byte[] encodedBytes = dictionary.EncodeToBytes();
// -- or --
dictionary.EncodeToStream(stream);

Parse the binary data back into a MPack object model (you can also cast back to an MPackMap or MPackArray after reading if you want dictionary/array methods):

var reconstructed = MPack.ParseFromBytes(encodedBytes);
// -- or --
var reconstructed = MPack.ParseFromStream(stream);

Turn MPack objects back into types that we understand with the generic To<>() method. Since we know the types of everything here we can just call To<bool>() to reconstruct our bool, but if you don't know you can access the instance enum MPack.ValueType to know what kind of value it is:

bool bool1 = reconstructed["bool1"].To<bool>();
var array1 = reconstructed["array1"] as MPackArray;
var array1_value1 = array1[0];
double double1 = reconstructed["double1"].To<double>();
//etc...
Credits

The following people/projects have made this possible:

  1. Me: [caelantsayler]at[gmail]dot[com]
  2. All of the people that make MessagePack happen: https://github.com/msgpack

jonathonl/goodform https://github.com/jonathonl/goodform

GoodForm

Form validation library. Includes MsgPack and JSON serializer/deserializer.

MsgPack Usage
std::stringstream ss;
goodform::variant var, var2;
var = goodform::object
  {
    {"compact", true},
    {"schema", 0}
  };

goodform::msgpack::serialize(var, ss);
goodform::msgpack::deserialize(ss, var2);

goodform::form form(var2);

struct
{
  bool compact;
  std::int32_t schema;
} mpack;

mpack.compact = form.at("compact").boolean().val();
mpack.schema = form.at("schema").int32().val();

if (form.is_good())
{
  std::cout << "{ \"compact\": " << std::boolalpha << mpack.compact << ", \"schema\": " << mpack.schema << " }" << std::endl;
}
JSON Usage
goodform::variant var;
std::stringstream ss;
ss << "{" << std::endl
  << "\"first_name\":\"John\", // This is a comment" << std::endl
  << "\"last_name\":\"Smith\", " << std::endl
  << "\"age\": 23," << std::endl
  << "\"gpa\": 4.0," << std::endl
  << "\"email\":\"[email protected]\"," << std::endl
  << "\"password_hash\":\"5f4dcc3b5aa765d61d8327deb882cf99\"," << std::endl
  << "\"interests\": [\"sailing\",\"swimming\",\"yoga\"]" << std::endl
  << "}" << std::endl;

goodform::json::deserialize(ss, var);

goodform::form form(var);

struct
{
  std::string first_name;
  std::string last_name;
  std::uint8_t age;
  float gpa;
  std::string email;
  std::string password_hash;
  bool subscribe_to_email_marketing;
  std::list<std::string> interests;
} form_data;


form_data.first_name = form.at("first_name").string().match(std::regex("^[a-zA-Z ]{1,64}$")).val();
form_data.last_name = form.at("last_name").string().match(std::regex("^[a-zA-Z ]{1,64}$")).val();
form_data.age = form.at("age").uint8().val();
form_data.gpa = form.at("gpa").float32().gte(0).lte(4.0).val();
form_data.email = form.at("email").string().match(std::regex("^.{3,256}$")).val();
form_data.password_hash = form.at("password_hash").string().match(std::regex("^[a-fA-F0-9]{32}$")).val();
form_data.subscribe_to_email_marketing = form.at("subscribe_to_email_marketing", true).boolean().val(); // Optional field defaults to true.

form.at("interests").array().for_each([&form_data](goodform::sub_form& sf, std::size_t i)
{
  form_data.interests.push_back(sf.string().val());
});

if (form.is_good())
{
  // Use validated form_data.
}
else
{
  // Handle error.
}

HEADS-project/arduino_msgpack https://github.com/HEADS-project/arduino_msgpack

arduino_msgpack

This Arduino library provides a light weight serializer and parser for messagepack.

Install

Download the zip, and import it with your Arduino IDE: Sketch>Include Library>Add .zip library

Usage

See the either the .h file, or the examples (led_controller and test_uno_writer).

In short:

  • functions like msgpck_what_next(Stream * s); watch the next type of data without reading it (without advancing the buffer of Stream s).
  • functions like msgpck_read_bool(Stream * s, bool *b) read a value from Stream s.
  • functions like msgpck_write_bool(Stream * s, bool b) write a value on Stream s.

Notes:

  • Stream are used as much as possible in order not to add to much overhead with buffers. Therefore you should be able to store the minimum number of value at a given time.
  • Map and Array related functions concern only their headers. Ex: If you want to write an array containing two elements you should write the array header, then write the two elements.
Limitations

Currently the library does not support:

  • 8 bytes float (Only 4 bytes floats are supported by default on every Arduino and floats are anyway not recommended on Arduino)
  • 2^32 char long (or longer) strings
  • 2^32 byte long (or longer) bins
  • extention types.

st3v0/msgpack-as3 https://github.com/st3v0/msgpack-as3

MessagePack for ActionScript3

msgpack-as3 is an implementation of the latest MessagePack specification for ActionScript3 language (Flash, Flex and AIR).

Basic Usage
Serialize and Deserialize

The usage of MsgPack class is very simple. You need create an object and call read and write methods.

// message pack object created
var msgpack:MsgPack = new MsgPack();

// encode an array
var bytes:ByteArray = msgpack.write([1, 2, 3, 4, 5]);

// rewind the buffer
bytes.position = 0;

// print the decoded object
trace(msgpack.read(bytes));
Flags

Currently there are three flags which you may use to initialize a MsgPack object:

  • MsgPackFlags.READ_STRING_AS_BYTE_ARRAY: message pack string data is read as byte array instead of string;
  • MsgPackFlags.ACCEPT_LITTLE_ENDIAN: MsgPack objects will work with little endian buffers (message pack specification defines big endian as default).
  • MsgPackFlags.SPEC2013_COMPATIBILITY: MsgPack will run in backwards compatibility mode.
var msg:MsgPack;

// use logical operator OR to set the flags.
msgpack = new MsgPack(MsgPackFlags.READ_STRING_AS_BYTE_ARRAY | MsgPackFlags.ACCEPT_LITTLE_ENDIAN);
Advanced Usage
Extensions

You can create your own Extension Workers by extending the ExtensionWorker Class and then assigning it to the MsgPack Factory.

The following example assigns a custom worker which extends the ExtensionWorker Class.

var msgpack:MsgPack = new MsgPack();

// Assign the new worker to the factory.
msgpack.factory.assign(new CustomWorker());

For more information regarding Extensions refer to the MessagePack specification.

Priorities

Worker priority behaves similar to how the Adobe Event Dispatcher priorities work. In MessagePack, deciding which worker will be use for serializing/deserializing depends on two(2) factors.

  1. The order in which the worker was assigned to the factory.
  2. The priority of the worker. Higher values take precedence.

All workers have a default priority of 0.

In the following example workerB will never be used because it's assign after workerA

var msgpack:MsgPack = new MsgPack();

var workerA:StringWorker = new StringWorker();
var workerB:DifferentStringWorker = new DifferentStringWorker();

msgpack.factory.assign(workerA);
msgpack.factory.assign(workerB);

However if we adjust the priority of workerB, then workerA will never be used.

var msgpack:MsgPack = new MsgPack();

var workerA:StringWorker = new StringWorker();
var workerB:DifferentStringWorker = new DifferentStringWorker(null, 1);

msgpack.factory.assign(workerA);
msgpack.factory.assign(workerB);
Credits

This application uses Open Source components. You can find the source code of their open source projects along with license information below. We acknowledge and are grateful to these developers for their contributions to open source.

Project: as3-msgpack https://github.com/loteixeira/as3-msgpack
Copyright (C) 2013 Lucas Teixeira
License (Apache V2.0) http://www.apache.org/licenses/LICENSE-2.0

JSON

MessagePack (hex)