Protocol Buffers

Protocol Buffers

Developer(s)	Google
Initial release	7 July 2008
Stable release	2.1.0 / 14 May 2009
Operating system	Any
Platform	Cross-platform
Development status	Active
Type	serialization format and library, IDL compiler
License	BSD
Website	http://code.google.com/apis/protocolbuffers/

Protocol Buffers is a serialization format with an interface description language developed by Google. The original Google implementation for C++, Java and Python is available under a free software, open source license. Various other language implementations are either available or in development.

The design goals for Protocol Buffers emphasized simplicity and performance. In particular, it was designed to be faster than XML (no reproducible comparisons are publicly available to confirm this, however). Protocol Buffers is very similar to Facebook’s Thrift protocol, except it does not include a concrete RPC stack to use for defined services. Since Protocol Buffers was open sourced, a number of RPC stacks have emerged to fill this gap.

Prior to the release as open source, Protocol Buffers had been widely used at Google for storing and interchanging all kinds of structured information. Protocol Buffers serve as a basis for a custom RPC system that is used for practically all inter-machine communication at Google.

Data structures and services are defined in the Proto Definition file (.proto) which is then compiled with protoc. This compilation generates code that matches the services. For example, example.proto will produce example.pb.cc and example.pb.h which will define C++ classes for each Message and Service example.proto defines.

Protocol Buffers allow serialization into any number of formats. In the officially supported implementations there is a full Reflection interface available, making it easy to serialize protos as XML and JSON.

Though the primary purpose of Protocol Buffers is to facilitate network communication, its simplicity and speed make Protocol Buffers a great replacement of data-centric C++ classes and structs.

Example

message Point {
  required int32 x = 1;
  required int32 y = 2;
  optional string label = 3;
}

message Line {
  required Point start = 1;
  required Point end = 2;
  optional string label = 3;
}

message Polyline {
  repeated Point point = 1;
  optional string label = 2;
}

This subsequently compiled with protoc. A C++ program can then use it like so:

#include "polyline.pb.h"  // generated by calling protoc polyline.proto (defined above)

Line* createNewLine(const std::string& name) {
  Line* line = new Line;
  line->mutable_start()->set_x(10);
  line->mutable_start()->set_y(20);
  line->mutable_end()->set_x(30);
  line->mutable_end()->set_y(40);
  line->set_label(name);
  return line;
}

Polyline* createNewPolyline() {
  Polyline* polyline = new Polyline;
  Point* point1 = polyline->add_point();
  point1->set_x(10);
  point1->set_y(10);
  Point* point2 = polyline->add_point();
  point2->set_x(10);
  point2->set_y(10);
  return polyline;
}

The "Point" message defines two mandatory data items, x and y. The data item label is optional. Each data item has a tag. The tag is defined after the equal sign, e.g. x has the tag 1.

The "Line" and "Polyline" messages demonstrate how composition works in Protocol Buffers (they both use Point). Polyline has a repeated field, which behaves like a vector.

See also

• ASN.1 Distinguished Encoding Rules
• Facebook’s Thrift
• Cisco’s Etch
• ZeroC's ICE
• SDXF: Structured Data Exchange Format
• Comparison of data serialization formats

External links

• Official project page at code.google.com
• Another official home page at Google Open Source project hosting