protobuf:字符编码

Encoding

字符编码

This topic explains how Protocol Buffers encodes data to files or to the wire.

本主题介绍协议缓冲区如何将数据编码到文件或连接。

This document describes the protocol buffer wire format, which defines the details of how your message is sent on the wire and how much space it consumes on disk. You probably don’t need to understand this to use protocol buffers in your application, but it’s useful information for doing optimizations.

本文档描述了协议缓冲区线格式，它定义了如何在线上发送消息以及它在磁盘上占用多少空间的详细信息。可能不需要了解这一点就可以在应用程序中使用协议缓冲区，但这对于进行优化非常有用。

If you already know the concepts but want a reference, skip to the Condensed reference card section.

​如果已经知道这些概念，但想要参考，请跳到简明参考卡部分。

Protoscope is a very simple language for describing snippets of the low-level wire format, which we’ll use to provide a visual reference for the encoding of various messages. Protoscope’s syntax consists of a sequence of tokens that each encode down to a specific byte sequence.

Protoscope是一种非常简单的语言，用于描述低级线格式的片段，我们将使用它为各种消息的编码提供可视化参考。Protoscope的语法由一系列标记组成，每个标记都编码为特定的字节序列。

For example, backticks denote a raw hex literal, like `70726f746f6275660a`. This encodes into the exact bytes denoted as hex in the literal. Quotes denote UTF-8 strings, like "Hello, Protobuf!". This literal is synonymous with `48656c6c6f2c2050726f746f62756621` (which, if you observe closely, is composed of ASCII bytes). We’ll introduce more of the Protoscope language as we discuss aspects of the wire format.

例如，反勾号表示原始十六进制文字，如`70726f746f6275660a`。这将编码为文本中表示为十六进制的确切字节。引号表示UTF-8字符串，如"Hello, Protobuf!"。此文字与`48656c6c6f2c2050726f746f62756621`同义（如果仔细观察，它由ASCII字节组成）。在讨论线格式的各个方面时，我们将介绍更多的Protoscope语言。

The Protoscope tool can also dump encoded protocol buffers as text. See https://github.com/protocolbuffers/protoscope/tree/main/testdata for examples.

​Protoscope工具还可以将编码的协议缓冲区转储为文本。请参阅https://github.com/protocolbuffers/protoscope/tree/main/testdata示例。

A Simple Message

一个简单的消息

Let’s say you have the following very simple message definition:

假设有以下非常简单的消息定义：

message Test1 {
  optional int32 a = 1;
}

In an application, you create a Test1 message and set a to 150. You then serialize the message to an output stream. If you were able to examine the encoded message, you’d see three bytes:

在一个应用程序中，创建一个Test1消息，并将a设置为150。然后将消息序列化为输出流。如果能够检查编码的消息，您将看到三个字节：

08 96 01

So far, so small and numeric – but what does it mean? If you use the Protoscope tool to dump those bytes, you’d get something like 1: 150. How does it know this is the contents of the message?

到目前为止，数字如此之小——但这意味着什么？如果你使用Protoscope工具来转储这些字节，会得到大约1:150的结果。它怎么知道这是消息的内容？

Base 128 Varints

基础的128变量

Variable-width integers, or varints, are at the core of the wire format. They allow encoding unsigned 64-bit integers using anywhere between one and ten bytes, with small values using fewer bytes.

可变宽度整数或变量是线格式的核心。它们允许使用1到10个字节对无符号64位整数进行编码，小值使用更少的字节。

Each byte in the varint has a continuation bit that indicates if the byte that follows it is part of the varint. This is the most significant bit (MSB) of the byte (sometimes also called the sign bit). The lower 7 bits are a payload; the resulting integer is built by appending together the 7-bit payloads of its constituent bytes.

变量中的每个字节都有一个连续位，指示后面的字节是否是变量的一部分。这是字节的最高有效位（MSB）（有时也称为符号位）。较低的7个比特是有效载荷；所得到的整数是通过将其组成字节的7比特有效载荷附加在一起而构建的。

So, for example, here is the number 1, encoded as `01` – it’s a single byte, so the MSB is not set:

因此，例如，这里是数字1，编码为“01”——它是一个单字节，因此MSB未设置：

0000 0001
^ msb

And here is 150, encoded as `9601` – this is a bit more complicated:

这里是150，编码为“9601”——这有点复杂：

10010110 00000001
^ msb    ^ msb

How do you figure out that this is 150? First you drop the MSB from each byte, as this is just there to tell us whether we’ve reached the end of the number (as you can see, it’s set in the first byte as there is more than one byte in the varint). Then we concatenate the 7-bit payloads, and interpret it as a little-endian, 64-bit unsigned integer:

怎么知道这是150？首先，从每个字节中删除MSB，因为它只是用来告诉我们是否已经到达数字的末尾（正如所看到的，它设置在第一个字节中，因为变量中有多个字节）。然后，我们将7位有效载荷连接起来，并将其解释为一个小端序、64位无符号整数：

10010110 00000001        // Original inputs. 原始输入。
 0010110  0000001        // Drop continuation bits. 删除连续位。
 0000001  0010110        // Put into little-endian order. 按小序排列。
 10010110                // Concatenate. 连接运算
 128 + 16 + 4 + 2 = 150  // Interpret as integer. 解释为整数。

Because varints are so crucial to protocol buffers, in protoscope syntax, we refer to them as plain integers. 150 is the same as `9601`.

因为变量对协议缓冲区至关重要，所以在protoscope语法中，我们将它们称为纯整数。150与“9601”相同。

Message Structure

消息结构

A protocol buffer message is a series of key-value pairs. The binary version of a message just uses the field’s number as the key – the name and declared type for each field can only be determined on the decoding end by referencing the message type’s definition (i.e. the .proto file). Protoscope does not have access to this information, so it can only provide the field numbers.

协议缓冲区消息是一系列键值对。消息的二进制版本只使用字段的编号作为键——每个字段的名称和声明类型只能在解码端通过引用消息类型的定义（即.proto文件）来确定。Protoscope无法访问这些信息，因此只能提供字段编号。

When a message is encoded, each key-value pair is turned into a record consisting of the field number, a wire type and a payload. The wire type tells the parser how big the payload after it is. This allows old parsers to skip over new fields they don’t understand. This type of scheme is sometimes called Tag-Length-Value, or TLV.

​当对消息进行编码时，每个键值对都会变成一个由字段号、线类型和有效负载组成的记录。连线类型告诉解析器负载的大小。这允许旧解析器跳过他们不理解的新字段。这种类型的方案有时被称为标签长度值，或TLV。

There are six wire types: VARINT, I64, LEN, SGROUP, EGROUP, and I32

有六种线类型：VARINT、I64、LEN、SGROUP、EGROUP和I32

ID	Name	Used For
0	VARINT	int32, int64, uint32, uint64, sint32, sint64, bool, enum
1	I64	fixed64, sfixed64, double
2	LEN	string, bytes, embedded messages, packed repeated fields
3	SGROUP	group start (deprecated)
4	EGROUP	group end (deprecated)
5	I32	fixed32, sfixed32, float

The “tag” of a record is encoded as a varint formed from the field number and the wire type via the formula (field_number << 3) | wire_type. In other words, after decoding the varint representing a field, the low 3 bits tell us the wire type, and the rest of the integer tells us the field number.

记录的“标记”被编码为由字段号和线类型通过公式（field_number<<3）|wire_type形成的变量。换句话说，在解码表示字段的变量后，低3位告诉我们线类型，其余整数告诉字段编号。

Now let’s look at our simple example again. You now know that the first number in the stream is always a varint key, and here it’s `08`, or (dropping the MSB):

现在让我们再次看一下我们的简单示例。现在知道，流中的第一个数字始终是variant键，这里它是“08”或（去掉MSB）：

000 1000

You take the last three bits to get the wire type (0) and then right-shift by three to get the field number (1). Protoscope represents a tag as an integer followed by a colon and the wire type, so we can write the above bytes as 1:VARINT.

取最后三位得到线类型（0），然后右移三位得到字段号（1）。Protoscope将标记表示为一个整数，后跟冒号和线类型，因此我们可以将上面的字节写成1:VARINT。

Because the wire type is 0, or VARINT, we know that we need to decode a varint to get the payload. As we saw above, the bytes `9601` varint-decode to 150, giving us our record. We can write it in Protoscope as 1:VARINT 150.

因为线类型是0，或者VARINT，所以我们知道我们需要解码一个variant来获得有效载荷。正如我们在上面看到的，字节“9601”变为150，这给了我们记录。我们可以在Protoscope中将其写成1:VARINT150。

Protoscope can infer the type for a tag if there is whitespace after the :. It does so by looking ahead at the next token and guessing what you meant (the rules are documented in detail in Protoscope’s language.txt). For example, in 1: 150, there is a varint immediately after the untyped tag, so Protoscope infers its type to be VARINT. If you wrote 2: {}, it would see the { and guess LEN; if you wrote 3: 5i32 it would guess I32, and so on.

​如果以下内容后面有空格，Protoscope可以推断标记的类型：。它通过向前看下一个标记并猜测你的意思来做到这一点（规则在Protoscope的language.txt中有详细的文档记录）。例如，在1:150中，在未键入的标记后面有一个variant，因此Protoscop推断其类型为varint。如果你写2:{}，它会看到{并猜测LEN；如果你写3:5i32，它会猜测I32，依此类推。

More Integer Types

更多整数类型

Bools and Enums

布尔和枚举

Bools and enums are both encoded as if they were int32s. Bools, in particular, always encode as either `00` or `01`. In Protoscope, false and true are aliases for these byte strings.

布尔和枚举都是像int32一样编码的。特别是布尔，总是编码为“00”或“01”。在Protoscope中，false和true是这些字节字符串的别名。

Signed Integers

有符号整数

As you saw in the previous section, all the protocol buffer types associated with wire type 0 are encoded as varints. However, varints are unsigned, so the different signed types, sint32 and sint64 vs int32 or int64, encode negative integers differently.

正如在上一节中看到的，与连线类型0相关联的所有协议缓冲区类型都被编码为变量。然而，变量是无符号的，因此不同的有符号类型sint32和sint64与int32或int64对负整数的编码不同。

The intN types encode negative numbers as two’s complement, which means that, as unsigned, 64-bit integers, they have their highest bit set. As a result, this means that all ten bytes must be used. For example, -2 is converted by protoscope into

intN类型将负数编码为2的补码，这意味着，作为无符号的64位整数，它们有其最高位集。因此，这意味着必须使用全部十个字节。例如，-2被protoscope转换为

11111110 11111111 11111111 11111111 11111111
11111111 11111111 11111111 11111111 00000001

This is the two’s complement of 2, defined in unsigned arithmetic as ~0 - 2 + 1, where ~0 is the all-ones 64-bit integer. It is a useful exercise to understand why this produces so many ones.

这是2的2的补码，在无符号算术中定义为~0 - 2 + 1，其中~0是全1 64位整数。这是一个有用的练习，以了解为什么这会产生这么多。

sintN uses the “ZigZag” encoding instead of two’s complement to encode negative integers. Positive integers p are encoded as 2 * p (the even numbers), while negative integers n are encoded as 2 * |n| - 1 (the odd numbers). The encoding thus “zig-zags” between positive and negative numbers. For example:

sintN使用“ZigZag”编码而不是二的补码来编码负整数。正整数p编码为2*p（偶数），而负整数n编码为2*|n|-1（奇数）。因此，编码在正数和负数之间“曲折”。例如：

Signed Original	Encoded As
0	0
-1	1
1	2
-2	3
…	…
0x7fffffff	0xfffffffe
-0x80000000	0xffffffff

In other words, each value n is encoded using

换句话说，每个值n使用

(n << 1) ^ (n >> 31)

for sint32s, or

对于sint32s，或

(n << 1) ^ (n >> 63)

for the 64-bit version.

对于64位版本。

When the sint32 or sint64 is parsed, its value is decoded back to the original, signed version.

当sint32或sint64被解析时，其值被解码回原始的签名版本。

In protoscope, suffixing an integer with a z will make it encode as ZigZag. For example, -500z is the same as the varint 999.

在protoscope中，在一个整数后面加上z将使其编码为ZigZag。例如，-500z与变量999相同。

Non-varint Numbers

非变量数字

Non-varint numeric types are simple – double and fixed64 have wire type I64, which tells the parser to expect a fixed eight-byte lump of data. We can specify a double record by writing 5: 25.4, or a fixed64 record with 6: 200i64. In both cases, omitting an explicit wire type implies the I64 wire type.

非变量数字类型很简单–double和fixed64的线类型为I64，这告诉解析器需要一个固定的8字节数据块。我们可以通过写5:25.4指定一个double记录，或者用6:200i64指定一个fixed64记录。在这两种情况下，省略显式导线类型意味着I64线类型。

Similarly float and fixed32 have wire type I32, which tells it to expect four bytes instead. The syntax for these consists of adding an i32 prefix. 25.4i32 will emit four bytes, as will 200i32. Tag types are inferred as I32.

类似地，float和fixed32的线类型为I32，这告诉它需要四个字节。它们的语法包括添加一个i32前缀。25.4i32将发出四个字节，200i32也是如此。标记类型推断为I32。

Length-Delimited Records

长度分隔的记录

Length prefixes are another major concept in the wire format. The LEN wire type has a dynamic length, specified by a varint immediately after the tag, which is followed by the payload as usual.

长度前缀是线格式中的另一个主要概念。LEN线类型具有动态长度，由紧跟在标签后面的变量指定，该变量后面通常是有效载荷。

Consider this message schema:

请考虑以下消息架构：

message Test2 {
  optional string b = 2;
}

A record for the field b is a string, and strings are LEN-encoded. If we set b to "testing", we encoded as a LEN record with field number 2 containing the ASCII string "testing". The result is `120774657374696e67`. Breaking up the bytes,

字段b的记录是字符串，并且字符串是LEN编码的。如果我们将b设置为“testing”，我们将编码为LEN记录，字段号为2，包含ASCII字符串“testing”。结果是“120774657374696e67”。分解字节，

12 07 [74 65 73 74 69 6e 67]

we see that the tag, `12`, is 00010 010, or 2:LEN. The byte that follows is the int32 varint 7, and the next seven bytes are the UTF-8 encoding of "testing". The int32 varint means that the max length of a string is 2GB.

我们看到标签“12”是0001010:LEN。接下来的字节是int32变量7，接下来的7个字节是“测试”的UTF-8编码。int32变量表示字符串的最大长度为2GB。

In Protoscope, this is written as 2:LEN 7 "testing". However, it can be incovenient to repeat the length of the string (which, in Protoscope text, is already quote-delimited). Wrapping Protoscope content in braces will generate a length prefix for it: {"testing"} is a shorthand for 7 "testing". {} is always inferred by fields to be a LEN record, so we can write this record simply as 2: {"testing"}.

在Protoscope中，这被写成2:LEN 7“测试”。但是，重复字符串的长度（在Protoscope文本中，该字符串已经用引号分隔）可能是不方便的。用大括号包装Protoscope内容将为其生成一个长度前缀：｛“testing”｝是7“test”的缩写。｛｝总是由字段推断为LEN记录，因此我们可以将此记录简单地写成2:｛“test”｝。

bytes fields are encoded in the same way.

字节字段以相同的方式进行编码。

Submessages

子消息

Submessage fields also use the LEN wire type. Here’s a message definition with an embedded message of our original example message, Test1:

子消息字段也使用LEN线类型。以下是一个消息定义，其中包含我们原始示例消息Test1的嵌入消息：

message Test3 {
  optional Test1 c = 3;
}

If Test1’s a field (i.e., Test3’s c.a field) is set to 150, we get ``1a03089601``. Breaking it up:

如果Test1的一个字段（即，Test3的c.a字段）设置为150，我们得到“1a03089601”。分解：

 1a 03 [08 96 01]

The last three bytes (in []) are exactly the same ones from our very first example. These bytes are preceded by a LEN-typed tag, and a length of 3, exactly the same way as strings are encoded.

​最后三个字节（在[]中）与我们第一个示例中的字节完全相同。这些字节前面有一个LEN类型的标记，长度为3，与字符串的编码方式完全相同。

In Protoscope, submessages are quite succinct. ``1a03089601`` can be written as 3: {1: 150}.

在Protoscope中，子消息非常简洁。“1a03089601”可以写成3:｛1:150｝。

Optional and Repeated Elements

可选和重复元素

Missing optional fields are easy to encode: we just leave out the record if it’s not present. This means that “huge” protos with only a few fields set are quite sparse.

缺少可选字段很容易编码：如果记录不存在，我们只会忽略它。这意味着只有几个场集的“巨大”质子是相当稀疏的。

repeated fields are a bit more complicated. Ordinary (not packed) repeated fields emit one record for every element of the field. Thus, if we have

​重复字段有点复杂。普通（非压缩）重复字段为字段的每个元素发出一条记录。因此，如果我们

message Test4 {
  optional string d = 4;
  repeated int32 e = 5;
}

and we construct a Test4 message with d set to "hello", and e set to 1, 2, and 3, this could be encoded as `220568656c6c6f280128022803`, or written out as Protoscope,

我们构造了一个Test4消息，其中d设置为“hello”，e设置为1、2和3，这可以编码为“220568656c6f280128022803”，也可以写成Protoscope，

4: {"hello"}
5: 1
5: 2
5: 3

However, records for e do not need to appear consecutively, and can be interleaved with other fields; only the order of records for the same field with respect to each other is preserved. Thus, this could also have been encoded as

然而，e的记录不需要连续出现，可以与其他字段交错出现；仅保留相同字段的记录相对于彼此的顺序。因此，这也可能被编码为

5: 1
5: 2
4: {"hello"}
5: 3

There is no special treatment for oneofs in the wire format.

线格式中的oneof没有特殊处理。

Last One Wins

最后一场胜利

Normally, an encoded message would never have more than one instance of a non-repeated field. However, parsers are expected to handle the case in which they do. For numeric types and strings, if the same field appears multiple times, the parser accepts the last value it sees. For embedded message fields, the parser merges multiple instances of the same field, as if with the Message::MergeFrom method – that is, all singular scalar fields in the latter instance replace those in the former, singular embedded messages are merged, and repeated fields are concatenated. The effect of these rules is that parsing the concatenation of two encoded messages produces exactly the same result as if you had parsed the two messages separately and merged the resulting objects. That is, this:

通常，一个编码的消息永远不会有一个以上的非重复字段实例。然而，解析器应该处理它们这样做的情况。对于数字类型和字符串，如果同一个字段多次出现，解析器会接受它看到的最后一个值。对于嵌入的消息字段，解析器会合并同一字段的多个实例，就像使用message::MergeFrom方法一样——也就是说，后一个实例中的所有奇异标量字段都会替换前一个实例，奇异嵌入消息会被合并，重复字段会被连接。这些规则的效果是，解析两个编码消息的串联会产生完全相同的结果，就好像分别解析了这两个消息并合并了结果对象一样。也就是说：

MyMessage message;
message.ParseFromString(str1 + str2);

is equivalent to this:

相当于：

MyMessage message, message2;
message.ParseFromString(str1);
message2.ParseFromString(str2);
message.MergeFrom(message2);

This property is occasionally useful, as it allows you to merge two messages (by concatenation) even if you do not know their types.

此属性偶尔很有用，因为它允许合并两条消息（通过串联），即使不知道它们的类型。

Packed Repeated Fields

压缩重复字段

Starting in v2.1.0, repeated fields of scalar type can be declared as “packed”. In proto2 this is done using the field option [packed=true]. In proto3 it is the default.

​从v2.1.0开始，标量类型的重复字段可以声明为“packed”。在proto2中，这是使用字段选项[packed=true]来完成的。在proto3中，它是默认的。

Instead of being encoded as one record per entry, they are encoded as a single LEN record that contains each element concatenated. To decode, elements are decoded from the LEN record one by one until the payload is exhausted. The start of the next element is determined by the length of the previous, which itself depends on the type of the field.

它们不是被编码为每个条目一个记录，而是被编码为包含每个串联元素的单个LEN记录。为了解码，从LEN记录中逐个解码元素，直到有效载荷耗尽。下一个元素的开始由上一个元素长度决定，上一个的长度本身取决于字段的类型。

For example, imagine you have the message type:

例如，假设消息类型为：

message Test5 {
  repeated int32 f = 6 [packed=true];
}

Now let’s say you construct a Test5, providing the values 3, 270, and 86942 for the repeated field f. Encoded, this gives us `3206038e029ea705`, or as Protoscope text,

现在假设您构建一个Test5，为重复字段f提供值3、270和86942。经过编码，这给了我们“3206038e029ea705”，或者作为Protoscope文本，

6: {3 270 86942}

Only repeated fields of primitive numeric types can be declared “packed”. These are types that would normally use the VARINT, I32, or I64 wire types.

只有基元数字类型的重复字段才能声明为“packed”。这些类型通常使用VARINT、I32或I64线类型。

Note that although there’s usually no reason to encode more than one key-value pair for a packed repeated field, parsers must be prepared to accept multiple key-value pairs. In this case, the payloads should be concatenated. Each pair must contain a whole number of elements. The following is a valid encoding of the same message above that parsers must accept:

请注意，尽管通常没有理由为打包的重复字段编码多个键值对，但解析器必须准备好接受多个键值配对。在这种情况下，应将有效载荷连接起来。每一对都必须包含整数个元素。以下是解析器必须接受的上述同一消息的有效编码：

6: {3 270}
6: {86942}

Protocol buffer parsers must be able to parse repeated fields that were compiled as packed as if they were not packed, and vice versa. This permits adding [packed=true] to existing fields in a forward- and backward-compatible way.

协议缓冲区解析器必须能够解析被编译为压缩的重复字段，就像它们没有被压缩一样，反之亦然。这允许以向前和向后兼容的方式将[packed=true]添加到现有字段中。

Maps

映射

Map fields are just a shorthand for a special kind of repeated field. If we have

映射字段只是一种特殊类型的重复字段的简写。如果我们有

message Test6 {
  map<string, int32> g = 7;
}

this is actually the same as

这实际上与

message Test6 {
  message g_Entry {
    optional string key = 1;
    optional int32 value = 2;
  }
  repeated g_Entry g = 7;
}

Thus, maps are encoded exactly like a repeated message field: as a sequence of LEN-typed records, with two fields each.

因此，映射的编码方式与重复的消息字段完全相同：作为LEN类型记录的序列，每个字段有两个字段。

Groups

组

Groups are a deprecated feature that should not be used, but they remain in the wire format, and deserve a passing mention.

组是一个不推荐使用的功能，但它们仍然是线格式，值得一提。

A group is a bit like a submessage, but it is delimited by special tags rather than by a LEN prefix. Each group in a message has a field number, which is used on these special tags.

组有点像子消息，但它是由特殊标签而不是LEN前缀分隔的。消息中的每个组都有一个字段号，用于这些特殊标签。

A group with field number 8 begins with an 8:SGROUP tag. SGROUP records have empty payloads, so all this does is denote the start of the group. Once all the fields in the group are listed, a corresponding 8:EGROUP tag denotes its end. EGROUP records also have no payload, so 8:EGROUP is the entire record. Group field numbers need to match up. If we encounter 7:EGROUP where we expect 8:EGROUP, the message is mal-formed.

字段编号为8的组以8:SGROUP标记开始。SGROUP记录的有效载荷为空，因此所有这些都表示组的开始。一旦列出了组中的所有字段，相应的8:EGROUP标记表示其结束。EGROUP记录也没有有效负载，因此8:EGROUP是整个记录。组字段编号需要匹配。如果我们在预期8:EGROUP的情况下遇到7:EGROUP，则消息格式错误。

Protoscope provides a convenient syntax for writing groups. Instead of writing

Protoscope为编写组提供了一种方便的语法。而不是写作

8:SGROUP
  1: 2
  3: {"foo"}
8:EGROUP

Protoscope allows

Protoscope允许

8: !{
  1: 2
  3: {"foo"}
}

This will generate the appropriate start and end group markers. The !{} syntax can only occur immediately after an un-typed tag expression, like 8:.

这将生成适当的开始和结束组标记。！｛｝语法只能立即出现在未键入的标记表达式之后，如8:。

Field Order

字段顺序

Field numbers may be declared in any order in a .proto file. The order chosen has no effect on how the messages are serialized.

proto文件中的字段号可以按任何顺序声明。所选顺序对消息的序列化方式没有影响。

When a message is serialized, there is no guaranteed order for how its known or unknown fields will be written. Serialization order is an implementation detail, and the details of any particular implementation may change in the future. Therefore, protocol buffer parsers must be able to parse fields in any order.

​当消息被序列化时，它的已知或未知字段的写入顺序没有保证。序列化顺序是一个实现细节，任何特定实现的细节将来都可能更改。因此，协议缓冲区解析器必须能够以任何顺序解析字段。

Implications

意义

• Do not assume the byte output of a serialized message is stable. This is especially true for messages with transitive bytes fields representing other serialized protocol buffer messages.
• 不要假定序列化消息的字节输出是稳定的。对于具有表示其他序列化协议缓冲区消息的可传递字节字段的消息来说尤其如此。
• By default, repeated invocations of serialization methods on the same protocol buffer message instance may not produce the same byte output. That is, the default serialization is not deterministic.
• 默认情况下，在同一协议缓冲区消息实例上重复调用序列化方法可能不会产生相同的字节输出。也就是说，默认的序列化不是确定性的。
  • Deterministic serialization only guarantees the same byte output for a particular binary. The byte output may change across different versions of the binary.
  • 确定性序列化只保证特定二进制文件具有相同的字节输出。字节输出可能在二进制文件的不同版本之间发生变化。
• The following checks may fail for a protocol buffer message instance foo:
• 对于协议缓冲区消息实例foo，以下检查可能会失败：
  • foo.SerializeAsString() == foo.SerializeAsString()
  • Hash(foo.SerializeAsString()) == Hash(foo.SerializeAsString())
  • CRC(foo.SerializeAsString()) == CRC(foo.SerializeAsString())
  • FingerPrint(foo.SerializeAsString()) == FingerPrint(foo.SerializeAsString())
• Here are a few example scenarios where logically equivalent protocol buffer messages foo and bar may serialize to different byte outputs:
• 以下是几个示例场景，其中逻辑等效的协议缓冲区消息foo和bar可以串行化为不同的字节输出：
  • bar is serialized by an old server that treats some fields as unknown.
  • bar由一个旧服务器序列化，该服务器将某些字段视为未知字段。
  • bar is serialized by a server that is implemented in a different programming language and serializes fields in different order.
  • bar由一个用不同编程语言实现的服务器序列化，并按不同顺序序列化字段。
  • bar has a field that serializes in a non-deterministic manner.
  • bar有一个以非确定性方式序列化的字段。
  • bar has a field that stores a serialized byte output of a protocol buffer message which is serialized differently.
  • bar有一个字段，用于存储协议缓冲区消息的序列化字节输出，该消息以不同方式序列化。
  • bar is serialized by a new server that serializes fields in a different order due to an implementation change.
  • bar是由一个新的服务器序列化的，由于实现的更改，该服务器以不同的顺序序列化字段。
  • foo and bar are concatenations of the same individual messages in a different order.
  • foo和bar是按不同顺序连接的相同单个消息。

Condensed Reference Card

简明参考卡

The following provides the most prominent parts of the wire format in an easy-to-reference format.

以下内容以易于参考的格式提供了线格式中最突出的部分。

message    := (tag value)*

tag        := (field << 3) bit-or wire_type;
                encoded as uint32 varint
value      := varint      for wire_type == VARINT,
              i32         for wire_type == I32,
              i64         for wire_type == I64,
              len-prefix  for wire_type == LEN,
              <empty>     for wire_type == SGROUP or EGROUP

varint     := int32 | int64 | uint32 | uint64 | bool | enum | sint32 | sint64;
                encoded as varints (sintN are ZigZag-encoded first)
i32        := sfixed32 | fixed32 | float;
                encoded as 4-byte little-endian;
                memcpy of the equivalent C types (u?int32_t, float)
i64        := sfixed64 | fixed64 | double;
                encoded as 8-byte little-endian;
                memcpy of the equivalent C types (u?int32_t, float)

len-prefix := size (message | string | bytes | packed);
                size encoded as int32 varint
string     := valid UTF-8 string (e.g. ASCII);
                max 2GB of bytes
bytes      := any sequence of 8-bit bytes;
                max 2GB of bytes
packed     := varint* | i32* | i64*,
                consecutive values of the type specified in `.proto`

See also the Protoscope Language Reference.

​另请参阅Protoscope语言参考。

Key

message := (tag value)*

A message is encoded as a sequence of zero or more pairs of tags and values.

消息被编码为零对或多对标签和值的序列。

tag := (field << 3) bit-or wire_type

A tag is a combination of a wire_type, stored in the least significant three bits, and the field number that is defined in the .proto file.

标记是存储在最低有效位的wire_type和.proto文件中定义的字段号的组合。

value := varint for wire_type == VARINT, ...

A value is stored differently depending on the wire_type specified in the tag.

值的存储方式不同，具体取决于标记中指定的wire_type。

varint := int32 | int64 | uint32 | uint64 | bool | enum | sint32 | sint64

You can use varint to store any of the listed data types.

可以使用varint来存储任何列出的数据类型。

i32 := sfixed32 | fixed32 | float

You can use fixed32 to store any of the listed data types.

可以使用fixed32来存储任何列出的数据类型。

i64 := sfixed64 | fixed64 | double

You can use fixed64 to store any of the listed data types.

可以使用fixed64来存储任何列出的数据类型。

len-prefix := size (message | string | bytes | packed)

A length-prefixed value is stored as a length (encoded as a varint), and then one of the listed data types.

以长度为前缀的值存储为长度（编码为变量），然后存储为列出的数据类型之一。

string := valid UTF-8 string (e.g. ASCII)

As described, a string must use UTF-8 character encoding. A string cannot exceed 2GB.

如上所述，字符串必须使用UTF-8字符编码。字符串不能超过2GB。

bytes := any sequence of 8-bit bytes

As described, bytes can store custom data types, up to 2GB in size.

如前所述，字节可以存储自定义数据类型，大小最高可达2GB。

packed := varint* | i32* | i64*

Use the packed data type when you are storing consecutive values of the type described in the protocol definition. The tag is dropped for values after the first, which amortizes the costs of tags to one per field, rather than per element.

当存储协议定义中描述的类型的连续值时，请使用压缩数据类型。对于第一个之后的值，标记被丢弃，这将标记的成本分摊到每个字段一个，而不是每个元素一个。