Эх сурвалжийг харах

Added background information about proto3 presence. (#7501)

Joshua Haberman 5 жил өмнө
parent
commit
4bbdffacf2

+ 50 - 0
docs/implementing_proto3_presence.md

@@ -5,6 +5,8 @@ proto3. Proto3 optional fields track presence like in proto2. For background
 information about what presence tracking means, please see
 information about what presence tracking means, please see
 [docs/field_presence](field_presence.md).
 [docs/field_presence](field_presence.md).
 
 
+## Document Summary
+
 This document is targeted at developers who own or maintain protobuf code
 This document is targeted at developers who own or maintain protobuf code
 generators. All code generators will need to be updated to support proto3
 generators. All code generators will need to be updated to support proto3
 optional fields. First-party code generators developed by Google are being
 optional fields. First-party code generators developed by Google are being
@@ -14,6 +16,7 @@ independently by their authors. This includes:
 - implementations of Protocol Buffers for other languges.
 - implementations of Protocol Buffers for other languges.
 - alternate implementations of Protocol Buffers that target specialized use
 - alternate implementations of Protocol Buffers that target specialized use
   cases.
   cases.
+- RPC code generators that create generated APIs for service calls.
 - code generators that implement some utility code on top of protobuf generated
 - code generators that implement some utility code on top of protobuf generated
   classes.
   classes.
 
 
@@ -21,6 +24,53 @@ While this document speaks in terms of "code generators", these same principles
 apply to implementations that dynamically generate a protocol buffer API "on the
 apply to implementations that dynamically generate a protocol buffer API "on the
 fly", directly from a descriptor, in languages that support this kind of usage.
 fly", directly from a descriptor, in languages that support this kind of usage.
 
 
+## Background
+
+Presence tracking was added to proto3 in response to user feedback, both from
+inside Google and [from open-source
+users](https://github.com/protocolbuffers/protobuf/issues/1606). The [proto3
+wrapper
+types](https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/wrappers.proto)
+were previously the only supported presence mechanism for proto3. Users have
+pointed to both efficiency and usability issues with the wrapper types.
+
+Presence in proto3 uses exactly the same syntax and semantics as in proto2.
+Proto3 Fields marked `optional` will track presence like proto2, while fields
+without any label (known as "singular fields"), will continue to omit presence
+information.  The `optional` keyword was chosen to minimize differences with
+proto2.
+
+Unfortunately, for the current descriptor protos and `Descriptor` API (as of
+3.11.4) it is not possible to use the same representation as proto2. Proto3
+descriptors already use `LABEL_OPTIONAL` for proto3 singular fields, which do
+not track presence. There is a lot of existing code that reflects over proto3
+protos and assumes that `LABEL_OPTIONAL` in proto3 means "no presence." Changing
+the semantics now would be risky, since old software would likely drop proto3
+presence information, which would be a data loss bug.
+
+To minimize this risk we chose a descriptor representation that is semantically
+compatible with existing proto3 reflection. Every proto3 optional field is
+placed into a one-field `oneof`. We call this a "synthetic" oneof, as it was not
+present in the source `.proto` file.
+
+Since oneof fields in proto3 already track presence, existing proto3
+reflection-based algorithms should correctly preserve presence for proto3
+optional fields with no code changes. For example, the JSON and TextFormat
+parsers/serializers in C++ and Java did not require any changes to support
+proto3 presence. This is the major benefit of synthetic oneofs.
+
+This design does leave some cruft in descriptors. Synthetic oneofs are a
+compatibility measure that we can hopefully clean up in the future. For now
+though, it is important to preserve them across different descriptor formats and
+APIs. It is never safe to drop synthetic oneofs from a proto schema. Code
+generators can (and should) skip synthetic oneofs when generating a user-facing
+API or user-facing documentation. But for any schema representation that is
+consumed programmatically, it is important to keep the synthetic oneofs around.
+
+In APIs it can be helpful to offer separate accessors that refer to "real"
+oneofs (see [API Changes](#api-changes) below). This is a convenient way to omit
+synthetic oneofs in code generators.
+
 ## Updating a Code Generator
 ## Updating a Code Generator
 
 
 When a user adds an `optional` field to proto3, this is internally rewritten as
 When a user adds an `optional` field to proto3, this is internally rewritten as