Update docs so we can generate better output from Sphinx. (#7295)

This change updates docstrings and comments so that they will produce nicer
formatting and cross-references from Sphinx. There are a few broad categories of
changes:

- Paramter and attribute docs are updated so that types will be recognized by
  Napoleon (https://sphinxcontrib-napoleon.readthedocs.io/en/latest/) This
  usually just means moving a colon in the docstring, so
  `name: (type) description` becomes `name (type): description`.

- References to other symbols can be cross-references if they have the right
  format. For example, "attr_name" might become ":attr:`attr_name`".
  https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#cross-referencing-python-objects

- For fenced code blocks, adding a double-colon `::` signifies a literal block.
  https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#literal-blocks

- Some bits of docstrings move from docstring to comments. For TODOs, this
  means we won't be putting stale (or otherwise unrelated) noise into the docs.
  For `Message.DESCRIPTOR`, the change means it gets appropriate documentation.

- There are some wording tweaks for consistency, and some new docstrings
  (especially for methods in Message).

For types, I used the convention of `list[Foo]` and `dict(foo, bar)`, which seem
to be common among other Python rst docstrings. Sphinx should generally
recognize both, and cross-links them correctly (both internally and to Python
library documentation). Upgrading to Python3-style type annotations would allow
us to use `sphinx-autodoc-typehints`; the changes in this commit are very
similar to typing-based hints.

python/google/protobuf/descriptor.py

@@ -173,13 +173,12 @@ class _NestedDescriptorBase(DescriptorBase):
     Args:
       options: Protocol message options or None
         to use default message options.
-      options_class_name: (str) The class name of the above options.
-
-      name: (str) Name of this protocol message type.
-      full_name: (str) Fully-qualified name of this protocol message type,
+      options_class_name (str): The class name of the above options.
+      name (str): Name of this protocol message type.
+      full_name (str): Fully-qualified name of this protocol message type,
         which will include protocol "package" name and the name of any
         enclosing types.
-      file: (FileDescriptor) Reference to file info.
+      file (FileDescriptor): Reference to file info.
       containing_type: if provided, this is a nested descriptor, with this
         descriptor as parent, otherwise None.
       serialized_start: The start index (inclusive) in block in the
@@ -223,56 +222,45 @@ class Descriptor(_NestedDescriptorBase):
 
   """Descriptor for a protocol message type.
 
-  A Descriptor instance has the following attributes:
-
-    name: (str) Name of this protocol message type.
-    full_name: (str) Fully-qualified name of this protocol message type,
-      which will include protocol "package" name and the name of any
-      enclosing types.
-
-    containing_type: (Descriptor) Reference to the descriptor of the
-      type containing us, or None if this is top-level.
-
-    fields: (list of FieldDescriptors) Field descriptors for all
-      fields in this type.
-    fields_by_number: (dict int -> FieldDescriptor) Same FieldDescriptor
-      objects as in |fields|, but indexed by "number" attribute in each
-      FieldDescriptor.
-    fields_by_name: (dict str -> FieldDescriptor) Same FieldDescriptor
-      objects as in |fields|, but indexed by "name" attribute in each
-      FieldDescriptor.
-    fields_by_camelcase_name: (dict str -> FieldDescriptor) Same
-      FieldDescriptor objects as in |fields|, but indexed by
-      "camelcase_name" attribute in each FieldDescriptor.
-
-    nested_types: (list of Descriptors) Descriptor references
-      for all protocol message types nested within this one.
-    nested_types_by_name: (dict str -> Descriptor) Same Descriptor
-      objects as in |nested_types|, but indexed by "name" attribute
-      in each Descriptor.
-
-    enum_types: (list of EnumDescriptors) EnumDescriptor references
-      for all enums contained within this type.
-    enum_types_by_name: (dict str ->EnumDescriptor) Same EnumDescriptor
-      objects as in |enum_types|, but indexed by "name" attribute
-      in each EnumDescriptor.
-    enum_values_by_name: (dict str -> EnumValueDescriptor) Dict mapping
-      from enum value name to EnumValueDescriptor for that value.
-
-    extensions: (list of FieldDescriptor) All extensions defined directly
-      within this message type (NOT within a nested type).
-    extensions_by_name: (dict, string -> FieldDescriptor) Same FieldDescriptor
-      objects as |extensions|, but indexed by "name" attribute of each
-      FieldDescriptor.
-
-    is_extendable:  Does this type define any extension ranges?
-
-    oneofs: (list of OneofDescriptor) The list of descriptors for oneof fields
-      in this message.
-    oneofs_by_name: (dict str -> OneofDescriptor) Same objects as in |oneofs|,
-      but indexed by "name" attribute.
-
-    file: (FileDescriptor) Reference to file descriptor.
+  Attributes:
+      name (str): Name of this protocol message type.
+      full_name (str): Fully-qualified name of this protocol message type,
+          which will include protocol "package" name and the name of any
+          enclosing types.
+      containing_type (Descriptor): Reference to the descriptor of the type
+          containing us, or None if this is top-level.
+      fields (list[FieldDescriptor]): Field descriptors for all fields in
+          this type.
+      fields_by_number (dict(int, FieldDescriptor)): Same
+          :class:`FieldDescriptor` objects as in :attr:`fields`, but indexed
+          by "number" attribute in each FieldDescriptor.
+      fields_by_name (dict(str, FieldDescriptor)): Same
+          :class:`FieldDescriptor` objects as in :attr:`fields`, but indexed by
+          "name" attribute in each :class:`FieldDescriptor`.
+      nested_types (list[Descriptor]): Descriptor references
+          for all protocol message types nested within this one.
+      nested_types_by_name (dict(str, Descriptor)): Same Descriptor
+          objects as in :attr:`nested_types`, but indexed by "name" attribute
+          in each Descriptor.
+      enum_types (list[EnumDescriptor]): :class:`EnumDescriptor` references
+          for all enums contained within this type.
+      enum_types_by_name (dict(str, EnumDescriptor)): Same
+          :class:`EnumDescriptor` objects as in :attr:`enum_types`, but
+          indexed by "name" attribute in each EnumDescriptor.
+      enum_values_by_name (dict(str, EnumValueDescriptor)): Dict mapping
+          from enum value name to :class:`EnumValueDescriptor` for that value.
+      extensions (list[FieldDescriptor]): All extensions defined directly
+          within this message type (NOT within a nested type).
+      extensions_by_name (dict(str, FieldDescriptor)): Same FieldDescriptor
+          objects as :attr:`extensions`, but indexed by "name" attribute of each
+          FieldDescriptor.
+      is_extendable (bool):  Does this type define any extension ranges?
+      oneofs (list[OneofDescriptor]): The list of descriptors for oneof fields
+          in this message.
+      oneofs_by_name (dict(str, OneofDescriptor)): Same objects as in
+          :attr:`oneofs`, but indexed by "name" attribute.
+      file (FileDescriptor): Reference to file descriptor.
+
   """
 
   if _USE_C_DESCRIPTORS:
@@ -345,6 +333,9 @@ class Descriptor(_NestedDescriptorBase):
 
   @property
   def fields_by_camelcase_name(self):
+    """Same FieldDescriptor objects as in :attr:`fields`, but indexed by
+    :attr:`FieldDescriptor.camelcase_name`.
+    """
     if self._fields_by_camelcase_name is None:
       self._fields_by_camelcase_name = dict(
           (f.camelcase_name, f) for f in self.fields)
@@ -393,53 +384,51 @@ class FieldDescriptor(DescriptorBase):
 
   """Descriptor for a single field in a .proto file.
 
-  A FieldDescriptor instance has the following attributes:
-
-    name: (str) Name of this field, exactly as it appears in .proto.
-    full_name: (str) Name of this field, including containing scope.  This is
+  Attributes:
+    name (str): Name of this field, exactly as it appears in .proto.
+    full_name (str): Name of this field, including containing scope.  This is
       particularly relevant for extensions.
-    camelcase_name: (str) Camelcase name of this field.
-    index: (int) Dense, 0-indexed index giving the order that this
+    index (int): Dense, 0-indexed index giving the order that this
       field textually appears within its message in the .proto file.
-    number: (int) Tag number declared for this field in the .proto file.
+    number (int): Tag number declared for this field in the .proto file.
 
-    type: (One of the TYPE_* constants below) Declared type.
-    cpp_type: (One of the CPPTYPE_* constants below) C++ type used to
+    type (int): (One of the TYPE_* constants below) Declared type.
+    cpp_type (int): (One of the CPPTYPE_* constants below) C++ type used to
       represent this field.
 
-    label: (One of the LABEL_* constants below) Tells whether this
+    label (int): (One of the LABEL_* constants below) Tells whether this
       field is optional, required, or repeated.
-    has_default_value: (bool) True if this field has a default value defined,
+    has_default_value (bool): True if this field has a default value defined,
       otherwise false.
-    default_value: (Varies) Default value of this field.  Only
+    default_value (Varies): Default value of this field.  Only
       meaningful for non-repeated scalar fields.  Repeated fields
       should always set this to [], and non-repeated composite
       fields should always set this to None.
 
-    containing_type: (Descriptor) Descriptor of the protocol message
+    containing_type (Descriptor): Descriptor of the protocol message
       type that contains this field.  Set by the Descriptor constructor
       if we're passed into one.
       Somewhat confusingly, for extension fields, this is the
       descriptor of the EXTENDED message, not the descriptor
       of the message containing this field.  (See is_extension and
       extension_scope below).
-    message_type: (Descriptor) If a composite field, a descriptor
+    message_type (Descriptor): If a composite field, a descriptor
       of the message type contained in this field.  Otherwise, this is None.
-    enum_type: (EnumDescriptor) If this field contains an enum, a
+    enum_type (EnumDescriptor): If this field contains an enum, a
       descriptor of that enum.  Otherwise, this is None.
 
     is_extension: True iff this describes an extension field.
-    extension_scope: (Descriptor) Only meaningful if is_extension is True.
+    extension_scope (Descriptor): Only meaningful if is_extension is True.
       Gives the message that immediately contains this extension field.
       Will be None iff we're a top-level (file-level) extension field.
 
-    options: (descriptor_pb2.FieldOptions) Protocol message field options or
+    options (descriptor_pb2.FieldOptions): Protocol message field options or
       None to use default field options.
 
-    containing_oneof: (OneofDescriptor) If the field is a member of a oneof
+    containing_oneof (OneofDescriptor): If the field is a member of a oneof
       union, contains its descriptor. Otherwise, None.
 
-    file: (FileDescriptor) Reference to file descriptor.
+    file (FileDescriptor): Reference to file descriptor.
   """
 
   # Must be consistent with C++ FieldDescriptor::Type enum in
@@ -579,6 +568,11 @@ class FieldDescriptor(DescriptorBase):
 
   @property
   def camelcase_name(self):
+    """Camelcase name of this field.
+
+    Returns:
+      str: the name in CamelCase.
+    """
     if self._camelcase_name is None:
       self._camelcase_name = _ToCamelCase(self.name)
     return self._camelcase_name
@@ -594,7 +588,7 @@ class FieldDescriptor(DescriptorBase):
     Args:
       proto_type: the Python proto type (descriptor.FieldDescriptor.TYPE_*)
     Returns:
-      descriptor.FieldDescriptor.CPPTYPE_*, the C++ type.
+      int: descriptor.FieldDescriptor.CPPTYPE_*, the C++ type.
     Raises:
       TypeTransformationError: when the Python proto type isn't known.
     """
@@ -608,24 +602,23 @@ class EnumDescriptor(_NestedDescriptorBase):
 
   """Descriptor for an enum defined in a .proto file.
 
-  An EnumDescriptor instance has the following attributes:
-
-    name: (str) Name of the enum type.
-    full_name: (str) Full name of the type, including package name
+  Attributes:
+    name (str): Name of the enum type.
+    full_name (str): Full name of the type, including package name
       and any enclosing type(s).
 
-    values: (list of EnumValueDescriptors) List of the values
+    values (list[EnumValueDescriptors]): List of the values
       in this enum.
-    values_by_name: (dict str -> EnumValueDescriptor) Same as |values|,
+    values_by_name (dict(str, EnumValueDescriptor)): Same as :attr:`values`,
       but indexed by the "name" field of each EnumValueDescriptor.
-    values_by_number: (dict int -> EnumValueDescriptor) Same as |values|,
+    values_by_number (dict(int, EnumValueDescriptor)): Same as :attr:`values`,
       but indexed by the "number" field of each EnumValueDescriptor.
-    containing_type: (Descriptor) Descriptor of the immediate containing
+    containing_type (Descriptor): Descriptor of the immediate containing
       type of this enum, or None if this is an enum defined at the
       top level in a .proto file.  Set by Descriptor's constructor
       if we're passed into one.
-    file: (FileDescriptor) Reference to file descriptor.
-    options: (descriptor_pb2.EnumOptions) Enum options message or
+    file (FileDescriptor): Reference to file descriptor.
+    options (descriptor_pb2.EnumOptions): Enum options message or
       None to use default enum options.
   """
 
@@ -664,7 +657,7 @@ class EnumDescriptor(_NestedDescriptorBase):
     """Copies this to a descriptor_pb2.EnumDescriptorProto.
 
     Args:
-      proto: An empty descriptor_pb2.EnumDescriptorProto.
+      proto (descriptor_pb2.EnumDescriptorProto): An empty descriptor proto.
     """
     # This function is overridden to give a better doc comment.
     super(EnumDescriptor, self).CopyToProto(proto)
@@ -674,14 +667,15 @@ class EnumValueDescriptor(DescriptorBase):
 
   """Descriptor for a single value within an enum.
 
-    name: (str) Name of this value.
-    index: (int) Dense, 0-indexed index giving the order that this
+  Attributes:
+    name (str): Name of this value.
+    index (int): Dense, 0-indexed index giving the order that this
       value appears textually within its enum in the .proto file.
-    number: (int) Actual number assigned to this enum value.
-    type: (EnumDescriptor) EnumDescriptor to which this value
-      belongs.  Set by EnumDescriptor's constructor if we're
+    number (int): Actual number assigned to this enum value.
+    type (EnumDescriptor): :class:`EnumDescriptor` to which this value
+      belongs.  Set by :class:`EnumDescriptor`'s constructor if we're
       passed into one.
-    options: (descriptor_pb2.EnumValueOptions) Enum value options message or
+    options (descriptor_pb2.EnumValueOptions): Enum value options message or
       None to use default enum value options options.
   """
 
@@ -713,14 +707,15 @@ class EnumValueDescriptor(DescriptorBase):
 class OneofDescriptor(DescriptorBase):
   """Descriptor for a oneof field.
 
-    name: (str) Name of the oneof field.
-    full_name: (str) Full name of the oneof field, including package name.
-    index: (int) 0-based index giving the order of the oneof field inside
+  Attributes:
+    name (str): Name of the oneof field.
+    full_name (str): Full name of the oneof field, including package name.
+    index (int): 0-based index giving the order of the oneof field inside
       its containing type.
-    containing_type: (Descriptor) Descriptor of the protocol message
-      type that contains this field.  Set by the Descriptor constructor
+    containing_type (Descriptor): :class:`Descriptor` of the protocol message
+      type that contains this field.  Set by the :class:`Descriptor` constructor
       if we're passed into one.
-    fields: (list of FieldDescriptor) The list of field descriptors this
+    fields (list[FieldDescriptor]): The list of field descriptors this
       oneof can contain.
   """
 
@@ -750,18 +745,19 @@ class ServiceDescriptor(_NestedDescriptorBase):
 
   """Descriptor for a service.
 
-    name: (str) Name of the service.
-    full_name: (str) Full name of the service, including package name.
-    index: (int) 0-indexed index giving the order that this services
+  Attributes:
+    name (str): Name of the service.
+    full_name (str): Full name of the service, including package name.
+    index (int): 0-indexed index giving the order that this services
       definition appears within the .proto file.
-    methods: (list of MethodDescriptor) List of methods provided by this
+    methods (list[MethodDescriptor]): List of methods provided by this
       service.
-    methods_by_name: (dict str -> MethodDescriptor) Same MethodDescriptor
-      objects as in |methods_by_name|, but indexed by "name" attribute in each
-      MethodDescriptor.
-    options: (descriptor_pb2.ServiceOptions) Service options message or
+    methods_by_name (dict(str, MethodDescriptor)): Same
+      :class:`MethodDescriptor` objects as in :attr:`methods_by_name`, but
+      indexed by "name" attribute in each :class:`MethodDescriptor`.
+    options (descriptor_pb2.ServiceOptions): Service options message or
       None to use default service options.
-    file: (FileDescriptor) Reference to file info.
+    file (FileDescriptor): Reference to file info.
   """
 
   if _USE_C_DESCRIPTORS:
@@ -788,14 +784,21 @@ class ServiceDescriptor(_NestedDescriptorBase):
       method.containing_service = self
 
   def FindMethodByName(self, name):
-    """Searches for the specified method, and returns its descriptor."""
+    """Searches for the specified method, and returns its descriptor.
+
+    Args:
+      name (str): Name of the method.
+    Returns:
+      MethodDescriptor or None: the desctiptor for the requested method, if
+      found.
+    """
     return self.methods_by_name.get(name, None)
 
   def CopyToProto(self, proto):
     """Copies this to a descriptor_pb2.ServiceDescriptorProto.
 
     Args:
-      proto: An empty descriptor_pb2.ServiceDescriptorProto.
+      proto (descriptor_pb2.ServiceDescriptorProto): An empty descriptor proto.
     """
     # This function is overridden to give a better doc comment.
     super(ServiceDescriptor, self).CopyToProto(proto)
@@ -805,15 +808,18 @@ class MethodDescriptor(DescriptorBase):
 
   """Descriptor for a method in a service.
 
-  name: (str) Name of the method within the service.
-  full_name: (str) Full name of method.
-  index: (int) 0-indexed index of the method inside the service.
-  containing_service: (ServiceDescriptor) The service that contains this
-    method.
-  input_type: The descriptor of the message that this method accepts.
-  output_type: The descriptor of the message that this method returns.
-  options: (descriptor_pb2.MethodOptions) Method options message or
-    None to use default method options.
+  Attributes:
+    name (str): Name of the method within the service.
+    full_name (str): Full name of method.
+    index (int): 0-indexed index of the method inside the service.
+    containing_service (ServiceDescriptor): The service that contains this
+      method.
+    input_type (Descriptor): The descriptor of the message that this method
+      accepts.
+    output_type (Descriptor): The descriptor of the message that this method
+      returns.
+    options (descriptor_pb2.MethodOptions or None): Method options message, or
+      None to use default method options.
   """
 
   if _USE_C_DESCRIPTORS:
@@ -844,24 +850,32 @@ class MethodDescriptor(DescriptorBase):
 class FileDescriptor(DescriptorBase):
   """Descriptor for a file. Mimics the descriptor_pb2.FileDescriptorProto.
 
-  Note that enum_types_by_name, extensions_by_name, and dependencies
-  fields are only set by the message_factory module, and not by the
-  generated proto code.
-
-  name: name of file, relative to root of source tree.
-  package: name of the package
-  syntax: string indicating syntax of the file (can be "proto2" or "proto3")
-  serialized_pb: (str) Byte string of serialized
-    descriptor_pb2.FileDescriptorProto.
-  dependencies: List of other FileDescriptors this FileDescriptor depends on.
-  public_dependencies: A list of FileDescriptors, subset of the dependencies
-    above, which were declared as "public".
-  message_types_by_name: Dict of message names and their descriptors.
-  enum_types_by_name: Dict of enum names and their descriptors.
-  extensions_by_name: Dict of extension names and their descriptors.
-  services_by_name: Dict of services names and their descriptors.
-  pool: the DescriptorPool this descriptor belongs to.  When not passed to the
-    constructor, the global default pool is used.
+  Note that :attr:`enum_types_by_name`, :attr:`extensions_by_name`, and
+  :attr:`dependencies` fields are only set by the
+  :py:mod:`google.protobuf.message_factory` module, and not by the generated
+  proto code.
+
+  Attributes:
+    name (str): Name of file, relative to root of source tree.
+    package (str): Name of the package
+    syntax (str): string indicating syntax of the file (can be "proto2" or
+      "proto3")
+    serialized_pb (bytes): Byte string of serialized
+      :class:`descriptor_pb2.FileDescriptorProto`.
+    dependencies (list[FileDescriptor]): List of other :class:`FileDescriptor`
+      objects this :class:`FileDescriptor` depends on.
+    public_dependencies (list[FileDescriptor]): A subset of
+      :attr:`dependencies`, which were declared as "public".
+    message_types_by_name (dict(str, Descriptor)): Mapping from message names
+      to their :class:`Desctiptor`.
+    enum_types_by_name (dict(str, EnumDescriptor)): Mapping from enum names to
+      their :class:`EnumDescriptor`.
+    extensions_by_name (dict(str, FieldDescriptor)): Mapping from extension
+      names declared at file scope to their :class:`FieldDescriptor`.
+    services_by_name (dict(str, ServiceDescriptor)): Mapping from services'
+      names to their :class:`ServiceDescriptor`.
+    pool (DescriptorPool): The pool this descriptor belongs to.  When not
+      passed to the constructor, the global default pool is used.
   """
 
   if _USE_C_DESCRIPTORS:

python/google/protobuf/descriptor_pool.py

@@ -38,7 +38,7 @@ For most applications protocol buffers should be used via modules generated by
 the protocol buffer compiler tool. This should only be used when the type of
 protocol buffers used in an application or library cannot be predetermined.
 
-Below is a straightforward example on how to use this class:
+Below is a straightforward example on how to use this class::
 
   pool = DescriptorPool()
   file_descriptor_protos = [ ... ]
@@ -91,10 +91,10 @@ def _NormalizeFullyQualifiedName(name):
   generated with a leading period. This function removes that prefix.
 
   Args:
-    name: A str, the fully-qualified symbol name.
+    name (str): The fully-qualified symbol name.
 
   Returns:
-    A str, the normalized fully-qualified symbol name.
+    str: The normalized fully-qualified symbol name.
   """
   return name.lstrip('.')
 
@@ -159,8 +159,8 @@ class DescriptorPool(object):
 
     Args:
       desc: Descriptor of a message, enum, service, extension or enum value.
-      desc_name: the full name of desc.
-      file_name: The file name of descriptor.
+      desc_name (str): the full name of desc.
+      file_name (str): The file name of descriptor.
     """
     for register, descriptor_type in [
         (self._descriptors, descriptor.Descriptor),
@@ -196,7 +196,7 @@ class DescriptorPool(object):
     """Adds the FileDescriptorProto and its types to this pool.
 
     Args:
-      file_desc_proto: The FileDescriptorProto to add.
+      file_desc_proto (FileDescriptorProto): The file descriptor to add.
     """
 
     self._internal_db.Add(file_desc_proto)
@@ -205,8 +205,8 @@ class DescriptorPool(object):
     """Adds the FileDescriptorProto and its types to this pool.
 
     Args:
-      serialized_file_desc_proto: A bytes string, serialization of the
-        FileDescriptorProto to add.
+      serialized_file_desc_proto (bytes): A bytes string, serialization of the
+        :class:`FileDescriptorProto` to add.
     """
 
     # pylint: disable=g-import-not-at-top
@@ -392,10 +392,10 @@ class DescriptorPool(object):
     """Gets a FileDescriptor by file name.
 
     Args:
-      file_name: The path to the file to get a descriptor for.
+      file_name (str): The path to the file to get a descriptor for.
 
     Returns:
-      A FileDescriptor for the named file.
+      FileDescriptor: The descriptor for the named file.
 
     Raises:
       KeyError: if the file cannot be found in the pool.
@@ -421,10 +421,11 @@ class DescriptorPool(object):
     """Gets the FileDescriptor for the file containing the specified symbol.
 
     Args:
-      symbol: The name of the symbol to search for.
+      symbol (str): The name of the symbol to search for.
 
     Returns:
-      A FileDescriptor that contains the specified symbol.
+      FileDescriptor: Descriptor for the file that contains the specified
+      symbol.
 
     Raises:
       KeyError: if the file cannot be found in the pool.
@@ -447,10 +448,11 @@ class DescriptorPool(object):
     """Gets the already built FileDescriptor containing the specified symbol.
 
     Args:
-      symbol: The name of the symbol to search for.
+      symbol (str): The name of the symbol to search for.
 
     Returns:
-      A FileDescriptor that contains the specified symbol.
+      FileDescriptor: Descriptor for the file that contains the specified
+      symbol.
 
     Raises:
       KeyError: if the file cannot be found in the pool.
@@ -495,10 +497,10 @@ class DescriptorPool(object):
     """Loads the named descriptor from the pool.
 
     Args:
-      full_name: The full name of the descriptor to load.
+      full_name (str): The full name of the descriptor to load.
 
     Returns:
-      The descriptor for the named type.
+      Descriptor: The descriptor for the named type.
 
     Raises:
       KeyError: if the message cannot be found in the pool.
@@ -513,10 +515,10 @@ class DescriptorPool(object):
     """Loads the named enum descriptor from the pool.
 
     Args:
-      full_name: The full name of the enum descriptor to load.
+      full_name (str): The full name of the enum descriptor to load.
 
     Returns:
-      The enum descriptor for the named type.
+      EnumDescriptor: The enum descriptor for the named type.
 
     Raises:
       KeyError: if the enum cannot be found in the pool.
@@ -531,10 +533,10 @@ class DescriptorPool(object):
     """Loads the named field descriptor from the pool.
 
     Args:
-      full_name: The full name of the field descriptor to load.
+      full_name (str): The full name of the field descriptor to load.
 
     Returns:
-      The field descriptor for the named field.
+      FieldDescriptor: The field descriptor for the named field.
 
     Raises:
       KeyError: if the field cannot be found in the pool.
@@ -548,10 +550,10 @@ class DescriptorPool(object):
     """Loads the named oneof descriptor from the pool.
 
     Args:
-      full_name: The full name of the oneof descriptor to load.
+      full_name (str): The full name of the oneof descriptor to load.
 
     Returns:
-      The oneof descriptor for the named oneof.
+      OneofDescriptor: The oneof descriptor for the named oneof.
 
     Raises:
       KeyError: if the oneof cannot be found in the pool.
@@ -565,10 +567,10 @@ class DescriptorPool(object):
     """Loads the named extension descriptor from the pool.
 
     Args:
-      full_name: The full name of the extension descriptor to load.
+      full_name (str): The full name of the extension descriptor to load.
 
     Returns:
-      A FieldDescriptor, describing the named extension.
+      FieldDescriptor: The field descriptor for the named extension.
 
     Raises:
       KeyError: if the extension cannot be found in the pool.
@@ -594,15 +596,15 @@ class DescriptorPool(object):
   def FindExtensionByNumber(self, message_descriptor, number):
     """Gets the extension of the specified message with the specified number.
 
-    Extensions have to be registered to this pool by calling
-    AddExtensionDescriptor.
+    Extensions have to be registered to this pool by calling :func:`Add` or
+    :func:`AddExtensionDescriptor`.
 
     Args:
-      message_descriptor: descriptor of the extended message.
-      number: integer, number of the extension field.
+      message_descriptor (Descriptor): descriptor of the extended message.
+      number (int): Number of the extension field.
 
     Returns:
-      A FieldDescriptor describing the extension.
+      FieldDescriptor: The descriptor for the extension.
 
     Raises:
       KeyError: when no extension with the given number is known for the
@@ -615,16 +617,16 @@ class DescriptorPool(object):
       return self._extensions_by_number[message_descriptor][number]
 
   def FindAllExtensions(self, message_descriptor):
-    """Gets all the known extension of a given message.
+    """Gets all the known extensions of a given message.
 
     Extensions have to be registered to this pool by calling
-    AddExtensionDescriptor.
+    :func:`Add` or :func:`AddExtensionDescriptor`.
 
     Args:
-      message_descriptor: descriptor of the extended message.
+      message_descriptor (Descriptor): Descriptor of the extended message.
 
     Returns:
-      A list of FieldDescriptor describing the extensions.
+      list[FieldDescriptor]: Field descriptors describing the extensions.
     """
     # Fallback to descriptor db if FindAllExtensionNumbers is provided.
     if self._descriptor_db and hasattr(
@@ -681,10 +683,10 @@ class DescriptorPool(object):
     """Loads the named service descriptor from the pool.
 
     Args:
-      full_name: The full name of the service descriptor to load.
+      full_name (str): The full name of the service descriptor to load.
 
     Returns:
-      The service descriptor for the named service.
+      ServiceDescriptor: The service descriptor for the named service.
 
     Raises:
       KeyError: if the service cannot be found in the pool.
@@ -698,10 +700,10 @@ class DescriptorPool(object):
     """Loads the named service method descriptor from the pool.
 
     Args:
-      full_name: The full name of the method descriptor to load.
+      full_name (str): The full name of the method descriptor to load.
 
     Returns:
-      The method descriptor for the service method.
+      MethodDescriptor: The method descriptor for the service method.
 
     Raises:
       KeyError: if the method cannot be found in the pool.
@@ -715,10 +717,10 @@ class DescriptorPool(object):
     """Finds the file in descriptor DB containing the specified symbol.
 
     Args:
-      symbol: The name of the symbol to search for.
+      symbol (str): The name of the symbol to search for.
 
     Returns:
-      A FileDescriptor that contains the specified symbol.
+      FileDescriptor: The file that contains the specified symbol.
 
     Raises:
       KeyError: if the file cannot be found in the descriptor database.

python/google/protobuf/internal/containers.py

@@ -33,9 +33,10 @@
 This file defines container classes which represent categories of protocol
 buffer field types which need extra maintenance. Currently these categories
 are:
-  - Repeated scalar fields - These are all repeated fields which aren't
+
+-   Repeated scalar fields - These are all repeated fields which aren't
     composite (e.g. they are of simple types like int32, string, etc).
-  - Repeated composite fields - Repeated fields which are composite. This
+-   Repeated composite fields - Repeated fields which are composite. This
     includes groups and nested messages.
 """
 
@@ -285,7 +286,6 @@ class RepeatedScalarFieldContainer(BaseContainer):
 
   def MergeFrom(self, other):
     """Appends the contents of another repeated field of the same type to this
-
     one. We do not check the types of the individual fields.
     """
     self._values.extend(other._values)
@@ -416,7 +416,6 @@ class RepeatedCompositeFieldContainer(BaseContainer):
 
   def MergeFrom(self, other):
     """Appends the contents of another repeated field of the same type to this
-
     one, copying each individual message.
     """
     self.extend(other._values)

python/google/protobuf/message.py

@@ -36,9 +36,19 @@
 
 __author__ = 'robinson@google.com (Will Robinson)'
 
-class Error(Exception): pass
-class DecodeError(Error): pass
-class EncodeError(Error): pass
+class Error(Exception):
+  """Base error type for this module."""
+  pass
+
+
+class DecodeError(Error):
+  """Exception raised when deserializing messages."""
+  pass
+
+
+class EncodeError(Error):
+  """Exception raised when serializing messages."""
+  pass
 
 
 class Message(object):
@@ -48,22 +58,23 @@ class Message(object):
   Protocol message classes are almost always generated by the protocol
   compiler.  These generated types subclass Message and implement the methods
   shown below.
+  """
 
-  TODO(robinson): Link to an HTML document here.
+  # TODO(robinson): Link to an HTML document here.
 
-  TODO(robinson): Document that instances of this class will also
-  have an Extensions attribute with __getitem__ and __setitem__.
-  Again, not sure how to best convey this.
+  # TODO(robinson): Document that instances of this class will also
+  # have an Extensions attribute with __getitem__ and __setitem__.
+  # Again, not sure how to best convey this.
 
-  TODO(robinson): Document that the class must also have a static
-    RegisterExtension(extension_field) method.
-    Not sure how to best express at this point.
-  """
+  # TODO(robinson): Document that the class must also have a static
+  #   RegisterExtension(extension_field) method.
+  #   Not sure how to best express at this point.
 
   # TODO(robinson): Document these fields and methods.
 
   __slots__ = []
 
+  #: The :class:`google.protobuf.descriptor.Descriptor` for this message type.
   DESCRIPTOR = None
 
   def __deepcopy__(self, memo=None):
@@ -99,7 +110,7 @@ class Message(object):
     appended. Singular sub-messages and groups are recursively merged.
 
     Args:
-      other_msg: Message to merge into the current message.
+      other_msg (Message): A message to merge into the current message.
     """
     raise NotImplementedError
 
@@ -110,7 +121,7 @@ class Message(object):
     message using MergeFrom.
 
     Args:
-      other_msg: Message to copy into the current one.
+      other_msg (Message): A message to copy into the current one.
     """
     if self is other_msg:
       return
@@ -127,15 +138,16 @@ class Message(object):
     This normally happens automatically when you assign a field of a
     sub-message, but sometimes you want to make the sub-message
     present while keeping it empty.  If you find yourself using this,
-    you may want to reconsider your design."""
+    you may want to reconsider your design.
+    """
     raise NotImplementedError
 
   def IsInitialized(self):
     """Checks if the message is initialized.
 
     Returns:
-      The method returns True if the message is initialized (i.e. all of its
-      required fields are set).
+      bool: The method returns True if the message is initialized (i.e. all of
+      its required fields are set).
     """
     raise NotImplementedError
 
@@ -148,40 +160,40 @@ class Message(object):
   def MergeFromString(self, serialized):
     """Merges serialized protocol buffer data into this message.
 
-    When we find a field in |serialized| that is already present
+    When we find a field in `serialized` that is already present
     in this message:
-      - If it's a "repeated" field, we append to the end of our list.
-      - Else, if it's a scalar, we overwrite our field.
-      - Else, (it's a nonrepeated composite), we recursively merge
-        into the existing composite.
 
-    TODO(robinson): Document handling of unknown fields.
+    -   If it's a "repeated" field, we append to the end of our list.
+    -   Else, if it's a scalar, we overwrite our field.
+    -   Else, (it's a nonrepeated composite), we recursively merge
+        into the existing composite.
 
     Args:
-      serialized: Any object that allows us to call buffer(serialized)
-        to access a string of bytes using the buffer interface.
-
-    TODO(robinson): When we switch to a helper, this will return None.
+      serialized (bytes): Any object that allows us to call
+        ``memoryview(serialized)`` to access a string of bytes using the
+        buffer interface.
 
     Returns:
-      The number of bytes read from |serialized|.
-      For non-group messages, this will always be len(serialized),
+      int: The number of bytes read from `serialized`.
+      For non-group messages, this will always be `len(serialized)`,
       but for messages which are actually groups, this will
-      generally be less than len(serialized), since we must
-      stop when we reach an END_GROUP tag.  Note that if
-      we *do* stop because of an END_GROUP tag, the number
+      generally be less than `len(serialized)`, since we must
+      stop when we reach an ``END_GROUP`` tag.  Note that if
+      we *do* stop because of an ``END_GROUP`` tag, the number
       of bytes returned does not include the bytes
-      for the END_GROUP tag information.
+      for the ``END_GROUP`` tag information.
 
     Raises:
-      message.DecodeError if the input cannot be parsed.
+      DecodeError: if the input cannot be parsed.
     """
+    # TODO(robinson): Document handling of unknown fields.
+    # TODO(robinson): When we switch to a helper, this will return None.
     raise NotImplementedError
 
   def ParseFromString(self, serialized):
     """Parse serialized protocol buffer data into this message.
 
-    Like MergeFromString(), except we clear the object first.
+    Like :func:`MergeFromString()`, except we clear the object first.
     """
     self.Clear()
     return self.MergeFromString(serialized)
@@ -189,18 +201,16 @@ class Message(object):
   def SerializeToString(self, **kwargs):
     """Serializes the protocol message to a binary string.
 
-    Arguments:
-      **kwargs: Keyword arguments to the serialize method, accepts
-        the following keyword args:
-        deterministic: If true, requests deterministic serialization of the
-          protobuf, with predictable ordering of map keys.
+    Keyword Args:
+      deterministic (bool): If true, requests deterministic serialization
+        of the protobuf, with predictable ordering of map keys.
 
     Returns:
       A binary string representation of the message if all of the required
       fields in the message are set (i.e. the message is initialized).
 
     Raises:
-      message.EncodeError if the message isn't initialized.
+      EncodeError: if the message isn't initialized (see :func:`IsInitialized`).
     """
     raise NotImplementedError
 
@@ -210,14 +220,12 @@ class Message(object):
     This method is similar to SerializeToString but doesn't check if the
     message is initialized.
 
-    Arguments:
-      **kwargs: Keyword arguments to the serialize method, accepts
-        the following keyword args:
-        deterministic: If true, requests deterministic serialization of the
-          protobuf, with predictable ordering of map keys.
+    Keyword Args:
+      deterministic (bool): If true, requests deterministic serialization
+        of the protobuf, with predictable ordering of map keys.
 
     Returns:
-      A string representation of the partial message.
+      bytes: A serialized representation of the partial message.
     """
     raise NotImplementedError
 
@@ -238,48 +246,116 @@ class Message(object):
   # keywords. So they would become lambda_ or yield_.
   # """
   def ListFields(self):
-    """Returns a list of (FieldDescriptor, value) tuples for all
-    fields in the message which are not empty.  A message field is
-    non-empty if HasField() would return true. A singular primitive field
-    is non-empty if HasField() would return true in proto2 or it is non zero
-    in proto3. A repeated field is non-empty if it contains at least one
-    element.  The fields are ordered by field number"""
+    """Returns a list of (FieldDescriptor, value) tuples for present fields.
+
+    A message field is non-empty if HasField() would return true. A singular
+    primitive field is non-empty if HasField() would return true in proto2 or it
+    is non zero in proto3. A repeated field is non-empty if it contains at least
+    one element. The fields are ordered by field number.
+
+    Returns:
+      list[tuple(FieldDescriptor, value)]: field descriptors and values
+      for all fields in the message which are not empty. The values vary by
+      field type.
+    """
     raise NotImplementedError
 
   def HasField(self, field_name):
-    """Checks if a certain field is set for the message, or if any field inside
-    a oneof group is set.  Note that if the field_name is not defined in the
-    message descriptor, ValueError will be raised."""
+    """Checks if a certain field is set for the message.
+
+    For a oneof group, checks if any field inside is set. Note that if the
+    field_name is not defined in the message descriptor, :exc:`ValueError` will
+    be raised.
+
+    Args:
+      field_name (str): The name of the field to check for presence.
+
+    Returns:
+      bool: Whether a value has been set for the named field.
+
+    Raises:
+      ValueError: if the `field_name` is not a member of this message.
+    """
     raise NotImplementedError
 
   def ClearField(self, field_name):
-    """Clears the contents of a given field, or the field set inside a oneof
-    group.  If the name neither refers to a defined field or oneof group,
-    ValueError is raised."""
+    """Clears the contents of a given field.
+
+    Inside a oneof group, clears the field set. If the name neither refers to a
+    defined field or oneof group, :exc:`ValueError` is raised.
+
+    Args:
+      field_name (str): The name of the field to check for presence.
+
+    Raises:
+      ValueError: if the `field_name` is not a member of this message.
+    """
     raise NotImplementedError
 
   def WhichOneof(self, oneof_group):
-    """Returns the name of the field that is set inside a oneof group, or
-    None if no field is set.  If no group with the given name exists, ValueError
-    will be raised."""
+    """Returns the name of the field that is set inside a oneof group.
+
+    If no field is set, returns None.
+
+    Args:
+      oneof_group (str): the name of the oneof group to check.
+
+    Returns:
+      str or None: The name of the group that is set, or None.
+
+    Raises:
+      ValueError: no group with the given name exists
+    """
     raise NotImplementedError
 
   def HasExtension(self, extension_handle):
+    """Checks if a certain extension is present for this message.
+
+    Extensions are retrieved using the :attr:`Extensions` mapping (if present).
+
+    Args:
+      extension_handle: The handle for the extension to check.
+
+    Returns:
+      bool: Whether the extension is present for this message.
+
+    Raises:
+      KeyError: if the extension is repeated. Similar to repeated fields,
+        there is no separate notion of presence: a "not present" repeated
+        extension is an empty list.
+    """
     raise NotImplementedError
 
   def ClearExtension(self, extension_handle):
+    """Clears the contents of a given extension.
+
+    Args:
+      extension_handle: The handle for the extension to clear.
+    """
     raise NotImplementedError
 
   def UnknownFields(self):
-    """Returns the UnknownFieldSet."""
+    """Returns the UnknownFieldSet.
+
+    Returns:
+      UnknownFieldSet: The unknown fields stored in this message.
+    """
     raise NotImplementedError
 
   def DiscardUnknownFields(self):
+    """Clears all fields in the :class:`UnknownFieldSet`.
+
+    This operation is recursive for nested message.
+    """
     raise NotImplementedError
 
   def ByteSize(self):
     """Returns the serialized size of this message.
+
     Recursively calls ByteSize() on all contained messages.
+
+    Returns:
+      int: The number of bytes required to serialize this message.
     """
     raise NotImplementedError
 

python/google/protobuf/service.py

@@ -73,6 +73,7 @@ class Service(object):
     In the blocking case, RpcException will be raised on error.
 
     Preconditions:
+
     * method_descriptor.service == GetDescriptor
     * request is of the exact same classes as returned by
       GetRequestClass(method).
@@ -82,6 +83,7 @@ class Service(object):
       RpcChannel which the stub is using.
 
     Postconditions:
+
     * "done" will be called when the method is complete.  This may be
       before CallMethod() returns or it may be at some point in the future.
     * If the RPC failed, the response value passed to "done" will be None.

python/google/protobuf/service_reflection.py

@@ -55,14 +55,14 @@ class GeneratedServiceType(type):
 
   The protocol compiler currently uses this metaclass to create protocol service
   classes at runtime. Clients can also manually create their own classes at
-  runtime, as in this example:
-
-  mydescriptor = ServiceDescriptor(.....)
-  class MyProtoService(service.Service):
-    __metaclass__ = GeneratedServiceType
-    DESCRIPTOR = mydescriptor
-  myservice_instance = MyProtoService()
-  ...
+  runtime, as in this example::
+
+    mydescriptor = ServiceDescriptor(.....)
+    class MyProtoService(service.Service):
+      __metaclass__ = GeneratedServiceType
+      DESCRIPTOR = mydescriptor
+    myservice_instance = MyProtoService()
+    # ...
   """
 
   _DESCRIPTOR_KEY = 'DESCRIPTOR'

python/google/protobuf/symbol_database.py

@@ -34,7 +34,7 @@ SymbolDatabase is the MessageFactory for messages generated at compile time,
 and makes it easy to create new instances of a registered type, given only the
 type's protocol buffer symbol name.
 
-Example usage:
+Example usage::
 
   db = symbol_database.SymbolDatabase()
 
@@ -72,7 +72,8 @@ class SymbolDatabase(message_factory.MessageFactory):
     Calls to GetSymbol() and GetMessages() will return messages registered here.
 
     Args:
-      message: a message.Message, to be registered.
+      message: A :class:`google.protobuf.message.Message` subclass (or
+        instance); its descriptor will be registered.
 
     Returns:
       The provided message.
@@ -87,7 +88,7 @@ class SymbolDatabase(message_factory.MessageFactory):
     """Registers the given message descriptor in the local database.
 
     Args:
-      message_descriptor: a descriptor.MessageDescriptor.
+      message_descriptor (Descriptor): the message descriptor to add.
     """
     if api_implementation.Type() == 'python':
       # pylint: disable=protected-access
@@ -97,10 +98,10 @@ class SymbolDatabase(message_factory.MessageFactory):
     """Registers the given enum descriptor in the local database.
 
     Args:
-      enum_descriptor: a descriptor.EnumDescriptor.
+      enum_descriptor (EnumDescriptor): The enum descriptor to register.
 
     Returns:
-      The provided descriptor.
+      EnumDescriptor: The provided descriptor.
     """
     if api_implementation.Type() == 'python':
       # pylint: disable=protected-access
@@ -111,10 +112,8 @@ class SymbolDatabase(message_factory.MessageFactory):
     """Registers the given service descriptor in the local database.
 
     Args:
-      service_descriptor: a descriptor.ServiceDescriptor.
-
-    Returns:
-      The provided descriptor.
+      service_descriptor (ServiceDescriptor): the service descriptor to
+        register.
     """
     if api_implementation.Type() == 'python':
       # pylint: disable=protected-access
@@ -124,10 +123,7 @@ class SymbolDatabase(message_factory.MessageFactory):
     """Registers the given file descriptor in the local database.
 
     Args:
-      file_descriptor: a descriptor.FileDescriptor.
-
-    Returns:
-      The provided descriptor.
+      file_descriptor (FileDescriptor): The file descriptor to register.
     """
     if api_implementation.Type() == 'python':
       # pylint: disable=protected-access
@@ -140,7 +136,7 @@ class SymbolDatabase(message_factory.MessageFactory):
     may be extended in future to support other symbol types.
 
     Args:
-      symbol: A str, a protocol buffer symbol.
+      symbol (str): a protocol buffer symbol.
 
     Returns:
       A Python class corresponding to the symbol.
@@ -161,7 +157,7 @@ class SymbolDatabase(message_factory.MessageFactory):
     messages, but does not register any message extensions.
 
     Args:
-      files: The file names to extract messages from.
+      files (list[str]): The file names to extract messages from.
 
     Returns:
       A dictionary mapping proto names to the message classes.

python/google/protobuf/text_format.py

@@ -30,7 +30,7 @@
 
 """Contains routines for printing protocol messages in text format.
 
-Simple usage example:
+Simple usage example::
 
   # Create a proto object and serialize it to a text proto string.
   message = my_proto_pb2.MyMessage(foo='bar')
@@ -155,23 +155,24 @@ def MessageToString(message,
       will be printed at the end of the message and their relative order is
       determined by the extension number. By default, use the field number
       order.
-    float_format: If set, use this to specify float field formatting
-      (per the "Format Specification Mini-Language"); otherwise, 8 valid digits
-      is used (default '.8g'). Also affect double field if double_format is
-      not set but float_format is set.
-    double_format: If set, use this to specify double field formatting
+    float_format (str): If set, use this to specify float field formatting
+      (per the "Format Specification Mini-Language"); otherwise, 8 valid
+      digits is used (default '.8g'). Also affect double field if
+      double_format is not set but float_format is set.
+    double_format (str): If set, use this to specify double field formatting
       (per the "Format Specification Mini-Language"); if it is not set but
-      float_format is set, use float_format. Otherwise, use str()
+      float_format is set, use float_format. Otherwise, use ``str()``
     use_field_number: If True, print field numbers instead of names.
-    descriptor_pool: A DescriptorPool used to resolve Any types.
-    indent: The initial indent level, in terms of spaces, for pretty print.
-    message_formatter: A function(message, indent, as_one_line): unicode|None
-      to custom format selected sub-messages (usually based on message type).
-      Use to pretty print parts of the protobuf for easier diffing.
+    descriptor_pool (DescriptorPool): Descriptor pool used to resolve Any types.
+    indent (int): The initial indent level, in terms of spaces, for pretty
+      print.
+    message_formatter (function(message, indent, as_one_line) -> unicode|None):
+      Custom formatter for selected sub-messages (usually based on message
+      type). Use to pretty print parts of the protobuf for easier diffing.
     print_unknown_fields: If True, unknown fields will be printed.
 
   Returns:
-    A string of the text formatted protocol buffer message.
+    str: A string of the text formatted protocol buffer message.
   """
   out = TextWriter(as_utf8)
   printer = _Printer(out, indent, as_utf8, as_one_line,
@@ -620,7 +621,8 @@ def Parse(text,
   If text contains a field already set in message, the value is appended if the
   field is repeated. Otherwise, an error is raised.
 
-  Example
+  Example::
+
     a = MyProto()
     a.repeated_field.append('test')
     b = MyProto()
@@ -640,18 +642,18 @@ def Parse(text,
   Caller is responsible for clearing the message as needed.
 
   Args:
-    text: Message text representation.
-    message: A protocol buffer message to merge into.
+    text (str): Message text representation.
+    message (Message): A protocol buffer message to merge into.
     allow_unknown_extension: if True, skip over missing extensions and keep
       parsing
     allow_field_number: if True, both field number and field name are allowed.
-    descriptor_pool: A DescriptorPool used to resolve Any types.
+    descriptor_pool (DescriptorPool): Descriptor pool used to resolve Any types.
     allow_unknown_field: if True, skip over unknown field and keep
       parsing. Avoid to use this option if possible. It may hide some
       errors (e.g. spelling error on field name)
 
   Returns:
-    The same message passed as argument.
+    Message: The same message passed as argument.
 
   Raises:
     ParseError: On text parsing problems.
@@ -677,18 +679,18 @@ def Merge(text,
   replace those in the message.
 
   Args:
-    text: Message text representation.
-    message: A protocol buffer message to merge into.
+    text (str): Message text representation.
+    message (Message): A protocol buffer message to merge into.
     allow_unknown_extension: if True, skip over missing extensions and keep
       parsing
     allow_field_number: if True, both field number and field name are allowed.
-    descriptor_pool: A DescriptorPool used to resolve Any types.
+    descriptor_pool (DescriptorPool): Descriptor pool used to resolve Any types.
     allow_unknown_field: if True, skip over unknown field and keep
       parsing. Avoid to use this option if possible. It may hide some
       errors (e.g. spelling error on field name)
 
   Returns:
-    The same message passed as argument.
+    Message: The same message passed as argument.
 
   Raises:
     ParseError: On text parsing problems.