Merge branch 'master' of https://github.com/grpc/grpc into channelz-get-top-channels

doc/keepalive.md

@@ -0,0 +1,50 @@
+# Keepalive User Guide for gRPC Core (and dependants)
+
+The keepalive ping is a way to check if a channel is currently working by sending HTTP2 pings over the transport. It is sent periodically, and if the ping is not acknowledged by the peer within a certain timeout period, the transport is disconnected.
+
+This guide documents the knobs within gRPC core to control the current behavior of the keepalive ping.
+
+The keepalive ping is controlled by two important channel arguments -
+* **GRPC_ARG_KEEPALIVE_TIME_MS**
+  * This channel argument controls the period (in milliseconds) after which a keepalive ping is sent on the transport.
+* **GRPC_ARG_KEEPALIVE_TIMEOUT_MS**
+  * This channel argument controls the amount of time (in milliseconds), the sender of the keepalive ping waits for an acknowledgement. If it does not receive an acknowledgement within this time, it will close the connection.
+
+The above two channel arguments should be sufficient for most users, but the following arguments can also be useful in certain use cases.
+* **GRPC_ARG_KEEPALIVE_PERMIT_WITHOUT_CALLS**
+  * This channel argument if set to 1 (0 : false; 1 : true), allows keepalive pings to be sent even if there are no calls in flight. 
+* **GRPC_ARG_HTTP2_MAX_PINGS_WITHOUT_DATA**
+  * This channel argument controls the maximum number of pings that can be sent when there is no other data (data frame or header frame) to be sent. GRPC Core will not continue sending pings if we run over the limit. Setting it to 0 allows sending pings without sending data.
+* **GRPC_ARG_HTTP2_MIN_SENT_PING_INTERVAL_WITHOUT_DATA_MS**
+  * If there is no data being sent on the transport, this channel argument controls the minimum time (in milliseconds) gRPC Core will wait between successive pings.
+* **GRPC_ARG_HTTP2_MIN_RECV_PING_INTERVAL_WITHOUT_DATA_MS**
+  * If there is no data being sent on the transport, this channel argument on the server side controls the minimum time (in milliseconds) that gRPC Core would expect between receiving successive pings. If the time between successive pings is less that than this time, then the ping will be considered a bad ping from the peer. Such a ping counts as a ‘ping strike’.
+On the client side, this does not have any effect.
+* **GRPC_ARG_HTTP2_MAX_PING_STRIKES**
+  * This arg controls the maximum number of bad pings that the server will tolerate before sending an HTTP2 GOAWAY frame and closing the transport. Setting it to 0 allows the server to accept any number of bad pings.
+
+### Defaults Values
+
+Channel Argument| Client|Server
+----------------|-------|------
+GRPC_ARG_KEEPALIVE_TIME_MS|INT_MAX (disabled)|7200000 (2 hours)
+GRPC_ARG_KEEPALIVE_TIMEOUT_MS|20000 (20 seconds)|20000 (20 seconds)
+GRPC_ARG_KEEPALIVE_PERMIT_WITHOUT_CALLS|0 (false)|0 (false)
+GRPC_ARG_HTTP2_MAX_PINGS_WITHOUT_DATA|2|2
+GRPC_ARG_HTTP2_MIN_SENT_PING_INTERVAL_WITHOUT_DATA_MS|300000 (5 minutes)|300000 (5 minutes)
+GRPC_ARG_HTTP2_MIN_RECV_PING_INTERVAL_WITHOUT_DATA_MS|N/A|300000 (5 minutes)
+GRPC_ARG_HTTP2_MAX_PING_STRIKES|N/A|2
+
+### FAQ
+* When is the keepalive timer started?
+  * The keepalive timer is started when a transport is done connecting (after handshake).
+* What happens when the keepalive timer fires?
+  * When the keepalive timer fires, gRPC Core would try to send a keepalive ping on the transport. This ping can be blocked if -
+    * there is no active call on that transport and GRPC_ARG_KEEPALIVE_PERMIT_WITHOUT_CALLS is false.
+    * the number of pings already sent on the transport without any data has already exceeded GRPC_ARG_HTTP2_MAX_PINGS_WITHOUT_DATA.
+    * the time expired since the previous ping is less than GRPC_ARG_HTTP2_MIN_SENT_PING_INTERVAL_WITHOUT_DATA_MS.
+  * If a keepalive ping is not blocked and is sent on the transport, then the keepalive watchdog timer is started which would close the transport if the ping is not acknowledged before it fires.
+* Why am I receiving a GOAWAY with error code ENHANCE_YOUR_CALM?
+  * A server sends a GOAWAY with ENHANCE_YOUR_CALM if the client sends too many misbehaving pings. For example -
+    * if a server has GRPC_ARG_KEEPALIVE_PERMIT_WITHOUT_CALLS set to false, and the client sends pings without there being any call in flight.
+    * if the client's GRPC_ARG_HTTP2_MIN_SENT_PING_INTERVAL_WITHOUT_DATA_MS setting is lower than the server's GRPC_ARG_HTTP2_MIN_RECV_PING_INTERVAL_WITHOUT_DATA_MS.

include/grpc/support/string_util.h

@@ -21,6 +21,8 @@
 
 #include <grpc/support/port_platform.h>
 
+#include <grpc/impl/codegen/gpr_types.h>
+
 #ifdef __cplusplus
 extern "C" {
 #endif

src/core/ext/filters/client_channel/client_channel.cc

@@ -3174,6 +3174,16 @@ static void try_to_connect_locked(void* arg, grpc_error* error_ignored) {
   GRPC_CHANNEL_STACK_UNREF(chand->owning_stack, "try_to_connect");
 }
 
+void grpc_client_channel_populate_child_refs(
+    grpc_channel_element* elem, grpc_core::ChildRefsList* child_subchannels,
+    grpc_core::ChildRefsList* child_channels) {
+  channel_data* chand = static_cast<channel_data*>(elem->channel_data);
+  if (chand->lb_policy != nullptr) {
+    chand->lb_policy->FillChildRefsForChannelz(child_subchannels,
+                                               child_channels);
+  }
+}
+
 grpc_connectivity_state grpc_client_channel_check_connectivity_state(
     grpc_channel_element* elem, int try_to_connect) {
   channel_data* chand = static_cast<channel_data*>(elem->channel_data);

src/core/ext/filters/client_channel/client_channel.h

@@ -21,6 +21,7 @@
 
 #include <grpc/support/port_platform.h>
 
+#include "src/core/ext/filters/client_channel/client_channel_channelz.h"
 #include "src/core/ext/filters/client_channel/client_channel_factory.h"
 #include "src/core/ext/filters/client_channel/resolver.h"
 #include "src/core/lib/channel/channel_stack.h"
@@ -39,6 +40,10 @@ extern grpc_core::TraceFlag grpc_client_channel_trace;
 
 extern const grpc_channel_filter grpc_client_channel_filter;
 
+void grpc_client_channel_populate_child_refs(
+    grpc_channel_element* elem, grpc_core::ChildRefsList* child_subchannels,
+    grpc_core::ChildRefsList* child_channels);
+
 grpc_connectivity_state grpc_client_channel_check_connectivity_state(
     grpc_channel_element* elem, int try_to_connect);
 

src/core/ext/filters/client_channel/client_channel_channelz.cc

@@ -64,6 +64,37 @@ void ClientChannelNode::PopulateConnectivityState(grpc_json* json) {
                          false);
 }
 
+void ClientChannelNode::PopulateChildRefs(grpc_json* json) {
+  ChildRefsList child_subchannels;
+  ChildRefsList child_channels;
+  grpc_json* json_iterator = nullptr;
+  grpc_client_channel_populate_child_refs(client_channel_, &child_subchannels,
+                                          &child_channels);
+  if (child_subchannels.size() > 0) {
+    grpc_json* array_parent = grpc_json_create_child(
+        nullptr, json, "subchannelRef", nullptr, GRPC_JSON_ARRAY, false);
+    for (size_t i = 0; i < child_subchannels.size(); ++i) {
+      json_iterator =
+          grpc_json_create_child(json_iterator, array_parent, nullptr, nullptr,
+                                 GRPC_JSON_OBJECT, false);
+      grpc_json_add_number_string_child(json_iterator, nullptr, "subchannelId",
+                                        child_subchannels[i]);
+    }
+  }
+  if (child_channels.size() > 0) {
+    grpc_json* array_parent = grpc_json_create_child(
+        nullptr, json, "channelRef", nullptr, GRPC_JSON_ARRAY, false);
+    json_iterator = nullptr;
+    for (size_t i = 0; i < child_subchannels.size(); ++i) {
+      json_iterator =
+          grpc_json_create_child(json_iterator, array_parent, nullptr, nullptr,
+                                 GRPC_JSON_OBJECT, false);
+      grpc_json_add_number_string_child(json_iterator, nullptr, "channelId",
+                                        child_subchannels[i]);
+    }
+  }
+}
+
 grpc_arg ClientChannelNode::CreateChannelArg() {
   return grpc_channel_arg_pointer_create(
       const_cast<char*>(GRPC_ARG_CHANNELZ_CHANNEL_NODE_CREATION_FUNC),

src/core/ext/filters/client_channel/client_channel_channelz.h

@@ -22,9 +22,17 @@
 #include <grpc/support/port_platform.h>
 
 #include "src/core/lib/channel/channel_args.h"
+#include "src/core/lib/channel/channel_stack.h"
 #include "src/core/lib/channel/channelz.h"
+#include "src/core/lib/gprpp/inlined_vector.h"
 
 namespace grpc_core {
+
+// TODO(ncteisen), this only contains the uuids of the children for now,
+// since that is all that is strictly needed. In a future enhancement we will
+// add human readable names as in the channelz.proto
+typedef InlinedVector<intptr_t, 10> ChildRefsList;
+
 namespace channelz {
 
 // Subtype of ChannelNode that overrides and provides client_channel specific
@@ -39,6 +47,9 @@ class ClientChannelNode : public ChannelNode {
   // channel connectivity.
   void PopulateConnectivityState(grpc_json* json) override;
 
+  // Override this functionality since client_channels have subchannels
+  void PopulateChildRefs(grpc_json* json) override;
+
   // Helper to create a channel arg to ensure this type of ChannelNode is
   // created.
   static grpc_arg CreateChannelArg();

src/core/ext/filters/client_channel/lb_policy.h

@@ -21,6 +21,7 @@
 
 #include <grpc/support/port_platform.h>
 
+#include "src/core/ext/filters/client_channel/client_channel_channelz.h"
 #include "src/core/ext/filters/client_channel/client_channel_factory.h"
 #include "src/core/ext/filters/client_channel/subchannel.h"
 #include "src/core/lib/gprpp/abstract.h"
@@ -143,6 +144,14 @@ class LoadBalancingPolicy
   /// consider whether this method is still needed.
   virtual void ExitIdleLocked() GRPC_ABSTRACT;
 
+  /// populates child_subchannels and child_channels with the uuids of this
+  /// LB policy's referenced children. This is not invoked from the
+  /// client_channel's combiner. The implementation is responsible for
+  /// providing its own synchronization.
+  virtual void FillChildRefsForChannelz(ChildRefsList* child_subchannels,
+                                        ChildRefsList* child_channels)
+      GRPC_ABSTRACT;
+
   void Orphan() override {
     // Invoke ShutdownAndUnrefLocked() inside of the combiner.
     GRPC_CLOSURE_SCHED(
@@ -196,6 +205,12 @@ class LoadBalancingPolicy
   grpc_pollset_set* interested_parties_;
   /// Callback to force a re-resolution.
   grpc_closure* request_reresolution_;
+
+  // Dummy classes needed for alignment issues.
+  // See https://github.com/grpc/grpc/issues/16032 for context.
+  // TODO(ncteisen): remove this as soon as the issue is resolved.
+  ChildRefsList dummy_list_foo;
+  ChildRefsList dummy_list_bar;
 };
 
 }  // namespace grpc_core

src/core/ext/filters/client_channel/lb_policy/grpclb/grpclb.cc

@@ -135,6 +135,9 @@ class GrpcLb : public LoadBalancingPolicy {
   void HandOffPendingPicksLocked(LoadBalancingPolicy* new_policy) override;
   void PingOneLocked(grpc_closure* on_initiate, grpc_closure* on_ack) override;
   void ExitIdleLocked() override;
+  // TODO(ncteisen): implement this in a follow up PR
+  void FillChildRefsForChannelz(ChildRefsList* child_subchannels,
+                                ChildRefsList* child_channels) override {}
 
  private:
   /// Linked list of pending pick requests. It stores all information needed to

src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc

@@ -58,6 +58,8 @@ class PickFirst : public LoadBalancingPolicy {
   void HandOffPendingPicksLocked(LoadBalancingPolicy* new_policy) override;
   void PingOneLocked(grpc_closure* on_initiate, grpc_closure* on_ack) override;
   void ExitIdleLocked() override;
+  void FillChildRefsForChannelz(ChildRefsList* child_subchannels,
+                                ChildRefsList* child_channels) override;
 
  private:
   ~PickFirst();
@@ -103,10 +105,23 @@ class PickFirst : public LoadBalancingPolicy {
     }
   };
 
+  // Helper class to ensure that any function that modifies the child refs
+  // data structures will update the channelz snapshot data structures before
+  // returning.
+  class AutoChildRefsUpdater {
+   public:
+    explicit AutoChildRefsUpdater(PickFirst* pf) : pf_(pf) {}
+    ~AutoChildRefsUpdater() { pf_->UpdateChildRefsLocked(); }
+
+   private:
+    PickFirst* pf_;
+  };
+
   void ShutdownLocked() override;
 
   void StartPickingLocked();
   void DestroyUnselectedSubchannelsLocked();
+  void UpdateChildRefsLocked();
 
   // All our subchannels.
   OrphanablePtr<PickFirstSubchannelList> subchannel_list_;
@@ -122,10 +137,17 @@ class PickFirst : public LoadBalancingPolicy {
   PickState* pending_picks_ = nullptr;
   // Our connectivity state tracker.
   grpc_connectivity_state_tracker state_tracker_;
+
+  /// Lock and data used to capture snapshots of this channels child
+  /// channels and subchannels. This data is consumed by channelz.
+  gpr_mu child_refs_mu_;
+  ChildRefsList child_subchannels_;
+  ChildRefsList child_channels_;
 };
 
 PickFirst::PickFirst(const Args& args) : LoadBalancingPolicy(args) {
   GPR_ASSERT(args.client_channel_factory != nullptr);
+  gpr_mu_init(&child_refs_mu_);
   grpc_connectivity_state_init(&state_tracker_, GRPC_CHANNEL_IDLE,
                                "pick_first");
   if (grpc_lb_pick_first_trace.enabled()) {
@@ -139,6 +161,7 @@ PickFirst::~PickFirst() {
   if (grpc_lb_pick_first_trace.enabled()) {
     gpr_log(GPR_INFO, "Destroying Pick First %p", this);
   }
+  gpr_mu_destroy(&child_refs_mu_);
   GPR_ASSERT(subchannel_list_ == nullptr);
   GPR_ASSERT(latest_pending_subchannel_list_ == nullptr);
   GPR_ASSERT(pending_picks_ == nullptr);
@@ -158,6 +181,7 @@ void PickFirst::HandOffPendingPicksLocked(LoadBalancingPolicy* new_policy) {
 }
 
 void PickFirst::ShutdownLocked() {
+  AutoChildRefsUpdater(this);
   grpc_error* error = GRPC_ERROR_CREATE_FROM_STATIC_STRING("Channel shutdown");
   if (grpc_lb_pick_first_trace.enabled()) {
     gpr_log(GPR_INFO, "Pick First %p Shutting down", this);
@@ -280,7 +304,61 @@ void PickFirst::PingOneLocked(grpc_closure* on_initiate, grpc_closure* on_ack) {
   }
 }
 
+void PickFirst::FillChildRefsForChannelz(
+    ChildRefsList* child_subchannels_to_fill, ChildRefsList* ignored) {
+  mu_guard guard(&child_refs_mu_);
+  for (size_t i = 0; i < child_subchannels_.size(); ++i) {
+    // TODO(ncteisen): implement a de dup loop that is not O(n^2). Might
+    // have to implement lightweight set. For now, we don't care about
+    // performance when channelz requests are made.
+    bool found = false;
+    for (size_t j = 0; j < child_subchannels_to_fill->size(); ++j) {
+      if ((*child_subchannels_to_fill)[j] == child_subchannels_[i]) {
+        found = true;
+        break;
+      }
+    }
+    if (!found) {
+      child_subchannels_to_fill->push_back(child_subchannels_[i]);
+    }
+  }
+}
+
+void PickFirst::UpdateChildRefsLocked() {
+  ChildRefsList cs;
+  if (subchannel_list_ != nullptr) {
+    for (size_t i = 0; i < subchannel_list_->num_subchannels(); ++i) {
+      if (subchannel_list_->subchannel(i)->subchannel() != nullptr) {
+        grpc_core::channelz::SubchannelNode* subchannel_node =
+            grpc_subchannel_get_channelz_node(
+                subchannel_list_->subchannel(i)->subchannel());
+        if (subchannel_node != nullptr) {
+          cs.push_back(subchannel_node->subchannel_uuid());
+        }
+      }
+    }
+  }
+  if (latest_pending_subchannel_list_ != nullptr) {
+    for (size_t i = 0; i < latest_pending_subchannel_list_->num_subchannels();
+         ++i) {
+      if (latest_pending_subchannel_list_->subchannel(i)->subchannel() !=
+          nullptr) {
+        grpc_core::channelz::SubchannelNode* subchannel_node =
+            grpc_subchannel_get_channelz_node(
+                latest_pending_subchannel_list_->subchannel(i)->subchannel());
+        if (subchannel_node != nullptr) {
+          cs.push_back(subchannel_node->subchannel_uuid());
+        }
+      }
+    }
+  }
+  // atomically update the data that channelz will actually be looking at.
+  mu_guard guard(&child_refs_mu_);
+  child_subchannels_ = std::move(cs);
+}
+
 void PickFirst::UpdateLocked(const grpc_channel_args& args) {
+  AutoChildRefsUpdater guard(this);
   const grpc_arg* arg = grpc_channel_args_find(&args, GRPC_ARG_LB_ADDRESSES);
   if (arg == nullptr || arg->type != GRPC_ARG_POINTER) {
     if (subchannel_list_ == nullptr) {
@@ -388,6 +466,7 @@ void PickFirst::UpdateLocked(const grpc_channel_args& args) {
 void PickFirst::PickFirstSubchannelData::ProcessConnectivityChangeLocked(
     grpc_connectivity_state connectivity_state, grpc_error* error) {
   PickFirst* p = static_cast<PickFirst*>(subchannel_list()->policy());
+  AutoChildRefsUpdater guard(p);
   // The notification must be for a subchannel in either the current or
   // latest pending subchannel lists.
   GPR_ASSERT(subchannel_list() == p->subchannel_list_.get() ||

src/core/ext/filters/client_channel/lb_policy/round_robin/round_robin.cc

@@ -69,6 +69,9 @@ class RoundRobin : public LoadBalancingPolicy {
   void HandOffPendingPicksLocked(LoadBalancingPolicy* new_policy) override;
   void PingOneLocked(grpc_closure* on_initiate, grpc_closure* on_ack) override;
   void ExitIdleLocked() override;
+  // TODO(ncteisen): implement this in a follow up PR
+  void FillChildRefsForChannelz(ChildRefsList* child_subchannels,
+                                ChildRefsList* child_channels) override {}
 
  private:
   ~RoundRobin();

src/core/ext/filters/client_channel/subchannel.cc

@@ -134,6 +134,9 @@ struct grpc_subchannel {
   bool backoff_begun;
   /** our alarm */
   grpc_timer alarm;
+
+  grpc_core::RefCountedPtr<grpc_core::channelz::SubchannelNode>
+      channelz_subchannel;
 };
 
 struct grpc_subchannel_call {
@@ -178,6 +181,7 @@ static void connection_destroy(void* arg, grpc_error* error) {
 
 static void subchannel_destroy(void* arg, grpc_error* error) {
   grpc_subchannel* c = static_cast<grpc_subchannel*>(arg);
+  c->channelz_subchannel.reset();
   gpr_free((void*)c->filters);
   grpc_channel_args_destroy(c->args);
   grpc_connectivity_state_destroy(&c->state_tracker);
@@ -374,9 +378,22 @@ grpc_subchannel* grpc_subchannel_create(grpc_connector* connector,
   c->backoff.Init(backoff_options);
   gpr_mu_init(&c->mu);
 
+  const grpc_arg* arg =
+      grpc_channel_args_find(c->args, GRPC_ARG_ENABLE_CHANNELZ);
+  bool channelz_enabled = grpc_channel_arg_get_bool(arg, false);
+  if (channelz_enabled) {
+    c->channelz_subchannel =
+        grpc_core::MakeRefCounted<grpc_core::channelz::SubchannelNode>();
+  }
+
   return grpc_subchannel_index_register(key, c);
 }
 
+grpc_core::channelz::SubchannelNode* grpc_subchannel_get_channelz_node(
+    grpc_subchannel* s) {
+  return s->channelz_subchannel.get();
+}
+
 static void continue_connect_locked(grpc_subchannel* c) {
   grpc_connect_in_args args;
   args.interested_parties = c->pollset_set;

src/core/ext/filters/client_channel/subchannel.h

@@ -21,6 +21,7 @@
 
 #include <grpc/support/port_platform.h>
 
+#include "src/core/ext/filters/client_channel/client_channel_channelz.h"
 #include "src/core/ext/filters/client_channel/connector.h"
 #include "src/core/lib/channel/channel_stack.h"
 #include "src/core/lib/gpr/arena.h"
@@ -115,6 +116,9 @@ grpc_subchannel_call* grpc_subchannel_call_ref(
 void grpc_subchannel_call_unref(
     grpc_subchannel_call* call GRPC_SUBCHANNEL_REF_EXTRA_ARGS);
 
+grpc_core::channelz::SubchannelNode* grpc_subchannel_get_channelz_node(
+    grpc_subchannel* subchannel);
+
 /** Returns a pointer to the parent data associated with \a subchannel_call.
     The data will be of the size specified in \a parent_data_size
     field of the args passed to \a grpc_connected_subchannel_create_call(). */

src/core/ext/transport/chttp2/transport/hpack_parser.cc

@@ -1622,7 +1622,7 @@ grpc_error* grpc_chttp2_header_parser_parse(void* hpack_parser,
                                             grpc_chttp2_transport* t,
                                             grpc_chttp2_stream* s,
                                             grpc_slice slice, int is_last) {
-  GPR_TIMER_SCOPE("grpc_chttp2_hpack_parser_parse", 0);
+  GPR_TIMER_SCOPE("grpc_chttp2_header_parser_parse", 0);
   grpc_chttp2_hpack_parser* parser =
       static_cast<grpc_chttp2_hpack_parser*>(hpack_parser);
   if (s != nullptr) {

src/core/lib/channel/channel_trace.cc

@@ -131,38 +131,6 @@ void ChannelTrace::AddTraceEventReferencingSubchannel(
 
 namespace {
 
-// returns an allocated string that represents tm according to RFC-3339, and,
-// more specifically, follows:
-// https://developers.google.com/protocol-buffers/docs/proto3#json
-//
-// "Uses RFC 3339, where generated output will always be Z-normalized and uses
-// 0, 3, 6 or 9 fractional digits."
-char* fmt_time(gpr_timespec tm) {
-  char time_buffer[35];
-  char ns_buffer[11];  // '.' + 9 digits of precision
-  struct tm* tm_info = localtime((const time_t*)&tm.tv_sec);
-  strftime(time_buffer, sizeof(time_buffer), "%Y-%m-%dT%H:%M:%S", tm_info);
-  snprintf(ns_buffer, 11, ".%09d", tm.tv_nsec);
-  // This loop trims off trailing zeros by inserting a null character that the
-  // right point. We iterate in chunks of three because we want 0, 3, 6, or 9
-  // fractional digits.
-  for (int i = 7; i >= 1; i -= 3) {
-    if (ns_buffer[i] == '0' && ns_buffer[i + 1] == '0' &&
-        ns_buffer[i + 2] == '0') {
-      ns_buffer[i] = '\0';
-      // Edge case in which all fractional digits were 0.
-      if (i == 1) {
-        ns_buffer[0] = '\0';
-      }
-    } else {
-      break;
-    }
-  }
-  char* full_time_str;
-  gpr_asprintf(&full_time_str, "%s%sZ", time_buffer, ns_buffer);
-  return full_time_str;
-}
-
 const char* severity_string(ChannelTrace::Severity severity) {
   switch (severity) {
     case ChannelTrace::Severity::Info:
@@ -186,9 +154,9 @@ void ChannelTrace::TraceEvent::RenderTraceEvent(grpc_json* json) const {
   json_iterator = grpc_json_create_child(json_iterator, json, "severity",
                                          severity_string(severity_),
                                          GRPC_JSON_STRING, false);
-  json_iterator =
-      grpc_json_create_child(json_iterator, json, "timestamp",
-                             fmt_time(timestamp_), GRPC_JSON_STRING, true);
+  json_iterator = grpc_json_create_child(json_iterator, json, "timestamp",
+                                         gpr_format_timespec(timestamp_),
+                                         GRPC_JSON_STRING, true);
   if (referenced_channel_ != nullptr) {
     char* uuid_str;
     gpr_asprintf(&uuid_str, "%" PRIdPTR, referenced_channel_->channel_uuid());
@@ -216,9 +184,9 @@ grpc_json* ChannelTrace::RenderJson() const {
   json_iterator =
       grpc_json_create_child(json_iterator, json, "numEventsLogged",
                              num_events_logged_str, GRPC_JSON_STRING, true);
-  json_iterator =
-      grpc_json_create_child(json_iterator, json, "creationTimestamp",
-                             fmt_time(time_created_), GRPC_JSON_STRING, true);
+  json_iterator = grpc_json_create_child(
+      json_iterator, json, "creationTimestamp",
+      gpr_format_timespec(time_created_), GRPC_JSON_STRING, true);
   grpc_json* events = grpc_json_create_child(json_iterator, json, "events",
                                              nullptr, GRPC_JSON_ARRAY, false);
   json_iterator = nullptr;

src/core/lib/channel/channelz.cc

@@ -41,53 +41,6 @@
 namespace grpc_core {
 namespace channelz {
 
-namespace {
-
-// TODO(ncteisen): move this function to a common helper location.
-//
-// returns an allocated string that represents tm according to RFC-3339, and,
-// more specifically, follows:
-// https://developers.google.com/protocol-buffers/docs/proto3#json
-//
-// "Uses RFC 3339, where generated output will always be Z-normalized and uses
-// 0, 3, 6 or 9 fractional digits."
-char* fmt_time(gpr_timespec tm) {
-  char time_buffer[35];
-  char ns_buffer[11];  // '.' + 9 digits of precision
-  struct tm* tm_info = localtime((const time_t*)&tm.tv_sec);
-  strftime(time_buffer, sizeof(time_buffer), "%Y-%m-%dT%H:%M:%S", tm_info);
-  snprintf(ns_buffer, 11, ".%09d", tm.tv_nsec);
-  // This loop trims off trailing zeros by inserting a null character that the
-  // right point. We iterate in chunks of three because we want 0, 3, 6, or 9
-  // fractional digits.
-  for (int i = 7; i >= 1; i -= 3) {
-    if (ns_buffer[i] == '0' && ns_buffer[i + 1] == '0' &&
-        ns_buffer[i + 2] == '0') {
-      ns_buffer[i] = '\0';
-      // Edge case in which all fractional digits were 0.
-      if (i == 1) {
-        ns_buffer[0] = '\0';
-      }
-    } else {
-      break;
-    }
-  }
-  char* full_time_str;
-  gpr_asprintf(&full_time_str, "%s%sZ", time_buffer, ns_buffer);
-  return full_time_str;
-}
-
-// TODO(ncteisen); move this to json library
-grpc_json* add_num_str(grpc_json* parent, grpc_json* it, const char* name,
-                       int64_t num) {
-  char* num_str;
-  gpr_asprintf(&num_str, "%" PRId64, num);
-  return grpc_json_create_child(it, parent, name, num_str, GRPC_JSON_STRING,
-                                true);
-}
-
-}  // namespace
-
 ChannelNode::ChannelNode(grpc_channel* channel, size_t channel_tracer_max_nodes,
                          bool is_top_level_channel)
     : channel_(channel),
@@ -114,6 +67,8 @@ void ChannelNode::RecordCallStarted() {
 
 void ChannelNode::PopulateConnectivityState(grpc_json* json) {}
 
+void ChannelNode::PopulateChildRefs(grpc_json* json) {}
+
 grpc_json* ChannelNode::RenderJson() {
   // We need to track these three json objects to build our object
   grpc_json* top_level_json = grpc_json_create(GRPC_JSON_OBJECT);
@@ -124,7 +79,8 @@ grpc_json* ChannelNode::RenderJson() {
                                          GRPC_JSON_OBJECT, false);
   json = json_iterator;
   json_iterator = nullptr;
-  json_iterator = add_num_str(json, json_iterator, "channelId", channel_uuid_);
+  json_iterator = grpc_json_add_number_string_child(json, json_iterator,
+                                                    "channelId", channel_uuid_);
   // reset json iterators to top level object
   json = top_level_json;
   json_iterator = nullptr;
@@ -152,22 +108,25 @@ grpc_json* ChannelNode::RenderJson() {
   json = data;
   json_iterator = nullptr;
   if (calls_started_ != 0) {
-    json_iterator =
-        add_num_str(json, json_iterator, "callsStarted", calls_started_);
+    json_iterator = grpc_json_add_number_string_child(
+        json, json_iterator, "callsStarted", calls_started_);
   }
   if (calls_succeeded_ != 0) {
-    json_iterator =
-        add_num_str(json, json_iterator, "callsSucceeded", calls_succeeded_);
+    json_iterator = grpc_json_add_number_string_child(
+        json, json_iterator, "callsSucceeded", calls_succeeded_);
   }
   if (calls_failed_) {
-    json_iterator =
-        add_num_str(json, json_iterator, "callsFailed", calls_failed_);
+    json_iterator = grpc_json_add_number_string_child(
+        json, json_iterator, "callsFailed", calls_failed_);
   }
   gpr_timespec ts =
       grpc_millis_to_timespec(last_call_started_millis_, GPR_CLOCK_REALTIME);
   json_iterator =
       grpc_json_create_child(json_iterator, json, "lastCallStartedTimestamp",
-                             fmt_time(ts), GRPC_JSON_STRING, true);
+                             gpr_format_timespec(ts), GRPC_JSON_STRING, true);
+  json = top_level_json;
+  json_iterator = nullptr;
+  PopulateChildRefs(json);
   return top_level_json;
 }
 
@@ -185,5 +144,13 @@ RefCountedPtr<ChannelNode> ChannelNode::MakeChannelNode(
       channel, channel_tracer_max_nodes, is_top_level_channel);
 }
 
+SubchannelNode::SubchannelNode() {
+  subchannel_uuid_ = ChannelzRegistry::RegisterSubchannelNode(this);
+}
+
+SubchannelNode::~SubchannelNode() {
+  ChannelzRegistry::UnregisterSubchannelNode(subchannel_uuid_);
+}
+
 }  // namespace channelz
 }  // namespace grpc_core

src/core/lib/channel/channelz.h

@@ -68,6 +68,8 @@ class ChannelNode : public RefCounted<ChannelNode> {
   // instead of lib/
   virtual void PopulateConnectivityState(grpc_json* json);
 
+  virtual void PopulateChildRefs(grpc_json* json);
+
   ChannelTrace* trace() { return trace_.get(); }
 
   void MarkChannelDestroyed() {
@@ -102,6 +104,24 @@ class ChannelNode : public RefCounted<ChannelNode> {
   ManualConstructor<ChannelTrace> trace_;
 };
 
+// Placeholds channelz class for subchannels. All this can do now is track its
+// uuid (this information is needed by the parent channelz class).
+// TODO(ncteisen): build this out to support the GetSubchannel channelz request.
+class SubchannelNode : public RefCounted<SubchannelNode> {
+ public:
+  SubchannelNode();
+  virtual ~SubchannelNode();
+
+  intptr_t subchannel_uuid() { return subchannel_uuid_; }
+
+ protected:
+  GPRC_ALLOW_CLASS_TO_USE_NON_PUBLIC_DELETE
+  GPRC_ALLOW_CLASS_TO_USE_NON_PUBLIC_NEW
+
+ private:
+  intptr_t subchannel_uuid_;
+};
+
 // Creation functions
 
 typedef RefCountedPtr<ChannelNode> (*ChannelNodeCreationFunc)(grpc_channel*,

src/core/lib/channel/channelz_registry.h

@@ -22,6 +22,7 @@
 #include <grpc/impl/codegen/port_platform.h>
 
 #include "src/core/lib/channel/channel_trace.h"
+#include "src/core/lib/channel/channelz.h"
 #include "src/core/lib/gprpp/inlined_vector.h"
 
 #include <stdint.h>
@@ -39,6 +40,7 @@ class ChannelzRegistry {
   // To be called in grpc_shutdown();
   static void Shutdown();
 
+  // Register/Unregister/Get for ChannelNode
   static intptr_t RegisterChannelNode(ChannelNode* channel_node) {
     RegistryEntry entry(channel_node, EntityType::kChannelNode);
     return Default()->InternalRegisterEntry(entry);
@@ -51,6 +53,20 @@ class ChannelzRegistry {
     return gotten == nullptr ? nullptr : static_cast<ChannelNode*>(gotten);
   }
 
+  // Register/Unregister/Get for SubchannelNode
+  static intptr_t RegisterSubchannelNode(SubchannelNode* channel_node) {
+    RegistryEntry entry(channel_node, EntityType::kSubchannelNode);
+    return Default()->InternalRegisterEntry(entry);
+  }
+  static void UnregisterSubchannelNode(intptr_t uuid) {
+    Default()->InternalUnregisterEntry(uuid, EntityType::kSubchannelNode);
+  }
+  static SubchannelNode* GetSubchannelNode(intptr_t uuid) {
+    void* gotten =
+        Default()->InternalGetEntry(uuid, EntityType::kSubchannelNode);
+    return gotten == nullptr ? nullptr : static_cast<SubchannelNode*>(gotten);
+  }
+
   // Returns the allocated JSON string that represents the proto
   // GetTopChannelsResponse as per channelz.proto.
   static char* GetTopChannels(intptr_t start_channel_id) {
@@ -60,6 +76,7 @@ class ChannelzRegistry {
  private:
   enum class EntityType {
     kChannelNode,
+    kSubchannelNode,
     kUnset,
   };
 

src/core/lib/gpr/string.cc

@@ -23,8 +23,10 @@
 #include <ctype.h>
 #include <limits.h>
 #include <stddef.h>
+#include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
+#include <time.h>
 
 #include <grpc/support/alloc.h>
 #include <grpc/support/log.h>
@@ -54,6 +56,32 @@ typedef struct {
   char* data;
 } dump_out;
 
+char* gpr_format_timespec(gpr_timespec tm) {
+  char time_buffer[35];
+  char ns_buffer[11];  // '.' + 9 digits of precision
+  struct tm* tm_info = localtime((const time_t*)&tm.tv_sec);
+  strftime(time_buffer, sizeof(time_buffer), "%Y-%m-%dT%H:%M:%S", tm_info);
+  snprintf(ns_buffer, 11, ".%09d", tm.tv_nsec);
+  // This loop trims off trailing zeros by inserting a null character that the
+  // right point. We iterate in chunks of three because we want 0, 3, 6, or 9
+  // fractional digits.
+  for (int i = 7; i >= 1; i -= 3) {
+    if (ns_buffer[i] == '0' && ns_buffer[i + 1] == '0' &&
+        ns_buffer[i + 2] == '0') {
+      ns_buffer[i] = '\0';
+      // Edge case in which all fractional digits were 0.
+      if (i == 1) {
+        ns_buffer[0] = '\0';
+      }
+    } else {
+      break;
+    }
+  }
+  char* full_time_str;
+  gpr_asprintf(&full_time_str, "%s%sZ", time_buffer, ns_buffer);
+  return full_time_str;
+}
+
 static dump_out dump_out_create(void) {
   dump_out r = {0, 0, nullptr};
   return r;

src/core/lib/gpr/string.h

@@ -21,6 +21,8 @@
 
 #include <grpc/support/port_platform.h>
 
+#include <grpc/impl/codegen/gpr_types.h>
+
 #include <stdbool.h>
 #include <stddef.h>
 
@@ -81,6 +83,14 @@ char* gpr_strjoin_sep(const char** strs, size_t nstrs, const char* sep,
 void gpr_string_split(const char* input, const char* sep, char*** strs,
                       size_t* nstrs);
 
+/* Returns an allocated string that represents tm according to RFC-3339, and,
+   more specifically, follows:
+   https://developers.google.com/protocol-buffers/docs/proto3#json
+
+   Uses RFC 3339, where generated output will always be Z-normalized and uses
+   0, 3, 6 or 9 fractional digits. */
+char* gpr_format_timespec(gpr_timespec);
+
 /* A vector of strings... for building up a final string one piece at a time */
 typedef struct {
   char** strs;

src/core/lib/gprpp/abstract.h

@@ -28,7 +28,10 @@
 
 // gRPC currently can't depend on libstdc++, so we can't use "= 0" for
 // pure virtual methods.  Instead, we use this macro.
-#define GRPC_ABSTRACT \
-  { GPR_ASSERT(false); }
+#define GRPC_ABSTRACT                                                        \
+  {                                                                          \
+    gpr_log(GPR_ERROR, "Function marked GRPC_ABSTRACT was not implemented"); \
+    GPR_ASSERT(false);                                                       \
+  }
 
 #endif /* GRPC_CORE_LIB_GPRPP_ABSTRACT_H */

src/core/lib/gprpp/inlined_vector.h

@@ -22,6 +22,7 @@
 #include <grpc/support/port_platform.h>
 
 #include <cassert>
+#include <cstring>
 
 #include "src/core/lib/gprpp/memory.h"
 
@@ -50,9 +51,33 @@ class InlinedVector {
   InlinedVector() { init_data(); }
   ~InlinedVector() { destroy_elements(); }
 
-  // For now, we do not support copying.
-  InlinedVector(const InlinedVector&) = delete;
-  InlinedVector& operator=(const InlinedVector&) = delete;
+  // copy constructor
+  InlinedVector(const InlinedVector& v) {
+    init_data();
+    copy_from(v);
+  }
+
+  InlinedVector& operator=(const InlinedVector& v) {
+    if (this != &v) {
+      clear();
+      copy_from(v);
+    }
+    return *this;
+  }
+
+  // move constructor
+  InlinedVector(InlinedVector&& v) {
+    init_data();
+    move_from(v);
+  }
+
+  InlinedVector& operator=(InlinedVector&& v) {
+    if (this != &v) {
+      clear();
+      move_from(v);
+    }
+    return *this;
+  }
 
   T* data() {
     return dynamic_ != nullptr ? dynamic_ : reinterpret_cast<T*>(inline_);
@@ -98,6 +123,33 @@ class InlinedVector {
 
   void push_back(T&& value) { emplace_back(std::move(value)); }
 
+  void copy_from(const InlinedVector& v) {
+    // if v is allocated, copy over the buffer.
+    if (v.dynamic_ != nullptr) {
+      reserve(v.capacity_);
+      memcpy(dynamic_, v.dynamic_, v.size_ * sizeof(T));
+    } else {
+      memcpy(inline_, v.inline_, v.size_ * sizeof(T));
+    }
+    // copy over metadata
+    size_ = v.size_;
+    capacity_ = v.capacity_;
+  }
+
+  void move_from(InlinedVector& v) {
+    // if v is allocated, then we steal its buffer, else we copy it.
+    if (v.dynamic_ != nullptr) {
+      dynamic_ = v.dynamic_;
+    } else {
+      memcpy(inline_, v.inline_, v.size_ * sizeof(T));
+    }
+    // copy over metadata
+    size_ = v.size_;
+    capacity_ = v.capacity_;
+    // null out the original
+    v.init_data();
+  }
+
   size_t size() const { return size_; }
   bool empty() const { return size_ == 0; }
 

src/core/lib/iomgr/ev_epollex_linux.cc

@@ -494,8 +494,9 @@ static void fd_orphan(grpc_fd* fd, grpc_closure* on_done, int* release_fd,
     is_fd_closed = true;
   }
 
+  // TODO(sreek): handle fd removal (where is_fd_closed=false)
   if (!is_fd_closed) {
-    gpr_log(GPR_DEBUG, "TODO: handle fd removal?");
+    GRPC_FD_TRACE("epoll_fd %p (%d) was orphaned but not closed.", fd, fd->fd);
   }
 
   /* Remove the active status but keep referenced. We want this grpc_fd struct
@@ -1564,7 +1565,7 @@ static void pollset_set_add_pollset_set(grpc_pollset_set* a,
     gpr_mu_unlock(b_mu);
   }
   // try to do the least copying possible
-  // TODO(ctiller): there's probably a better heuristic here
+  // TODO(sreek): there's probably a better heuristic here
   const size_t a_size = a->fd_count + a->pollset_count;
   const size_t b_size = b->fd_count + b->pollset_count;
   if (b_size > a_size) {

src/core/lib/iomgr/tcp_posix.cc

@@ -26,6 +26,7 @@
 #include "src/core/lib/iomgr/tcp_posix.h"
 
 #include <errno.h>
+#include <limits.h>
 #include <stdbool.h>
 #include <stdlib.h>
 #include <string.h>
@@ -513,7 +514,11 @@ static void tcp_read(grpc_endpoint* ep, grpc_slice_buffer* incoming_buffer,
 }
 
 /* returns true if done, false if pending; if returning true, *error is set */
+#if defined(IOV_MAX) && IOV_MAX < 1000
+#define MAX_WRITE_IOVEC IOV_MAX
+#else
 #define MAX_WRITE_IOVEC 1000
+#endif
 static bool tcp_flush(grpc_tcp* tcp, grpc_error** error) {
   struct msghdr msg;
   struct iovec iov[MAX_WRITE_IOVEC];

src/core/lib/json/json.cc

@@ -18,10 +18,12 @@
 
 #include <grpc/support/port_platform.h>
 
+#include <inttypes.h>
 #include <string.h>
 
 #include <grpc/support/alloc.h>
 #include <grpc/support/log.h>
+#include <grpc/support/string_util.h>
 
 #include "src/core/lib/json/json.h"
 
@@ -84,3 +86,11 @@ grpc_json* grpc_json_create_child(grpc_json* sibling, grpc_json* parent,
   child->key = key;
   return child;
 }
+
+grpc_json* grpc_json_add_number_string_child(grpc_json* parent, grpc_json* it,
+                                             const char* name, int64_t num) {
+  char* num_str;
+  gpr_asprintf(&num_str, "%" PRId64, num);
+  return grpc_json_create_child(it, parent, name, num_str, GRPC_JSON_STRING,
+                                true);
+}

src/core/lib/json/json.h

@@ -91,4 +91,9 @@ grpc_json* grpc_json_create_child(grpc_json* sibling, grpc_json* parent,
                                   const char* key, const char* value,
                                   grpc_json_type type, bool owns_value);
 
+/* Creates a child json string object from the integer num, then links the
+   json object into the parent's json tree */
+grpc_json* grpc_json_add_number_string_child(grpc_json* parent, grpc_json* it,
+                                             const char* name, int64_t num);
+
 #endif /* GRPC_CORE_LIB_JSON_JSON_H */

templates/tools/dockerfile/test/bazel/Dockerfile.template

@@ -28,6 +28,8 @@
     openjdk-8-jdk ${'\\'}
     vim
   
+  <%include file="../../python_deps.include"/>
+
   <%include file="../../bazel.include"/>
   
   RUN mkdir -p /var/local/jenkins

test/core/gprpp/inlined_vector_test.cc

@@ -17,20 +17,32 @@
  */
 
 #include "src/core/lib/gprpp/inlined_vector.h"
+#include <grpc/support/log.h>
 #include <gtest/gtest.h>
 #include "src/core/lib/gprpp/memory.h"
 #include "test/core/util/test_config.h"
 
 namespace grpc_core {
 namespace testing {
+namespace {
+
+template <typename Vector>
+static void FillVector(Vector* v, int len, int start = 0) {
+  for (int i = 0; i < len; i++) {
+    v->push_back(i + start);
+    EXPECT_EQ(i + 1UL, v->size());
+  }
+  EXPECT_EQ(static_cast<size_t>(len), v->size());
+  EXPECT_LE(static_cast<size_t>(len), v->capacity());
+}
+
+}  // namespace
 
 TEST(InlinedVectorTest, CreateAndIterate) {
   const int kNumElements = 9;
   InlinedVector<int, 2> v;
   EXPECT_TRUE(v.empty());
-  for (int i = 0; i < kNumElements; ++i) {
-    v.push_back(i);
-  }
+  FillVector(&v, kNumElements);
   EXPECT_EQ(static_cast<size_t>(kNumElements), v.size());
   EXPECT_FALSE(v.empty());
   for (int i = 0; i < kNumElements; ++i) {
@@ -42,9 +54,7 @@ TEST(InlinedVectorTest, CreateAndIterate) {
 TEST(InlinedVectorTest, ValuesAreInlined) {
   const int kNumElements = 5;
   InlinedVector<int, 10> v;
-  for (int i = 0; i < kNumElements; ++i) {
-    v.push_back(i);
-  }
+  FillVector(&v, kNumElements);
   EXPECT_EQ(static_cast<size_t>(kNumElements), v.size());
   for (int i = 0; i < kNumElements; ++i) {
     EXPECT_EQ(i, v[i]);
@@ -71,19 +81,13 @@ TEST(InlinedVectorTest, ClearAndRepopulate) {
   const int kNumElements = 10;
   InlinedVector<int, 5> v;
   EXPECT_EQ(0UL, v.size());
-  for (int i = 0; i < kNumElements; ++i) {
-    v.push_back(i);
-    EXPECT_EQ(i + 1UL, v.size());
-  }
+  FillVector(&v, kNumElements);
   for (int i = 0; i < kNumElements; ++i) {
     EXPECT_EQ(i, v[i]);
   }
   v.clear();
   EXPECT_EQ(0UL, v.size());
-  for (int i = 0; i < kNumElements; ++i) {
-    v.push_back(kNumElements + i);
-    EXPECT_EQ(i + 1UL, v.size());
-  }
+  FillVector(&v, kNumElements, kNumElements);
   for (int i = 0; i < kNumElements; ++i) {
     EXPECT_EQ(kNumElements + i, v[i]);
   }
@@ -93,10 +97,7 @@ TEST(InlinedVectorTest, ConstIndexOperator) {
   constexpr int kNumElements = 10;
   InlinedVector<int, 5> v;
   EXPECT_EQ(0UL, v.size());
-  for (int i = 0; i < kNumElements; ++i) {
-    v.push_back(i);
-    EXPECT_EQ(i + 1UL, v.size());
-  }
+  FillVector(&v, kNumElements);
   // The following lambda function is exceptionally allowed to use an anonymous
   // capture due to the erroneous behavior of the MSVC compiler, that refuses to
   // capture the kNumElements constexpr, something allowed by the standard.
@@ -108,6 +109,161 @@ TEST(InlinedVectorTest, ConstIndexOperator) {
   const_func(v);
 }
 
+// the following constants and typedefs are used for copy/move
+// construction/assignment
+const size_t kInlinedLength = 8;
+typedef InlinedVector<int, kInlinedLength> IntVec8;
+const size_t kInlinedFillSize = kInlinedLength - 1;
+const size_t kAllocatedFillSize = kInlinedLength + 1;
+
+TEST(InlinedVectorTest, CopyConstructerInlined) {
+  IntVec8 original;
+  FillVector(&original, kInlinedFillSize);
+  IntVec8 copy_constructed(original);
+  for (size_t i = 0; i < original.size(); ++i) {
+    EXPECT_EQ(original[i], copy_constructed[i]);
+  }
+}
+
+TEST(InlinedVectorTest, CopyConstructerAllocated) {
+  IntVec8 original;
+  FillVector(&original, kAllocatedFillSize);
+  IntVec8 copy_constructed(original);
+  for (size_t i = 0; i < original.size(); ++i) {
+    EXPECT_EQ(original[i], copy_constructed[i]);
+  }
+}
+
+TEST(InlinedVectorTest, CopyAssignementInlinedInlined) {
+  IntVec8 original;
+  FillVector(&original, kInlinedFillSize);
+  IntVec8 copy_assigned;
+  FillVector(&copy_assigned, kInlinedFillSize, 99);
+  copy_assigned = original;
+  for (size_t i = 0; i < original.size(); ++i) {
+    EXPECT_EQ(original[i], copy_assigned[i]);
+  }
+}
+
+TEST(InlinedVectorTest, CopyAssignementInlinedAllocated) {
+  IntVec8 original;
+  FillVector(&original, kInlinedFillSize);
+  IntVec8 copy_assigned;
+  FillVector(&copy_assigned, kAllocatedFillSize, 99);
+  copy_assigned = original;
+  for (size_t i = 0; i < original.size(); ++i) {
+    EXPECT_EQ(original[i], copy_assigned[i]);
+  }
+}
+
+TEST(InlinedVectorTest, CopyAssignementAllocatedInlined) {
+  IntVec8 original;
+  FillVector(&original, kAllocatedFillSize);
+  IntVec8 copy_assigned;
+  FillVector(&copy_assigned, kInlinedFillSize, 99);
+  copy_assigned = original;
+  for (size_t i = 0; i < original.size(); ++i) {
+    EXPECT_EQ(original[i], copy_assigned[i]);
+  }
+}
+
+TEST(InlinedVectorTest, CopyAssignementAllocatedAllocated) {
+  IntVec8 original;
+  FillVector(&original, kAllocatedFillSize);
+  IntVec8 copy_assigned;
+  FillVector(&copy_assigned, kAllocatedFillSize, 99);
+  copy_assigned = original;
+  for (size_t i = 0; i < original.size(); ++i) {
+    EXPECT_EQ(original[i], copy_assigned[i]);
+  }
+}
+
+TEST(InlinedVectorTest, MoveConstructorInlined) {
+  IntVec8 original;
+  FillVector(&original, kInlinedFillSize);
+  IntVec8 tmp(original);
+  auto* old_data = tmp.data();
+  IntVec8 move_constructed(std::move(tmp));
+  for (size_t i = 0; i < original.size(); ++i) {
+    EXPECT_EQ(original[i], move_constructed[i]);
+  }
+  // original data was inlined so it should have been copied, not moved.
+  EXPECT_NE(move_constructed.data(), old_data);
+}
+
+TEST(InlinedVectorTest, MoveConstructorAllocated) {
+  IntVec8 original;
+  FillVector(&original, kAllocatedFillSize);
+  IntVec8 tmp(original);
+  auto* old_data = tmp.data();
+  IntVec8 move_constructed(std::move(tmp));
+  for (size_t i = 0; i < original.size(); ++i) {
+    EXPECT_EQ(original[i], move_constructed[i]);
+  }
+  // original data was allocated, so it should been moved, not copied
+  EXPECT_EQ(move_constructed.data(), old_data);
+}
+
+TEST(InlinedVectorTest, MoveAssignmentInlinedInlined) {
+  IntVec8 original;
+  FillVector(&original, kInlinedFillSize);
+  IntVec8 move_assigned;
+  FillVector(&move_assigned, kInlinedFillSize, 99);  // Add dummy elements
+  IntVec8 tmp(original);
+  auto* old_data = tmp.data();
+  move_assigned = std::move(tmp);
+  for (size_t i = 0; i < original.size(); ++i) {
+    EXPECT_EQ(original[i], move_assigned[i]);
+  }
+  // original data was inlined so it should have been copied, not moved.
+  EXPECT_NE(move_assigned.data(), old_data);
+}
+
+TEST(InlinedVectorTest, MoveAssignmentInlinedAllocated) {
+  IntVec8 original;
+  FillVector(&original, kInlinedFillSize);
+  IntVec8 move_assigned;
+  FillVector(&move_assigned, kAllocatedFillSize, 99);  // Add dummy elements
+  IntVec8 tmp(original);
+  auto* old_data = tmp.data();
+  move_assigned = std::move(tmp);
+  for (size_t i = 0; i < original.size(); ++i) {
+    EXPECT_EQ(original[i], move_assigned[i]);
+  }
+  // original data was inlined so it should have been copied, not moved.
+  EXPECT_NE(move_assigned.data(), old_data);
+}
+
+TEST(InlinedVectorTest, MoveAssignmentAllocatedInlined) {
+  IntVec8 original;
+  FillVector(&original, kAllocatedFillSize);
+  IntVec8 move_assigned;
+  FillVector(&move_assigned, kInlinedFillSize, 99);  // Add dummy elements
+  IntVec8 tmp(original);
+  auto* old_data = tmp.data();
+  move_assigned = std::move(tmp);
+  for (size_t i = 0; i < original.size(); ++i) {
+    EXPECT_EQ(original[i], move_assigned[i]);
+  }
+  // original data was allocated so it should have been moved, not copied.
+  EXPECT_EQ(move_assigned.data(), old_data);
+}
+
+TEST(InlinedVectorTest, MoveAssignmentAllocatedAllocated) {
+  IntVec8 original;
+  FillVector(&original, kAllocatedFillSize);
+  IntVec8 move_assigned;
+  FillVector(&move_assigned, kAllocatedFillSize, 99);  // Add dummy elements
+  IntVec8 tmp(original);
+  auto* old_data = tmp.data();
+  move_assigned = std::move(tmp);
+  for (size_t i = 0; i < original.size(); ++i) {
+    EXPECT_EQ(original[i], move_assigned[i]);
+  }
+  // original data was allocated so it should have been moved, not copied.
+  EXPECT_EQ(move_assigned.data(), old_data);
+}
+
 }  // namespace testing
 }  // namespace grpc_core
 

test/cpp/util/test_credentials_provider.cc

@@ -63,6 +63,8 @@ class DefaultCredentialsProvider : public CredentialsProvider {
       SslCredentialsOptions ssl_opts = {test_root_cert, "", ""};
       args->SetSslTargetNameOverride("foo.test.google.fr");
       return SslCredentials(ssl_opts);
+    } else if (type == grpc::testing::kGoogleDefaultCredentialsType) {
+      return grpc::GoogleDefaultCredentials();
     } else {
       std::unique_lock<std::mutex> lock(mu_);
       auto it(std::find(added_secure_type_names_.begin(),

test/cpp/util/test_credentials_provider.h

@@ -33,6 +33,7 @@ const char kInsecureCredentialsType[] = "INSECURE_CREDENTIALS";
 // property "transport_security_type".
 const char kTlsCredentialsType[] = "ssl";
 const char kAltsCredentialsType[] = "alts";
+const char kGoogleDefaultCredentialsType[] = "google_default_credentials";
 
 // Provide test credentials of a particular type.
 class CredentialTypeProvider {

tools/dockerfile/test/bazel/Dockerfile

@@ -26,6 +26,22 @@ RUN apt-get update && apt-get -y install \
   openjdk-8-jdk \
   vim
 
+#====================
+# Python dependencies
+
+# Install dependencies
+
+RUN apt-get update && apt-get install -y \
+    python-all-dev \
+    python3-all-dev \
+    python-pip
+
+# Install Python packages from PyPI
+RUN pip install --upgrade pip==10.0.1
+RUN pip install virtualenv
+RUN pip install futures==2.2.0 enum34==1.0.4 protobuf==3.5.2.post1 six==1.10.0 twisted==17.5.0
+
+
 #========================
 # Bazel installation
 RUN echo "deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8" > /etc/apt/sources.list.d/bazel.list

tools/doxygen/Doxyfile.c++

@@ -784,6 +784,7 @@ doc/http-grpc-status-mapping.md \
 doc/http2-interop-test-descriptions.md \
 doc/internationalization.md \
 doc/interop-test-descriptions.md \
+doc/keepalive.md \
 doc/load-balancing.md \
 doc/naming.md \
 doc/server-reflection.md \

tools/doxygen/Doxyfile.c++.internal

@@ -784,6 +784,7 @@ doc/http-grpc-status-mapping.md \
 doc/http2-interop-test-descriptions.md \
 doc/internationalization.md \
 doc/interop-test-descriptions.md \
+doc/keepalive.md \
 doc/load-balancing.md \
 doc/naming.md \
 doc/server-reflection.md \

tools/doxygen/Doxyfile.core

@@ -786,6 +786,7 @@ doc/http-grpc-status-mapping.md \
 doc/http2-interop-test-descriptions.md \
 doc/internationalization.md \
 doc/interop-test-descriptions.md \
+doc/keepalive.md \
 doc/load-balancing.md \
 doc/naming.md \
 doc/server-reflection.md \

tools/doxygen/Doxyfile.core.internal

@@ -786,6 +786,7 @@ doc/http-grpc-status-mapping.md \
 doc/http2-interop-test-descriptions.md \
 doc/internationalization.md \
 doc/interop-test-descriptions.md \
+doc/keepalive.md \
 doc/load-balancing.md \
 doc/naming.md \
 doc/server-reflection.md \

tools/internal_ci/linux/grpc_python_bazel_test.cfg

@@ -0,0 +1,23 @@
+# Copyright 2018 gRPC authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Config file for the internal CI (in protobuf text format)
+
+# Location of the continuous shell script in repository.
+build_file: "grpc/tools/internal_ci/linux/grpc_bazel.sh"
+timeout_mins: 240
+env_vars {
+  key: "BAZEL_SCRIPT"
+  value: "tools/internal_ci/linux/grpc_python_bazel_test_in_docker.sh"
+}

tools/internal_ci/linux/grpc_python_bazel_test_in_docker.sh

@@ -0,0 +1,27 @@
+#!/usr/bin/env bash
+# Copyright 2018 gRPC authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# Test full Bazel
+#
+# NOTE: No empty lines should appear in this file before igncr is set!
+set -ex -o igncr || set -ex
+
+mkdir -p /var/local/git
+git clone /var/local/jenkins/grpc /var/local/git/grpc
+(cd /var/local/jenkins/grpc/ && git submodule foreach 'cd /var/local/git/grpc \
+&& git submodule update --init --reference /var/local/jenkins/grpc/${name} \
+${name}')
+cd /var/local/git/grpc/test
+bazel test --spawn_strategy=standalone --genrule_strategy=standalone //src/python/...

tools/internal_ci/linux/grpc_tsan_on_foundry.sh

@@ -14,5 +14,5 @@
 # limitations under the License.
 
 export UPLOAD_TEST_RESULTS=true
-EXTRA_FLAGS="--copt=-gmlt --strip=never --copt=-fsanitize=thread --linkopt=-fsanitize=thread --test_timeout=3600 --action_env=TSAN_OPTIONS=suppressions=test/core/util/tsan_suppressions.txt:halt_on_error=1:second_deadlock_stack=1"
+EXTRA_FLAGS="--copt=-gmlt --strip=never --copt=-fsanitize=thread --linkopt=-fsanitize=thread --test_timeout=3600 --action_env=TSAN_OPTIONS=suppressions=test/core/util/tsan_suppressions.txt:halt_on_error=1:second_deadlock_stack=1  --runs_per_test_detects_flakes --runs_per_test=2"
 github/grpc/tools/internal_ci/linux/grpc_bazel_on_foundry_base.sh "${EXTRA_FLAGS}"

tools/internal_ci/linux/pull_request/grpc_tsan_on_foundry.sh

@@ -0,0 +1,17 @@
+#!/usr/bin/env bash
+# Copyright 2017 gRPC authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+EXTRA_FLAGS="--copt=-gmlt --strip=never --copt=-fsanitize=thread --linkopt=-fsanitize=thread --test_timeout=3600 --action_env=TSAN_OPTIONS=suppressions=test/core/util/tsan_suppressions.txt:halt_on_error=1:second_deadlock_stack=1"
+github/grpc/tools/internal_ci/linux/grpc_bazel_on_foundry_base.sh "${EXTRA_FLAGS}"

tools/interop_matrix/client_matrix.py

@@ -252,6 +252,12 @@ LANG_RELEASE_MATRIX = {
         {
             'v1.10.0': None
         },
+        {
+            'v1.11.3': None
+        },
+        {
+            'v1.12.4': None
+        },
     ],
     'ruby': [
         {