From 0d0ba3a765e2296a4313d89b368e8abf68d9f932 Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Tue, 6 Dec 2016 15:27:39 -0500 Subject: gRPC --- chapter/1/gRPC.md | 122 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 122 insertions(+) create mode 100644 chapter/1/gRPC.md (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md new file mode 100644 index 0000000..49bdee5 --- /dev/null +++ b/chapter/1/gRPC.md @@ -0,0 +1,122 @@ +--- +layout: page +title: "gRPC" +by: "Paul Grosu (Northeastern U.), Muzammil Abdul Rehman (Northeastern U.), Eric Anderson (Google, Inc.), Vijay Pai (Google, Inc.), and Heather Miller (Northeastern U.)" +--- + +

+

gRPC

+

+ +

+

Paul Grosu (Northeastern U.), Muzammil Abdul Rehman (Northeastern U.), Eric Anderson (Google, Inc.), Vijay Pai (Google, Inc.), and Heather Miller (Northeastern U.)

+

+ +
+ +

Abstract

+ +gRPC has been built from a collaboration between Google and Square as a public replacement of Stubby, ARCWire and Sake {% cite Apigee %}. The gRPC framework is a form of an Actor Model based on an IDL (Interface Description Language), which is defined via the Protocol Buffer message format. With the introduction of HTTP/2 the internal Google Stubby and Square Sake frameworks are now been made available to the public. By working on top of the HTTP/2 protocol, gRPC enables messages to be multiplexed and compressed bi-directionally as premptive streams for maximizing capacity of any microservices ecosystem. Google has also a new approach to public projects, where instead of just releasing a paper describing the concepts will now also provide the implementation of how to properly interpret the standard. + + +

Introduction

+ +In order to understand gRPC and the flexibity of enabling a microservices ecosystem to become into a Reactive Actor Model, it is important to appreciate the nuances of the HTTP/2 Protocol upon which it is based. Afterward we will describe the gRPC Framework - focusing specifically on the gRPC-Java implementation - with the scope to expand this chapter over time to all implementations of gRPC. At the end we will cover examples demonstrating these ideas, by taking a user from the initial steps of how to work with the gRPC-Java framework. + +

1 HTTP/2

+ +The HTTP 1.1 protocol has been a success for some time, though there were some key features which began to be requested by the community with the increase of distributed computing, especially in the area of microservices. The phenomenon of creating more modularized functional units that are organically constructed based on a share-nothing model with a bidirectional, high-throughput request and response methodology demands a new protocol for communication and integration. Thus the HTTP/2 was born as a new standard, which is a binary wire protocol providing compressed streams that can be multiplexed for concurrency. As many microservices implementations currently scan header messages before actually processing any payload in order to scale up the processing and routing of messages, HTTP/2 now provides header compression for this purpose. One last important benefit is that the server endpoint can actually push cached resources to the client based on anticipated future communication, dramatically saving client communication time and processing. + +

1.1 HTTP/2 Frames

+ +The HTTP/2 protocol is now a framed protocol, which expands the capability for bidirectional, asynchronous communication. Every message is thus part of a frame that will have a header, frame type and stream identifier aside from the standard frame length for processing. Each stream can have a priority, which allows for dependency between streams to be achieved forming a priority tree. The data can be either a request or response which allows for the bidirectional communication, with the capability of flagging the communication for stream termination, flow control with priority settings, continuation and push responses from the server for client confirmation. Below is the format of the HTTP/2 frame {% cite RFC7540 %}: + +

+
+ Figure 1: The encoding a HTTP/2 frame. +

+ +

1.2 Header Compression

+ +The HTTP header is one of the primary methods of passing information about the state of other endpoints, the request or response and the payload. This enables endpoints to save time when processing a large quantity to streams, with the ability to forward information along without wasting time to inspect the payload. Since the header information can be quite large, it is possible to now compress the them to allow for better throughput and capacity of stored stateful information. + + +

1.3 Multiplexed Streams

+ +As streams are core to the implementation of HTTP/2, it is important to discuss the details of their implemenation in the protocol. As many streams can be open simultanously from many endpoints, each stream will be in one of the following states. Each stream is multiplexed together forming a chain of streams that are transmitted over the wire, allowing for asynchronous bi-directional concurrency to be performed by the receiving endpoint. Below is the lifecycle of a stream {% cite RFC7540 %}: + +

+
+ Figure 2: The lifecycle of a HTTP/2 stream. +

+ +

1.4 Flow Control of Streams

+ +Since many streams will compete for the bandwidth of a connection, in order to prevent bottlenecks and collisions in the transmission. This is done via the WINDOW_UPDATE payload for every stream - and the overall connection as well - to let the sender know how much room the receiving endpoint has for processing new data. + +

2 Protocol Buffers with RPC

+ +Though gRPC was built on top of HTTP/2, an IDL had to be used to perform the communication between endpoints. The natural direction was to use Protocol Buffers is the method of stucturing data for serialization between a server and client. At the time of the start of gRPC development only version 2.0 (proto2) was available, which only implemented data structures without any request/response mechanism. An example of a Protocol Buffer data structure would look something like this: + +``` +// A message containing the user's name. +message Hello { + string name = 1; +} +``` +

+ Figure 3: Protocol Buffer version 2.0 representing a message data-structure. +

+ +Thus the language had to be updated to support gRPC and the development of a service message with a request and a response definition was added for version version 3.0 of Protocol Buffers. The updated implementation would look as follows {% cite HelloWorldProto %}: + +``` +// The request message containing the user's name. +message HelloRequest { + string name = 1; +} + +// The response message containing the greetings +message HelloReply { + string message = 1; +} + +// The greeting service definition. +service Greeter { + // Sends a greeting + rpc SayHello (HelloRequest) returns (HelloReply) {} +} +``` +

+ Figure 4: Protocol Buffer version 3.0 representing a message data-structure with the accompanied RPC definition. +

+ +Notice the addition of a service, where the RPC call would use one of the messages as the structure of a Request with the other being the Response message format. + +Once of these Proto file get generated, one would then use them to compile with gRPC to for generating the Client and Server files representing the classical two endpoints of a RPC implementation. + +

3 gRPC

+ +gRPC was built on top of HTTP/2, and we will cover the specifics of gRPC-Java, but expand it to all the implementations with time. gRPC is a framework for + +

+
+ Figure 5: gRPC allows for asynchronous language-agnostic message passing via Protocol Buffers. +

+ + +## References + +[Apigee]: https://www.youtube.com/watch?v=-2sWDr3Z0Wo +[Authentication]: http://www.grpc.io/docs/guides/auth.html +[Benchmarks]: http://www.grpc.io/docs/guides/benchmarking.html +[CoreSurfaceAPIs]: https://github.com/grpc/grpc/tree/master/src/core +[ErrorModel]: http://www.grpc.io/docs/guides/error.html +[gRPC]: https://github.com/grpc/grpc/blob/master/doc/g_stands_for.md +[gRPC-Companies]: http://www.grpc.io/about/ +[gRPC-Languages]: http://www.grpc.io/docs/ +[gRPC-Protos]: https://github.com/googleapis/googleapis/ +[Netty]: http://netty.io/ +[RFC7540]: http://httpwg.org/specs/rfc7540.html +[HelloWorldProto]: https://github.com/grpc/grpc/blob/master/examples/protos/helloworld.proto + -- cgit v1.2.3 From 5594d046d3fdf98fc96adf629f0dfc1529b56d68 Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Tue, 6 Dec 2016 15:30:03 -0500 Subject: gRPC --- chapter/1/gRPC.md | 1 + 1 file changed, 1 insertion(+) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index 49bdee5..ae8d2c1 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -12,6 +12,7 @@ by: "Paul Grosu (Northeastern U.), Muzammil Abdul Rehman (Northeastern U.), Eri

Paul Grosu (Northeastern U.), Muzammil Abdul Rehman (Northeastern U.), Eric Anderson (Google, Inc.), Vijay Pai (Google, Inc.), and Heather Miller (Northeastern U.)

+

Abstract

-- cgit v1.2.3 From 7a4919f1267279165276e13e24fb95b8698ac457 Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Tue, 6 Dec 2016 15:35:54 -0500 Subject: submit --- chapter/1/gRPC.md | 1 - 1 file changed, 1 deletion(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index ae8d2c1..49bdee5 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -12,7 +12,6 @@ by: "Paul Grosu (Northeastern U.), Muzammil Abdul Rehman (Northeastern U.), Eri

Paul Grosu (Northeastern U.), Muzammil Abdul Rehman (Northeastern U.), Eric Anderson (Google, Inc.), Vijay Pai (Google, Inc.), and Heather Miller (Northeastern U.)

-

Abstract

-- cgit v1.2.3 From db03fdb08aedc5125580c7ec8baa7215f08fea92 Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Tue, 6 Dec 2016 18:24:23 -0500 Subject: submit --- chapter/1/gRPC.md | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index 49bdee5..741f265 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -40,7 +40,6 @@ The HTTP/2 protocol is now a framed protocol, which expands the capability for b The HTTP header is one of the primary methods of passing information about the state of other endpoints, the request or response and the payload. This enables endpoints to save time when processing a large quantity to streams, with the ability to forward information along without wasting time to inspect the payload. Since the header information can be quite large, it is possible to now compress the them to allow for better throughput and capacity of stored stateful information. -

1.3 Multiplexed Streams

As streams are core to the implementation of HTTP/2, it is important to discuss the details of their implemenation in the protocol. As many streams can be open simultanously from many endpoints, each stream will be in one of the following states. Each stream is multiplexed together forming a chain of streams that are transmitted over the wire, allowing for asynchronous bi-directional concurrency to be performed by the receiving endpoint. Below is the lifecycle of a stream {% cite RFC7540 %}: @@ -68,6 +67,18 @@ message Hello { Figure 3: Protocol Buffer version 2.0 representing a message data-structure.

+This message will also be encoded for highest compression when sent over the wire. For example, let us say that the message is the string "Hi". + + + + + + + + + +
TypeMeaningUsed For
0Varintint32, int64, uint32, uint64, sint32, sint64, bool, enum
164-bitfixed64, sfixed64, double
2Length-delimitedstring, bytes, embedded messages, packed repeated fields
3Start groupgroups (deprecated)
4End groupgroups (deprecated)
532-bitfixed32, sfixed32, float
+ Thus the language had to be updated to support gRPC and the development of a service message with a request and a response definition was added for version version 3.0 of Protocol Buffers. The updated implementation would look as follows {% cite HelloWorldProto %}: ``` -- cgit v1.2.3 From 48ff73298ef9c06266fd031e5eceeb8128df0206 Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Tue, 6 Dec 2016 18:26:40 -0500 Subject: submit --- chapter/1/gRPC.md | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index 741f265..62a3795 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -69,15 +69,10 @@ message Hello { This message will also be encoded for highest compression when sent over the wire. For example, let us say that the message is the string "Hi". - - - - - - - - -
TypeMeaningUsed For
0Varintint32, int64, uint32, uint64, sint32, sint64, bool, enum
164-bitfixed64, sfixed64, double
2Length-delimitedstring, bytes, embedded messages, packed repeated fields
3Start groupgroups (deprecated)
4End groupgroups (deprecated)
532-bitfixed32, sfixed32, float
+

+
+ Table 1: Tag values for Protocol Buffer types. +

Thus the language had to be updated to support gRPC and the development of a service message with a request and a response definition was added for version version 3.0 of Protocol Buffers. The updated implementation would look as follows {% cite HelloWorldProto %}: -- cgit v1.2.3 From 2b0e3e9ff9f08b00aed2268853208a6b9926695b Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Tue, 6 Dec 2016 18:47:16 -0500 Subject: submit --- chapter/1/gRPC.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index 62a3795..fb3c2a0 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -67,13 +67,19 @@ message Hello { Figure 3: Protocol Buffer version 2.0 representing a message data-structure.

-This message will also be encoded for highest compression when sent over the wire. For example, let us say that the message is the string "Hi". +This message will also be encoded for highest compression when sent over the wire. For example, let us say that the message is the string "Hi". Every Protocol Buffer type has a value, and in this case a string has a value of `2`, as noted in the Table 1 {% cite Protobuf-Types %}.


Table 1: Tag values for Protocol Buffer types.

+One will notice that there is a number associated with each field element in the Protocol Buffer definition, which represents its tag. In Figure 3, the field `name` has a tag of `1`. When a message gets encoded each field will start with a one byte value (8 bits), where the least-significant 3-bit value encode the type and the rest the tag. In this case tag which is `1`, with a type of 2. Thus the encoding will be `00001 010`, which has a hexdecimal value of `A`. The following byte is the length of the string which is `2`, followed by the string as `48` and `69` representing `H` and `i`. Thus the whole tranmission will look as follows: + +``` +A 2 48 69 +``` + Thus the language had to be updated to support gRPC and the development of a service message with a request and a response definition was added for version version 3.0 of Protocol Buffers. The updated implementation would look as follows {% cite HelloWorldProto %}: ``` @@ -125,4 +131,4 @@ gRPC was built on top of HTTP/2, and we will cover the specifics of gRPC-Java, b [Netty]: http://netty.io/ [RFC7540]: http://httpwg.org/specs/rfc7540.html [HelloWorldProto]: https://github.com/grpc/grpc/blob/master/examples/protos/helloworld.proto - +[Protobuf-Types]: https://developers.google.com/protocol-buffers/docs/encoding -- cgit v1.2.3 From 47c8ad4b134cfd11b8cf1b7003d7314fed42b71d Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Tue, 6 Dec 2016 19:01:59 -0500 Subject: submit --- chapter/1/gRPC.md | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index fb3c2a0..622498d 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -105,17 +105,23 @@ service Greeter { Notice the addition of a service, where the RPC call would use one of the messages as the structure of a Request with the other being the Response message format. -Once of these Proto file get generated, one would then use them to compile with gRPC to for generating the Client and Server files representing the classical two endpoints of a RPC implementation. +Once of these Proto file gets generated, one would then use them to compile with gRPC to for generating the Client and Server files representing the classical two endpoints of a RPC implementation.

3 gRPC

-gRPC was built on top of HTTP/2, and we will cover the specifics of gRPC-Java, but expand it to all the implementations with time. gRPC is a framework for +gRPC was built on top of HTTP/2, and we will cover the specifics of gRPC-Java, but expand it to all the implementations with time. gRPC is a cross-platform framework that allows integration across many languages as denoted in Figure 5 {% cite gRPC-Overview %}.


Figure 5: gRPC allows for asynchronous language-agnostic message passing via Protocol Buffers.

+The officially supported languages are listed in Table 2 {% cite gRPC-Languages %}. + +

+
+ Table 2: Officially supported languages by gRPC. +

## References @@ -132,3 +138,5 @@ gRPC was built on top of HTTP/2, and we will cover the specifics of gRPC-Java, b [RFC7540]: http://httpwg.org/specs/rfc7540.html [HelloWorldProto]: https://github.com/grpc/grpc/blob/master/examples/protos/helloworld.proto [Protobuf-Types]: https://developers.google.com/protocol-buffers/docs/encoding +[gRPC-Overview]: http://www.grpc.io/docs/guides/ +[gRPC-Languages]: http://www.grpc.io/about/#osp -- cgit v1.2.3 From 66187e549607d715b684b0a4df7e589d057287a0 Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Tue, 6 Dec 2016 19:31:57 -0500 Subject: submit --- chapter/1/gRPC.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index 622498d..99a81e0 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -123,6 +123,18 @@ The officially supported languages are listed in Table 2 {% cite gRPC-Languages Table 2: Officially supported languages by gRPC.

+There are benchmarks being performed on a daily basis to ensure that gRPC performs optimally under high-throughput conditions as illustrated in Figure 6. + +

+
+ Figure 6: Benchmark showing the queries-per-second on two virtual machines with 32 cores each. +

+ + +

3.1 gRPC Java

+ +The Java implementation of gRPC has been built with Mobile platform in mind and requires support of JDK 6.0 and higher. There are + ## References [Apigee]: https://www.youtube.com/watch?v=-2sWDr3Z0Wo -- cgit v1.2.3 From 93948d701c8e531c42caeb538b7fcb352e8d19ab Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Tue, 6 Dec 2016 19:32:39 -0500 Subject: submit --- chapter/1/gRPC.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index 99a81e0..96d4609 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -123,7 +123,7 @@ The officially supported languages are listed in Table 2 {% cite gRPC-Languages Table 2: Officially supported languages by gRPC.

-There are benchmarks being performed on a daily basis to ensure that gRPC performs optimally under high-throughput conditions as illustrated in Figure 6. +There are benchmarks being performed on a daily basis to ensure that gRPC performs optimally under high-throughput conditions as illustrated in Figure 6 {% cite gRPC-Benchmark %}.


@@ -152,3 +152,4 @@ The Java implementation of gRPC has been built with Mobile platform in mind and [Protobuf-Types]: https://developers.google.com/protocol-buffers/docs/encoding [gRPC-Overview]: http://www.grpc.io/docs/guides/ [gRPC-Languages]: http://www.grpc.io/about/#osp +[gRPC-Benchmark]: http://www.grpc.io/docs/guides/benchmarking.html \ No newline at end of file -- cgit v1.2.3 From 0598f53e1a16ab7731f081a77f15542192c28b07 Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Tue, 6 Dec 2016 21:25:40 -0500 Subject: submit --- chapter/1/gRPC.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index 96d4609..0dc78c3 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -130,10 +130,17 @@ There are benchmarks being performed on a daily basis to ensure that gRPC perfor Figure 6: Benchmark showing the queries-per-second on two virtual machines with 32 cores each.

+Most of the public Google APIs have been ported to support gRPC and their definitions can be analyzed at the following location: + +

+ https://github.com/googleapis/googleapis/tree/master/google
+

+ +

3.1 gRPC Java

-The Java implementation of gRPC has been built with Mobile platform in mind and requires support of JDK 6.0 and higher. There are +The Java implementation of gRPC been built with Mobile platform in mind and to provide that capability it requires JDK 6.0 to be supported. Though the core of gRPC is built for data centers specifically to support C/C++ for the Linux platform, the Java and Go implementations are two very reliable platform to experiment the microservice ecosystem implementations. ## References -- cgit v1.2.3 From c4a9f8305ae1a99721030b9d3009a64453e17121 Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Tue, 6 Dec 2016 21:32:44 -0500 Subject: submit --- chapter/1/gRPC.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index 0dc78c3..c3333c2 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -130,12 +130,12 @@ There are benchmarks being performed on a daily basis to ensure that gRPC perfor Figure 6: Benchmark showing the queries-per-second on two virtual machines with 32 cores each.

-Most of the public Google APIs have been ported to support gRPC and their definitions can be analyzed at the following location: +Most of the public Google APIs - including the Speech API, Vision API, BigTable, Pub/Sub, etc. - have been ported to support gRPC, and their definitions can be found at the following location:

- https://github.com/googleapis/googleapis/tree/master/google
-

- +
+ Figure 7: The public Google APIs have been updated for gRPC, and be found at https://github.com/googleapis/googleapis/tree/master/google +

3.1 gRPC Java

-- cgit v1.2.3 From 2d03731c4d9a4d2a0342e613cb8d7ad7e5fc3f7a Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Tue, 6 Dec 2016 21:35:14 -0500 Subject: submit --- chapter/1/gRPC.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index c3333c2..59090d0 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -130,7 +130,7 @@ There are benchmarks being performed on a daily basis to ensure that gRPC perfor Figure 6: Benchmark showing the queries-per-second on two virtual machines with 32 cores each.

-Most of the public Google APIs - including the Speech API, Vision API, BigTable, Pub/Sub, etc. - have been ported to support gRPC, and their definitions can be found at the following location: +Most of the public Google APIs - including the Speech API, Vision API, Bigtable, Pub/Sub, etc. - have been ported to support gRPC, and their definitions can be found at the following location:


@@ -140,13 +140,13 @@ Most of the public Google APIs - including the Speech API, Vision API, BigTable,

3.1 gRPC Java

-The Java implementation of gRPC been built with Mobile platform in mind and to provide that capability it requires JDK 6.0 to be supported. Though the core of gRPC is built for data centers specifically to support C/C++ for the Linux platform, the Java and Go implementations are two very reliable platform to experiment the microservice ecosystem implementations. +The Java implementation of gRPC been built with Mobile platform in mind and to provide that capability it requires JDK 6.0 to be supported. Though the core of gRPC is built with data centers in mind - specifically to support C/C++ for the Linux platform - the Java and Go implementations are two very reliable platform to experiment the microservice ecosystem implementations. ## References -[Apigee]: https://www.youtube.com/watch?v=-2sWDr3Z0Wo -[Authentication]: http://www.grpc.io/docs/guides/auth.html -[Benchmarks]: http://www.grpc.io/docs/guides/benchmarking.html +`[Apigee]: https://www.youtube.com/watch?v=-2sWDr3Z0Wo` [Apigee]: https://www.youtube.com/watch?v=-2sWDr3Z0Wo +`[Authentication]: http://www.grpc.io/docs/guides/auth.html` [Authentication]: http://www.grpc.io/docs/guides/auth.html +`[Benchmarks]: http://www.grpc.io/docs/guides/benchmarking.html` [Benchmarks]: http://www.grpc.io/docs/guides/benchmarking.html [CoreSurfaceAPIs]: https://github.com/grpc/grpc/tree/master/src/core [ErrorModel]: http://www.grpc.io/docs/guides/error.html [gRPC]: https://github.com/grpc/grpc/blob/master/doc/g_stands_for.md -- cgit v1.2.3 From fc4ac30854c08373e0608260b48265627fc382fe Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Tue, 6 Dec 2016 21:36:00 -0500 Subject: submit --- chapter/1/gRPC.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index 59090d0..0bb7ede 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -144,9 +144,9 @@ The Java implementation of gRPC been built with Mobile platform in mind and to p ## References -`[Apigee]: https://www.youtube.com/watch?v=-2sWDr3Z0Wo` [Apigee]: https://www.youtube.com/watch?v=-2sWDr3Z0Wo -`[Authentication]: http://www.grpc.io/docs/guides/auth.html` [Authentication]: http://www.grpc.io/docs/guides/auth.html -`[Benchmarks]: http://www.grpc.io/docs/guides/benchmarking.html` [Benchmarks]: http://www.grpc.io/docs/guides/benchmarking.html +`` [Apigee]: https://www.youtube.com/watch?v=-2sWDr3Z0Wo +`` [Authentication]: http://www.grpc.io/docs/guides/auth.html +`` [Benchmarks]: http://www.grpc.io/docs/guides/benchmarking.html [CoreSurfaceAPIs]: https://github.com/grpc/grpc/tree/master/src/core [ErrorModel]: http://www.grpc.io/docs/guides/error.html [gRPC]: https://github.com/grpc/grpc/blob/master/doc/g_stands_for.md -- cgit v1.2.3 From 7d22d28bce2beb45907663b8aff9e04aeae2cbbd Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Tue, 6 Dec 2016 21:36:34 -0500 Subject: submit --- chapter/1/gRPC.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index 0bb7ede..2afd338 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -144,9 +144,12 @@ The Java implementation of gRPC been built with Mobile platform in mind and to p ## References -`` [Apigee]: https://www.youtube.com/watch?v=-2sWDr3Z0Wo -`` [Authentication]: http://www.grpc.io/docs/guides/auth.html -`` [Benchmarks]: http://www.grpc.io/docs/guides/benchmarking.html +``[Apigee]: https://www.youtube.com/watch?v=-2sWDr3Z0Wo + +``[Authentication]: http://www.grpc.io/docs/guides/auth.html + +``[Benchmarks]: http://www.grpc.io/docs/guides/benchmarking.html + [CoreSurfaceAPIs]: https://github.com/grpc/grpc/tree/master/src/core [ErrorModel]: http://www.grpc.io/docs/guides/error.html [gRPC]: https://github.com/grpc/grpc/blob/master/doc/g_stands_for.md -- cgit v1.2.3 From 7694f5c94f38a35e457649f11d571818d6eec6df Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Tue, 6 Dec 2016 21:36:51 -0500 Subject: submit --- chapter/1/gRPC.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index 2afd338..ddac5d7 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -144,11 +144,11 @@ The Java implementation of gRPC been built with Mobile platform in mind and to p ## References -``[Apigee]: https://www.youtube.com/watch?v=-2sWDr3Z0Wo +`[Apigee]: https://www.youtube.com/watch?v=-2sWDr3Z0Wo -``[Authentication]: http://www.grpc.io/docs/guides/auth.html +`[Authentication]: http://www.grpc.io/docs/guides/auth.html -``[Benchmarks]: http://www.grpc.io/docs/guides/benchmarking.html +`[Benchmarks]: http://www.grpc.io/docs/guides/benchmarking.html [CoreSurfaceAPIs]: https://github.com/grpc/grpc/tree/master/src/core [ErrorModel]: http://www.grpc.io/docs/guides/error.html -- cgit v1.2.3 From 3027b93e08cf101f078b5a1604b302fa6d1ccee5 Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Tue, 6 Dec 2016 21:37:17 -0500 Subject: submit --- chapter/1/gRPC.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index ddac5d7..20a0ce0 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -144,11 +144,11 @@ The Java implementation of gRPC been built with Mobile platform in mind and to p ## References -`[Apigee]: https://www.youtube.com/watch?v=-2sWDr3Z0Wo +` `[Apigee]: https://www.youtube.com/watch?v=-2sWDr3Z0Wo -`[Authentication]: http://www.grpc.io/docs/guides/auth.html +` `[Authentication]: http://www.grpc.io/docs/guides/auth.html -`[Benchmarks]: http://www.grpc.io/docs/guides/benchmarking.html +` `[Benchmarks]: http://www.grpc.io/docs/guides/benchmarking.html [CoreSurfaceAPIs]: https://github.com/grpc/grpc/tree/master/src/core [ErrorModel]: http://www.grpc.io/docs/guides/error.html -- cgit v1.2.3 From e0f2b97b972a58fff80bf0989729f1aa97bc7fba Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Tue, 6 Dec 2016 21:38:29 -0500 Subject: submit --- chapter/1/gRPC.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index 20a0ce0..e4eed89 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -134,7 +134,7 @@ Most of the public Google APIs - including the Speech API, Vision API, Bigtable,


- Figure 7: The public Google APIs have been updated for gRPC, and be found at https://github.com/googleapis/googleapis/tree/master/google + Figure 7: The public Google APIs have been updated for gRPC, and be found at https://github.com/googleapis/googleapis/tree/master/google

@@ -150,7 +150,7 @@ The Java implementation of gRPC been built with Mobile platform in mind and to p ` `[Benchmarks]: http://www.grpc.io/docs/guides/benchmarking.html -[CoreSurfaceAPIs]: https://github.com/grpc/grpc/tree/master/src/core +` `[CoreSurfaceAPIs]: https://github.com/grpc/grpc/tree/master/src/core [ErrorModel]: http://www.grpc.io/docs/guides/error.html [gRPC]: https://github.com/grpc/grpc/blob/master/doc/g_stands_for.md [gRPC-Companies]: http://www.grpc.io/about/ -- cgit v1.2.3 From bf05b2a7a974fc7af352f7a2e68f08cc610353ac Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Tue, 6 Dec 2016 21:38:48 -0500 Subject: submit --- chapter/1/gRPC.md | 3 --- 1 file changed, 3 deletions(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index e4eed89..af26980 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -145,11 +145,8 @@ The Java implementation of gRPC been built with Mobile platform in mind and to p ## References ` `[Apigee]: https://www.youtube.com/watch?v=-2sWDr3Z0Wo - ` `[Authentication]: http://www.grpc.io/docs/guides/auth.html - ` `[Benchmarks]: http://www.grpc.io/docs/guides/benchmarking.html - ` `[CoreSurfaceAPIs]: https://github.com/grpc/grpc/tree/master/src/core [ErrorModel]: http://www.grpc.io/docs/guides/error.html [gRPC]: https://github.com/grpc/grpc/blob/master/doc/g_stands_for.md -- cgit v1.2.3 From 6bf06870225d78ea9457ae87840bae822b50d893 Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Tue, 6 Dec 2016 21:39:43 -0500 Subject: submit --- chapter/1/gRPC.md | 40 ++++++++++++++++++++++++++++------------ 1 file changed, 28 insertions(+), 12 deletions(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index af26980..20028ea 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -145,18 +145,34 @@ The Java implementation of gRPC been built with Mobile platform in mind and to p ## References ` `[Apigee]: https://www.youtube.com/watch?v=-2sWDr3Z0Wo + ` `[Authentication]: http://www.grpc.io/docs/guides/auth.html + ` `[Benchmarks]: http://www.grpc.io/docs/guides/benchmarking.html + ` `[CoreSurfaceAPIs]: https://github.com/grpc/grpc/tree/master/src/core -[ErrorModel]: http://www.grpc.io/docs/guides/error.html -[gRPC]: https://github.com/grpc/grpc/blob/master/doc/g_stands_for.md -[gRPC-Companies]: http://www.grpc.io/about/ -[gRPC-Languages]: http://www.grpc.io/docs/ -[gRPC-Protos]: https://github.com/googleapis/googleapis/ -[Netty]: http://netty.io/ -[RFC7540]: http://httpwg.org/specs/rfc7540.html -[HelloWorldProto]: https://github.com/grpc/grpc/blob/master/examples/protos/helloworld.proto -[Protobuf-Types]: https://developers.google.com/protocol-buffers/docs/encoding -[gRPC-Overview]: http://www.grpc.io/docs/guides/ -[gRPC-Languages]: http://www.grpc.io/about/#osp -[gRPC-Benchmark]: http://www.grpc.io/docs/guides/benchmarking.html \ No newline at end of file + +` `[ErrorModel]: http://www.grpc.io/docs/guides/error.html + +` `[gRPC]: https://github.com/grpc/grpc/blob/master/doc/g_stands_for.md + +` `[gRPC-Companies]: http://www.grpc.io/about/ + +` `[gRPC-Languages]: http://www.grpc.io/docs/ + +` `[gRPC-Protos]: https://github.com/googleapis/googleapis/ + +` `[Netty]: http://netty.io/ + +` `[RFC7540]: http://httpwg.org/specs/rfc7540.html + +` `[HelloWorldProto]: https://github.com/grpc/grpc/blob/master/examples/protos/ +helloworld.proto + +` `[Protobuf-Types]: https://developers.google.com/protocol-buffers/docs/encoding + +` `[gRPC-Overview]: http://www.grpc.io/docs/guides/ + +` `[gRPC-Languages]: http://www.grpc.io/about/#osp + +` `[gRPC-Benchmark]: http://www.grpc.io/docs/guides/benchmarking.html -- cgit v1.2.3 From 880349fca82d4504dde53d0dd706238b803046d9 Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Tue, 6 Dec 2016 21:46:21 -0500 Subject: submit --- chapter/1/gRPC.md | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index 20028ea..0063958 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -137,11 +137,28 @@ Most of the public Google APIs - including the Speech API, Vision API, Bigtable, Figure 7: The public Google APIs have been updated for gRPC, and be found at https://github.com/googleapis/googleapis/tree/master/google

+

3.2 Authentication

-

3.1 gRPC Java

+There are two methods of authentication that are available in gRPC: + +* SSL/TLS +* Google Token (via OAuth2) + +gRPC is flexible in that once can implement their custom authentication if that is preferred. + +

3.3 gRPC Java

The Java implementation of gRPC been built with Mobile platform in mind and to provide that capability it requires JDK 6.0 to be supported. Though the core of gRPC is built with data centers in mind - specifically to support C/C++ for the Linux platform - the Java and Go implementations are two very reliable platform to experiment the microservice ecosystem implementations. +

3... Downloading gRPC Java

+ +The easiest way to download the gRPC-Java implemenation is by performing the following command: + + +

3... Hello World Demonstration

+ + + ## References ` `[Apigee]: https://www.youtube.com/watch?v=-2sWDr3Z0Wo -- cgit v1.2.3 From 0125b8bba4eea09ec018db6dfcb529ac71e1f6e2 Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Tue, 6 Dec 2016 21:47:59 -0500 Subject: submit --- chapter/1/gRPC.md | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index 0063958..5c13c98 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -116,13 +116,6 @@ gRPC was built on top of HTTP/2, and we will cover the specifics of gRPC-Java, b Figure 5: gRPC allows for asynchronous language-agnostic message passing via Protocol Buffers.

-The officially supported languages are listed in Table 2 {% cite gRPC-Languages %}. - -

-
- Table 2: Officially supported languages by gRPC. -

- There are benchmarks being performed on a daily basis to ensure that gRPC performs optimally under high-throughput conditions as illustrated in Figure 6 {% cite gRPC-Benchmark %}.

@@ -137,6 +130,16 @@ Most of the public Google APIs - including the Speech API, Vision API, Bigtable, Figure 7: The public Google APIs have been updated for gRPC, and be found at https://github.com/googleapis/googleapis/tree/master/google

+ +

3.1 Supported Languages

+ +The officially supported languages are listed in Table 2 {% cite gRPC-Languages %}. + +

+
+ Table 2: Officially supported languages by gRPC. +

+

3.2 Authentication

There are two methods of authentication that are available in gRPC: @@ -146,6 +149,8 @@ There are two methods of authentication that are available in gRPC: gRPC is flexible in that once can implement their custom authentication if that is preferred. + +

3.3 gRPC Java

The Java implementation of gRPC been built with Mobile platform in mind and to provide that capability it requires JDK 6.0 to be supported. Though the core of gRPC is built with data centers in mind - specifically to support C/C++ for the Linux platform - the Java and Go implementations are two very reliable platform to experiment the microservice ecosystem implementations. -- cgit v1.2.3 From 507b194bd0e1222787127bc902ac022f435784f3 Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Tue, 6 Dec 2016 21:49:20 -0500 Subject: submit --- chapter/1/gRPC.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index 5c13c98..4b0c04b 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -116,14 +116,14 @@ gRPC was built on top of HTTP/2, and we will cover the specifics of gRPC-Java, b Figure 5: gRPC allows for asynchronous language-agnostic message passing via Protocol Buffers.

-There are benchmarks being performed on a daily basis to ensure that gRPC performs optimally under high-throughput conditions as illustrated in Figure 6 {% cite gRPC-Benchmark %}. +To ensure scalability, benchmarks are run on a daily basis to ensure that gRPC performs optimally under high-throughput conditions as illustrated in Figure 6 {% cite gRPC-Benchmark %}.


Figure 6: Benchmark showing the queries-per-second on two virtual machines with 32 cores each.

-Most of the public Google APIs - including the Speech API, Vision API, Bigtable, Pub/Sub, etc. - have been ported to support gRPC, and their definitions can be found at the following location: +To standardize, most of the public Google APIs - including the Speech API, Vision API, Bigtable, Pub/Sub, etc. - have been ported to support gRPC, and their definitions can be found at the following location:


-- cgit v1.2.3 From b42f4f4f4108ded7f89d80bed9a0c72fdc4ac013 Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Tue, 6 Dec 2016 22:04:40 -0500 Subject: submit --- chapter/1/gRPC.md | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index 4b0c04b..d4f1601 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -147,9 +147,21 @@ There are two methods of authentication that are available in gRPC: * SSL/TLS * Google Token (via OAuth2) -gRPC is flexible in that once can implement their custom authentication if that is preferred. +gRPC is flexible in that once can plug in their custom authentication system if that is preferred. +

3.3 gRPC Mechanism

+In its simplest form gRPC has a structured set of steps one goes about using it, which has this general flow: + +1. Download gRPC for the language of interest. + +2. Implement the Request and Response definition in a ProtoBuf file. + +3. Compile the ProtoBuf file and run the code-generators for the the specific language. This will generate the Client and Server endpoints. + +4. Customize the Client and Server code for the desired implementation. + +Most of these will require tweaking the Protobuf file and testing the throughput to ensure that the network and CPU capacities are optimally maximized.

3.3 gRPC Java

-- cgit v1.2.3 From f1689419b0626987a53ca9c3e178d47b0067abf3 Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Tue, 6 Dec 2016 22:07:24 -0500 Subject: submit --- chapter/1/gRPC.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index d4f1601..4d06e75 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -149,7 +149,7 @@ There are two methods of authentication that are available in gRPC: gRPC is flexible in that once can plug in their custom authentication system if that is preferred. -

3.3 gRPC Mechanism

+

3.3 Development Cycle

In its simplest form gRPC has a structured set of steps one goes about using it, which has this general flow: -- cgit v1.2.3 From 46d75a8eb88b7edbb30a14ba8834f6b4bc2ed68a Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Tue, 6 Dec 2016 22:23:26 -0500 Subject: submit --- chapter/1/gRPC.md | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index 4d06e75..08962b1 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -163,7 +163,18 @@ In its simplest form gRPC has a structured set of steps one goes about using it, Most of these will require tweaking the Protobuf file and testing the throughput to ensure that the network and CPU capacities are optimally maximized. -

3.3 gRPC Java

+

3.4 The gRPC Framework (Stub, Channel and Transport Layer)

+ +One starts by initializing a communication Channel between Client to a Server and storing that as a Stub. The Credentials are provided to the Channel when being initialized. These form a Context for the Client's connection to the Server. Then a Request can be built based on the definition in the Protobuf file. The Request and associated expectedResponse is executed by the service constructed in the Protobuf file. The Response is them parsed for any data coming from the Channel. + +The connection can be asynchronous and bi-directionally streaming so that data is constantly flowing back and available to be read when ready. This allows one to treat the Client and Server as endpoints where one can even adjust not just the flow but also intercept to filter and thus request the data of interest. + +That stub can be referenced later in order + + +The Java implementation of gRPC been built with Mobile platform in mind and to + +

3... gRPC Java

The Java implementation of gRPC been built with Mobile platform in mind and to provide that capability it requires JDK 6.0 to be supported. Though the core of gRPC is built with data centers in mind - specifically to support C/C++ for the Linux platform - the Java and Go implementations are two very reliable platform to experiment the microservice ecosystem implementations. -- cgit v1.2.3 From 3e366732f47c850dd9734dfabfada204e9bef7be Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Tue, 6 Dec 2016 22:28:38 -0500 Subject: submit --- chapter/1/gRPC.md | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index 08962b1..7d9cd34 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -167,12 +167,9 @@ Most of these will require tweaking the Protobuf file and testing the throughput One starts by initializing a communication Channel between Client to a Server and storing that as a Stub. The Credentials are provided to the Channel when being initialized. These form a Context for the Client's connection to the Server. Then a Request can be built based on the definition in the Protobuf file. The Request and associated expectedResponse is executed by the service constructed in the Protobuf file. The Response is them parsed for any data coming from the Channel. -The connection can be asynchronous and bi-directionally streaming so that data is constantly flowing back and available to be read when ready. This allows one to treat the Client and Server as endpoints where one can even adjust not just the flow but also intercept to filter and thus request the data of interest. +The connection can be asynchronous and bi-directionally streaming so that data is constantly flowing back and available to be read when ready. This allows one to treat the Client and Server as endpoints where one can even adjust not just the flow but also intercept and decoration to filter and thus request and retrieve the data of interest. -That stub can be referenced later in order - - -The Java implementation of gRPC been built with Mobile platform in mind and to +The Transport Layer performs the retrieval and placing of binary protocol on the wire. For gRPC-Java has three implementations, though a user can implement their own: Netty, OkHttp, and inProcess.

3... gRPC Java

-- cgit v1.2.3 From c776ce785f3a35059b6da451cec4d6aac943f185 Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Wed, 7 Dec 2016 01:08:55 -0500 Subject: submit --- chapter/1/gRPC.md | 27 ++++++++++++++++++++++----- 1 file changed, 22 insertions(+), 5 deletions(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index 7d9cd34..76a285b 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -80,7 +80,7 @@ One will notice that there is a number associated with each field element in the A 2 48 69 ``` -Thus the language had to be updated to support gRPC and the development of a service message with a request and a response definition was added for version version 3.0 of Protocol Buffers. The updated implementation would look as follows {% cite HelloWorldProto %}: +Thus the language had to be updated to support gRPC and the development of a service message with a request and a response definition was added for version version 3.0.0 of Protocol Buffers. The updated implementation would look as follows {% cite HelloWorldProto %}: ``` // The request message containing the user's name. @@ -100,7 +100,7 @@ service Greeter { } ```

- Figure 4: Protocol Buffer version 3.0 representing a message data-structure with the accompanied RPC definition. + Figure 4: Protocol Buffer version 3.0.0 representing a message data-structure with the accompanied RPC definition.

Notice the addition of a service, where the RPC call would use one of the messages as the structure of a Request with the other being the Response message format. @@ -171,13 +171,30 @@ The connection can be asynchronous and bi-directionally streaming so that data i The Transport Layer performs the retrieval and placing of binary protocol on the wire. For gRPC-Java has three implementations, though a user can implement their own: Netty, OkHttp, and inProcess. -

3... gRPC Java

+

3.5 gRPC Java

The Java implementation of gRPC been built with Mobile platform in mind and to provide that capability it requires JDK 6.0 to be supported. Though the core of gRPC is built with data centers in mind - specifically to support C/C++ for the Linux platform - the Java and Go implementations are two very reliable platform to experiment the microservice ecosystem implementations. -

3... Downloading gRPC Java

+

3.5.1 Downloading gRPC Java

-The easiest way to download the gRPC-Java implemenation is by performing the following command: +The easiest way to download the gRPC-Java implementation is by performing the following command: + +``` +git clone -b v1.0.0 https://github.com/grpc/grpc-java.git +``` + +Next compile on a Windows machine using Gradle using the following steps - and if you are using any Firewall software it might be necessary to temporarily disable it while compiling gRPC-Java as sockets are used for the tests: + +``` +cd grpc-java +set GRADLE_OPTS=-Xmx2048m +set JAVA_OPTS=-Xmx2048m +set DEFAULT_JVM_OPTS="-Dfile.encoding=utf-8" +echo skipCodegen=true > gradle.properties +gradlew.bat build -x test +cd examples +gradlew.bat installDist +```

3... Hello World Demonstration

-- cgit v1.2.3 From b8eb0b556606b2ef1ad98771382f99e7df6c20b4 Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Wed, 7 Dec 2016 01:10:03 -0500 Subject: submit --- chapter/1/gRPC.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index 76a285b..ecdaa73 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -196,8 +196,15 @@ cd examples gradlew.bat installDist ``` +If you are having issues with Unicode translation of Git on Windows you can try the following commands after entering the `examples` folder: -

3... Hello World Demonstration

+``` +https://raw.githubusercontent.com/benelot/grpc-java/feb88a96a4bc689631baec11abe989a776230b74/examples/src/main/java/io/grpc/examples/routeguide/RouteGuideServer.java + +copy RouteGuideServer.java src\main\java\io\grpc\examples\routeguide\RouteGuideServer.java +``` + +

3 Hello World Demonstration

-- cgit v1.2.3 From 255d5413ffdb8aea348b3d143377a51755f6b701 Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Wed, 7 Dec 2016 01:21:07 -0500 Subject: submit --- chapter/1/gRPC.md | 28 ++++++++++++++++++++++++++-- 1 file changed, 26 insertions(+), 2 deletions(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index ecdaa73..d39a4f6 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -199,14 +199,38 @@ gradlew.bat installDist If you are having issues with Unicode translation of Git on Windows you can try the following commands after entering the `examples` folder: ``` -https://raw.githubusercontent.com/benelot/grpc-java/feb88a96a4bc689631baec11abe989a776230b74/examples/src/main/java/io/grpc/examples/routeguide/RouteGuideServer.java +wget https://raw.githubusercontent.com/benelot/grpc-java/feb88a96a4bc689631baec11abe989a776230b74/examples/src/main/java/io/grpc/examples/routeguide/RouteGuideServer.java copy RouteGuideServer.java src\main\java\io\grpc\examples\routeguide\RouteGuideServer.java ``` -

3 Hello World Demonstration

+

3.5.2 Running the Hello World Demonstration

+Make sure you open two Command (Terminal) windows, each within the `grpc-java\examples\build\install\examples\bin` folder. In the first of the two windows type the following command: +``` +hello-world-server.bat +``` + +You should see the following: + +

+
+ Figure 8: The Hello World gRPC Server. +

+ +In the second of the two windows type the following command: + +``` +hello-world-client.bat +``` + +You should see the following response: + +

+
+ Figure 9: The Hello World gRPC Client and the response from the Server. +

## References -- cgit v1.2.3 From 7669eaecd48e61685361666d97edc7fbbdf56970 Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Wed, 7 Dec 2016 01:22:18 -0500 Subject: submit --- chapter/1/gRPC.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index d39a4f6..3bea7e1 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -196,7 +196,7 @@ cd examples gradlew.bat installDist ``` -If you are having issues with Unicode translation of Git on Windows you can try the following commands after entering the `examples` folder: +If you are having issues with Unicode (UTF-8) translation when using Git on Windows, you can try the following commands after entering the `examples` folder: ``` wget https://raw.githubusercontent.com/benelot/grpc-java/feb88a96a4bc689631baec11abe989a776230b74/examples/src/main/java/io/grpc/examples/routeguide/RouteGuideServer.java -- cgit v1.2.3 From 5cbc7c29567b3c893e4069f2a6bc0d5ad27b7489 Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Wed, 7 Dec 2016 01:22:51 -0500 Subject: submit --- chapter/1/gRPC.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index 3bea7e1..643edaa 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -216,7 +216,7 @@ You should see the following:


- Figure 8: The Hello World gRPC Server. + Figure 8: The Hello World gRPC Server.

In the second of the two windows type the following command: @@ -229,7 +229,7 @@ You should see the following response:


- Figure 9: The Hello World gRPC Client and the response from the Server. + Figure 9: The Hello World gRPC Client and the response from the Server.

## References -- cgit v1.2.3 From 63e77966fd35ce0cbc88a70615069122ba33a1d9 Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Wed, 7 Dec 2016 16:06:48 -0500 Subject: submit --- chapter/1/gRPC.md | 31 ++++++++++++++++++++++++++++--- 1 file changed, 28 insertions(+), 3 deletions(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index 643edaa..953cfdd 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -175,6 +175,31 @@ The Transport Layer performs the retrieval and placing of binary protoc The Java implementation of gRPC been built with Mobile platform in mind and to provide that capability it requires JDK 6.0 to be supported. Though the core of gRPC is built with data centers in mind - specifically to support C/C++ for the Linux platform - the Java and Go implementations are two very reliable platform to experiment the microservice ecosystem implementations. +There are several moving parts to understanding how gRPC-Java works. The first important step is to ensure that the Client and Server stub inferface code get generated by the Protobuf plugin compiler. This is usually placed in your Gradle build file called `build.gradle` as follows: + +``` + compile 'io.grpc:grpc-netty:1.0.1' + compile 'io.grpc:grpc-protobuf:1.0.1' + compile 'io.grpc:grpc-stub:1.0.1' +``` + +When you build using Gradle, then the appropriate base code gets generated for you, which you can override to build your preferred implementation of the Client and Server. + +Since one has to implement the HTTP/2 protocol, the chosen method was to have a Metadata class that will convert the key-value pairs into HTTP/2 Headers and vice-versa for the Netty implementation via GrpcHttp2HeadersDecoder and GrpcHttp2OutboundHeaders. + +Another key insight is to understand that the code that handles the HTTP/2 conversion for the Client and the Server are being done via the NettyClientHandler.java and NettyServerHandler.java classes shown in Figures 8 and 9. + +

+
+ Figure 8: The Client Tranport Handler for gRPC-Java. +

+ +

+
+ Figure 9: The Server Tranport Handler for gRPC-Java. +

+ +

3.5.1 Downloading gRPC Java

The easiest way to download the gRPC-Java implementation is by performing the following command: @@ -183,7 +208,7 @@ The easiest way to download the gRPC-Java implementation is by performing the fo git clone -b v1.0.0 https://github.com/grpc/grpc-java.git ``` -Next compile on a Windows machine using Gradle using the following steps - and if you are using any Firewall software it might be necessary to temporarily disable it while compiling gRPC-Java as sockets are used for the tests: +Next compile on a Windows machine using Gradle (or Maven) using the following steps - and if you are using any Firewall software it might be necessary to temporarily disable it while compiling gRPC-Java as sockets are used for the tests: ``` cd grpc-java @@ -216,7 +241,7 @@ You should see the following:


- Figure 8: The Hello World gRPC Server. + Figure 10: The Hello World gRPC Server.

In the second of the two windows type the following command: @@ -229,7 +254,7 @@ You should see the following response:


- Figure 9: The Hello World gRPC Client and the response from the Server. + Figure 10: The Hello World gRPC Client and the response from the Server.

## References -- cgit v1.2.3 From e25b8418bcd2297d0ecc6a350e9d4c11219c6c5c Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Wed, 7 Dec 2016 16:28:15 -0500 Subject: submit --- chapter/1/gRPC.md | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index 953cfdd..e61e432 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -49,6 +49,30 @@ As streams are core to the implementation of HTTP/2, it is important to discuss Figure 2: The lifecycle of a HTTP/2 stream.

+To better understand this diagram, it is important to define some of the terms in it: + +PUSH_PROMISE - This is being performed by one endpoint to alert another that it will be sending some data over the wire. + +RST_STREAM - This makes termination of a stream possible. + +PRIORITY - This is sent by an endpoint on the priority of a stream. + +END_STREAM - This flag denotes the end of a DATA frame. + +HEADERS - This frame will open a stream. + +Idle - This is a state that a stream can be in when it is opened by receiving a HEADERS frame. + +Reserved (Local) - To be in this state is means that one has sent a PUSH_PROMISE frame. + +Reserved (Remote) - To be in this state is means that it has been reserved by a remote endpoint. + +Open - To be in this state means that both endpoints can send frames. + +Closed - This is a terminal state. + +Half-Closed (Local) - This means that no frames can be sent except for WINDOW_UPDATE, PRIORITY, and RST_STREAM. +

1.4 Flow Control of Streams

Since many streams will compete for the bandwidth of a connection, in order to prevent bottlenecks and collisions in the transmission. This is done via the WINDOW_UPDATE payload for every stream - and the overall connection as well - to let the sender know how much room the receiving endpoint has for processing new data. -- cgit v1.2.3 From e8b2aa3c2fd421f407aa4ad5979cc33ce18e0b77 Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Wed, 7 Dec 2016 16:31:11 -0500 Subject: submit --- chapter/1/gRPC.md | 2 ++ 1 file changed, 2 insertions(+) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index e61e432..d645046 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -73,6 +73,8 @@ To better understand this diagram, it is important to define some of the terms i Half-Closed (Local) - This means that no frames can be sent except for WINDOW_UPDATE, PRIORITY, and RST_STREAM. +Half-Closed (Remote) - This means that a frame is not used by the remote endpoint to send frames of data. +

1.4 Flow Control of Streams

Since many streams will compete for the bandwidth of a connection, in order to prevent bottlenecks and collisions in the transmission. This is done via the WINDOW_UPDATE payload for every stream - and the overall connection as well - to let the sender know how much room the receiving endpoint has for processing new data. -- cgit v1.2.3 From aedd469d5ec714759ab0a7cf91c426631288be03 Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Wed, 7 Dec 2016 16:35:14 -0500 Subject: submit --- chapter/1/gRPC.md | 4 ++++ 1 file changed, 4 insertions(+) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index d645046..fceeaa6 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -283,6 +283,10 @@ You should see the following response: Figure 10: The Hello World gRPC Client and the response from the Server.

+

4 Conclusion

+ +This chapter presented an overview of the concepts behing gRPC, HTTP/2 and will be expanded in both breadth and language implementations. The area of microservices one can see how a server endpoint can actually spawn more endpoints where the message content is the protobuf definition for new endpoints to be generated for load-balancing like for the classical Actor Model. + ## References ` `[Apigee]: https://www.youtube.com/watch?v=-2sWDr3Z0Wo -- cgit v1.2.3 From 0e9470119ea56635ea096777f4656f75138b5ed1 Mon Sep 17 00:00:00 2001 From: Paul Grosu Date: Wed, 7 Dec 2016 16:36:41 -0500 Subject: submit --- chapter/1/gRPC.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'chapter/1/gRPC.md') diff --git a/chapter/1/gRPC.md b/chapter/1/gRPC.md index fceeaa6..f6c47b7 100644 --- a/chapter/1/gRPC.md +++ b/chapter/1/gRPC.md @@ -81,7 +81,7 @@ Since many streams will compete for the bandwidth of a connection, in order to p

2 Protocol Buffers with RPC

-Though gRPC was built on top of HTTP/2, an IDL had to be used to perform the communication between endpoints. The natural direction was to use Protocol Buffers is the method of stucturing data for serialization between a server and client. At the time of the start of gRPC development only version 2.0 (proto2) was available, which only implemented data structures without any request/response mechanism. An example of a Protocol Buffer data structure would look something like this: +Though gRPC was built on top of HTTP/2, an IDL had to be used to perform the communication between endpoints. The natural direction was to use Protocol Buffers is the method of stucturing key-value-based data for serialization between a server and client. At the time of the start of gRPC development only version 2.0 (proto2) was available, which only implemented data structures without any request/response mechanism. An example of a Protocol Buffer data structure would look something like this: ``` // A message containing the user's name. @@ -100,7 +100,7 @@ This message will also be encoded for highest compression when sent over the wir Table 1: Tag values for Protocol Buffer types.

-One will notice that there is a number associated with each field element in the Protocol Buffer definition, which represents its tag. In Figure 3, the field `name` has a tag of `1`. When a message gets encoded each field will start with a one byte value (8 bits), where the least-significant 3-bit value encode the type and the rest the tag. In this case tag which is `1`, with a type of 2. Thus the encoding will be `00001 010`, which has a hexdecimal value of `A`. The following byte is the length of the string which is `2`, followed by the string as `48` and `69` representing `H` and `i`. Thus the whole tranmission will look as follows: +One will notice that there is a number associated with each field element in the Protocol Buffer definition, which represents its tag. In Figure 3, the field `name` has a tag of `1`. When a message gets encoded each field (key) will start with a one byte value (8 bits), where the least-significant 3-bit value encode the type and the rest the tag. In this case tag which is `1`, with a type of 2. Thus the encoding will be `00001 010`, which has a hexdecimal value of `A`. The following byte is the length of the string which is `2`, followed by the string as `48` and `69` representing `H` and `i`. Thus the whole tranmission will look as follows: ``` A 2 48 69 -- cgit v1.2.3