Wednesday 29 October 2014

Google Protobuf - Get it working on Linux


Note: This article is posted while keeping in mind that the reader has already gone through the Proto file syntax i.e. what variable are used data types etc, last but not the least what protobuf is? If answer to this question is NO, Please visit: https://developers.google.com/protocol-buffers/

Getting Goggle Protobuf working is quite a heck so to make it easy I thought it would be worth sharing a quick read:

Steps to configure a sample maven java project in eclipse using protobuf in LINUX:
1. Install the protobuf compiler as below:
sudo apt-get install protobuf-compiler

2. Maven dependency that needs to be inserted into the pom.xml
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>2.6.0</version>
</dependency>

3. First of all we have to create a .proto file and then compile it using the protobuf compiler which results in .java class:
Syntax:  
protoc --java_out=<path where you want your respective java files to be created> <the actual .proto file's path>
Example: 
protoc --java_out=src/com/piyush/demo src/com/piyush/demo/addressbook.proto
protoc --java_out=src/ src/com/piyush/demo/my_object.proto

4. Earlier step can be automated using "maven-antrun-plugin", however get this running is quite a heck so please refer the below thread for this:
Links:
We can use maven-antrun-plugin in following ways:
  • For Single proto file compilation:
<echo>Generate</echo>
<exec executable="protoc">
<arg value="--java_out=src/" />
<arg value="src/com/piyush/demo/addressbook.proto" />
</exec>
  • For Multiple proto files compilation:
        <tasks>
                <echo>Generate</echo>
                    <path id="proto.path">
                        <fileset dir="src/com/piyush/demo/proto">
                            <include name="**/*.proto" />
                            </fileset>
                       </path>
                        <pathconvert pathsep=" " property="proto.files" refid="proto.path" />
                        <exec executable="protoc" failonerror="true">
                            <arg value="--java_out=src/" />
                            <arg value="-I${project.basedir}/src/com/piyush/demo/proto" />
                            <arg line="${proto.files}" />
                        </exec>
        </tasks>


Tips & Tricks:
1. Message name should be in camel case.

2. Beware that the name of the message and java_outer_classname should not be same in the proto file. For Example below proto file won't compile:
option java_package = "com.xyz.zrtb.simulator.protos";
option java_outer_classname = "A";
message a {
   optional string id = 1;
   optional string name = 2;
   repeated string cat = 4;
   optional string domain = 3;
}
Below will compile fine:
option java_package = "com.xyz.zrtb.simulator.protos";
option java_outer_classname = "OuterA";
message A {
   optional string id = 1;
   optional string name = 2;
   repeated string cat = 4;
   optional string domain = 3;
}


3. Importing proto file in one another is quite a heck and so it is good if you get it right at the very first place:
Import relies on the on the parameter '--proto_path' so If you've a package com.abc.pqr.model and defined '--proto_path' as below:
--proto_path=${project.basedir}/src/main/java/com/abc/pqr/model
THEN you can import just by writing xxx.proto However If you've defined '--proto_path' as below:
--proto_path=${project.basedir}/
THEN you can import by writing src/main/java/com/abc/pqr/model/xxx.proto and many such combinations.

4. The maven-antrun-plugin is problematic and can be a blocker, so below is the ready to use pom.xml configuration:
<build>
     <pluginManagement>
         <plugins>
            <!-- force a dependency on JDK 5.0 -->
            <plugin>
               <groupId>org.apache.maven.plugins</groupId>
               <artifactId>maven-compiler-plugin</artifactId>
               <configuration>
                  <source>1.7</source>
                  <target>1.7</target>
               </configuration>
            </plugin>
            <!--This plugin's configuration is used to store Eclipse m2e settings only. It has no influence on the Maven build
          itself. -->
            <plugin>
               <groupId>org.eclipse.m2e</groupId>
               <artifactId>lifecycle-mapping</artifactId>
               <version>1.0.0</version>
               <configuration>
                  <lifecycleMappingMetadata>
                     <pluginExecutions>
                        <pluginExecution>
                           <pluginExecutionFilter>
                              <groupId>org.apache.maven.plugins</groupId>
                              <artifactId>maven-antrun-plugin</artifactId>
                              <versionRange>[1.7,)</versionRange>
                              <goals>
                                 <goal>run</goal>
                              </goals>
                           </pluginExecutionFilter>
                           <action>
                              <ignore />
                           </action>
                        </pluginExecution>
                     </pluginExecutions>
                  </lifecycleMappingMetadata>
               </configuration>
            </plugin>
         </plugins>
        <pluginManagement>
      <plugins>
         <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-antrun-plugin</artifactId>
            <version>1.7</version>
            <executions>
               <execution>
                  <id>generate-sources</id>
                  <phase>generate-sources</phase>
                  <configuration>
                     <target>
                        <echo>Generate</echo>
                        <path id="proto.path">
                           <fileset dir="src/main/java/com/piyush/zrtb/simulator/proto">
                              <include name="**/*.proto" />
                           </fileset>
                        </path>
                        <pathconvert pathsep=" " property="proto.files" refid="proto.path" />
                        <exec executable="protoc" failonerror="true">
                           <arg value="--java_out=src/main/java" />
                           <arg value="-I${project.basedir}/src/main/java/" />
                           <arg line="${proto.files}" />
                        </exec>
                     </target>
                     <sourceRoot>src/main/java</sourceRoot>
                  </configuration>
                  <goals>
                     <goal>run</goal>
                  </goals>
               </execution>
            </executions>
         </plugin>
      </plugins>
      <extensions />
   </build>

Links to go through:
  • https://code.google.com/p/protoclipse/
  • http://www.siafoo.net/user/stou/blog/2010/01/29/Protocol-Buffers-and-Eclipse
  • http://www.masterzen.fr/2011/12/25/protobuf-maven-m2e-and-eclipse-are-on-a-boat/
  • http://techtraits.com/noproto/
  • http://xmeblog.blogspot.in/2013/12/sending-protobuf-serialized-data-using.html
  • http://sleeplessinslc.blogspot.in/2010/03/restful-representation-with-google.html
  • http://tutorials.jenkov.com/maven/maven-tutorial.html
  • https://developers.google.com/protocol-buffers/docs/proto#generating

No comments:

Post a Comment