Social Icons

Pages

Featured Posts

Thursday, November 6, 2014

Gerrit Code Review: Installation, Configuration & Usage

Recently I had to build a new development team within a large company. As part of it, had to implement best practices and processes for the development team. The challenge was to find tools, frameworks, and processes that are critical to the productivity and agility of the team as well as to work with the existing infrastructure, frameworks and tools within the organization. During the course of this, I had to select a code review tool and evaluated a number of them:
  1. Crucible
  2. Review Board
  3. Phabricator
  4. Gerrit
Each of one has their own merits and pitfalls. Here I will explain a quick setup and configuration of Gerrit as I had to go through a number of sites and documentation to figure out how to get this done. Gerrit would like to be the owner of your Git repository. If this can be implemented, everything will work great. But in our case, we have a centrally hosted Git repository and all the checkins should be available in this central repo. The solution to this problem is to setup Gerrit Replication. This is not a great idea. If people are directly pushing to the master repository, AND Gerrit is pushing, this will mess up the code base. The solution is to have one-way pull and push to different repositories as described below:
Here are the steps involved in getting this work.

Install Gerrit

Install the server with all the defaults
java -jar gerrit.war init -d ~/gerrit-server

Setup User

By default, Gerrit will be installed with OpenID. Login into Gerrit UI using any authentication (e.g. Google). The first user to login will be added to the Administrator group.
  1. Create SSH Keys
  2. Login to Gerrit UI and copy the key generated in the previous step (e.g. id_rsa.pub) into your Account Setup
  3. Setup a username to connect to Git Server hosted by the Gerrit (e.g. gerrit-user)
  4. Test connectivity from your local client to Gerrit Server
    ssh gerrit-user@gerrit-server -p 29418
    
    This will give a nice welcome message and inform that interactive shells are disabled

Import and existing project

To import an existing project, create a new project in Gerrit Server and then run import.
  1. Create a new project called gerrit-test
    ssh -p 29418 gerrit-user@gerrit-server gerrit create-project --name gerrit-test
    
  2. Create access to gerrit-user to newly created project gerrit-test
    1. Log into Gerrit UI and create a new user group called gerrit-test-group
    2. Go to Projects->List->gerrit-test->Access
    3. Grant access to refs/* for Create Reference permission to user group gerrit-test-group
  3. Import project
    cd dev/gerrit-test
    push ssh://gerrit-user@gerrit-server:29418/gerrit-test *:*
    

Setup local Git Repo for Gerrit

Install Gerrit hooks
cd dev/gerrit-test
scp -p -P 29418 gerrit-user@gerrit-server:hooks/commit-msg .git/hooks

Setup Gerrit server replication to push changes to central repository

  1. Setup the local ssh keys to connect to the Git server
  2. Setup the key for git host
    git-server> ssh-keygen -t rsa git-host >> ~/.ssh/known_hosts
    
  3. Add a new file called replication.config in the gerrit/etc directory with the following contents
    [remote "central-git-repo"]
      url = git@git-host:/{name}.git
    
  4. Shutdown and restart the Gerrit Server

Test

  1. Make some changes in to a local file
  2. Commit the changes
  3. Push the changes to Gerrit
    git push ssh://gerrit-user@gerrit-server:29418/gerrit-test HEAD:refs/for/master
    
  4. Login to Gerrit UI
  5. Approve the changes with +2
  6. Gerrit will push the changes to the master git repository. Verify the latest changes are indeed replicated by Gerrit Server

Thursday, October 30, 2014

Identifying duplicate rows in a table

select column-name from table group by column-name having count(*) > 1;

Thursday, October 23, 2014

JAX-RS 2.0 and HTTP PATCH

When designing RESTful services, one of the scenario comes across is partial updates. Typical REST operations use the whole resource. Normally updates use PUT to modify the resource. More often, you need to do a GET before you modify the resource. This approach has a number of side effects. Increased chatter with the server, bandwidth issues, performance etc. If the resource is really big and you are only updating a few attributes, PUT is not a good idea.

Enter HTTP Patch. This is meant to address this, partial updates of resources.

A sample patch request

RFC 6902 defines a JSON document format for patching
{"op" : "test",    "path" : "/firstName", "value" : "John"},
{"op" : "replace", "path" : "/zipcode",   "value" : "94555"},
{"op" : "add",     "path" : "/phone",     "value" : "888-999-1234"},
{"op" : "remove",  "path" : "/hobby"},
{"op" : "copy", "from"    : "/primayAddress", "to" : "/secondaryAddress"},
{"op" : "move", "from"    : "/mobilePhone",   "to" : "/primaryPhone"}
In this post, I discuss a generic framework to patch a single resource.

Environment

  1. JAX-RS 2.0
  2. Jackson JSON Processor
  3. json-patch (https://github.com/fge/json-patch) implementing RFC6902

Model

public class User {
    private long id;
    private String username;
    private String firstName;
    private String phone;
    private String primaryAddress;
    private String secondaryAddress;
    private String mobilePhone;
    private String primaryPhone;
    private String hobby;

   // .. boiler plate code ..
}

Patch Framework

Steps involved in applying patch for a single resource:
  1. Define PATCH annotation
  2. Implement JAX-RS Resource
  3. Implement PATCH API
    • Read existing resource from back-end data store
    • Convert the resource to JSON
    • Use json-patch to apply the patch
    • Convert the patched JSON to Java class
    • Save the patched object to back-end data store

Implementation

@Target({ElementType.METHOD})
@Retention(RetentionPolicy.RUNTIME)
@HttpMethod("PATCH")
@Documented
@NameBinding
public @interface PATCH {
}

@Produces('application/json")
public class UserResource {
    @Inject private UserManager userManager;

    @PATCH
    @Path("/users/{username}")
    @Transactional
    public Response patch(@PathParam("username") String username, String json) {
        // get the user from DB
        User user = userManger.findByUsername(username);

         try {
             if (user != null) {
                 ObjectMapper mapper = new ObjectMapper();

                 // convert JSON string to a Java class
                 JsonPatch patch = mapper.readValue(json, JsonPatch.class);

                 // convert User to a JSON object
                 JsonNode userJson = mapper.valueToTree(user);

                 // apply patch
                 JsonNode patched = patch.apply(userJson);

                 // convert the patched object to User
                 user = mapper.readValue(patched.toString(), User.class);

                 // save the patched object
                 user = userManager.save(user);
             }
         }  catch (Exception ex) {
             throw new RuntimeException(ex);
         }
         return Response.ok().build();
    }
}


Monday, September 29, 2014

MySQL Import and Export

To import and export and entire database schema and data, follow these steps:
mysqldump -h host -u user -p --databases database-name > dump.sql # Export from a database
sed '/^\/\*\!50013 DEFINER/d' dump.sql > dump_0.sql # Fix the permission issues in case you have views
mysql -u user -p database-name < dump_0.sql         # Import to database

Tuesday, September 23, 2014

Spring Security and Ajax Session Expiry

One of the requirement in a recent project was to capture the Ajax session expiry and send a JSON response back to the client if the session was expired. There are a few different ways to handle this in Spring Security. Here is one approach

<bean id="ajaxRequestMatcher" class="com.mycompany.AjaxRequestMatcher"/>

  <bean id="loginEntryPoint"
        class="org.springframework.security.web.authentication.LoginUrlAuthenticationEntryPoint">
    <constructor-arg value="/login.jsp" />
  </bean>

  <bean id="ajaxEntryPoint"
        class="com.mycompany.AjaxAuthenticationEntryPoint" />
  
  <bean id="authenticationRequestCache"
        class="org.springframework.security.web.savedrequest.HttpSessionRequestCache">
    <property name="requestMatcher" ref="ajaxRequestMatcher" />
  </bean>

  <bean id="authenticationEntryPoint"
        class="org.springframework.security.web.authentication.DelegatingAuthenticationEntryPoint">
    <constructor-arg>
      <map>
        <entry key-ref="ajaxRequestMatcher" value-ref="ajaxEntryPoint" />
      </map>
    </constructor-arg>
    <property name="defaultEntryPoint" ref="loginEntryPoint" />
  </bean>

  <security:http entry-point-ref="authenticationEntryPoint">
    <security:intercept-url
        access="IS_AUTHENTICATED_REMEMBERED"
        pattern="/secure/**"/>

    <security:form-login
        login-page="/login.jsp"/>
    <security:logout/>

  </security:http>

Java Classes

public class AjaxRequestMatcher implements RequestMatcher {
    public boolean matches(HttpServletRequest request) {
        return "XMLHttpRequest".equals(request.getHeader("X-Requested-With"));
    }
}

public class AjaxAuthenticationEntryPoint extends LoginUrlAuthenticationEntryPoint {
    @Override
    public void commence(HttpServletRequest request,
                         HttpServletResponse response,
                         AuthenticationException authException)
        throws IOException, ServletException {    

        // form json string
        String json = ....

        response.setContentType(MediaType.APPLICATION_JSON_VALUE);
        response.setHeader("Cache-control", "no-cache");
        response.setStatus(HttpServletResponse.SC_FORBIDDEN);

        response.getWriter().print(json);
        response.getWriter().flush();
        response.getWriter().close();
    }
}