Wednesday, November 20, 2013

SQLDeveloper wont start on Mavericks

After upgrading to Mavericks, the SQLDeveloper version 3.2.20.09.87 wont start. The solution is to create a file named jdk containing one line /usr in the directory ~/.sqldeveloper.

Monday, November 18, 2013

Apache Load Balancing & Fail Over with mod_proxy

Apache mod_proxy and the extension mod_proxy_balancer can be used for load balancing and fail over. The following setup is used for two downstream Tomcat servers with a candidate web app deployed at the root.

httpd.conf (On Mac OS, Mavericks)

LoadModule proxy_module libexec/apache2/mod_proxy.so
LoadModule proxy_ajp_module libexec/apache2/mod_proxy_ajp.so
LoadModule proxy_balancer_module libexec/apache2/mod_proxy_balancer.so
ProxyRequests Off
<Proxy balancer://cluster>
    BalancerMember ajp://localhost:8009  loadfactor=1 ping=10
    BalancerMember ajp://localhost:18009 loadfactor=2 ping=10
lt;/Proxy>
ProxyPass          /status   !
ProxyPass          /balancer !
ProxyPass          /         balancer://cluster/
ProxyPassReverse   /         balancer://cluster/

ProxyStatus On
<Location /status>
    SetHandler server-status
    Order Deny,Allow
    Deny from all
    Allow from all
</Location>

<Location /balancer>
    SetHandler balancer-manager
    Order Deny,Allow
    Deny from all
    Allow from all
</Location>

Friday, November 15, 2013

Mac OS username not in sudoers file

Recently run into this error when trying to run a command with sudo and got this error:

'username' not in sudoers file

The solution is to log in as a user with admin privilege and edit the /etc/sudoers file. Add the user whom you want to provide sudo privilege into this file as a separate line. For example, add the last line and replace the username with the your username.
# User privilege specification
root    ALL=(ALL) ALL
%admin  ALL=(ALL) ALL
username ALL=(ALL) ALL ---> Added

Tuesday, October 22, 2013

Hibernate Row Level Security using Apache Shiro

Apache Shiro is a feature rich and simpler security framework that can be easily integrated with any enterprise level systems. It comes with a default implementation of permissions and it is much easier to implement any custom security framework on top of it. The features offered by Shiro are good enough for most of the security requirements.

This posting describes a simple way to implement row level security using Hibernate ORM and Apache Shiro in a multi-tenant environment.

Design Approach


  1. Each entity in the model that requires row level security should have a tenant id as part of the entity.
  2. Define the user level permissions using the Shiro permission syntax: entity-type:operation:instance-id
  3. Use hibernate filters to add additional predicates to the queries.
  4. Set the values of hibernate filters using the currently authenticated users permissions.

Notes


  1. Shiro do not have direct way of obtaining the current roles or permissions for the current subject. So this information is obtained using the Realm interface.
  2. Use caching to improve performance. Shiro supports out of the box support caching. Simplest way is to use EHCache.

The Model


The model contains two entities. A Company and a list of Employees. An authenticated user can only see Employees for the Company he/she has granted permission to.
@MappedSuperclass
@FilterDef(name = "rlsFilter",
           defaultCondition = "fk_company_id in (:companies)",
           parameters = {@ParamDef(name="companies", type="long")})
@Filter(name = "rlsFilter")
public abstract class SecuredEntity {
    @ManyToOne
    @JoinColumn(name = "fk_company_id", nullable=false)
    public Company getCompany() {
        return this.company;
    }
    public void setCompany(Company c) {
        this.company = c;
    }
    private Company company;
}

@Entity
@Table(name = "employee")
public class Employee extends SecuredEntity {
    @Id
    @GeneratedValue
    public long getId() {
        return this.id;
    }
    public void setId(long newId) {
        this.id = newId;
    }
    @Column(name = "email")
    public String getEmail() {
        return this.email;
    }
    public void setEmail(String neEmail) {
        this.email = newEmail;
    }
    @Column(name = "full_name")
    public String getFullname() {
        return this.fullname;
    }
    public void setFullname(String name) {
        this.fullname = name;
    }

    private long id;
    private String email;
    private String fullname;
}

@Entity
@Table(name = "company")
public class Company {
    @Id
    @GeneratedValue
    public long getId() {
        return this.id;
    }
    public void setId(long newId) {
        this.id = newId;
    }
    public String getName() {
        return this.name;
    }
    public void setName(String nm) {
        this.name = nm;
    }
    private long id;
    private String name;
} 

Permission Model


The permission model consists of a User representing a user in the system and a set of Permissions. The Permissions will follow the Shiro StringPermission syntax: resource-type:operation:instance-id. For example, company:*:1234 means permission for company with id 1234 for all operations.
@Entity
@Table(name = "user")
public class User {
    @Id
    @GeneratedValue
    public long getId() {
        return this.id;
    }
    public void setId(long newId) {
        this.id = newId;
    }
    @Column(name = "user_name")
    public String getUsername() {
        return this.username;
    }
    public void setUsername(String nm) {
        this.username = nm;
    }
    @Column(name = "user_pass")
    public String getPassword() {
        return this.password;
    }
    public void setPassword(String pwd) {
        this.password = pwd;
    }
    @ManyToMany
    @JoinTable(name = "user_permission",
               joinColumns = @JoinColumn(name = "fk_user_id"),
               inverseJoinColumn(name = "fk_permission_id"))
    public Set<Permission> getPermissions() {
        return this.permissions;
    }
    public void setPermission(Set<Permission> perms) {
        this.permissions = perms;
    }
    private long id;
    private String username;
    private String password;
    private Set<Permission> permissions = new HashSet<String>();
}

@Entity
@Table(name = "permission")
public class Permission {
    @Id
    @GeneratedValue
    public long getId() {
        return this.id;
    }
    public void setId(long newId) {
        this.id = newId;
    }
    public String getValue() {
        return this.value;
    }
    public void setValue(String val) {
        this.value = val;
    }
    private long id;
    private String value;
}

Shiro Plumbing


  1. Create a custom Realm class. Implement the authentication and authorization methods based on the permission model.
  2. Get the permissions for the currently logged in user
public class MyShiroRealm extends AuthorizingRealm {
    public boolean supports(AuthenticationToken token) {
        return token ! null
            &&  UsernamePasswordToken.class.isAssignableFrom(token.getClass());
    }
    public AuthorizationInfo doGetAuthorizationInfo(PrincipalCollection principals) {
        User user = (User) principals.getPrimaryPrincipal();

        SimpleAuthorizationInfo auth = new SimpleAuthorizationInfo();

        for (Permisison p : user.getPermissions()) {
            auth.addStringPermission(p.getPermission());
        }
    }
    public AuthenticationInfo getAuthenticationInfo(AuthenticationToke token) {
        UsernamePasswordToken authToken = (UsernamePasswordToken) token;

        User user = userDao.getUser(authToken.getUsername());
        if (user != null) {
            auth = new SimpleAuthenticationInfo(user, user.getPassword(),
                                                realmName);
        } else {
            throw new AccountException("Account not found for user: "
                                       + authToken.getUsername())
        }
        return auth;
    }
    public Collection<String> getPermissions() {
        Subject subject = SecurityUtils.getSubject();
        AuthorizationInfo auth = getAuthorizationInfo(subject.getPrincipals());

        return auth != null
            ?  auth.getStringPermissions()
            :  new ArrayList<String>();
    }

    private String realmName = "MyShiroRealm";
}

Hibernate Integration


Hibernate layer integrates the Shiro permissions using filters as explained above. This can be done when a new session is obtained. If you are using OpenSessionInViewFilter, then subclass it and enable the filter soon after the session is opened.

  1. To provide individual permissions, follow the syntax: company:*:123
  2. To provide permissions to all companies, define permission string as: company:*
  3. If the user has permission to ALL companies, then the filter will be disabled
  4. If user has no company permission, then the filter will be executed with a bogus company id of -2.
public class EmployeeDao {
    protected Session getSession() {
        Session session = sessionFactory.getCurrentSession();

        Collecton<Long> companies = getAuhorizedCompanies();
        if (!companies.isEmpty()) {
            if (companies.size() == 1
                &&  companies.iterator().next() == ALL_COMPANIES) {

                session.disableFilter("rlsFilter");
            } else {
                Filter filter = sesson.enableFilter("rlsFilter");
                filter.setParameterList("companies", companies);
            }
        } else {
            companies.add(NO_COMPANIES);
            Filter filter = sesson.enableFilter("rlsFilter");
            filter.setParameterList("companies", companies);
        }
        return session;
    }
    protected List<Long> getAuthorizedCompanies() {
        List<Long> companies = new ArrayList<Long>();

        List<String> permissions = getPermissions();
        for (String p : permissions) {
            String[] parts = s.split(":");

            if (parts != null  &&  parts.length > 0) {
                if (parts[0].equalsIgnoreCase("company")) {
                    if (parts.length == 3) {
                        if (parts[2].equals("*")) {
                            companies.add(new Long(ALL_COMPANIES));
                            break;

                        } else {
                            companies.add(Long.parseLong(parts[2]));
                        }

                    } else if (parts.length < 3) {
                        companies.add(new Long(ALL_COMPANIES));
                        break;
                    }
                }
            }
        }
        return companies;
    }
    protected List<String> getPermissions() {
        List<String> permissions = new ArrayList<String>();

        RealmSecurityManager mgr =
            (RealmSecurityManager) SecurityUtils.getSecurityManager();

        if (mgr != null) {
            for (Realm r : mgr.getRealms()) {
                Collection<String> list = ((MyShiroRealm) r).getPermissions();

                if (list != null) {
                    permisisons.addAll(list);
                }
            }
        }
        return permissions;
    }

    public List<Employee> getAllEmployees() {
        Session session = getSession();
        // do the hibernate query
    }
    private SessionFactory sessionFactory;

    private static final long ALL_COMPANIES = -1;
    private static final long NO_COMPANIES  = -2;
}

Testing


  1. Create some users
  2. Assign some permissions
  3. Execute a hibernate based query (eg: getAllEmployees)
  4. Watch the SQL query executed and see the in clause predicate

Saturday, September 14, 2013

Java/JSON Polymorphic Mapping Using Jackson

Jackson has a number of flexible ways to map Java objects to and from JSON. Some of them are as follows:

  1. @JsonProperty to change the name of the property output to JSON
  2. @JsonSerialize/@JsonDeserialize to change the type of the object or to provide custom serializer
  3. @JsonIgnore to ignore properties
  4. Mixins to control runtime annotations during serialization & deserialization

Here in this post, I focus on serialization and deserialization of polymorphic types using @JsonPropertyInfo to inject an extra property called type to distinguish between different object types.

Model

import com.fasterxml.jackson.annotation.*;

@JsonTypeInfo(use=JsonTypeInfo.Id.NAME,
              include=JsonTypeInfo.As.PROPERTY,
              property="type")
@JsonSubTypes({
        @JsonSubTypes.Type(value=Dog.class, name="dog"),
        @JsonSubTypes.Type(value=Cat.class, name="cat"),
    })
public abstract class Animal {
    public Animal() {
    }
    public Animal(String name) {
        this.name = name;
    }
    public String getName() {
        return this.name;
    }
    public void setName(String s) {
        this.name = s;
    }
    public Animal getParent() {
        return this.parent;
    }
    public void setParent(Animal p) {
        this.parent = p;
    }
    private String name;
    private Animal parent;
}


@JsonTypeName("dog")
public class Dog extends Animal {
    public Dog() {
    }
    public Dog(String name) {
        super(name);
    }
}

@JsonTypeName("cat")
public class Cat extends Animal {
    public Cat() {
    }
    public Cat(String name) {
        super(name);
    }
}

Driver


Now create a Zoo class holding the animals. And the driver to test the model
import java.util.ArrayList;
import java.util.List;

import com.fasterxml.jackson.databind.ObjectMapper;

public class Zoo {
    public static void main(String... args) throws Exception {
        Zoo zoo = createZoo();

        ObjectMapper mapper = new ObjectMapper();
        String json1 = mapper.writerWithDefaultPrettyPrinter().writeValueAsString(zoo);
        System.out.println("+++>>>" + json1);

        zoo = mapper.readValue(json1, Zoo.class);

        String json2 = mapper.writerWithDefaultPrettyPrinter().writeValueAsString(zoo);

        System.out.println("Equals: " + json1.equals(json2));
    }

    private static Zoo createZoo() {
        Zoo zoo = new Zoo();

        Dog dog1 = new Dog("Dog1");
        Dog dog2 = new Dog("Dog2");
        Cat cat1 = new Cat("Cat1");
        Cat cat2 = new Cat("Cat2");

        dog2.setParent(dog1);
        cat2.setParent(cat1);

        zoo.add(dog1);
        zoo.add(dog2);
        zoo.add(cat1);
        zoo.add(cat2);

        return zoo;
    }
    public void add(Animal animal) {
        animals.add(animal);
    }

    public void setAnimals(List<Animal> list) {
        this.animals = list;
    }
    public List<Animal> getAnimals() {
        return this.animals;
    }
    private List<Animal> animals = new ArrayList<Animal>();
}

Result

{
 "animals" : [ {
    "type" : "dog",
    "name" : "Dog1",
    "parent" : null
  }, {
    "type" : "dog",
    "name" : "Dog2",
    "parent" : {
      "type" : "dog",
      "name" : "Dog1",
      "parent" : null
    }
  }, {
    "type" : "cat",
    "name" : "Cat1",
    "parent" : null
  }, {
    "type" : "cat",
    "name" : "Cat2",
    "parent" : {
      "type" : "cat",
      "name" : "Cat1",
      "parent" : null
    }
  } ]
}

Equals: true

Thursday, June 7, 2012

Intel SS4200-E RAID Failure & Solution

For years, I have a RAID setup at home using Intel Entry Storage System SS4200-E. I have four 1TB disks setup in RAID 5 configuration. A couple of days ago, there was a power failure. When I rebooted the storage, initially it went on to re-build the arrays. But after more than 24 hours, the status on the web interface was still showing 0% rebuild complete. On top of it, I was not able to access the dashboard through the web interface.

So I powered down the storage, wait for 5 minutes or so for the disks to cool down and powered backup. My expectation was the array will start the rebuilt process over again. Instead, all the four lights on the front panel turned amber, meaning all four disks have failed!!. Once again, I cannot access the dashboard. I power down the system a couple more times, expecting the results could be different.

Unfortunately, there is nothing much I was able to do at this point. Then I noticed a warning in the disk status under settings in the web interface "disk has been replaced with a disk containing data from another system" against each of the four drives. So the system thinks that the disks have been swapped with and not assembling the RAID properly.

Digging further, I came across this thread http://communities.intel.com/message/32955 and decided to try it out.

  1. Login to the device using the web interface
  2. Bring up the hidden support.html page by manually pointing to the URL http://your-device-ip/support.html
  3. Turn on SSH access
  4. Restart the device
  5. Login to the device using SSH. The username was root and the password should be sohoYOUR-ADMIN-PASSWORD. For example, if your admin password is 1234, then the ssh password for root is soho1234
  6. Look for the process e2fsck by running the command ps -aef | grep -i e2fsck
  7. Kill this process. kill -9 pid-of-e2fsck-process
Suddenly, all the lights turned solid blue and there was no data loss. I was able to access all my data without any problems! It would have been such a nightmare if I lose years worth of data. Now I am a not re-booting the storage until I found out a permanent solution for this process kicking in during startup.

Update-1: Looks like the e2fsck process will run only when rebooted manually by pressing the power button. If rebooted cleanly using the web interface, this process is not found to be running. But the disks are still showing amber after any type of reboot. So the quick fix seems to be to reboot using the power button, kill the e2fsck process using SSH to get access to the disks and data. There should be some other way to fix this.

Update-2: After killing the process, ran the e2fsck command manually. e2fsck -f /dev/evms/md0vol1. Provide yes to all the questions asked. Once this check is complete, I was able to successfully reboot the system without any issues and the disks were not going to amber state.

References

  1. http://communities.intel.com/message/32955
  2. http://communities.intel.com/thread/26862
  3. http://communities.intel.com/message/70720
  4. http://serverfault.com/questions/118791/how-do-you-get-e2fsck-to-show-progress-information

Thursday, May 24, 2012

Migrating from Gitosis to Gitolite on Ubuntu 12.04

Ubuntu 12.04 (Precise Pangolin) removed the support for Gitosis Git server. I have been using to host my projects locally. This forced me to look for alternatives. This link http://askubuntu.com/questions/126097/ubuntu-12-04-gitosis-no-longer-available discusses the alternatives.

I decided to go ahead and use Gitolite, which also provides migration from Gitosis. Here are the steps to migrate from Gitosis to Gitolite.

git-server: Where the git repository is hosted
git-client: A windows git client. I am using command line git from Cygwin
gitadmin: The user account under which the git repository is managed.
gituser: The windows user name under which the git client operations are performed.

Copy the existing public key to the server.
git-client> scp /home/gituser/.ssh/id_rsa.pub gitadmin@git-server:/tmp/  
Disable the gitosis and install gitolite
git-server> sudo apt-get install gitolite
git-server> su gitadmin
git-server> cd /home/gitadmin
git-server> mv .ssh/autorized_keys .ssh/autorized_keys.back
git-server> cp -r respositories repositories.gitosis
git-server> cd repsitories/gitosis-admin.git/hooks
git-server> mv post-update post-update.sample
git-server> cd /tmp
git-server> gl-setup id_rsa.pub
Make changes to the gitolite configurations
git-client> git clone git://github.com/sitaramc/gitolite
git-client> git clone gitadmin@git-server:gitolite-admin.git
git-client> gitolite/convert-gitosis-conf < gitosis-admin/gitosis.conf >> gitolite-admin/conf/gitolite.conf
git-client> cd gitolite-admin/conf
git-client> vi gitolite.conf
Modify the gitolite.conf and delete the gitosis-admin repo. Also modify any user names which do not have a dot '.' in them as discussed in the migration document here: http://sitaramc.github.com/gitolite/gsmigr.html. In my setup, the sample configuration will look like:



Copy the existing keys
git-client> gitosis-admin/keydir/*  gitolite-admin/keydir
Delete any duplicate keys as discussed here: http://sitaramc.github.com/gitolite/gsmigr.html

Commit the changes
git-client> cd gitolite-admin
git-client> git commit -am 'Moved to gitolite repository'
git-client> git push