Social Icons

Pages

Featured Posts

Friday, July 17, 2015

Upgrading CrashPlan App on Qnap TS-251

I started to notice lately that my Qnap is not backing up to CrashPlan. I had set up my Qnap TS-251 and have Qnap app properly installed and configured. It was backing up properly, but for the last couple of months it has not been backing up, even though there are data changes in my NAS. Digging further, I noticed CrashPlan app tries to upgrade to a new version and the upgrade is failing. Backup was not happening when there is a pending upgrade.

When ever I tried to do an upgrade from the UI, CrashPlan service completely stops. I had to manually restart the CrashPlan service, but then it will go back to do the upgrade!!

Digging further, I noticed that there are issues with the upgrade script and it fails. I had to do the following steps to upgrade the Qnap app and restart my backup.

Environment

  • Qnap Version: TS-251 4.1.4 (2015/05/22)
  • CrashPlan app version: 4.2.0_35

Upgrading CrashPlan App

The following steps were needed to this upgrade working

  1. Start the CrashPlan app using the Qnap UI
  2. ssh to the Qnap Server
  3. Stop the CrashPlan service
  4. Modify the config file
  5. Modify the startup script
  6. Change the upgrade script
  7. Manually run the upgrade script(s)
  8. Stop the service using Qnap UI
  9. Modify the config file
  10. Start the CrashPlan app from Qnap UI
  11. Configure the client
  12. Adopt the existing computer if needed
  13. Start the backup
ssh your-qnap-ip
cd /share/CACHEDDEV1_DATA/.qpkg/CrashPlan/conf
/etc/init.d/crashplan.sh stop
Modify the my.service.xml and change the <location> and <serviceHost> tag values to:
<location>0.0.0.0:4242</location>
<serviceHost>0.0.0.0</serviceHost>
Modify the /etc/init.d/crashplan.sh
  1. Comment out the line which modifies the <location> in the my.service.xml file. Look for the line starting /bin/sed -i "s/<location.* and put a '#' at the begining of the line
  2. Repeat the same to comment out the place it replaces the <serviceHost>

Modify the upgrade scripts Go to the CrashPlan/upgrade directory and find out the latest directory created by the upgrade script. E.g. Run 'ls -lart' and take the last directory and go into that directory Modify the upgrade.sh with the following:
  1. Remove the "-v" parameter from the RM command E.g. RM = "rm -f"
  2. Modify the INIT_SCRIPT value E.g. INIT_SCRIPT = /etc/init.d/crashplan.sh
  3. Modify the INSTALL_VARS value E.g. INSTALL_VARS = ../../crashplan.vars
Run the upgrade script
sh upgrade.sh
Repeat the same modification to the upgrade.sh for any other upgrades. Typically, the upgrades are packaged into jar files and they will be downloaded to the upgrade directory. When one upgrade is done, the system automatically starts the next one. But for this also I had to modify the upgrade script. So need to stop the crash plan and the upgrade script before modifying it.
/etc/init.d/crashplan.sh
ps -aef | grep -i upgrade
# And kill the upgrade process if any, modify the upgrade script and run it
sh upgrade.sh
Finally modify the conf/my.service.xml and /etc/init.d/crashplan.sh as described before, if the upgrade script has modified these files.

To configure the client, (on MacOS)
  1. Modify the /Applications/CrashPlan.app/Contents/Resources/Java/conf/ui.properties and add the serviceHost to the IP address of the Qnap NAS.
  2. Copy the values from /share/CACHEDEV1_DATA/.qpkg/CrashPlan/var/.ui_info from the Qnap server to the /Library/Application Support/CrashPlan/.ui_info file on the Mac client

Friday, April 24, 2015

Testing POP3 from command line

Without SSL

To test a POP3 account from command line follow these steps. This will connect to the POP3 server using telnet, list the available emails, and retrieve the first one.
telnet <your-email-server>:110
Connected to your-email-server-ip...
Escape character is '^]'.
user <your-mail-server-username>
pass <your-mail-server-pass>
list
retr 1
quit

With SSL

If you are getting the following error, you will need to access your POP3 server using SSL
user <user-mail-server-username>
- ERR Command is not valid in this state.
To use SSL, use OpenSSL to connect to the server
openssl s_client -crlf -connect <your-email-server>:995
....
+ OK The Microsoft Exchange POP3 service is ready.
user <your-mail-server-username>
+OK
pass <your-mail-server-pass>
+OK User successfully logged on.
list
retr 1
quit

Friday, January 9, 2015

Docker and MacOS using Brew

Docker is an very light-weight Linux Container (LXC) management and configuration tool. One can very quickly spin, configure and provision VMs using Docker.

Install Docker

To install Docker on a MacOS, using Brew:
  1. Install Cask
  2. Install VirtualBox
  3. Install Docker
brew update &&  brew upgrade
brew install caskroom/cask/brew-cask
brew cask install virtualbox
brew install docker
brew install boot2docker
boot2docker init

Run Docker

boot2docker up
This will give the IP and Socket of the Docker daemon. For example:
    export DOCKER_HOST=tcp://192.168.2.103:2376
    export DOCKER_CERT_PATH=/Users/johndoe/.boot2docker/certs/boot2docker-vm
    export DOCKER_TLS_VERIFY=1

Use Docker

On another terminal, setup the environment variables and spin off a VM
export DOCKER_HOST=tcp://192.168.2.103:2376
export DOCKER_CERT_PATH=/Users/johndoe/.boot2docker/certs/boot2docker-vm
export DOCKER_TLS_VERIFY=1

docker images
docker run hello-world
docker run -it ubuntu bash

Shut down Docker

boot2docker down

Thursday, November 6, 2014

Gerrit Code Review: Installation, Configuration & Usage

Recently I had to build a new development team within a large company. As part of it, had to implement best practices and processes for the development team. The challenge was to find tools, frameworks, and processes that are critical to the productivity and agility of the team as well as to work with the existing infrastructure, frameworks and tools within the organization. During the course of this, I had to select a code review tool and evaluated a number of them:
  1. Crucible
  2. Review Board
  3. Phabricator
  4. Gerrit
Each of one has their own merits and pitfalls. Here I will explain a quick setup and configuration of Gerrit as I had to go through a number of sites and documentation to figure out how to get this done. Gerrit would like to be the owner of your Git repository. If this can be implemented, everything will work great. But in our case, we have a centrally hosted Git repository and all the checkins should be available in this central repo. The solution to this problem is to setup Gerrit Replication. This is not a great idea. If people are directly pushing to the master repository, AND Gerrit is pushing, this will mess up the code base. The solution is to have one-way pull and push to different repositories as described below:
Here are the steps involved in getting this work.

Install Gerrit

Install the server with all the defaults
java -jar gerrit.war init -d ~/gerrit-server

Setup User

By default, Gerrit will be installed with OpenID. Login into Gerrit UI using any authentication (e.g. Google). The first user to login will be added to the Administrator group.
  1. Create SSH Keys
  2. Login to Gerrit UI and copy the key generated in the previous step (e.g. id_rsa.pub) into your Account Setup
  3. Setup a username to connect to Git Server hosted by the Gerrit (e.g. gerrit-user)
  4. Test connectivity from your local client to Gerrit Server
    ssh gerrit-user@gerrit-server -p 29418
    
    This will give a nice welcome message and inform that interactive shells are disabled

Import and existing project

To import an existing project, create a new project in Gerrit Server and then run import.
  1. Create a new project called gerrit-test
    ssh -p 29418 gerrit-user@gerrit-server gerrit create-project --name gerrit-test
    
  2. Create access to gerrit-user to newly created project gerrit-test
    1. Log into Gerrit UI and create a new user group called gerrit-test-group
    2. Go to Projects->List->gerrit-test->Access
    3. Grant access to refs/* for Create Reference permission to user group gerrit-test-group
  3. Import project
    cd dev/gerrit-test
    push ssh://gerrit-user@gerrit-server:29418/gerrit-test *:*
    

Setup local Git Repo for Gerrit

Install Gerrit hooks
cd dev/gerrit-test
scp -p -P 29418 gerrit-user@gerrit-server:hooks/commit-msg .git/hooks

Setup Gerrit server replication to push changes to central repository

  1. Setup the local ssh keys to connect to the Git server
  2. Setup the key for git host
    git-server> ssh-keygen -t rsa git-host >> ~/.ssh/known_hosts
    
  3. Add a new file called replication.config in the gerrit/etc directory with the following contents
    [remote "central-git-repo"]
      url = git@git-host:/{name}.git
    
  4. Shutdown and restart the Gerrit Server

Test

  1. Make some changes in to a local file
  2. Commit the changes
  3. Push the changes to Gerrit
    git push ssh://gerrit-user@gerrit-server:29418/gerrit-test HEAD:refs/for/master
    
  4. Login to Gerrit UI
  5. Approve the changes with +2
  6. Gerrit will push the changes to the master git repository. Verify the latest changes are indeed replicated by Gerrit Server

Thursday, October 30, 2014

Identifying duplicate rows in a table

select column-name from table group by column-name having count(*) > 1;