Godel Escher Bach: Craziness for life?

Just finished reading Godel Escher Bach by Douglas Hofstadter. It is a classic book which won Pulitzer Award.

The essence of the book is how meaning can arise from meaningless symbols that creates a system with meaning. Not only is this explored in Eastern Philosophy but has been a constant topic of meditation and discussion.

Hofstadter uses the following tools for thinking:

• Isomorphism - Brain and Universe!
• Recursion - Factorial of a number.
• Paradox - All Indian are liars! (I am an Indian)
• Infinity - Cantor's different type of infinities and continuum hypothesis.
• Formal System - Principia Mathematica

I am not going to give anymore details about the book but let you explore it.
Below are the links for more resources that will help understanding GEB and will take you to a journey that will make you a little more crazy that what you are.

• MIT Open Courseware
• Bach Fugue
• MC Escher Paintings

Further reading:

Google IO Conference

I am attending Google IO Conference this year and will be in San Francisco on May 27th and 28th.
Really excited about this conference as I am really interested in new app engine for java sessions. Also looking forward to the sessions on GIS and integrating EC2 with app engine. Martin Fowler, Ola Bini and Rebecca Parson are fellow ThoughtWorkers who are speaking at the conference.

Software Development Trends for 2008

Here is an article in SDTimes that sums up the trend that was in software development during 2008. Here is the link

Soocial.com

I was reading AWS blog and saw an article on soocial.com using EC2.
Soocial is a free one address synchronization solution that includes Macs, Gmail, mobile phones and others. The site is built on Ruby which makes it way to cooler (I am biased towards Rails as I am using it personally).
I started using it and it is awesome. I think this might be the end of MobileMe service from Apple.
Though still in beta mode and Blackberry not yet implemented as Blackberry does not have SyncML (protocol for information synchronization). Lets see how they implement or whether Blackberry will have SyncML client of its own. Need to do more research on this topic as this is totally new to me and I am stuck with Blackberry with little over a year more to go.

I would recommend getting an account.

Rails Deployment: Capistrano + GitHub + Amazon EC2

I have been working on my personal stamp project for a few days now and got some basic application running. As till now, I am able to create entries for stamp and have a cart like application. Since the application is running, I thought why not deploy it and remain agile. So I started working on rails deployment. There were many options including building my own powerful server (which I am planning to anyways) but I wanted to have a quick start. What a better place then Amazon Web Services Elastic Cloud Computing (EC2) where I started a Ubuntu image and started customizing it. I used git as version control for my project and used github to host my repository. Also used Capistrano for rails deployment. In this post, I will give details steps of how to combine the power of Capistrano, github, MySQL (other database can be used) and Amazon EC2 to deploy your rails application quickly therefore remaining agile.
I have assumed that you have an AWS account and have you machine setup to run ec2 tools or have ElasticFox plugin for Firefox.

CONFIGURING EC2 INSTANCE
I used Ubuntu gutsy image and started an instance.
ssh to the instance. Generally, ruby is already installed on the instance so we should start installing other ruby packages.
Install RubyGems from source

wget http://rubyforge.org/frs/download.php/43985/rubygems-1.3.0.tgz
tar -xvzf rubygems-1.3.0.tgz
cd rubygems-1.3.0
sudo ruby setup.rb

Create a symbolic link to gem1.8

sudo ln -s /usr/bin/gem1.8 /usr/bin/gem

and update RubyGems

sudo gem update –system

Use RubyGems to install rails and other packages. Also install mysql server and driver plus

sudo apt-get install mysql-server libmysql-ruby libmysqlclient15-dev libc6-dev git-core

During mysql-server installation root password will be asked. It is recommended that a password be assigned.

sudo gem install rails mongrel termios mongrel_cluster mysql

Now is the time to create database in mysql for production and grant privileges. Login into

mysql as root and using root password.
mysql –u root –p
mysql>CREATE DATABASE myapp;
mysql> GRANT ALL PRIVILEGES ON myapp.* TO ‘myapp’@’localhost’ IDENTIFIED BY 'myapp_password';

Create directory for deploy. I created /var/www

INSTALLING APACHE ON EC2 INSTANCE

wget http://mirror.olnevhost.net/pub/apache/httpd/httpd-2.2.10.tar.gz
tar –xvf httpd-2.2.10.tar.gz
cd httpd-2.2.10
./configure --prefix=/usr/local/apache --enable-proxy --enable-proxy-http --enable-proxy-balancer --enable-dav --enable-rewrite --enable-so --enable-http --enable-expires --enable-headers --enable-mods=deflate_module --with-php --with-mysql --with-susexec --disable-info --without-berkeley-db --enable-dav=shared --enable-dav-lock=shared --with-included-apr
make
sudo make install

GITHUB AND GIT

At this point, we should have our git repository at github. Go to GitHub and create an account. Follow the instruction to add source control to your application. At this point we will need to create public-private keypair for local machine as well as each production server. Put the private key in the EC2 instance and add the public key to the github account for both local and EC2 instance. Put the private key in .ssh folder. You can use

ssh-keygen

to generate keys on EC2 instance and you local server. Do not change the name of the keypair and let it be named to the default (id_rsa and id_rsa.pub). Also don’t associate any paraphrase for the key.

CONFIGURING MONGREL TO RUN IN CLUSTER AND CONFIGURING OTHER FILES

Run the following command

sudo mongrel_rails cluster::configure –e production –p 8000 –a 127.0.0.1 –N 2 –c /deploy/path/current

Here config/mongrel_cluster.yml file will be created with configuration information for mongrel to start. –N here is for starting 2 mongrel servers. Also make sure that you add current to the end of the deploy path above.
Change database.yml and change username, password and database for production.
In config/environment.rb comment out the following line

RAILS_GEM_VERSION = '2.1.2' unless defined? RAILS_GEM_VERSION

This will cause the gem to use the latest version installed rather than a previous version which might cause some problems when you have update gem version.
In config/environments/production.rb comment out the following

config.action_view.cache_template_loading

INSTALLING CAPISTRANO ON LOCAL MACHINE

On your local machine install Capistrano
sudo gem install capistrano
This will install Capistrano 2.5.2 Now we need to “capify” the application. Goto to application’s root directory and type

capify

This will create two files
1. Capfile: This is the main file which cap need similar to rakefile for rake
2. Config/deploy.rb: This contains all the configuration data for rails application deployment

require 'mongrel_cluster/recipes'
set :application, "app_name
set :repository, "git://github.com/username/projectname.git "

# If you aren't deploying to /u/apps/#{application} on the target
# servers (which is the default), you can specify the actual location
# via the :deploy_to variable:
set :deploy_to, "/var/www/#{application}"
set :mongrel_conf, "#{current_path}/config/mongrel_cluster.yml"

# If you aren't using Subversion to manage your source code, specify
# your SCM below:
set :scm, :git
server "application.com", :app, :web, :db, :primary => true
set :user, 'root'
ssh_options[:port] = 22
ssh_options[:verbose] = :debug
ssh_options[:username] = 'root'
ssh_options[:host_key] = 'ssh-dss'
ssh_options[:keys] = "path/to/key"
ssh_options[:paranoid] = false

set :use_sudo, false

DEPLOYMENT
For the first time when you are deploying for the first time, run the command

cap deploy:cold

subsequently

cap deploy

After this, one can bundle the instance and create an image so that more instance can be started any time.

This summarizes the configuration and deployment of rails application on the cloud. Using GitHub and Amazon EC2, rails deployment is quick and cheap. One can start with EC2 and then add real servers + more instances.

Next to come is MySQL on Elastic Block Storage (Amazon EBS) so that database is persistent.

MySQL ALTER TABLE

Here is something interesting that causes a lot of confusion overtime while changing the structure of a database table or adding index. Problem arises when a query is extremely slow due to insufficient or non-existent indexes. The table is heavily used and contains millions of rows. Site has heavy traffic. Adding a required index is only the solution. But a common misconception is during ALTER TABLE to add an index all the selects and writes will be blocked. This is not true. During alter table only writes are blocked for the entire time but selects are allowed for most part of ALTER TABLE except during the RENAME part. Here is the detail of how ALTER TABLE works :

- open and lock the table
- create a copy of the table
- add index to the copied table
- rename the copied table to new table
- drop the old table

During the rename part there is an upgradation to name lock. It waits for all the selects to finish and then renames it. Particular problems are seen when one expects the ALTER TABLE to finish in less than a second but it is taking a lot longer. Seeing at the processlist will show that there are selects going on from the table. The processlist will not show any sign of lock or something but will be stuck at renaming table for the ALTER TABLE query. All the write queries on the table will be in lock state though.

This can be very useful in maintanence of high traffic site in devising the time for updating indexes etc.

Part of History!

Done with my first project in Boston and I am very proud so say that I was a part of change that happened on Nov 4th in USA. Who would have guessed that my first project at ThoughtWorks will be of such huge importance. I cannot blog about my projects but already enough hint is given for the project that I worked on.
From technology point of view, had very good exposure to PHP, MySQL and Cloud Computing (Amazon Web Services). It is great how scalability is becoming relatively easier with Elastic Cloud Computing (EC2) especially for start ups. There was a very good section on The Economist about the cloud computing from corporate point of view and how it is going to change. Well, definitely so much exciting stuff going on that who can stop!
Working on my personal project on Ruby on Rails these days. I was for long trying to create a website for philately enthusiasts, which will try to incorporate a lot of functionality that as a philatelist I always wanted to have in an application. Presently, a very simple site is up and running on my Mac. I will be in touch with fellow philatelist, whom I know, to test the functionality and recommend enhancements and addition to the site. Hopefully, looking for my beta mode in spring of 2009.

On My Bookshelf

Guns, Germs, and Steel: The Fates of Human Societies by Jared Diamond








The Drunkard's Walk: How Randomness Rules Our Lives by Leonard Mlodinow







Refactoring: Improving The Design Of Existing Code by Martin Fowler

Boston

It is good to be writing again. Things changed again and now I am in Boston on a project.
My training in Bangalore was fantastic. Made the best of friends and went home (Lucknow, India) for a week after almost 2 years.

Came back and had Ruby training in ThoughtWorks Chicago office and then moved to Boston. The project, I am working on, is one of the best project I can work on in terms of the client. It is going to be a world changing event and it feels good to be a part of the history.

I have been involved in high performance database optimization that involved some very intricate MySQL optimization. Really enjoying the database stuff.

Behavior Driven Development

Today we were introduced to Behavior Driven Development (BDD). So what is BDD. Think of the Tetris game. A piece is falling in the well with some garbage at the bottom. Depending upon the orientation of the piece falling from top, a line can disappear or garbage can increase. Similar is the case of the Schroedinger Cat where depending upon the observation the state of cat is dead or alive.

Owing to this fact, the same event can have different outcome depending upon the context. It gives importance to the fact that why we are writing code rather than the technical details and therefore makes the code simple to understand and translate between development team and business.

An example can be: Given that you are a Java Developer when you are on a project then you should code in Java.

So we were shown how to do BDD. A single test method may have name something like this shouldreturnthis(), this gives a very good idea of what we are supposed to test.

This is the basic that I learned today, thanks to Elizabeth Keogh.

One more thing, I learnt was that we can use Single Responsibility Principle (SRP) to get away with getter and setter methods. It helps in better cohesion when we return the required thing from a class rather than accessing and changing the private instance variables of a class. Tommorow is the session on SRP and will get more information on that. Very excited for tomorrow’s session……