I have been experimenting quite a bit with the Google App Engine (GAE) recently.  The Google App Engine is a managed application hosting platform; you develop your applications in Python upload the to Google’s Infrastructure and they manage the scaling, hosting and the database system.  The database system isn’t a RDBS, rather it is based on BigTable (Google’s Technology) and is accessed through a DJANGO style interface.

Having never used Python before and having a couple of ideas that I wanted to implement in "the cloud" I had a choice of either Amazon Elastic Cloud or Google App Engine.  Whilst both solutions target "In the Cloud" application hosting they are drastically different in how they approach the solution.  For a start Amazon Elastic cloud is an entire infrastructure and you manage how you want to scale the application architecture, the GAE on the other hand is entirely managed by Google.

Google’s pricing at the moment, whilst defined, hasn’t been implemented, however they have stated that there will always be a free version which, in my opinion, has very reasonable quotas (enough for 5 million page impressions a month).

I decided to go with Google’s offering simply because of the simplicity of the system and the price point for the amount of processing I will most likely be performing (free…. Yay!!)

So far I have three completed applications and I will describe a little about each and how the GAE helps me.  Each of the services is really simple, there is nothing complex going on.

Overall, I am really impressed with the service, for the usage my applications get the hosting is free so I really can’t complain.  Each of these services took about a day to create and were up and running as soon I uploaded them.  I suspect the sites would have taken a lot longer to create for the Amazon EC2, purely because I would have had to set all the images up and get them running with all the correct software.

Eweri

Eweri.com is a URL shortening service that mimics the functionality of tinyurl.com.  These types of services are ideal for services like Twitter where there is only a limited amount of space for your messages.

A user will submit a URL through the site, the Google App Engine maps the URL to a Request Handler that I have defined.  The request handler checks to see if the URL exists on the system, if it does the application will return the key of the URL in the database as the shortened Eweri URL, so for instance if the URL www.kinlan.co.uk is the 5th element in the data-store the the Eweri URL will be eweri.com/5.  If the URL doesn’t exist in the database the Request Handler will create a new entity to put in the data-store and then return the associated key to the user as the eweri.com style URL.

So far this service has seen a moderate amount of use, it has stores about 300 URLs at the moment and is nowhere near the quota limit and I don’t seeing it reaching it for a long time even if I suddenly get a massive traffic spike.

I found it really easy to develop this application and using this as an example to learn Python and the GAE it was made in to a really fun exercise.  Furthermore, was really easy to do iterative development with the SDK so I could easily add new features in to the system.  For instance it too me about an hour to create an REST API that is now being used by TweetDeck (as one on the many Url Shortening services)

So in summary, apart from the cost of the domain name (bought from www.switchmedia.co.uk) the entire application is running for free.

Know Your Privacy

KnowYourPrivacy.com is a privacy information and advisory tool for Facebook.  It is designed to show you how much information you give away when to potentially unknown third-parties when installing applications into your Facebook Profile.

The tool gives you guidance and information about the ways that fraudsters could easily steal your data and use it in ways that can harm you.

I have run this application for a while now, but it was hosted by another provider.  The application wasn’t getting much use, but when it was getting used it would constantly timeout.  I decided to try and use the pyFacebook Module and re-implement this tool on the Google App Engine.

The tool is conceptually very simple.  A user will install the application into their facebook profile, the tool will make a series of requests to Facebook for information that is visible to any third-party application and then present the results back to the user.

The GAE allows you to install nearly any module as long as it is pure Python (i.e. no C based extensions), this was a good start, because the Facebook authentication is a nightmare to use.  The pyFacebook module simplifies this and allows you to focus on developing the functionality of your application.  To add a module into the GAE you simply install

There are only about 150 monthly users of the application, so again it is a relatively unloaded system, however, because Google manage the scaling for you, I strongly believe that the GAE is the ideal platform for developing Facebook applications that won’t buckle under the load when they become popular.

The Thumb Thing

TheThumbThing.com is a web-site thumbnail service.

I created this service because I needed a way of displaying thumbnail images of web-sites next to links in some of the sites that I created.  Unfortunately, whilst a lot of these services already exist, they are either a) usage restricted, or b) have very high usage costs for the amount of images that I need to display.

This site has two major components, the first is the image hosting and the web-site thumbnail screen-shot queue (this is hosted by the GAE), the second is the software that takes screen-shots of web pages.

GAE is ideal for the first component, however you can’t have active processes on the GAE (such as scheduled events) or even binaries that are not python based, it is currently not possible to host the second component that takes the screen shot of the web sites.  I am lucky at the moment, I have the resource to be able to take the screen shots (mainly using webkit2png.py and a customer IE tool), so on my machines at home, I check the queue (http://www.thethumbthing.com/list) stored on the GAE and then PUT the results back into the GAE (including the image).

The GAE is very good at hosting the interface part of the site (including the images), however, if the demand creating the images exceeds what I can process then I might have to look at another method to generate the website thumbnails.

The performance of the GAE is excellent for hosting the images, currently I am receiving about 10 requests per second and I have not had to do any special code to manage the scaling of the site to handle the load.