Monday, July 19, 2010

Install RealPlayer on BackTrack Linux 4

Okay I couldn't find a post for this on the net, so here it is. If you notice the real player version available from real.com is 11 which doesn't get installed on BackTrack 4 because of backtrack having some older versions of packages while RealPlayer requires newer versions. This itself is odd since RealPlayer 11 was released in 2007 while BackTrack got released this year (2010). Wonders, wonders. Any way, instead of messing your system upgrading backtrack packages using other repositories (which then might not be compatible with BT4), the safe bet is to install real player 10. To do this, edit /etc/apt/sources.list file and add the following line to it:

deb http://archive.canonical.com/ubuntu dapper-commercial main

save the file, do apt-get update and then apt-get install realplay
and you're done.

ref: http://www.debianadmin.com/install-opera-web-broweser-and-realplayer-10-in-ubuntu.html

Monday, July 12, 2010

My 2 cents on web development using python

If you don't already know, I just love programming in python. I was told back in 1996-97 to use python but those were my early programming days on windows with VB so the idea of python went over my head. It took me around 5-7 years of development to realize the benefits of python and open source. So when I converted myself toward Linux and open source, python became my favorite programming language and though there have been a few others that I tried (Ruby, Perl) and have worked more often with (PHP), python still remains my first choice.

Every now and then I get the need to do web development in python; be it using django on some project or using GAE (Google AppEngine) to launch a site I always felt the need for something better in the web development area. The biggest problem for me seemed the issue of learning a whole new language for templating and a complete framework for creating a website. The framework bit wasn't that bad but why would I want to learn yet another programming language (which I might not like eventually) when I want to program in python?

About 6 months back, I decided to create a product (more on that some other day) and obviously python was the choice for programming. The issue was that this product required using templates; not for generating HTML, but instead for generating html, javascript and php code. This time I carefully examined all the available options like database layers and ORMs like SQLAlchemy, SQLObject etc; templating languages like jinja, genshi, mako and their older variants etc. There wasn't a need for a web framework so I was saved the trouble of deciding on one of djano, pylons, zope, turbogears and some micro-frameworks. The database layer doesn't give much choice so the obvious choice for me was SQLAlchemy. Its reflection capabilities really were critical for the project.

The templating language category had many many options and after careful consideration I decided on mako. Sure the biggest factor for me was that I could use python syntax and functions inside mako instead of learning a whole new templating language syntax and its associated functions/filters etc, etc. Both of these choices have worked well for me in the project.

Now a few days ago I again needed to make a choice as I decided to create a Intranet browser based application that involved mostly reporting and graphs. So the time to choose was upon me, having previously worked with django and GAE I didn't really want those kind of features. Zope seemed too huge and too complex. Some micro-frameworks like flask, bottle and others were also good candidates but then I decided on pylons. It by default supports SQLAlchemy and Mako. Though we can use any other option for ORM and templating language but the default choice says quite a bit about how the pylons developers want pylons to work. Being in favor or Mako and SQLAlchemy myself, it seemed more natural.

Now a bit about pylons, unless you have developed for the web a lot and have developed big projects with many many files with lots of lots of code; the benefits of pylons might not become clear to you. In fact, it might seem quite complicated (specially if you are coming from a php background). That is quite understandable. Pylons uses MVC (Model View Controller) approach towards web development. Don't panic if by hearing MVC, old Visual C developers come to mind :) PHP developers might need some getting used to changing 3-5 files for just getting changes done to a single page but once you get the hang of it (which can only be done with some practice, not just reading), you'll feel that developing this way is much better. A pylons project is structured in a way that it encourages code reuse with very little effort. Most importantly, the benefits start increasing as the size of project starts growing.

The extra effort placed upfront in a pylons project is well worth it in the long run. Just remember, you don't have to use everything pylons has to offer in your very first project. If you like to generate your html yourself, you can skip using the form helpers etc. Don't feel like going through the ORM, use the database directly and use plain old SQL if you like. The choice really is yours. Once you get used to some of the features then trying others would be a good idea.


Saturday, July 10, 2010

Using sqlalchemy reflection with pylons

Okay I have been working on a project using pylons. I already have a mysql database that I wanted to use with my project. I didn't want to create the model classes in pylons under the myapp/model folder. The solution for me was to reflect database tables using sqlalchemy. Reflection is the process through which sqlalchemy reads table information from databases.

Reflection in sqlalchemy is simple can can be done with code like:

from sqlalchemy import *
engine = create_engine('mysql://username:password@localhost:3600/mydb')
meta = MetaData()
meta.bind = engine
meta.reflect()

for tname in meta.tables:
print "Table: %s" % tname
T = meta.tables[tname]
for c in T.columns:
print " %s" % c



The problem with pylons is that sqlalchemy is used within the project structure and we need to make changes to a few files in order to get reflection working.

The database I am working with has 4 tables users, dbs, db_courses, summary_stats. I am going to describe the procedure for these tables (of course, you can modify the code to work with your tables).

First step is to add code to model/__init__.py file


import sqlalchemy as sa

class users(object):
pass

class dbs(object):
pass

class db_courses(object):
pass

class summary_stats(object):
pass

t_dbs = None
t_users = None
t_db_courses = None
t_summary_stats = None

def init_model(engine):
"""Call me before using any of the tables or classes in the model"""
global t_dbs, t_users, t_db_courses, t_summary_stats

t_dbs = sa.Table("dbs", Base.metadata, autoload=True, autoload_with=engine)
t_users = sa.Table("users", Base.metadata, autoload=True, autoload_with=engine)
t_db_courses = sa.Table("db_courses", Base.metadata, autoload=True, autoload_with=engine)
t_summary_stats = sa.Table("summary_stats", Base.metadata, autoload=True, autoload_with=engine)

sa.orm.mapper(dbs, t_dbs)
sa.orm.mapper(users, t_users)
sa.orm.mapper(db_courses, t_db_courses)
sa.orm.mapper(summary_stats, t_summary_stats)

Session.configure(bind=engine)
Base.metadata.reflect(bind=engine)

This automatically fetches table information from the DB and allows for using the table classes dbs, users, db_courses and summary_tables just like if we had created the table structure in those classes (without reflection)

Note that in controllers the sqlalchemy MetaData object is accessible through appname.model.meta.Base.metadata

Fetching records from any table is now very simple. For example to fetch all users following code will do:

from myapp.model import *
users = meta.Session.query(model.users)

Friday, July 9, 2010

RAID 6 on Linux

A recent hard drive failure on one of my development PCs got me thinking about RAID. Though I thankfully had most of the code backed up to a local SVN server. Still not having to rebuild a system again is a much better option.

Doing some research related to RAID (RAID levels, Software Vs Hardware RAID), I decided to go with software RAID using RAID 5 or RAID 6. A couple of blog posts here got me more interested.

http://louwrentius.blogspot.com/2009/06/linux-raid-6-performance-using-software.html

http://louwrentius.blogspot.com/2009/06/11-gbs-using-linux-software-raid.html

Now I don't plan to create a 20 TB or so NAS (Network Attached Storage). I plan to use 4x500 GB WD SATA drives. Using RAID 5 or 6 I should be able to get between 1.5 to 1.8 TB of storage from these with 1 or 2 disk fault tolerance.

Instead of doing all the software setup by hand, I plan to use a Linux NAS distro. For the time being OpenFiler seems most promising. I'll be posting my progress regarding this in the coming days so check back later to see how it went......