One Decade of Programming
July 26, 2009 at 11:33 am | In General, Suggested Reading, funny, rant | Leave a CommentSometime around July 1999 was the time I wrote my first “Hello World” program. Yes, its been 10 years since I started programming, and I dont mean L R L T of Logo. In fact, I wrote some BASIC code as early as 1995-96, but I will skip that for reasons beyond the scope of this post.
I remember liking programming for lots of reasons, but one which I particularly remember. Unlike the other subjects in school, like math or physics, programming had no boundaries or a legacy to deal with. By legacy, I mean there were no equations, constraints and I didn’t have to reciprocate what some mathematician proved 400 years ago. I have always enjoyed freedom and programming gave me that freedom to express myself. I guess, I can claim that most good programmers have taken up programming because it lets them play God or be creationists of some complex entity.
Anyway, there are a lot of interesting and fun facts through these ten years and I shall pen some down.
- BASIC was my first programming language. PASCAL was probably my first compiled language.
- I wrote my first sorting routine ( an act I call, truly understanding the programming abstraction) sometime during Jan 2000.
- I got my first computer in 2001. AMD Athlon 1.7 GHz with 256 MB Ram and 40 Gb harddisk . “OMG 40GB” , was my friends reaction.
- I disliked C early on, due to my PASCAL roots, but grew fond of it later on when I completed my first large scale project ( A linux text editor, which I proudly wanted to call, “Better than VI“). This is also where I wrote my first recursive functionality which resulted in people actually wanting to read and understand my code.
- My first exposure to Linux and OSS was in 2002, I think it was RedHat linux 7.1 with running Xwindows.
- I almost lost my entire project due to a floppy malfunction. USB’s were expensive and uncommon then. CD writers expensive.
- I learnt about data structures in 2002. My lecturer, one of the few good ones, was kind enough to teach us the intuition behind data structures and write the implementations ourselves. Our other lecturers taught programming from a book and expected us to replicate the programs written in the book.
- I would like to thank my data structures lecturer who spent hours helping me debug and correct programs. It was probably the first time I looked at code from an outside perspective and yes, I realized the importance of indenting code then and there. I have never not indented my code again. (Remember we are still in 2002).
- I learnt about OO during christmas vacations of 2002. Loved it. Loved the paradigm shift in the way I thought about programs
- My first OO program was one class with 50+ methods in it (Yup, there was nothing OO about it, but its a start)
- My first large scale OO project was building a Paint like utility using C++. I am still proud of this work because I learnt about programming abstractions and class reponsibilities here. I still remember writing my first button class which was fully reponsible for itself.
- I learnt PERL in 2004 and wrote my first web application. An online messaging, calendar and collaboration utility for friends and corporates. ( yes, this was my first social app). When I saw Myspace very soon after that ( MySpace was still pretty bare in terms of features then) the only difference I saw was that other people could see who my contacts(friends) were and network with them.
- When I first discovered databases (2003), I thought, “wow, somebody made my file I/O’s really easy”. It was also an amazement because I had thought of a unified file writing and reading mechanism to solve all my I/O woes. Problem was getting it to work with many languages. I couldn’t get my head around it and thats when I learnt about databases. Imagine my surprise and rejoice. The first database I used was Oracle and later MySQL ( it wasn’t even relational then). My database project earned me 100/100 in my finals.
- Compiler Design was one of my favorite subjects. I loved the fact that I could understand how they build programming languagues. Lex Yacc was probably the best thing that ever happened to programming. Prateeksha and I wrote the specifications for a shift reduce parser for C++. We used an A1 sized sheet of paper to write down the shift reduce matrix. After two full days writing the Shift Reduce rules, we realized we had messed up somewhere.
- I wrote a prototype of my CASE tool in 2004. My proud entry to the world of .NET. I had my official copy of Visual Studio.NET 2003 and .NET 1.1 . This CASE tool would eventually become my fnal semester project and also enter Microsoft’s Imagine Cup. We reached the Nationals for the competition. I called it Dzine.
- Even though I had learnt java in 2004, I didnt use J2EE till late 2005. when I started writing simple J2EE apps. I also figured that javascript, that language I had used to validate my controls in HTML (in 2004) , had suddenly taken the world by storm. Everybody was talking about it. Web 2.0.
- I sat for two whole days to figure out how yahoo mail was autocompleting email addresses I typed. After two days and a whole lot of searching, I learnt about AJAX. I was already on Gmail by then and didnt really understand the underlying plumbing that was holding the application together.
- Within the next 4 months I put AJAX and J2EE together to work on Samparkh with Prateeksha. I wrote an online chat application using AJAX ( inspired by Meebo). Remember this was a time when firebug wasn’t around and I used a tool called Venkman, which I am sure many of you haven’t even heard of.
- Then, Grad school happened and so did Microsoft, and the list of wonderful projects that I did during the two years I spent there.
- Special mention to BigKahuna, which took almost two years to perfect and won the Google Product Engineering Competition 2009.
Am I a good programmer? I dont know. But, I will continue to remain a programmer. Most of my friends know that I dont take sides. By sides, I mean OSS v/s Closed source, Linux vs Windows , Google vs Yahoo, and this is because of the vast and varied experiences I have had with all these different entities during programming. They all have a special place in my heart and I cannot choose one over the other. Programming is changing fast, and all I can hope for is that that I dont wear out of ideas or skills to call myself a programmer in the years to come.
A list of all the wonderful projects that I have pursued over the years is available here.
Uncertainty in programming – the lochness of the programming world
March 11, 2009 at 5:09 pm | In Tips,Tricks and code, python, rant | 1 CommentProgramming has come many a mile since the 70’s. A wide array of languages, methodologies, frameworks and other similar artifacts have made the life of a programmer really simple. These artifacts have incrementally solved problems faced by programmers and slowly, but steadily, wrapped the programmers view of a program into a set of abstractions. One of the first abstractions that was built, looking at the history of programming languages, was the ability to hide the underlying differences in hardware, system software and present a unified way of programing and manipulating the system. This is what we call modern day high level programming language.
If the programming language, an abstraction of the real machine code, ever helped solve a problem, it was that of uncertainty. Take an example of the piece of code given below.
// sample code to add two numbers
int a=10,b=20,c=0;
c= a+b;
Console.WriteLine(c.toString());
When I run this code on any machine, I am assured to get the value of c to be equal to be 30. I know when I access the variable “c” the next time, I will find it contains the value of 30. I know that two instructions from now, variables a,b,c will be available for further manipulation.
My recent attempts at programming on the cloud has taught me several lessons, the most important one being, programming to deploy on a cloud is almost like writing programs that you can never be certain about. You can never maintain application state. This means no static variables, no relational datastore, no freedom to write into the filesystem etc. Think about it for a second and it will make sense why these seemingly harmless actions are prohibited. Filesystem access is a big no-no anywhere, but as for static variables, persistent classes, singletons etc, running this on many actual/virtual machines means, all these entities with their values have to be moved/replicated across the cloud. This becomes a non trivial problem especially when the state keeps constantly changing. I could live with all these restrictions by coding, painful but effective, workarounds. What I can’t do is, work with uncertainty. Here is an example :
from google.appengine.api import memcache
def get_greetings(self):
"""get_greetings()
Checks the cache to see if there are cached greetings.
If not, call render_greetings and set the cache
Returns:
A string of HTML containing greetings.
"""
greetings = memcache.get("greetings")
if greetings is not None:
return greetings
else:
greetings = self.render_greetings()
if not memcache.add("greetings", greetings, 10):
logging.error("Memcache set failed.")
return greetings
The code is an example on using the built in caching mechanism on appengine. Notice the line of code given below; its supposed to return the value of the item in the cache with the key greetings
greetings = memcache.get("greetings")
Here’s the question: what is the guarantee that the value, which I inserted into the cache with a large timeout, is actually available. Whenever I write this line of code, do I have to write the failsafe code also(line 15,19) ? I am trying to model state using variables in the cache, mainly because its the next best thing to persistent classes and is less expensive (computationally and financially) than the key/value datastore. How do I reliably do this ? I cant trust that the cache will be available and have to keep on constantly updating the failsafe mechanism ( in case of appengine, the datastore) which is inefficient and highly taxing on the application. What has given rise to this situation is the environment of the cloud. Its not a new problem by any means. With the introduction of new languages, language constructs and other programmatic abstractions, this kind of uncertainty in programming has always reared its ugly head. The lochness of the programming world. And it will continue to do so; which is why we will have constructs like the assert(). My greatest worry is that I don’t see an elegant solution in the foreseeable future.
New Programming Paradigms
March 11, 2009 at 5:06 pm | In Tips,Tricks and code, rant | Leave a CommentOver the last two or three years, I have seen introduction of many new psuedo programming languages(if I can call it that) that help users build applications over the web. Most of these languages are built to work with or as a service. I shall wildly switch between a web service and also the langauge to interact with that webservice; so get the message when I switch from one to another. Let me take one of these languages called YQL. A sample instruction would look like this:
/* Get the latest 10 photos from flickr where the photo name contains cat */ select * from flickr.photos.search where text='Cat' limit 10
As you can clearly see the language makes querying a service and receiving its response really really simple. This is how most new psuedo languages are. They work with service end points and emulate an existing programming language’s syntax to do that. These languages are built with mashup’s in mind. The dangers of such an offering are already imminent. Services are good as long as they are up and live. Take for example any of the Google or yahoo Api’s and you will find wrappers written by people in such pseudo langauges to make your life simple. Even in the enterprise space there are such languages being built which query custom services and makes building applications really really simple.
Another observation of mine involves loose typing in these languages. Most new languages are loosely typed. Most of them take from python which lets the user take care of the typing. SQL by far has been the most emulated language amongst these pseudo langauges. Take for example JoSql to add SQL like capabilities to operations like file handling or Linq in .NET which exposes a sql like interface to datastructures. These improvisations have dramatically reduced time to turn ideas into code and rapidly prototype the application.
There are limitations to using such improvisations; some that even I can vouch for. Loosely typed and unstructured languages are good as long as you are not working on large scale systems. If you are hacking up a solution to a problem that you are facing, these pseudo languages look to be real problem solvers but when it comes to working in teams, projects that need to go into production, you start getting into big problems. Though I am a python fanboy, I faced problems when I was working on python and perl on a large project with a team. Interfaces would be unclear, poor documentation would literally spell doom and tons of other problems that we never thought we would face. There are others who complain of the very same thing. I am guessing we will see a flood of such languages in the future thanks largely to applications evolving slowly into services and it will be difficult to guage the quality of these services. Twitter’s API tried to make their service more stable but the mechanism they chose didn’t satisfy many developers. Lets hope we figure out a way to make these more reliable and stable. I guess its the developers call to be judicious about what language and service to choose when building applications.
Open source sucks and why ??
July 31, 2008 at 5:23 pm | In rant | Leave a CommentI got your attention didn’t I ? Now read on.
The problem with open source software is its lack of quality. And how is this lack of quality downgraded even more ? its due to the lack of proper documentation and support. Its pitiable looking at the so called open source enthusiasts who advocate open source like they live and breathe it, but when it comes time to get the basics of software engineering in place, you see them stepping back from the limelight. The problem can be attributed to the fact that institutes teach programming but doesn’t teach quality standards. Its gives you a very good idea why there are probably 300 projects that do the exact same thing on sourceforge and its so because the people who actually want to use one software don’t get enough help, documentation on it.
Have you ever tried looking at code that you wrote a year ago ? I have code that I wrote sometime in 2003 and trust me , I cannot understand a line in it. Why ? you cannot remember ever function that you wrote in life. Is it so hard to understand? Im sure most of you guys have taken classes in software engineering, its impossible to have a good software system without support and documentation. I shall take the example of a project that I am planning release soon.
Thanks to about a dozen people who contributed to the code, there is a lot of redundant code and almost half of the code, is not being used. What do we do now ? do we release it in the shape that it is ? Im sure the download size will probably double. Also, a lot of segments of code are not documented. How will it look when people see their names flashing at the top of a code file next to the @Author with lines and lines of spaghetti code with no documentation or explanation of their intent.
Even with the valiant attempts by a professor, our testing strategy has largely been designed not to test, but to con people into thinking its a testing strategy. Is it that hard to test ? No , it isn’t ; its just uncool work. Our project also contains a multitude of technologies like jsp’s , java, javascript and there is absolutely no architecture diagram or a data / control flow diagram. Most of the JSP’s and Javascript is still undocumented let alone being tested.
The last bit is that usage. Its not as simple as gcc project -> ./a.out . There are a lot of things that need to be configured and specific data formats, size restrictions and tons of other configuration information that needs to go into the system. But there isn’t a single document or a written account of how to go about setting up this project and executing it. Im sure even Donald Knuth wont be able to properly configure and run our project (No offense to Knuth, just used him as a figurehead) . Who will do the job of documenting it so that people who will actually want to use the project, get the documentation that they require ?
Its strange, the same people that commend Apache projects for their wonderful documentation and support forums do not put in a similar effort for their projects. I am not even talking about the technically inept people, I am talking about people who can do technology well. If the future patriarch’s of technology do not follow the basics of getting a project right then how can the next generation of programmers survive with such unkempt software. If you cannot contribute to making open source better, then please stop advocating it. Its better naive people pay for software which gives them support and service.
In our effort to program the cool, we have forgotten the basics of engineering. Its a shame they call us engineers !!
Tags: Opensource, software engineering, documentation, testing
Coming the full GCircle
June 26, 2008 at 2:30 pm | In rant | Leave a CommentRemember early 2004 and the release of Gmail ? The “ooh so exclusive, invite only” mailbox that was so cool. I remember. How hard I tried to get an invite, how I rejoiced when I layed my eyes on it; How I tried to figure out why is the page not navigating or how is that my contacts are autofilling the to field but I cannot find them in the source. How, almost after 4 whole months of searching, I learnt the buzzword that would graduate me from mere novice to a professional in the web space. AJAX.
After that started the GCraze. Almost any entity that had a G before it was sought out to be the most watched property on the web. This craze still holds; the only difference is that the craze has moved from web developers to general public. Most G properties are running only because of the G in front, sort of like the Y! of the late 90s. Call it fate or just too much popularity, I think a lot of people are coming the full circle.
Over the recent few days I have seen a lot of discontent among all the G users. I shall list most of the ones I remember, but you will get the idea..
- Mail : probably the most sought after mail service in the recent past has its own share of problems. The first being slow connections. You must see Gmail running at our university, when 250 people share a 16Mbps connection, Gmail takes on a life of its own. Mails dont open, conversations are not updated and the chat feature is a pure disaster. The biggest problem I have with Gmail are the intrusive ads next to my mails.
- Orkut and Google Talk: Orkut was the social network for India, so claim many people. But if it wasn’t for the G tag, I would have liked to see how far Orkut would have gone. I know this having competed with Orkut for a year with my social networking venture samparkh. The problem with social networks with no personal value add to it ( unlike del.icio.us or flickr) is that, it tends to grow boring, and spam and other marketing material get the better of the network . You have no choice but to opt out. I consider Gtalk as a piece of art , a wonderful piece of software who got the design and the user experience part just right. But Orkut integration did one very awful thing, it increased my contacts on Gtalk from a mere 10 to about 200, which I really didnt want. Some say I missed a warning to the same, but now there is no undoing it. Lot of my friends are shifting back to Ymsgr and Windows messenger because they see more peaceful days there.
So thanks to these problems and more, there is a reverse migration to older properties who over time have become better and people are rediscovering their long lost loved ones. Google is doing all the right things with other properties though. I love the reader and just cant do without it and thanks to the sharing and the friends aspect of it, its almost like a new paradigm of use of RSS. Docs also is really cool, though I still like using Zoho, coz its home made and is just purely awesome. Thats about it I guess.
ps : Inspiration for this post comes from my valiant struggle for almost 2 whole hours to get Gmail to work only to see a “Your Account is experiencing Errors” or something. And then I try Yahoo Mail and its so wonderfully fast and awesome and guess what, hotmail works like a charm too.
update: My Jinx continues. I have the Gods pissed. I decide to look at my shared items page and I see my own blogpost has gone missing wtf
Tags: google, gmail, google talk, gtalk, orkut
Firefox 3 – Review
June 18, 2008 at 5:37 pm | In rant | 1 CommentSo, I downloaded the latest installment of Firefox and tried it out. A brief about me, I have been a firefox user for a long time now and usually wait for newer versions of the browser to be shipped. The addons feature when launched was truly the best concept ever, then came the integrated search bar, spell check and a host of other wonderful features. I admit, I use firefox purely for its addons and nothing else (Actually the other reason is that my primitive designs really look good on firefox , IE and me dont really get along on the design playfield )
Well the 3rd installment really didn’t get me excited. Except for the full text like search on the address bar, I really didnt see any difference. The so called great features of FF3 viz offline support and apis for the same, social features etc are completely missing. So there is faster javascript and stuff, but for people in the third world countries ( at least in terms of bandwidth) like ours, pages take an eternity to load and a slight improvement in JS doesnt really do much until underlying infrastructure is changed.
The 3rd installment of FF has let me down. I still recommend people to use Flock. Their features and extensions on firefox are really great and do wonders for web users.
Do your bit for the environment
June 6, 2008 at 2:16 pm | In rant | Leave a CommentThe World Environment day has again highlighted the energy crisis the world is facing and its almost certain that the next biggest market is energy. Technology for efficient consumption of energy, be it electrical or fuel driven, will be at the peak of global demand. Many companies are getting into the energy space in the hopes of cashing in on this phenomenon. Though we may not live to see the true implications of the energy crisis, its almost guaranteed that future generations will find it really hard to survive the energy crisis. There are some things that we as individuals can do to help preserve the environment – here are a few that are not all that hard to follow
- Follow yearly maintenance schedules to ensure that your furnace and air conditioner run efficiently.
- Unplug rarely used appliances such as a TV located in the spare bedroom.
- Dry your clothes on a laundry line instead of using a clothes dryer
- Replace all incandescent light bulbs with compact fluorescent lights (CFL). CFL’s last up to 10 to 15 times longer and are 75 percent more efficient than incandescent bulbs.
- Turn off lights when leaving a room.
- Switch off your monitor or close the lid of your laptop when not in use. Remember always to shut down the PC before going home from Office.
- Set up your monitor to go to sleep when idle for more than 10 minutes.
- Reduce the brightness in your monitor.
- Do not unnecessarily leave electrical items like your phone, camera on continuous charging.
- Ventilate your room and use fans and Air conditioners sparingly.
- Always check the emission of your vehicle, remember to tune your engine optimallyto cut emission levels.
- Share a carpool or drive with a friend to work or college.
- Don’t waste paper, use a computer wherever necessary.
I also found out that I emit 3.056 Tonnes of Carbon dioxide every year
This is my environment day post ![]()
The possible end to outsourcing
May 12, 2008 at 7:46 am | In rant | Leave a CommentThe last 20 years have been more than fruitful for Bangalore and India in general as the preferred destination for outsourcing, but the tide may soon ebb. The cost advantage that India presented to the west isn’t there anymore. True globalization of the Indian work force has meant multinational and international companies are recruiting in India, offering international salaries and providing immigration opportunities. The impact of this globalization is the increase in salaries that Indian companies have to match to find skilled labor. The increased opportunities also mean very alarmingly high attrition rate amongst tier 2 and lower companies. In business processing outsourcing and ITES sector the attrition rate is almost twice that of Software. The rise in salaries especially in cities like Bangalore and Hyderabad have created high income inequalities amongst the lower and the middle class. The increased money and spending power has also raised costs of living. Its estimated that about 5000 people are entering Bangalore everyday in search of opportunities, but that’s easy math when you consider the population of India; This inflow of people have put severe infrastructure constraints in these cities. Clogged roads and sky high infrastructure costs are common in these cities. Companies are finding it hard to find breathing space in the city and the instability in the government means there is no end point to the woes faced by the IT companies.
For a new company trying to establish a presence in Bangalore, the costs of infrastructure are extremely high and the sheer amount of companies plus the income inequality created by the increased wages will assure that only high salaries will ensure loyal employees. That’s not even guaranteed. There is a resource crunch, high infrastructure costs and productivity numbers are beginning to decline. The cost advantage is clearly lost. Indian companies are setting shop in tier cities like Tumkur, Pune etc where the infrastructure costs are significantly lower and the local talent pool is still unadulterated by the outsourcing boom. This still isn’t the solution as the reason why cities like Bangalore saw the boom was due to high concentration of engineering workforce and also the output of engineers in the southern region thanks to the 1000 odd engineering colleges in the region. The setting up of companies here will only guarantee reduced costs and not skilled labor, which means poor quality software.
This situation isn’t new, it happened at route128 and also in the valley. This was the prime reason for outsourcing and now it seems we have come full circle. Falling dollar prices, recent slowdown in the US economy and the opportunities presented by countries in South East Asia, China and Russia which are currently centers of low costs means India faces severe competition. India’s economy is riding on the success of outsourcing exports and the currency has become costlier. Outsourcing has become a victim of its own success and only time will tell what fate lies ahead for India.
I recently wrote an essay on the software bottleneck and the reasons behind some of measures to resolve the bottleneck which include outsourcing, open source etc. If you were interested in the reading material above, be sure to read the essay.
Convergence
March 21, 2008 at 4:46 pm | In rant | Leave a CommentConvergence refers to the coming together of markets, technologies or concepts to benefit out of each other. This coming together when complete diminishes the lines between these two entities and it becomes really difficult to relate to each of them independently. The wikipedia definition states :
Technological convergence refers to a trend where some technologies having distinct functionalities evolve to technologies that overlap, i.e. multiple products come together to form one product, with the advantages of each initial component.
I recently wrote an essay on digital convergence for a course im taking. The essay talks about how telephony and computing converged to become the phenomenon we know as the internet. The essay talks in detail the history of both these fields and highlights significant landmarks and events that laid the foundation to this ocnvergence. Its an interesting read for anybody wanting to learn about the history of the internet.
Are days of the RDBMS numbered ?
December 19, 2007 at 8:01 pm | In Architecture - Design, Trends-Predictions, rant | Leave a CommentMost programmers know databases and its importance. Thanks to the new generation of software as a service and web services, traditional RDBMS’s are sparingly used and the number is bound to deteriorate further as enterprises adopt the Saas platform.
Data has far outgrown the domains of just text. Today we talk of mutlimedia data, urls, semantic data and many more application specific formats. Information on the Web is in JSON, REST , XML , Microformats etc. With this vareity in data formats and representations comes the inherent need for flexibility in storage and querying of such information. Almost all database users know of the conceptual modelling required for the design of any database, the key principle being that more tighter the model, more efficient the database. The integrity of the database is only as good as the integrity of the data. But you cannot talk of data integrity with the kind of formats available today.
Clearly markup data dominates the web . Though databases have developed features to better support , store and validate markup data , the initial design of databases was never to store the wide variety of loosely organized data. Querying of such markup data is fruitless and so is the attempt to index, sort , aggregate this data. To develop a custom database capable of all the above mentioned operations could be a solution, but the given the non standardized nature of this data and its probability of change, you would have a tough time scouring the web to search for changes. Plus these databases will not be semantically inter operable.
Developers are taking notice of a new scheme of storing data, I call it the bucket store. The design is roughly the same as that of a hash table, where data blocks are stored in buckets and hashes are used to index or refer to these buckets. A little improvisation in terms of adding upper layers like domains, groups and so on to complement the schema, table in a database is done to make the data easily classifiable. The advantage with this scheme is heterogeneity in data formats and the absence of constraints.
Several products are offering such services at dirt cheap prices. Take Amazon’s S3 or the recently launched Simpledb or CouchDb which offers a host it yourself version of this storage. Amazon S3 has businesses running on top of it; of the many I can recall Slideshare running on S3. With the advent of more mashups and heterogeneous data being churned out by the web more of such non DBMS related storage options will be employed. Given that this paradigm does implement all the enterprise important features like security, access control , backups, transactions etc and mature modeling methodologies that can rival the ER are proposed , I don’t see any problem in this becoming the most viable and cost effective option for data storage.
Blog at WordPress.com. | Theme: Pool by Borja Fernandez.
Entries and comments feeds.


