TechNayak(mirror)

The last 20 years have been more than fruitful for Bangalore and India in general as the preferred destination for outsourcing, but the tide may soon ebb. The cost advantage that India presented to the west isn’t there anymore. True globalization of the Indian work force has meant multinational and international companies are recruiting in India, offering international salaries and providing immigration opportunities. The impact of this globalization is the increase in salaries that Indian companies have to match to find skilled labor. The increased opportunities also mean very alarmingly high attrition rate amongst tier 2 and lower companies. In business processing outsourcing and ITES sector the attrition rate is almost twice that of Software. The rise in salaries especially in cities like Bangalore and Hyderabad have created high income inequalities amongst the lower and the middle class. The increased money and spending power has also raised costs of living. Its estimated that about 5000 people are entering Bangalore everyday in search of opportunities, but that’s easy math when you consider the population of India; This inflow of people have put severe infrastructure constraints in these cities. Clogged roads and sky high infrastructure costs are common in these cities. Companies are finding it hard to find breathing space in the city and the instability in the government means there is no end point to the woes faced by the IT companies.

For a new company trying to establish a presence in Bangalore, the costs of infrastructure are extremely high and the sheer amount of companies plus the income inequality created by the increased wages will assure that only high salaries will ensure loyal employees. That’s not even guaranteed. There is a resource crunch, high infrastructure costs and productivity numbers are beginning to decline. The cost advantage is clearly lost. Indian companies are setting shop in tier cities like Tumkur, Pune etc where the infrastructure costs are significantly lower and the local talent pool is still unadulterated by the outsourcing boom. This still isn’t the solution as the reason why cities like Bangalore saw the boom was due to high concentration of engineering workforce and also the output of engineers in the southern region thanks to the 1000 odd engineering colleges in the region. The setting up of companies here will only guarantee reduced costs and not skilled labor, which means poor quality software.

This situation isn’t new, it happened at route128 and also in the valley. This was the prime reason for outsourcing and now it seems we have come full circle. Falling dollar prices, recent slowdown in the US economy and the opportunities presented by countries in South East Asia, China and Russia which are currently centers of low costs means India faces severe competition. India’s economy is riding on the success of outsourcing exports and the currency has become costlier. Outsourcing has become a victim of its own success and only time will tell what fate lies ahead for India.

I recently wrote an essay on the software bottleneck and the reasons behind some of measures to resolve the bottleneck which include outsourcing, open source etc. If you were interested in the reading material above, be sure to read the essay.

Convergence refers to the coming together of markets, technologies or concepts to benefit out of each other. This coming together when complete diminishes the lines between these two entities and it becomes really difficult to relate to each of them independently. The wikipedia definition states :

Technological convergence refers to a trend where some technologies having distinct functionalities evolve to technologies that overlap, i.e. multiple products come together to form one product, with the advantages of each initial component.

I recently wrote an essay on digital convergence for a course im taking. The essay talks about how telephony and computing converged to become the phenomenon we know as the internet. The essay talks in detail the history of both these fields and highlights significant landmarks and events that laid the foundation to this ocnvergence. Its an interesting read for anybody wanting to learn about the history of the internet.

Click here to download my essay on Convergence

Most programmers know databases and its importance. Thanks to the new generation of software as a service and web services, traditional RDBMS’s are sparingly used and the number is bound to deteriorate further as enterprises adopt the Saas platform.

Data has far outgrown the domains of just text. Today we talk of mutlimedia data, urls, semantic data and many more application specific formats. Information on the Web is in JSON, REST , XML , Microformats etc. With this vareity in data formats and representations comes the inherent need for flexibility in storage and querying of such information. Almost all database users know of the conceptual modelling required for the design of any database, the key principle being that more tighter the model, more efficient the database. The integrity of the database is only as good as the integrity of the data. But you cannot talk of data integrity with the kind of formats available today.

Clearly markup data dominates the web . Though databases have developed features to better support , store and validate markup data , the initial design of databases was never to store the wide variety of loosely organized data. Querying of such markup data is fruitless and so is the attempt to index, sort , aggregate this data. To develop a custom database capable of all the above mentioned operations could be a solution, but the given the non standardized nature of this data and its probability of change, you would have a tough time scouring the web to search for changes. Plus these databases will not be semantically inter operable.

Developers are taking notice of a new scheme of storing data, I call it the bucket store. The design is roughly the same as that of a hash table, where data blocks are stored in buckets and hashes are used to index or refer to these buckets. A little improvisation in terms of adding upper layers like domains, groups and so on to complement the schema, table in a database is done to make the data easily classifiable. The advantage with this scheme is heterogeneity in data formats and the absence of constraints.

Several products are offering such services at dirt cheap prices. Take Amazon’s S3 or the recently launched Simpledb or CouchDb which offers a host it yourself version of this storage. Amazon S3 has businesses running on top of it; of the many I can recall Slideshare running on S3. With the advent of more mashups and heterogeneous data being churned out by the web more of such non DBMS related storage options will be employed. Given that this paradigm does implement all the enterprise important features like security, access control , backups, transactions etc and mature modeling methodologies that can rival the ER are proposed , I don’t see any problem in this becoming the most viable and cost effective option for data storage.

Is it true? Is Indian IT industry surviving because of lack of Semantic Interoperability?

Incorrect Semantic Interoperability describes mismatch in the format and representation of data belonging to two parallel applications which prohibit them from interacting with each other or prohibits possible migration. Take the example of ERPs like SAP and Oracle Apps, they essentially perform the same computations and solve the same problems, but do they interoperate? No. The data representations for both these applications are bespoke, which makes it unique to one product or a line of products. It is reasonable considering the fact that market domination is obtained by developing custom formats and providing custom decoders, but there is a bigger concern. For large players, data is collated and there is usually a Business Intelligence solution in place and with different products being deployed at different centres, the additional overhead of conversion of the data is imminent. Also two separate applications cannot talk to each other though they are probably linked up sequentially, you have to bring in additional middleware for format conversion and make a common bus.

Efforts like RSS, SOAP and others have been successful only to a certain extent that the format is correct but semantics are still loose. It has to come together sometime, if not now then later, when the data is huge. And the answer is yes, the Indian IT industry survives because of such semantic interoperability. Most of the workforce is maintaining these enterprise bridges (that holds together these different applications) if not building them. A majority of the work involved in the service sector has to do with writing compatibility plugins or writing migration scripts and patch ups. If it was for SI , we wouldn’t have jobs.

Gone are the days of the digital divide, there is a new kind of divide amongst many computer professionals now. Its the generation gap. Its hard to comprehend this statement, but anybody, whose is exposed to at least 5 years of industry dynamics, will know exactly what I am talking about. Call it Moore’s law affecting software or just plain old generation gap, there is a clear demarcation between people who appreciate new concepts and those who prefer things the 90s way.

There are a set of people that like the innovation happening on the web front and are adopting 2.0 technologies like there is no tomorrow. Everything from office automation to project management is now managed online on productivity service providers. Concepts like wiki, blogs, forums etc are fast appearing as mainstream applications in organizations. Surely as technology evolves and takes new shape, we will see a dramatic shift in adoption of these new tools .

In contrast , there are the other people who have been around for a long time and have seen a lot of productivity applications. To these people, technology is nothing more than a fast changing fad and prefer to stick to their old time favorites. Take people who have seen the main frame era, such folk just don’t appreciate concepts like distributed computing, virtual servers etc. Quotes like ” our mainframes never needed mirroring”, are common. People who still live reminiscing innovation of their times like spreadsheets and ERP’s.

It may be hard to believe but these form the majority of the so called power users of organizations and these legacy softwares( pun intended) , are maintained and supported just for their usage. Its distrubing to know that enterprise software lags open source software by at least 3 years , in terms of innovation. This lag can clearly be accounted to the legacy lovers who insist on using their accustomed softwares. Where does product development go in such a case. Office 2007 is seeing very slow adoption due to a change in the usability. Will this set of users be responsible for the sluggishness of product development? who will convince these users to adopt newer software? more importantly how? What will these users demand 20 years from now?

Its a strange question, but yes its an emerging market.

Recently, the field of performance has been taken by storm. Right from the people in my company who came to improve performance of our websites to the people who gave talks about performance in unconferences held in the city, performance seems to be the thing to talk about.

A recent trip to the Yahoo Developer network portal also showed some glaringly visible tributes to the field of performance. Continue reading Performance and its importance for websites…

I wrote about personalization some time back and about how we should actually be approaching this problem. Google has got their act into place and are making your own light weight personalization meter, but its for ads

Google is going to put a cookie in your browser that will record information everytime you read an ad served by Google. Continue reading Personalization is one cookie away…

There was an interesting concept put forward to me by my professor - Open World Computing. A software that is not bound by any restrictions or constraints. A software that learns and adapts to its environment. Think of a person who is taken from a metro and put in a village. Does the person fail and give up like a computer program ? No. the adaptability of living beings is something so hard to understand that it can take probably another 1000 years to just simulate a living being, let alone learn its qualities.

Think of the same thing in software. A software that is programmed based on generic constraints and the software dynamically learns from its environment through input devices like sensors and then adapts to the changed environment. The classical shortest path problem can be taken as an example. If a crawler running through the shortest path is lifted from its path and put somewhere else, will it be ale to comprehend the change and then quickly adapt or will it be lost. What if the graph changes and produces a lot of cycles or what if the problem statement changes during the course of the program.

I know that a very few people are relating to what they are reading, but the belief that a machine can learn and adapt is what scientists are trying to prove everyday. Machine learning is taboo after movies like iRobot and Terminator, but trust me, we are far far away from something like that. If we can solve a subset of problems of adaptability through an expert system, that will be an achievement in itself.

But how do you go about designing software with such requirements. I would go one step ahead and call it no requirements or changing requirements with no defined thresholds. No current methodologies like OO or aspects can cater to such a requirement. Probably a new scheme of designing learning software has to be developed. Taking tips from AI and the Turing thesis, a perfect turing machine is what is required. Its a hard call but we will get there one day.

Note : I blog on my personal space at riteshnayak.com/blog . This is a mirror of the content.

Everybody’s writing about the new social organizer phenomenon, So I thought I could add my two cents to it.

What if you want to hire someone to build an online community? Somebody to create and maintain a virtual world in which all the players in an industry feel like they need to be part of it? It would help if that person understood technology, at least well enough to know what it could do. They would need to be able to write. But they also have to be able to seduce stragglers into joining the group in the first place, so they have to be able to understand a marketplace, do outbound selling and non-electronic communications.

Seth Godin writes about the Online Community Organizer as the job of the future.

Continue reading Online Community Organizer - a job for the future…

Note : I blog on my personal space at riteshnayak.com/blog . This is a mirror of the content.

The recent opening up of the facebook platform has created a rage in the industry. Facebook themselves , after opening up to public signups, have had an increase of 80% in their monthly uniques. The facebook platform is just another icing in the cake.

There are innumerable number of apps that are being created on the facebook platform. Existing applications like wordpress are taking heed and porting their apps into the platform. Some of the initial apps have been taken over and VC’s are announcing seed fund for apps built on the facebook platform. Its like everyone wants a part of the 26 Million uniques visiting the site and want to cash in on the phenomenon. So what did facebook do right ?

Continue reading Community as a Service - implications of the facebook platform…