Blog

“Simple is better than complex.
Complex is better than complicated.” – Zen of Python

So what makes something Simple, Complex or Complicated?

It varies – A problem could be simple if you are Einstein but unfathomable for many of us. In case you are on the other side: hi Einstein!

Generally speaking, a problem is simple if it has just a few components and our mind grasps it easily. A complex problem can have many components and our mind takes some time to understand it, say, 20 minutes or more just to understand the problem. A complicated problem is a beast (sometimes beasts). You will keep missing few parts of the problem even after you get it. It is very difficult for our mind to keep the entire context of such a problem.

Why bother about it?

We are software developers and we love solving problems once and for all so that other humans won’t have to. The more complex a software becomes, the more difficult it becomes to maintain. This makes the software more likely to break due to changes and difficult to debug. It can cripple the team and bring the development pace to crawl.

Complexity is inevitable

Every non-trivial and useful software will gain weight as it keeps expanding its problem horizon. That is to say, complexity is inevitable for such a software because we will keep adding features and enhancements to make it more useful to our dear users.

Why should we fight against architectural complexity?

Since architectural complexity of a software already depends on the complexity of the problem it solves, what we should not do is make the software unnecessarily complex. As obvious as that sounds, it is important to keep it in mind, because it’s very tempting to step over the line and make something more complex than it should be. Why? Kendra Cherry has explained it better in this article. Though all the four “obstacles in problem-solving” listed in that article are very relevant for software developers, I must mention two of them right here. These are faced very frequently in our field:

“Assumptions: When dealing with a problem, people often make assumptions about the constraints and obstacles that prevent certain solutions.”

You bet. We assume 100,000 users are going to sign up within a couple of days after we launch. We assume that a simple network call will take so long that we have to keep the data prefetched and maintain a cache. We assume a lot of such constraints and add unnecessary complexity to the architecture which is not needed right at this moment. According to Kendra the second obstacle is:

“Irrelevant or Misleading Information: When you are trying to solve a problem, it is important to distinguish between information that is relevant to the issue and irrelevant data that can lead to faulty solutions. When a problem is very complex, the easier it becomes to focus on misleading or irrelevant information.”

I couldn’t agree more. It is easier to focus on these questions instead of the real problem at hand: should I be using NoSQL or MySQL? If NoSQL, will it be MongoDB or CouchDB. Enlighten me but I have never heard of a software that failed because it was using CouchDB instead of MongoDB (or vice versa).

Keeping things simple is hard, but important. While architecting and developing a software, you have to be careful about everything you are adding to it. Always ask yourself – Is that absolutely necessary? I would like to end this post with the following quotation:

“It is not a daily increase, but a daily decrease. Hack away at the inessentials.” – Bruce Lee

P.S. Please go and read the whole Zen of Python if you have not already.

Image Credit: dno1967b on flickr.

Comments(0)

MySQL, the world’s most used open source RDBMS has been one great success story. And now, a new breed of database management system – NoSQL, which has no adherence to the widely accepted RDBMS, is showing some promise. One such project is MongoDB, part of the NoSQL family of database systems.

With developers looking for alternatives to the traditional relational database, MongoDB (one of the fastest growing alternatives) offers a number of compelling advantages over the others. Replicating the MySQL success story can be really hard for MongoDB, and with popularity comes a lot of criticism. Robert Scoble summed it up nicely in his comment: the best technologies gather passionate haters!

Does CloudMagic use MongoDB?

We do not use NoSQL at the heart of the product, but we did happen to find a valuable use case of MongoDB – running our backend analytics. It gives a good combination of flexibility and ability to query the database (though the querying capability can not be compared to a RDBMS, it’s still quite useful).

How we do it?

We parse the logs using shell commands and PHP to form a JSON and then dump it into a MongoDB Collection. You might be wondering why MongoDB as this could have been achieved using MySQL (or any RDBMS) itself. Well, the major benefit is that no two requests are alike, so the JSON formed does not have the same keys. This is where the flexibility of NoSQL comes in handy. With no concept of column names here, all you have to do is to insert a JSON in a collection and you are done.

Why MongoDB?

One can also create a key/value table in MySQL, but imagine having 200,000 requests per day with approximately 30 key/value parameters, viz 6 million rows. It’s just difficult to get desired results from such a table using a single query, you will need to use multiple joins of the same table. Whereas in MongoDB you can get the result in a single command. Speaks volumes of what MongoDB is capable of.

Map/reduce gives another edge, MongoDB uses JavaScript as query language thereby allowing us to write functions and manipulate the data in the database server itself. Map/reduce is actually quite powerful, you can implement many custom conditions to handle the data and it can also act as an alternative to GROUP BY in SQL.

To get the analytics for a given day, we query all the requests for that day and get certain numbers like ‘Daily Active Users’, ‘Number Of Searches’, ‘Number Of Previews’ etc. We do create intermediate MongoDB collections for simplification so that each request follows a certain standard. This intermediate collection holds information of selective request types like search, preview, snapshots which makes it even more manageable.

With each CloudMagic app upgrade or feature implementation, the request data and format might change, but there is no major change in the analytics database, this makes a our lives a lot simpler.

Fascinated by MongoDB? We sure are. Share your experiments with MongoDB. Drop a comment below or join the discussion on hacker news.

Image courtesy: Junya Ogura on flickr

Comments(0)

This post first appeared in PandoDaily.

With the advent of Web 2.0 and webapps like Gmail (circa 2003), everyone started talking out how all desktop apps would be eventually converted to webapps.

Prior to Gmail, people didn’t believe that such a paradigm shift was possible. Websites were not considered as heavy-duty as desktop apps for a lot of use cases.

Gmail gave us new hope. Around the same time tabbed browsing became popular. The combination of tabbed browsing and rich webapps was supposed to deliver a death blow to traditional desktop apps – aka “thick clients”.

This combination promised that we will finally get rid of the nagging problems associated with desktop apps. Some of the serious problems that webapps addressed were:

• Each tab of a browser was completely sandboxed. To try a new app you didn’t have to install anything. It promised us freedom from DLL Hell. Business apps were no longer dependent on arcane ODBC drivers!
• Keeping multiple tabs open and switching back and forth was very convenient making us super productive.
• App delivery and upgrades were simplified. The executable code that made up the app was distributed and managed from central servers.
• New apps didn’t impact the performance of the OS. No virus problems (with the notable exception of webpages embedding ActiveX controls!). No Registry conflicts. No spyware.

The future looked bright. All hail the browser. “The browser is the OS” became a popular meme back then.

But there was one big problem. Those webapps were simply not as capable as desktop apps. JavaScript (JS), the language primarily used for the UI was not designed to be an industrial strength language for developing rich UI. Complex projects resulted in thousands of lines of spaghetti JS. Frameworks like Google Web Toolkit were introduced to bring some sanity into the entire process.

All of a sudden, the future didn’t look so bright after all.

A few years later, we saw another wave of optimism with HTML5.  HTML5 held the promise to solve world hunger. Everyone was dreaming of an utopian world where you could write rich apps once and those apps would run on different platforms with little or no modification.

HTML5 was a standard. No single company controlled HTML5 and developers would no longer be held hostage by Microsoft APIs. No vendor lock-ins. For the first time in the entire history of computing, it seemed possible to learn one front-end technology and deliver apps on all platforms. The promise once Java made, but failed to deliver – especially for the UI layer.

Ironically, the problem with HTML5 is that it was a “standard”. It was evolving rather slowly. It was a democracy led development, not a dictator led development. Historically, some of the most successful technology companies were dictator led. Notable examples are Apple, Microsoft and Oracle.

Meanwhile, all of us continued to love our browsers and its tabs for a lot of computing activities.

Then the iPhone launched. With the launch of iPhone, we were unknowingly transitioned to a new era. In my opinion, the best way to define this era is – “The OS is the new browser – apps are the tabs.”

The new era retained the richness and power of desktop apps and the delivery and sandboxing advantages of webapps. Additionally, you can use industrial strength languages C, Java and C# to develop apps.

iOS, Android and Windows 8 are designed ground-up to act more and more like browsers. There are some striking similarities.

• Launching an app became as trivial as launching a new tab
• Switching between apps became similar to switching between tabs
• Installation and upgrades are seamlessly delivered through central servers. Those central servers are called App stores.
• Complete sandboxing of apps. One app cannot access the data of another app. No DLL hell. Virus and spyware were contained.

However, nothing is perfect. This model introduced a new set of challenges.

In the classic browser and webapp model, seamless transition from one webapp to another was done naturally, using hyperlinks and URL parameters. In the app world, such seamless linking between apps is not popular yet. However, things are changing. Many apps now support URL schemes. For example, from the Twitter mobile web, you can launch the Twitter app with the right context. Over time, this integration will evolve and you can pass data from one app to another in more sophisticated ways.

Developing apps for multiple platforms is expensive when compared with HTML5 apps, that have a common codebase.

If development cost is the only reason you are settling down with HTML5, you are not working hard enough to create a killer browser tab for your app. You are going to miss out. Users don’t care about the cost implications of the developer. They care about the quality of their experience. Products that are winning provide the best experience.

Cover Image courtesy: Windows Blog

Comments(0)