Full Marathon?

Legend has it that in the year 490 BC, an Athenian soldier named Pheidippides ran from the town of Marathon, Greece to Athens, a distance of about 42 kilometers, to announce that the Greeks have been victorious over the invading Persians. Pheidippides ran non-stop the whole way that when he arrived in Athens to pronounce the great news, he only managed to exclaim “Victory!” before collapsing to his death.

Fast-forward to 1896 and the marathon became one of the first Olympic sports in the very first running of the Olympics. At that time, the race was only pegged at 40 kilometers, and only 17 runners lined up at the starting line to compete in the first ever Olympic marathon. Out of those 17 valiant competitors, only 8 runners crossed the finish line. The winning time was just a hair under 3 hours by a guy named Spyridon Louis who wore leather shoes donated by his fellow villagers.

Fast-forward again another 121 years. Legend has it that Ram Bhamidi will attempt to run 42.195 KM in October 2017. All he wants is for you to pray that he does not collapse like Pheidippides did!

Growth of 20/Twenty Social Intelligence Platform

Under the Hood

We have often been asked about the challenges we faced in scaling up our technology stack to manage big data. I have attempted to address this in this post which is the first of a series of blog posts on this and similar topics.

20/Twenty was created ground-up as the most intuitive and easy to use cloud based (SaaS) Social Intelligence platform in the world.  Based on our deep understanding of what marketers needed and the awesome designs we created, we signed up our first client even before the product was officially launched. The pressure to quickly deliver the first version of the product was intense

From an engineering point of view, there’s a huge amount of data that we pull (Think Big Data!), process, augment and then visualize in the platform all on a near real-time basis. Imagine someone tweeting and it appears on our platform within a few seconds along with augmented information including Gender, Sentiment, Engagement, Spam score etc.

The evolution of 20/Twenty has already seen a few stages of growth. The graph below shows how 20/Twenty data has grown over the last 2 years since our product launch. This is a really cool growth for a startup like Circus Social both from a business perspective as well as from an engineering standpoint. We used several tricks from the books and a few practical hacks to ensure our ability to fetch, process, augment and visualize high volumes of data continued to become better, though this journey was not without pain!

 

Stage 1

We created over 200 custom marketing applications in our previous Avatar at Circus Social working with some of the biggest brands in the world. We used the same open source technologies (PHP / MySQL) to create the first version of 20/Twenty. This worked well and as our data grew in the first few months, we continued to grow vertically by adding more capacity (CPU/RAM).

Most of the queries from the application were read queries whereas a bulk of “write operations” were being performed by our data crawlers. We therefore created an efficient master-slave architecture where the application would read from the slaves and the crawler scripts would write into the master. This worked well in general but the exponential increase in the volume of data meant that certain queries were running extremely slow and impacting the user experience.

Stage 2

Since our data volume was growing exponentially and the relational aspects of the database were not the core of our application, we realized that sooner or later, we would have to move to a NoSQL database. However, the performance issues that were cropping up had to be sorted quickly and without a downtime. We quickly realized that we needed a dedicated search engine and MySQL was not good enough for this purpose.

We explored several options and Elasticsearch came to our rescue here. Elasticsearch is a distributed, RESTful search and analytics engine that centrally stores your data in a manner which can be retrieved / read really fast by your applications. Our awesome tech team deployed this in a matter of days. The improvement in performance was remarkable. The plan worked and we cheered!

Stage 3

Word spread in Singapore and Asia about how good our platform was (and our sales team did a good job too!) and we continued to sign up new clients. The volume of data continued to grow for existing clients as well as new clients. The tech stack of MySQL and ElasticSearch did not let us down but we wanted to create an architecture that would scale infinitely, if there’s a thing like that.

In Stage 3, we moved the core of our database from MySQL to Cassandra (Elasticsearch was now interacting with Cassandra) and the backend code from PHP to Node.js. We also migrated most of our front end code to Angular.js for better performance. This was a major architectural change on a live application being used by several clients so we created a parallel production like environment and ran it parallelly for several weeks to ensure everything was working as desired before switching over.

While we did the above, we continued to work on cool new features on the product and opened up our data API’s to a few clients who wanted a deeper integration with their own applications. Other tools we used during this and other stages were Postman, Github and JIRA.

As we scale further from here, we will probably have newer and more exciting technology challenges and we will keep posting about them. If you are excited to work on some of these, do write to us at careers@circussocial.com

This article first appeared on the Circus Social Blog here

2016 – The “Running” year!

After 3 years of half-hearted and unsuccessful attempts (2013, 2014, 2015) at making a habit out of running, I can now claim that 2016 was indeed a breakout year. I think there were 3 main reasons for this.

1. My wife ran Pinkathon in January 2016 (her first official run) and this was a great motivation for me to make another serious attempt at running.

2. I suffered from typhoid in late February / early March 2016 and this was devastating on my limbs. I resolved to take control of my body.

3. I joined a running group ( Soles of Bangalore ) after my first official run in May 2016. The support system of a running group helped enormously in staying motivated. I started looking forward to the Sunday morning runs whenever I was in town even if it meant waking up at 5 AM or earlier, something I loathed before!

Finally, the pic is from the finish line of my first ever Half Marathon, a feat unimaginable at the beginning of 2016.

 

 

FirstCry Acquires BabyOye from Mahindra Retail!

The recent stock filings by Mahindra Retail confirm that it has “sold” its BabyOye business to FirstCry.com. With this acquisition, FirstCry.com seems to be the only original baby products business in India to survive and even thrive.

Here is a chronology of events in this industry

  • 2009: Hushbabies.com is founded
  • 2010: FirstCry.com is founded. BabyOye.com is founded
  • 2011: Hoopos.com is founded.
  • Apr 2013 : Babyoye.com acquires Hoopos.com. Hoopos name is eventually dropped
  • Sep 2013: Hushbabies.com shuts down.
  • Feb 2015 Mahindra Acquires BabyOye and drops its own brand Mom&Me. Expands retail stores to 120 stores
  • Jan 2016: Ratan Tata invests in FirstCry.
  • Oct 2016: Firstcry acquires BabyOye from Mahindra in an all-stock deal. Mahindra also invests an additional amount in FirstCry.

With this acquisition, FirstCry.com is arguably the largest online and offline (300 stores) baby products company in India. It is a great story of a vertical e-commerce business that grew to become a successful omni-channel business. This is also a defining moment in the Indian ecommerce industry!

This article also appeared here

Debut Half Marathon

Behind that smile lies the satisfaction of completing my debut Half Marathon and the pain endured in running 21.1 KM.

I have come a long way from my Typhoid days in March 2016 when I was barely able to walk and resolving to take control of my body to completing my first HM today in a fabulous timing of 02:16:45 (#awesomedebut) with two 10K runs in between.

Here’s to many more such smiles!

Transparency in Digital Advertising – Where Art Thou?

In the last week or so, there were two news items that caught my eye, both related to digital advertising.

1. Facebook overestimated video metrics for two years, report says

Big ad buyers and marketers are upset with Facebook Inc. after learning the tech giant vastly overestimated average viewing time for video ads on its platform for two years, according to people familiar with the situation.

If you had allocated media spends to video ads on Facebook, chances are that you overestimated the impact those ads had and may have even put in more money on that basis.

2. Dentsu Apologizes After Overcharging Companies for Internet Ads

Advertising giant Dentsu Inc. on Friday said it overcharged at least 111 companies for internet ads. The company said it was still investigating the scale of the improper billing but has so far uncovered 633 cases of potential overcharging valued at a total of ¥230 million ($2.3 million). In some cases, promised ads never appeared online, it said.

One of the benefits of digital advertising is the ability to accurately track your return on investment (ROI) compared to offline channels which may not be that accurate. However, the optimisation process can be quite frustrating when the metrics you are looking at are inaccurate to start with.  Transparency in media spends is essential to understand what is working and what’s not and will allow advertisers to evaluate their campaigns better.

The recent cases may not be a bolt from the blue for the industry experts but hopefully, this will stir the transparency debate in a meaningful way and lead to more accountability.

This article also appeared here

Last Man Standing – The Indian Ecommerce Story

last_man_standing_intertitle

(Image Source)

There’s an interesting game being played in India for a few years now.  It’s called the “Last Man Standing” game.  Some of the players playing this game include Flipkart, Snapdeal and more recently Amazon.

Since the second innings of e-commerce which started in India around  2007-08, several e-commerce companies have already shut shop. This has been especially so in the last 2-3 years where smaller players with shallow pockets could not sustain the deep discounting offered by the bigger players.   There are others which have been forcefully merged with another company with common investors.

Simple trading logic tells us that if you buy something for Rs. 100 (including all associated costs) and Sell it for Rs. 110, you make a profit of Rs. 10.

E-commerce  in India works differently.   Let’s take the example of Flipkart.  In the Financial year 2013-14 (Source VCCircle) , Flipkart had losses of Rs. 400 Crores on a Revenue of Rs. 179 Crores.  Since Flipkart is a marketplace, Revenue refers to the actual Revenue and not the Gross Merchandise Value which is the price of products sold.  So for every Rs. 1 in Flipkart revenue, the company actually lost Rs. 2.23.

Flipkart is not alone in this game.  Amazon lost 321 Crores on a revenue of 169 Crores.  Snapdeal lost 265 Crores on a Revenue of 154 Crores.  That’s Rs. 1.9 (Amazon) and Rs. 1.7 (Snapdeal) loss per Rs. 1 in Revenue

The bottom line is that every e-commerce player in India continues to lose money and justifies that ( I believe) as a customer acquisition cost.

Nice Game! How does this continue and when does this end?  Well, to continue the game, the investors need to keep pumping in more and more money till you are the only company left and then you don’t need to do it any more and can start dictating terms and hopefully make profits.

Look at how much money Flipkart has raised till now – $2 Billion.

Also notice how desperate Flipkart has been to raise more money in 2014.  This game is not over yet.  Those lines are going to get steeper in 2015.  The amount of money that needs to be pumped into Flipkart will be several billion dollars more before we reach a stage where Flipkart can confidently say that they are the last man (or 1 of 2-3 men) standing.

While there is something fundamentally wrong with the way e-commerce is being approached in India, it is the consumer who is laughing all the way to the bank right now!!

The Revival – Ram Bhamidi’s Blog

It’s exactly 5 years since I started blogging and almost 4 years since my last blog post.  It’s time for a revival of the blog, don’t you think?

In the last 4 years:

1. I quit my  job at People Interactive ( Senior Vice President at Shaadi.com and Makaan.com ) and relocated to Bangalore which is the hub of India’s entrepreneurial activity

2. Grew  Circus Social to a team to 30+ people with my partner Shalu Wasu.  We are a marketing solutions and products company based out of Singapore and Bangalore and have done over 170 campaigns for over 40 brands, have launched 2 products and working on the 3rd one and hoping 2015 will be a fabulous year for further growth

3. Had another baby.

4. Met a lot of new people and made a lot of new friends!!

5. Lost my previous blog as I was too busy with work and didn’t renew my hosting (Sigh!) I pieced it together again using the archives at the Wayback Machine.  Thank you guys for doing a wonderful job of archiving the internet.

I hope to revive my blog by posting atleast once a fortnight in 2015.   Have a fantastic 2015!

109350-strip_

(Image Source)

Egyptian Revolution and Social Media

The French revolution lasted 10 years (1789-99) and was a period of radical upheaval in the French and European history that brought major social and political changes including the collapse of the French monarchy. Compare this with what happened in Egypt where the protests lasted a little over 2 weeks and resulted in the eviction of Mubarak.

Most revolutions are started by unhappy citizens. What shape they take after that and how long they last depends on several factors including of course the mode of communication to organize mass protests.

In recent times, the internet has emerged as a powerful platform for communication and social media has helped accelerate the speed of communication. Realizing the potential damage that could be caused by the internet, the Egyptian government shut down all access to the Internet on the first day of protests (in addition to blocking phone calls).

The revolution in Egypt was just waiting to happen and was of course influenced by the uprising in Tunisia. The revolution may have led to the same outcome even without the internet and social media. However, social media has become an integral part of our communication and definitely accelerated the process.

Long live Social Media!

Trains, Planes and Harrowed Passengers

I read 2 news items today with great interest

1. China has officially clocked the world’s fastest train at a mind blowing 481 km/h. This is the world’s fastest electric unmodified commercial train

2. Greedy Airlines in India would like to charge as much as Rs. 40,000 for domestic flights.

Let’s put this in context.  If you had to travel from Delhi to Mumbai (about 1150 KM air distance), the new chinese train will take 2 hours and 24 minutes.  Compare this with about 2 hours of flying time for a flight.  If you are flying, you will also check in earlier (90 minutes) plus wait for your baggage to arrive (30 minutes).  Railway stations are usually in the heart of the city whereas airports are not.  This could mean another 1-2 hours of travel time.  It will therefore take you about 5-6 hours of time if you were flying compared to 2 hours 24 minutes by train.

This speed of 481 kmph is of course top speed and one can not expect to travel at this speed throughout.  Even if you assumed an average speed of 360 kmph and a train distance of 1400KM, it will take you  less than 4 hours to get to Delhi from Mumbai.

Trains have other benefits too.  They are environmentally friendly compared to Planes and the journey is much more comfortable as well.

There are of infrastructural challenges which need to be addressed but I am all for getting these high speed trains to India.