Showing posts from April, 2013

Elephant in the Cloud

Today, I came across an interesting issue with cloud. Its called LFN aka Elephant. LFN refers to a TCP/IP issue called Long Fat Network. You can google for it and get the interesting technical details on this subject. What this means for a cloud user is simply that he or she is going to have to wait for ages before his or her fat file gets uploaded to the cloud server. It does not really matter if you have 2 Mbps or 20 Mbps. The latency in the network prevents the TCP from being able to transfer the files at the maximum available bandwidth. 

And this is serious issue for all cloud users! I tested it across many cloud service providers -  Amazon, Rackspace and Opsource. Over my 4 Mbps, I could not get more than  100 kbps while transferring over scp. 

This file has the relevant test info for copying speeds on various cloud operators.

So, whats the Solution?

Solution#1: Simple and Quick

Split the files into smaller chunks. I used hjsplit - it has software for windows as well as Linux. eg. hjs…