My LiveDrive Data Nightmare

TLDR; If you are not backing up your data, do it now. If you back up your data but have not tested if you can recover it, do it now. If you are using LiveDrive to back up your data, you might not be as prepared as you think.

My Previous Data Life

For a very long time I have stored my data in an external drive. Why? Mostly because I do not want the main output from my employment to be locked in a single computer. Often it is very convenient to be able to access the data from another computer. Also, when there is a fire alarm, my drive is the only thing I take. I know that the cloud is a good alternative, but I like to have the data locally and there is something unsettling about trusting Google or Dropbox with your entire digital life and work.

Of course, external drives fail, so I have purchased a subscription to a service which backs up my data on-line, away from my physical location, for over 12 years now.

The Incident

One day I arrive home, set up my USB hard drive on my computer, and the drive does not start. Or rather, it tries to partially start, but fails in a sequence of odd ways.
No biggie… I should be able to recover all files. It is going to take quite a while (it’s close to 1TB), but I should be covered!

I am Prepared… or am I?

Well, after wiping off the sweat from my brow, and taking a glass of water, I set out to the unpleasant and boring (but necessary) task of recovering my data. The most urgent staff first in my computer’s fixed hard drive first, and then the rest on a new external drive that I have ordered. Everything should be in my LiveDrive, right?

Well, the first unpleasant surprise is that LiveDrive downloads my files really really really slow. A full day of data recovery does not even start to make a dent in the 0.8 GB of data (approx) that I had backed up. I’m in a good connection, but maybe it is that the data is stored in Europe (where I used to live), and has to go under the water to get to me. I try a few things to solve this, including a VPN, but it turns out that this is not the problem. The service is extremely slow seemingly because it goes file by file, and it checks the status of each file individually, blocking the download until one file has been done. The download is decent when the file is large, but when you have a folder full of small files (e.g., Thunderbird’s e-mail folders, it slows down to a crawl. Unfortunately, the livedrive interface only lets you select folders, or files, recursively, but in one folder at a time. Additionally, every time you select what to download and it finishes, you have to select again from the backup set starting at the root folder. This goes on for a few days, with much effort and desperation, since my whole research is in there. It is hard and really tedious work of clicking buttons and check marks, taking notes of what is backed up and what is not, what folders should be priority, and trying to avoid those folders with lots of small files. I’m able to recover about 30% of my files this way, but with much effort.

The second unpleasant surprise is that some (important) folders simply do not download. This problem seems to be independent of the issues mentioned above. After some back and forth with LiveDrive’s help desk I am explained that some of the data stored is corrupted and therefore not downloadable. Their technical team are working on it, and they will update me of the progress. After some back and forth with them reminding them regularly for about 3 months I simply desist. After a year and a half I’m still to hear from them.

Contact with LiveDrive’s help desk results in reasonable response times, but not typically very useful. Comments about the problems of the interface are put in feature requests for the (presumably distant) future, while comments about the slow down seem to simply be above the pay grade of the helpdesk staff. They are generally pleasant. After they recognize that some of the data is corrupted, they offer to refund me what I have paid them so far “as a courtesy”.

What I Have Learned

  • Not all backup companies are the same. LiveDrive might be OK if you are just backing up only photos or videos, but the moment that you have to backup a real computer folder, e.g., with folders containing program state, e-mail folders, bookmarks, software that you have built, et cetera, the system is not viable any more.
  • The interface seems adequate at the beginning, affording most of the actions that you might expect to need (e.g., to find a file in the folder tree and ask for it to be downloaded). However, the reality is that when it comes to actual work, there are small details that can make the difference between hours of tedious work and minutes. As they say, God is in the details.
  • Losing much of your data can be debilitating. It is not only your set up of things; it is that you cannot trust any more to find the things where you left them. As an academic, I rely a lot on my drive, which is my “prosthetic memory”, for a multitude of things (creating a new talk, reporting activities, applying for tenure…). Not knowing whether you are going to find what you thought you had is sometimes even worse that knowing that you do not have it at all. It forces you to do a lot of clicking, it makes you feel insecure about your memory, and it reminds you every time of those negative feelings and the feeling of helplessness of the whole process. I’m not prone to depression, but there are very few other things that have made me feel so s****y about my work.
  • Hardware drive recovery is expensive and very disappointing when what comes back is minimal.

How I feel about the whole thing

More than a year and a half after the incident, I still have not completely recovered. My skin still crawls when I think back about it, and sometimes thinking back is unavoidable. A silver lining: I have learned to let go of some data, and perhaps not being so reliant on storing things in drives (although I still like to have most of my stuff in my own drive).

I also think that LiveDrive giving back my payments is like an insurance company giving you back your premiums if your house goes on fire. I have no animosity against the people in the helpdesk who helped me. They are probably underpaid, and it feels like the rest of the company is just in “automatic pilot mode”, without really any interest in improving the service, but more in milking their existing infrastructure and software (which is clearly not fit for purpose).

What I Recommend

If your data is not backed up, please please please please, back it up somehow now. At least the most important stuff should not be difficult to put on dropbox, google drive, etc.

If your data is backed up in a service, set aside some time to give it a serious test. Consider the following questions: how long would you consider acceptable to recover your data? How much work would you want to do for what you are investing now?

An exemplary move

In April this year (2011) Kate and I moved our residence from Calgary, Canada, to Cellardyke, Scotland. Our budget for the move was limited, and, although we decided not to move any big pieces of furniture, we have quite a big collection of books. Moving is something that we dreaded since we knew we had to do it in January. So… how do you go about it?

Well, we started by looking on-line for moving companies. This is one of those sectors on the internet where most of what you find is aggregated sites that ask you to fill forms, and supposedly, they will send your way a number of quotes. Generally useless because most of the time you get nothing back, but sometimes even dangerous, because we got at least one quote (from Euro Transport International) that we later found by doing a quick search on-line was basically a scam (check this out for scary stories on how a move can become a nightmare).

Finally, we thought that it would be a lot safer to go local, and it paid off. Before anyone says anything, this post is just the result of grateful clients to a wonderfully executed service, and we have absolutely no conflict of interest here. The service from Highland Movers which operate as STARLINE OVERSEAS MOVING for international relocations was wonderful, friendly, courteous, timely, and dead-on on the estimation. We chose to do groupage to make it cheaper, but it still took shorter than expected. Every box arrived in perfect state, dry, and the movers back here were also friendly and very quick. In other words, if you are in western Canada, and need to move abroad, you should call these guys. Thanks Robin!!

So you have an idea, we moved about 195 cubic feet (equivalent to approx 80 banker’s boxes), and before insurance, the total price came below 4000 $CAN. It took about two months to arrive, and it could have been less if the local movers had trusted Kate that “a 40 foot truck will not fit through our narrow street”. After the initial payment there came no extra charges, not even from customs (at least so far).

Anyways, I thought I would put this out in the interwebs: if you are moving, please, check your mover with the BBB, do not trust anyone that does not want to do a visual inspection of your stuff, and choose local if you can. Also, if they are not very responsive on e-mail through the whole process, that’s probably a bad sign (I count at least 20 e-mails sent by our moving manager in the last few months).

Getting busier

The good weather is gone (for now), but our place is just as beautiful in the rain. Today, I will try for a run in the evening, but for now you can check out the photosynth (a kind of panorama) that I made just north of Cellardyke during my run:
http://photosynth.net/view.aspx?cid=d19a39aa-d4df-4600-aa7a-87bc3b63daf9&m=true&i=0:0:0&t=False

I’ll try to keep these coming! 🙂

Newbie in Scotland

Kate and I have been in Scotland for almost a week. So far, we are loving it. The weather is wonderful, our place is amazing, and the people have been wonderfully welcoming. We live now in Cellardyke, which is a tiny fishing village, about ten miles from St Andrews.

For our first impressions of Scotland, it’s best that you check Kate’s funny account of our landing and first days.

Tomorrow I will start work at the SACHI group. Meanwhile, you check out some of the photographs that I made around where we live.

NOTE: because of the University of Calgary e-mail policies, I am not allowed to keep or redirect my ucalgary e-mail, so please, use the gmail address instead. Thanks and sorry for the inconvenience!