How to move a large file library to another cloud — without a week of downloads

by

Big migrations fail in boring, specific ways. Here's what actually goes wrong, and the small number of things that have to be right.

Anyone can move ten files between clouds. Moving half a terabyte — tens of thousands of files, nested folders, the odd 4 GB video in the middle — is a different animal, and it fails in ways that only show up at scale.

I've watched these migrations break, so here's the honest list of what goes wrong and what has to be true for it not to.

1. The "download everything first" approach collapses under its own weight.

Pulling a large library onto your laptop to re-upload it means your local disk has to be bigger than the library, your machine has to stay awake for hours or days, and one dropped Wi-Fi connection can leave you unsure what actually made it. The fix isn't a bigger laptop. It's not involving the laptop at all — the transfer should run cloud to cloud, server-side, so it continues whether your computer is on, asleep, or in another country.

2. A migration that can't resume is a migration that will fail.

At ten files, a failure is an annoyance. At forty thousand, a failure at file 38,000 that makes you start over is a catastrophe. What matters is that a re-run is incremental: it compares what's already at the destination — by size and modified time — and skips it, so interrupted transfers pick up where they stopped instead of re-copying everything. You should be able to run it, walk away, and run it again tomorrow to mop up whatever didn't finish, cheaply.

3. "Move" quietly means "copy, then delete" — and the delete is where people lose data.

The scariest migrations are the ones that clean up the source as they go. One wrong folder mapping and a delete lands on the wrong side. The only version of this I trust shows me a dry run first: the exact list of what will be copied, what will be updated, and what will be deleted — before a single byte moves. If a tool wants to delete your originals, it owes you that preview. Mine wouldn't ship without it, because I wanted it for my own data first.

4. Big files break naive tools.

A library is mostly small files and a few large ones, and the large ones are where cheap tools choke — timeouts, memory blowups, half-written uploads. Streaming the bytes through instead of buffering the whole file in memory is what lets a 5 GB video ride along with the 5 KB text files without special handling.

5. You should never have to trust the middleman with your data.

A migration tool sits between two clouds that hold your files. The right amount for it to keep is: nothing. Files should stream directly from source to destination and never be stored in the middle; the only thing worth remembering between runs is names, sizes and timestamps, so the next incremental pass knows what to skip.

Get those five right and a scary migration becomes a boring one — which, for your files, is exactly what you want. Boring is the goal. Boring means nothing was lost.

That's the checklist I built Syncrix around: cloud-to-cloud so your laptop stays out of it, incremental so it resumes, a dry-run preview before anything is deleted, streaming so big files just work, and nothing kept on our servers. It's the tool I wanted the last time I had to move a library and didn't trust anything to do it.

If you've got a specific migration coming up — provider, rough size, the one folder you're terrified of — describe it in the comments and I'll tell you honestly whether this handles it.

1 view

Add a comment

Replies

Be the first to comment