Technology

How-to store several terabytes of data cheaply

I love travel photography, it is my passion and part of my profession. In last couple of years, I have undertaken 22 trips and generated around 1.3TB of RAW and edited images and videos. Then there are photos and videos I shoot when I am not traveling and also tons of data from the websites I run.

Few days ago, I ran out of storage space on my 3TB WD HDD, even though I was using it exclusively for photos and videos.

3tb-hdd-full

I wish I had the money to buy a 10TB HDD and not worry about the storage issue for the next 3-4 years. However, Rs. 48,000 is no small amount and spending that amount of money on a single HDD doesn’t really solves the issue in the long run. Buying a 4TB HDD for 10-11k seems more feasible and better bang for buck. However, that would have been useful for me until the end of this year or at max, until middle of next year.

I neither wished to throw excess money at this problem, nor wished to have a collection of working HDDs in the storage bin, because they were now too small for my need.

I wanted a solution which was cheap and easy to implement, and was future expandable.

Enter, Windows Storage Spaces.

After a lot of research online about software and hardware based solutions for my storage requirement, I came across Windows Storage Spaces.

Storage Spaces has been around since Windows 7 and in simplest term, is a software based solution, which lets you pool two or more HDDs into a storage pool.

You can setup your storage space pool to have parity, two way or three way mirror if you want data redundancy and protection.

However, I wanted neither of these. I just needed a lot of storage space, even at the risk of losing all the data in case of single drive failure. I could afford to do this, since I have multiple backups of all my data and I am also in the process of implementing cloud based backup.

Getting ready to deploy Storage Spaces

Since I needed to delete all partitions of the HDDs I wanted to pool in storage spaces, I needed to back up all the data I had on to my external 4TB WD My Book HDD.

ready-for-transfer

Due to the amount of data involved (3TB), this process alone took over a day.

However, once all the data was safely copied to the external drive, I could delete partition D and E and begin setting up storage spaces.

Setting up Storage Spaces in Windows 10

In order to setup Storage Spaces, you need to type storage spaces in the search box and click on the storage spaces app.

storage-spaces

Once you do that, Storage Spaces App will open.

storage-space

Here you need to click on “Create a new pool and storage space” and you will be presented with a screen, showing all the HDDs, which do not have partitions and thus can be added to storage pool.

create-storage-space

Select the drive you want to add to the pool, in my case it was the 2TB and 3TB Western Digital Hard Drives and click on create pool.

create-storage-space

Now you get to assign name and drive letter to the storage pool and also choose the type of storage space you want (simple, parity, two way or three way mirror). I chose simple, since I wanted the combined storage space of both my HDDs and didn’t mind losing data, in case either of the drives failed.

Now comes the fun part and what makes Storages Spaces such a good option. Even though combined formatted capacity of both my HDDs is just 4.54TB, I can create a storage pool much larger than that!

This way, once I am about to run out of the installed capacity of my HDDs, windows would present me with a warning message and I can simply add another HDD to the pool and I would neither need to delete the pool nor lose any of the data in the process! Essentially this gives you almost limitless data storage, as long as you can keep adding HDDs to the pool. For practical purposes, I chose a 15TB partition size.

storage-space-settings

Within seconds my 15TB storage pool was created.

new-storage-space

storage-pool-created

And it was time to copy all the data back from my external drive to the storage pool, which once again took an entire day.

transfering-data-back

Once all the files were transferred back to the storage pool, I was using nearly 3TB of space with 1.5TB of free space remaining, before I needed to add another HDD to the mix. I am guessing, I can keep using this pool for another year or two, before I need to purchase another HDD.

backed-up

storage-space-pool

Conclusion

It is fairly easy to solve a problem by throwing money at it. However, that is not really an option if you’re on a budget and can’t afford to pay astronomical prices of large capacity HDDs.

With Windows Storage Spaces, I managed to solve an immediate storage issue without spending even a single rupee. I know it isn’t the ideal situation and there is greater risk of data loss. However, with the money I saved, I can use it later on for my NAS setup, which I guess I will need to implement by early next year.

Since I now have a fast enough internet connection, I am also using cloud backup storage to store copy of my data online, so as it give me data protection in case my internal as well as external HDDs fail simultaneously. In fact I will be sharing my new backup strategy in a coming blogpost soon.

Leave a Reply