How To Replace A Failing Disk in FreeNAS (or Increase Storage Space)

My initial design for Project Gibson was a single vdev (FreeNAS’s term for a group of drives acting as a single device) with 6 drives in a single pool, and I would extend as I go. When planning vdevs, they should consist of drives of similar size, as the RAIDZ2 protocol will treat all the disks in the vdev as having the same capacity as the smallest drive in the vdev. For example: A vdev containing drives that are 4T, 3T, and 2T will treat them all as being 2T; replace the 2T drive with another 4T drive and the array will still only be seen as 3T each. When I set up FreeNAS, I consolidated all of my data on the smallest drives I had and created my first vdev with the 6 largest drives available.

A Word About ZFS RAID Protocols

I chose RAIDZ2 as it is a nice balance between space availability, read-write times, redundancy, and longevity. The RAIDZ2 is a double-parity system that distributes data in such a way that any two drives within a vdev could fail and the dataset would still survive. After having suffered data loss from accidental disk drops, and knowing that I would be storing more critical backups on this array, I opted for a little more protection at the cost of less storage space.

Once the initial vdev was set up and data was transferred from the smaller drives, I began to assemble a second vdev of the smaller drives. My plan is to, over time, replace failed drives within the vdevs with larger capacity drives until the entire system is uniform. As storage will eventually come down in price, I can begin replacing drives with even larger capacity units as my storage needs grow.

Extending a ZFS Pool

Once my second set of 6 drives was ready, I installed them as normal, then fired the system back up. Under the Storage > Pools dialog, click the cog icon (Settings) to open the “Pool Actions” dialog, then click “Add Vdevs”. In the Pool Manager dialog, you’ll be presented with two lists: Available Disks on the left and Data Vdevs on the right. The newly installed drives will be listed under Available Disks, select each of them and click the right arrow to add them to a new vdev.

FreeNAS Pool Manager
FreeNAS Pool Manager

With the new drives added to a new vdev, you can click “Add Vdevs” to add the new vdev to the existing pool. Once back on the Pools dialog, click the cog again and click “Extend Pool” to stripe the available space across the two vdevs, adding the second vdev’s available space to the total. This procedure is covered in more detail in the FreeNAS User Guide, section 9.2, but this will serve as a general guide to the process.

Replacing (Or Upgrading) Disks In A ZFS Pool

I have 2 use cases for replacing disks in a pool: The first is obviously replacing a failed drive and the second is replacing a smaller drive with a larger capacity one. Both of these involve the same offline-replace-online procedure, so I’ll cover them at the same time. This is where my decision to use RAIDZ2 really becomes apparent, as the procedure puts the parity of the vdevs in a weakened state by removing a drive instead of simply connecting another drive to the motherboard (I only have 12 drive connections available). With RAIDZ1, any mishaps during the process would render the data on the vdev unrecoverable, so with RAIDZ2 I have some parity during the replacement process–just in case.

To replace a disk within a pool, first navigate to the Storage > Pools dialog in the web interface, click the cog in the right corner, then click “Status”. You will be presented with a list of physical drives within the pool.

FreeNAS Pool Status dialog
FreeNAS Pool Status dialog. Yes, my pool is named “balabushka” after George Balabushka and is probably the only thing with his name that I will ever own.

Locate the disk you need to replace from the list, then click the stacked dots (Options) to the right of the line item and then “Offline”. This will take the drive offline for replacement. I usually cross-reference the drive label with the Storage > Disks dialog to get the disk’s serial number (which I have printed on the exposed side of the physical disks in the case) to make identifying the physical disk easier. Once you have offlined the drive and noted its serial, shut down the system and replace the physical drive. Boot the system back up, reconnect to the web interface, and return to the Storage > Pools > Status dialog (under the settings menu). Click the options menu for the offline drive and click “Replace”. Select the serial number of the new disk and click “Replace Disk”. The vdev will remain in a compromised state until the resilvering process completes and the data is restored in its original form on the new disk.

The smaller vdev has experienced more failed disks than the larger array, so my upgrade plan has served well here. As disks wear out, I am replacing them with new 4T disks (although FreeeNAS only sees them as 1T for the moment). Eventually, the smaller vdev will consist of 6x 4T disks to match the first, and I will finally have doubled my available storage. No telling how long that will take, though, but I still have plenty of space to spare for now, and I’m weighing the option of adding my stack of 2.5″ USB drives in a third vdev. More on that if and when it comes.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.