I'm not like 'heavy' on this topic. I studied it a while ago but I haven't tried it yet.
Read with grains of salt handy....

FreeNAS uses the ZFS file system adopted from Solaris.
As far as I know it doesn't natively support other file systems. (??)
It's touted reliability is based primarily on ZFS and not the OS so much.

The RAID-5 equivalent is called RAID-Z.
There are also RAID-Z2 (like RAID-6) and RAID-Z3.
RAID Z2 and 6 both allow two drive failures without any lost data but have a 4 drive minimum.
Supposedly Z is more reliable than 5 and Z2 more reliable than 6.
Also ZFS can supposedly fix Bit-Rot errors on-the-fly.

Ubuntu adopted ZFS in 16.04 LTS. It's an add-in module.
Later Ubuntu retained ZFS in 18.04 LTS and 18.10.
Ubuntu also supports other file systems and has a lot more options than a stripped down OS with one purpose.
No fiddling with the OS on a USB stick, which I like.

I think Ubuntu and ZFS would be a good combination for a server and a NAS in the same box.

Think along these lines.
If you set up a Win Server with the OS and Server files on (say) a RAID-1 array.
Then stored all your data on a second set of drives in RAID-5 or 6.
A server and a NAS in one box.

I have collected all the parts to build one with an integrated 10Gigabit router but I'm still undecided about using WinServer or Ubuntu.
I used Ubuntu before without any real problems but that was only for a short time and years ago.

I do not know this for sure.
It seems to me that if Ubuntu supports ZFS then it would work to pull files off a drive removed from a ZFS array.
.

You could do the same by using FreeBSD instead of Ubuntu.
FreeNAS is basically a stripped down BSD.
.

I recently successfully migrated my ZFS pool from one machine to another. The procedure consisted of three simple steps:

1) zpool export on the old machine
2) unplugging the drives from the old machine and connecting them to the new machine
3) zpool import on the new machine

Based on this, I can say that if your FreeNAS install fails, you would be able to migrate your data no problem.

See also: https://docs.oracle.com/cd/E19253-01/819-5461 … azru/index.html

(BTW, my home server runs Ubuntu 18.04 with ZFS-on-Linux.)

You can put FreeNAS on a USB disk but I recommend not to. I've been running FreeNAS on my home server since 2008 and USB disks are fine but there is a higher rate of failure than an SSD and flakiness.

When I was running FreeNAS off a USB stick the following occurred:
USB stick failure I wasn't using a cheap stick either and it was bought new but I'm sure they are all made in the same factory anyway.
FreeNAS started requiring EFI/UEFI for booting from USB
Flakiness caused by onboard USB controller
Lower level of performance

One of the best things about FreeNAS is that the install is very small, the configuration file can be backed up and restore with one click. So let's say your USB or SSD stops working and you have your config file. Install from the ISO, load your config and you are back up. If you never backed up your config then you'll have to input all your settings but you can easily mount your data.

If your PC gets fried but your hard drives are still good then you can take your hard drives, stick them in another computer, install FreeNAS or whatever OS supports ZFS and access your data. Since it's ZFS software raid you don't need to worry about buying the same controller card or putting the right hard drive in the right port.

I wouldn't recommend RAID 5 if you have alot of drives and data. Only use RAID 5 if you are using an external NAS. Use RAIDZ2 or RAIDZ3. Whatever you use make sure you scrub regularly and you have a spare HD.

Don't store the FreeNAS OS on a RAID drive unless you are serving customers. At home it's a waste of time since you can get FreeNAS back up in 15m. You can put it on a RAID1 if you want but it's just a waste.

You can use any OS that supports ZFS to read ZFS although periodically there are updates to ZFS so I suppose it's possible if you try to read a newer version of ZFS with an older LiveCD then there might be an issue.

Use FreeNAS with ZFS. Stay away from NTFS. Bitrot is a killer and NTFS is flaky as hell compare to ZFS. Once you use ZFS you'll wonder why MS hasn't done anything with NTFS. (They did come up with ReFS but not much going on there)

You don't have to use FreeNAS but I prefer it for my server because it's minimal install, minimal security updates, prevents me from doing crazy things with it like web browsing or playing games. It's a file/media/VM servers. That's it.

I've been using FreeNAS since 2008 on a C2Q Q6600 (Upgraded the server this year) and during that time I probably only reinstalled three times. Once because I had to switch from USB to SSD. The other because I redid my storage and wanted to see what the latest install was like just for the heck of it and the third just because I could. Two of those times I used the backed up configuration file and the third I input all the settings myself.

Current Specs

SERVER

PROCESSOR COOLER
Noctua NH-U9 TR4-SP3, Premium CPU cooler for AMD TR4/SP3

MEMORY
4x Supermicro Certified MEM-DR432L-SL03-ER26 32GB DDR4-2666 LP ECC REG DIMM

MOTHERBOARD
SUPERMICRO MBDH11SSL-I Rev 1.01

PROCESSOR
AMD EPYC 7351P 16-Core

ADDON CARDS
ASUS Hyper M.2 x16 PCIe 3.0 x4 Expansion Card V2 supports 4 NVMe M.2 (2242/2260/2280/22110) up to 128 Gbps for Intel ...
Intel Ethernet Converged Network Adapter X540-T1 10GBE

CASE
SUPERMICRO CSE-743T-650B Black 4U Case 11 Bays
Supermicro CSE-M35TQB 5-in-3 Hot Swap SAS/SATA Mobile Rack (Black)
Replaced case fans with Noctua Fans

POWER SUPPLY
Supermicro 1200w ATX12V Power Supply - 1 kW, 1.20 kW - Internal - 110 V AC, 220 V AC (PWS-1K25P-PQ)

HARD DRIVE
2x Seagate IronWolf Pro 16TB NAS (ST16000NE000)
11X Seagate IronWolf 12TB NAS (ST12000VN0007)
1x Supermicro 64GB 520 MB/s Solid State Drive (SSD-DM064-SMCMVN1)
4x Samsung 970 EVO Plus Series - 1TB PCIe NVMe - M.2 Internal SSD (MZ-V7S1T0B/AM) for ASUS Hyper M.2 x16
1x Samsung SSD 970 EVO 2TB - NVMe PCIe M.2 2280 SSD (MZ-V7E2T0BW) for onboard M.2

I used ZFS briefly about 10... 15... years ago when I was an admin for a dev shop that had government contracts, correspondingly large budgets, and lots of Sun hardware running Solaris. It's a fantastic file system, and it's really cool that it got ported to Linux and BSD. It has always been super flexible, WRT redundant pools -- much more than RAID. And, as expressed above, since it's software redundancy, there's no dependency on specific hardware to recover the data. You just need the disks and a compatible ZFS driver in the OS. The bit-rot protection is a really nice feature, and I'm envious of it.

I don't use FreeNAS ... no particular reason, I just built my own Gentoo box back in... I dunno when it was, but the Wii was still hard to find at retail stores, and Blu-ray was new. I use software RAID 5 and IBM's JFS file system, because it performed better than ext3 for large volumes back then.

I'm still using Gentoo, RAID 5, and JFS now -- having upgraded from 4x1TB to 4x2TB to 4x3TB and now to 4x4TB while rarely ever rebooting or taking the volume offline. (In fact, once this bit me because my Linux kernel didn't support >2TB drives with the software RAID driver, but I didn't know that, because it worked fine when I hot-plugged the drives, then crashed hard on the next boot when our power failed. Updated the kernel and all was well.) I have had a few close-calls when I did something stupid, but I was always able to force the soft-RAID driver to mount the volumes (even when they weren't at the same serial # anymore), check the FS journal, and get back to good with a minimum of effort. I have lost a few volumes with hardware RAID controllers that "for my own safety" refused to proceed when things looked awry. For this reason, I definitely advocate for soft-RAID now.

I also use my previous generation's drives as back-ups. I've had various different schemes over the years, but have settled on (for the moment) an external USB SATA dock to mount one drive at a time, copy as much data on to it as will fit, then plug in the next drive and continue. The lost space from having one of the four drives dedicated to redundancy on the online volume about equals the capacity of using all four of the backup drives, so it usually works out OK. I have previously set these up in a spare chassis to have an online backup that could've been automated, but I dislike dedicating the space to it. Speaking of automation, I would like to say that I use a backup utility to manage this, but I don't. Too lazy. I just end up using rsync on top-level folders that will fit on to a given disk. It's fun because I get to look up the syntax again every time I remember to refresh the backup.

Mind you, I wouldn't recommend someone do it my way. It's manual, prone to error, and more trouble than it's worth. FreeNAS or the like would be a much better solution, but I'm committed (or at least I should've been. badump-tish) My only point is, it has worked fine for something like a decade, maybe two -- what year is it again? -- and has been easy to un-$%#@ when I $%#@ it up. So, absolutely go for it. Keep a back up, though. Always. As long as you have a backup, you won't use it. Just, make sure you find a better way to manage it than I have.

Many interesting points being made, thanks you.

ShovelKnight wrote:

1) zpool export on the old machine 2) unplugging the drives from the old machine and connecting them to the new machine 3) zpool […]
Show full quote

1) zpool export on the old machine
2) unplugging the drives from the old machine and connecting them to the new machine
3) zpool import on the new machine

Based on this, I can say that if your FreeNAS install fails, you would be able to migrate your data no problem.

DosFreak wrote:

One of the best things about FreeNAS is that the install is very small, the configuration file can be backed up and restore with one click. So let's say your USB or SSD stops working and you have your config file. Install from the ISO, load your config and you are back up.

DosFreak, is backing up and restoring config file the same with zpool export and zpool import commands mentioned by ShovelKnight beforehand, or is it a different thing altogether?

DosFreak wrote:

If you never backed up your config then you'll have to input all your settings but you can easily mount your data.

Then I assume FreeNAS configuration file is small enough and simple enough, that you don't need to memorize (or write down) every single detail of your RAID volumes to recover them in a new FreeNAS instance. Instead, you can just mount them in a new FreeNAS. Am I correct?

Anyway, quite an impressive spec you have. Does FreeNAS really need a 16-core CPU?

When using the config file you don't have to do anything to mount your pool.
Without it you'll have to use the FreeNAS interface or CLI to import the existing pool from your disks. FreeNAS will scan your drives for it.

FreeNAS worked fine for my uses with my C2Q Q6600 with 16GB of ram with 80TB of data from 2008 to 2019 for file serving.

The limitation with CPU power is not for file serving but is if you want to transcode multiple media files using Plex. With a Q6600 transcoding one blu-ray would bump the CPU utilization way up so multiple blu-ray streams with that low of a CPU likely wouldn't be a good idea. Of course this was a 1080p blu-ray with a high bitrate.
Also if you are running multiple VMs then you would want a fast processor.

FreeNAS caches files in memory so it's recommended you use ECC ram. If you are hosting a file server with dozens of users then you need alot of memory but for a home user 16GB or less is fine.

The only time I noticed a performance issue with file serving with FreeNAS was when I was backing up to my external NAS and timeouts and slowness would occur when accessing the storage this was due to the onbaord SATA controllers of the 2008 motherboard which couldn't handle the load. Replacing with an SAS controller (while still using software raid) fixed that issue.

On my current motherboard this isn't an issue using the onboard ports.

DosFreak wrote:

When using the config file you don't have to do anything to mount your pool.
Without it you'll have to use the FreeNAS interface or CLI to import the existing pool from your disks. FreeNAS will scan your drives for it.

Ah, that's good! Thank you. Recoverability is pretty good, then.

DosFreak wrote:

FreeNAS worked fine for my uses with my C2Q Q6600 with 16GB of ram with 80TB of data from 2008 to 2019 for file serving. […]
Show full quote

FreeNAS worked fine for my uses with my C2Q Q6600 with 16GB of ram with 80TB of data from 2008 to 2019 for file serving.

The limitation with CPU power is not for file serving but is if you want to transcode multiple media files using Plex. With a Q6600 transcoding one blu-ray would bump the CPU utilization way up so multiple blu-ray streams with that low of a CPU likely wouldn't be a good idea. Of course this was a 1080p blu-ray with a high bitrate.
Also if you are running multiple VMs then you would want a fast processor.

FreeNAS caches files in memory so it's recommended you use ECC ram. If you are hosting a file server with dozens of users then you need alot of memory but for a home user 16GB or less is fine.

The only time I noticed a performance issue with file serving with FreeNAS was when I was backing up to my external NAS and timeouts and slowness would occur when accessing the storage this was due to the onbaord SATA controllers of the 2008 motherboard which couldn't handle the load. Replacing with an SAS controller (while still using software raid) fixed that issue.

On my current motherboard this isn't an issue using the onboard ports.

I see, thanks for the information. I'll definitely try it.