VOGONS

Common searches


First post, by DosFreak

User metadata
Rank l33t++
Rank
l33t++

This thread is for helping others to build their own TrueNAS servers, thread may be cleaned up over time. Please leave divergent discussions to other threads.

First we'll start with:

SERVER SPECS

CASE
Ideally you'd use a case that has removable hard drives that can also show the status of those drives (power on, failure state) and also so you can label the drives with which port it is attached to so when there is a drive failure you can pull the right hard drive.
If all you need is M.2 NVME or 2.5" you can use internal bays or PCIe just make sure you label them!
You want a case that has good airflow but obviously if you have a ton of hard drives taking up space in the front of your case then that may not be possible, I've had 13x hard drives in the front of mine for years and although the temp of the HDs is high it is still within spec which is all that really matters.
Also look for a case that is easy to work on and that will last your for 10+ years. I've been using my SUPERMICRO CSE-743T-650B since 2007.

PSU
Size this appropriately , you don't want this too low but also not too high due to heat and energy bills. I always size mine too high since I think I may do more with my server (add a graphics card for passthru or transcoding) but never do.

MOTHERBOARD
You'll see people state that you need tons of memory for TrueNAS which is not true. 8GB or less can be fine depending on your uses. Something to keep in mind though is that just like SQL TrueNAS will actually use your memory so the more memory you have then the more in can store in memory. For a home server this usually isn't that big of a deal. For TrueNAS you should be buying a server motherboard anyway which should have the potential for alot of memory so you can upgrade later.
Find a board that has a lot of SATA ports, PCIe cards would be more heat, energy, affect air flow, potential failures, speed or compatibility issues.
If you want to run TrueNAS from a DOM then make sure there is a power connector for a SATA DOM or a USB2 or USB3 header on the board. You can also waste an onboard M.2 if you want but TrueNAS runs fine from a DOM. If you are using a USB or DOM I'd recommend storing the system dataset elsewhere.
It's likely that you'll use up all your SATA ports so you may need or want to use your PCI express for storage. I use "ASUS Hyper M.2 X16 PCIe" which store 4x m.2 nvme drives but requires that your motherboard supports bifurcation. You can find pci cards that don't require bifurcation, looks for the ones that have 2x slots and dedicate 4x to each m.2 and as a last resort use the ones with bridge chips.
Make sure the motherboard is either headless with an IPMI interface or has onboard video. No sense in wasting heat or energy on a video card.
Use a motherboard with 10GBE, you may not need it now but you don't want to have to spend money later and eat up a PCIe slot, if you find a motherboard you like that doesn't have it then make sure that you'll have enough PCIe slots to use it.

PROCESSOR
You don't want a processor that's too low or too high but you do need to take into account what you want to do with the server.
If it's pure file storage with no VMs or transcoding or dedupe then go lower.
If you plan on using any of the above then go higher.
If you are building a new server with a server class motherboard you'd have to actually go out of your way to find something that's not sufficient.
If you are repurposing an old motherboard with an old processor then it'll likely be only good enough for file storage.

OS
You can install TrueNAS to whatever you want as long as it's bootable. Something to keep in mind is that TrueNAS has a "System Dataset", if you are booting TrueNAS from USB then change the system dataset to some other location. If you are using a NAS and don't want your drives to spin up alot then also change this to an SSD if you have the option.

MEMORY
ECC memory should be used with TrueNAS but is not a requirement.
Memory is cheap and your files are important so use ECC memory.
Again TrueNAS doesn't need alot of memory for home usage, if you are using VMs or dedupe then add more.
If you are unsure then start with 8GB (or whatever makes sense since you don't want to throw away that stick when you add more memory) and only upgrade if you need to, not because you want to.

STORAGE
For my data I use raidz3 which uses 3x drives for parity out of 13x Seagate 16TB drives. It's recommended to not have so many drives in one pool for performance reasons but I can only put so many drives in this case and I want to keep everything in one case so 13x drives in one pool works for me.
For this pool for dedupe I have a Asus Hyper M.2 x16 PCI 4.0 card with 4x Seagate FireCuda 530 1TB that is used for my Special VDEV to store my DDT for dedupe.
For my Windows VMs that I access via SMB I use a ASUS Hyper M.2 x16 PCIe 3.0 x4 with 2x Sabrent Rocket Q 8TB M.2 (I store the original VMs on my raidz3)
For my TrueNAS bhyve VMs I store them on my 1x onboard M.2 Samsung SSD 970 EVO 2TB

BOOT DRIVE
Should be at least 8GB
It's preferred that this not be to USB but I haven't had any issues as long as I set the "System Dataset" to SSD/Platter HD.

DATA DRIVES
If you want the most affordable then this can't be SSD, use SSD for applications that need it such as Databases, Virtual Machines, etc.
If this is just for home use you may not even need a SSD depending on your usage.
Options are Mirroring,raidz1,raidz2,raidz3 (# = number of disk failures)
Only use mirroring if two drives or for performance reasons.
Only use raidz1 if you are limited in the number of drives you can use.
So plan on using either raidz2 or raidz3. If you have a lot of data and a lot of drives go with raidz3.
If you have too many drives you should create multiple pools. I've been fine with raidz3 with 13 drives for years but that is pushing it a bit, you usually see less than 12 recommended in a pool.
For home use the main factor will be cost so likely most home users will have a single pool instead of multiple pools, so plan for that.

DATA
You'll need to figure out how much data you'll need to store and to take into account how much it will grow over time.
The more data you have then you'll need larger drives and more of them.

VMs
If you plan to run VMs using bhyve on TrueNAS then you should use a SSD so account for that.
If the VM is simple to setup and backed up then you don't need to mirror them just use a single drive.
Bhyve is not as feature complete as other VM products and kind of annoying but it may be good enough for your uses. I use it for a Windows and Linux VM where I have the online game clients installed.

VMs DATA
You can setup a zvol (a volume) for your VM as a different drive and this can be stored wherever. Likely your SSD space is minimal so you may need to store this on the same drive as your other data. For my uses which is storing online games this works fine.

CASE: SUPERMICRO CSE-743T-650B Black 4U Case 11 Bays (Replaced case fans with Noctua Fans)
CASE ADDON: Supermicro CSE-M35TQB 5-in-3 Hot Swap SAS/SATA Mobile Rack (Black)
PSU: Supermicro 1200w ATX12V Power Supply - 1 kW, 1.20 kW - Internal - 110 V AC, 220 V AC (PWS-1K25P-PQ)
MOTHERBOARD: SUPERMICRO MBDH11SSL-I Rev 1.01
PROCESSOR: AMD EPYC 7351P 16-Core
PROCESSOR COOLER: Noctua NH-U9 TR4-SP3, Premium CPU cooler for AMD TR4/SP3
MEMORY: 4x Supermicro Certified MEM-DR432L-SL03-ER26 32GB DDR4-2666 LP ECC REG DIMM
ADDON CARD: ASUS Hyper M.2 x16 PCIe 3.0 x4 Expansion Card V2 supports 4 NVMe M.2 (2242/2260/2280/22110) up to 128 Gbps for Intel ...
ADDON CARD: ASUS Hyper M.2 X16 PCIe 4.0 X4 Expansion Card Supports 4 NVMe M.2 (2242/2260/2280/22110) up to 256Gbps for AMD 3rd Ryzen sTRX40, AM4 Socket
NIC: Intel Ethernet Converged Network Adapter X540-T1 10GBE
HARD DRIVE: 13x Seagate IronWolf Pro 16TB NAS (ST16000NE000)
DOM: 1x Supermicro 64GB 520 MB/s Solid State Drive (SSD-DM064-SMCMVN1)
ADDON CARD: 2x Sabrent Rocket Q 8TB M.2 for ASUS Hyper M.2 x16
ADDON CARD: 4x Seagate FireCuda 530 1TB Solid State Drive - M.2 PCIe Gen4 ×4 NVMe 1.4
ONBOARD M.2: 1x Samsung SSD 970 EVO 2TB - NVMe PCIe M.2 2280 SSD (MZ-V7E2T0BW) for onboard M.2

Last edited by DosFreak on 2022-09-13, 15:28. Edited 18 times in total.

How To Ask Questions The Smart Way
Make your games work offline

Reply 1 of 12, by DosFreak

User metadata
Rank l33t++
Rank
l33t++

BACKUP DEVICE SPECS

OFFSITE

QNAP TS-670 (TrueNAS Scale)
Usage: Backups and Plex
Filesystem: zfs raidz1
OWC 16GB (2x8GB) ram Upgrade PC3-12800 DDR3L 1600MHz sodimm 204 Pin CL11
SAMSUNG FIT Plus USB 3.1 Flash Drive 128GB - (MUF-128AB/AM)
1x QNAP QM2-2P-2p10g1tb (system dataset and jails) 10GbE w/ 2x Samsung 970 Evo
2x Noctua NF-A9 PWM, Premium Quiet Fan, 4-Pin (92mm, Brown)
Duttek USB Header to USB Cable,Motherboard USB 2.0 Adapter Cable (Not Connected)
4x Seagate Exos X20 20TB
2x Seagate Exos X18 18TB

QNAP TVS-671 (TrueNAS Scale)
Usage: Backups and Plex
Filesystem: zfs raidz1
SAMSUNG FIT Plus USB 3.1 Flash Drive 128GB - (MUF-128AB/AM)
1x QNAP QM2-2P-384 M.2 to PCI Express Adapter (system dataset and docker)
1x QNAP QXG-10G1T 10GbE
2x Noctua NF-A9 PWM, Premium Quiet Fan, 4-Pin (92mm, Brown)
Duttek USB Header to USB Cable,Motherboard USB 2.0 Adapter Cable (Not Connected)
Cablecc 2pcs 9Pin/10Pin Motherboard Female Header to Dual USB 2.0 Female Adapter Vertical Type (Not Connected)
6x Seagate IronWolf Pro 16TB

Synology DS1512+ (DSM)
Usage: Backups
Filesystem: btrfs
5x Seagate IronWolf Pro 16TB

ONSITE

QNAP TVS-x72XT (TrueNAS Core)
Usage: Backups
Filesystem: zfs raidz1
2x32GB G.Skill RipJaws Series 260-Pin SO-DIMM PC4-21300 DDR4 2666 CL18-18-18-43 1.20V Dual Channel Memory Model F4-2666C18D-64GRS
1x QNAP QM2-2P-384 M.2 to PCI Express Adapter w/ 2x Seagate FireCuda 530 1TB (Dedupe special VDEV)
2x Samsung SSD 970 EVO Plus 1TB (Onboard m.2) w/ 2x be quiet! BZ003 MC1 Pro M.2 SSD Cooler (TrueNAS)
8x Seagate IronWolf Pro 16TB

Last edited by DosFreak on 2022-09-06, 21:59. Edited 28 times in total.

How To Ask Questions The Smart Way
Make your games work offline

Reply 3 of 12, by DosFreak

User metadata
Rank l33t++
Rank
l33t++

RESERVED

BACKUP METHODS
Rsync (TrueNAS Task, Client Task or CLI)
sudo rsync -rvc --inplace /run/media/folder/ /media/folder/

Robocopy (Windows Task, CLI)
Robocopy "\\192.168.0.1\" "\\192.168.0.2\" /MIR /TBD /COPY:DAT /FFT /XD .RECYCLE /XD @RECYCLE /DCOPY:AT /MT:32

Replication (TrueNAS Task)

Last edited by DosFreak on 2022-08-07, 14:08. Edited 3 times in total.

How To Ask Questions The Smart Way
Make your games work offline

Reply 4 of 12, by DosFreak

User metadata
Rank l33t++
Rank
l33t++

Wiped Quts Hero from my QNAP TVS-x72XT today and installed TrueNAS. No longer long boot times and wondering if storage manager will ever load.
Coping over 100TB of data to it now and also setup dedupe on it so we'll see how that goes.

Added specs for offsite storage to above.
Can't remember what SSD I have on two of the QNAP so will need to find out.

Next step for this thread will be on setting up TrueNAS on the hardware.

How To Ask Questions The Smart Way
Make your games work offline

Reply 5 of 12, by Intel486dx33

User metadata
Rank l33t
Rank
l33t

What is the Best hardware for a budget home NAS under $1000 ?…

CPU - Type ?…( cores, threads, i5, i7, Xeon )
RAM - Speed and Amount ?
Motherboard - ?
Hard drive type - ? ( HGST, WD, Seagate )
Network card - 2.5ghz.
Network switch - Type, Model, and Brand ?
Ethernet Cable - Cat-6 or Cat-7 ?

WiFi card ?…..Will a NAS work with a WiFi card connection ?

Can you connect to a TrueNAS Server with a Client like Hummingbird or Xhost ?

Reply 6 of 12, by DosFreak

User metadata
Rank l33t++
Rank
l33t++

Which room is the NAS going to be placed?
What do you plan to do with the NAS? Backups, video streaming, VMs,etc
Is it going to be ATX or smaller?
How much data are you going to store?
How many users and what bandwidth are needed to access the server?
As far as management of my server I use IPMI and the TrueNAS web interface and SSH.

How To Ask Questions The Smart Way
Make your games work offline

Reply 7 of 12, by DosFreak

User metadata
Rank l33t++
Rank
l33t++

Still working this. Did some tests and 10GbE does same me a considerable amount of time especially with the /MT switch with robocopy. So bought a combined M.2 / 10GbE NIC for my TVS-670. Unfortunately FreeBSD (TrueNAS Core) doesn't support so had to move to Scale on this NAS which I'm doing right now. I already had to run Scale on my other NAS due to some oddity with TrueNAS Core and it's bootloader try to scan zfs data drives (this bios has no ability to disable drives from booting) so the system wouldn't boot for hours.

How To Ask Questions The Smart Way
Make your games work offline

Reply 8 of 12, by DosFreak

User metadata
Rank l33t++
Rank
l33t++

TrueNAS Scale working great on both QNAP. Sticking with TrueNAS Core on my server as long as possible since I like a secure OS.

Added some info on Boot and Data drives, need to add info on configuring but doesn't seem like much interest in TrueNAS here.

How To Ask Questions The Smart Way
Make your games work offline

Reply 9 of 12, by Thermalwrong

User metadata
Rank Oldbie
Rank
Oldbie

Thanks for putting this guide together 😀 I've been half-assedly planning an upgrade from my current HP Micro server with xpenology & some RAID1s to a TrueNAS setup. I was going to use an Intel 4th gen i5 with an ITX board but then read up and understood that ZFS really needs ECC.
So now it's getting ever closer to ready with an 8 port HBA (already reflashed to IT mode), an Asrock AM4 ITX motherboard and some ECC memory. All in a 6-bay ITX case from a vendor in China, since it has to fit the same footprint as my Microserver.
I keep putting it off though, there's a lot to TrueNAS to take in - seeing a current guide in one place is a big help. If I'm reading this right, then TrueNAS Scale is better if I want to run VMs on the system as well, but it can have trouble with some hardware configurations. Heh, I think me being on the fence is what put me off setting this up a few months ago.
Along with, initially I was thinking I'd put in 3 of the 6x 4TB drives til I need more space, then add in the rest maybe as a separate pool later.
I only really need 8TB of backed up storage right now, but if RAIDZ2 is that much better than Z1 then maybe one big pool would be better.

Reply 10 of 12, by DosFreak

User metadata
Rank l33t++
Rank
l33t++

In can feel overwhelming but likely it's just fear of the unknown. Best to jump in and start swimming. 😀

For VMs TrueNAS Core with bhyve might be sufficient for your needs but you'd likely be better off with KVM on TrueNAS Scale if you plan on using VMs alot. It's possibly that you may not even need KVM if docker is sufficent for what you want to do.

As long as you can backup and restore the data quickly then you could do 3 drives for now if you wanted but if you plan on using more drives anyway it would make more sense to do it to get it done.
The extra drives means more redundancy and the extra space would also allow you to enable snapshots.

How To Ask Questions The Smart Way
Make your games work offline

Reply 11 of 12, by DosFreak

User metadata
Rank l33t++
Rank
l33t++

Found another odd bootloader issue but this time with Scale instead of core.
On both Scale installs on both QNAPs I receive a "Compression algorithm not supported" before the TrueNAS scale boot menu appears.
On only one of the QNAP devices does this result in a hang and then a reboot before the boot menu appears.
The error appears very quickly before the boot menu when it doesn't hang.
For the QNAP where it constantly reboots before the boot meny I've tried different USB ports and different flash drives as well as an M.2 to USB adapter. Same issue.
Can't boot from M.2 unless i find a M.2 with an oprom. Don't feel like using clover.
Probably going to see if I can boot from a m.2 sata tomorrow.
Last resort will be to yank the hard drives and try booting from a HD, don't plan on actually using the system this way just want to see if it has the same isue.

Can't go back to TrueNAS core on this QNAP since it's bootloader freaks out when it sees ZFS data drives and the BIOS won't let me disable them.

As usual with Linux you can't really find anything for this issue except for people blaming bad USB drives. Sigh.

How To Ask Questions The Smart Way
Make your games work offline

Reply 12 of 12, by DosFreak

User metadata
Rank l33t++
Rank
l33t++

Figured it out. Had to disable the serial console in the TrueNAS settings.
I did notice the serial settings in the grub config earlier but didn't see any reason to mess with it since it didn't seem to relate to the error, that's what I get for trying to make sense of things. Problem solved after hours of troubleshooting yay.

How To Ask Questions The Smart Way
Make your games work offline