VOGONS

Common searches


First post, by DosFreak

User metadata
Rank l33t++
Rank
l33t++

So I've had my home server since about 2007 that I've upgraded over the years. For a file/plex server it does the job but I can tell I'm running it past it's limits with the amount of data I'm using and I can really only transcode one or two blurays 1080p blu-ray at the same time. I'd also like to be able to do more with this server than just server files and I'd like to make it last another 10yrs. I've purposely gone somewhat overkill on the upgrades.

Motherboard/CPU
Was looking at EPYC vs Threadripper but all I see for threadripper is gaming boards. I like the simplicity of the EPYC boards and 32core is overkill BUT I want to run Cisco VIRL for a test network, Multiple VMs, GPU passthru and I want it to last another 10yrs. Could drop down to the 16core.... I'd also like to move the VMs off my desktop as well and I'm running out of space on my desktop storing all of the GOG, Steam, EPIC, Bethesda games would make more since to keep them synced straight to the server instead of downloading the desktop and then backing up to the server.

Memory
Currently using 16GB below FreeNAS recommendations but not using dedupe or compression. FreeNAS recommends 32GB minimum for the amount of data I'm using. Also want to run Cisco VIRL which likely needs another 20GB plus whatever VMs I run.

Current Specs:
GIGABYTE GA-X38-DQ6 LGA 775 Intel X38 ATX Ultra Durable 2, Ultra Cooling Intel Motherboard
Intel Core 2 Quad Q6600 Kentsfield Quad-Core 2.4 GHz LGA 775 BX80562Q6600 Processor
SUPERMICRO CSE-743T-650B Black 4U Case w/ 650w PSU
Replaced case fans with Noctua Fans
SUPERMICRO CSE-M35T-1B 3 x 5.25" to 5 x 3.5" Hot-swap SATA HDD Trays
3x KINGWIN KF-1000-BK 3.5" Internal hot swap rack
PCI Geforce videocard
LSI 9305-24i x8 lane, PCIe 3.0 Full Height SAS SAS 9305 12 Gb/s SAS Host Bus Adapter
1x SSD for Freenas OS
11x 12TB Seagate Ironwolf HD's

UPGRADES:
SUPERMICRO MBDH11SSL-NC w/ AMD EPYC 7551P 32-Core CPU Installed DDR4 ATX Server Motherboard
NEMIX RAM 128GB 8x16GB DDR4-2933 PC4-23400 2Rx8 ECC Registered Memory
Noctua NH-U9 TR4-SP3, Premium CPU cooler for AMD TR4/SP3
Supermicro SSD-DM064-SMCMVN1 64GB SATA DOM
SAMSUNG 860 Pro Series 2.5" 4TB SATA III 3D NAND Internal Solid State Drive (SSD) MZ-76P4T0BW
Would like a 4TB M.2 instead of a 2.5" SSD but only NF1 is available but doubt it's compatible. May see if there is a PCI express card for that.....
10G NIC
Cisco SMB SG350XG-2F10-K9 12 Port Stackable Managed 10 Gb Ethernet Switch

How To Ask Questions The Smart Way
Make your games work offline

Reply 1 of 4, by BushLin

User metadata
Rank Member
Rank
Member

You seem like you have an actual use for a high number of threads but if you want dissuading:

AMD are just about to launch 7nm versions of all their current lineup.
Investing in a large single SATA drive when the proposed system has so many PCIe lanes seems a bit bizarre, it won't even saturate a 10Gb network link.

What really stands out is you're jumping from a dated consumer quad core to massively scaled, modern enterprise system. Per thread performance on the Core2 was beasted back when Sandy Bridge came out in 2011. Literally any of AMDs non-budget offerings will be a big upgrade. I'm not saying you should cheap out totally but consider that just the standard Ryzen consumer chips are going to be at least twice the cores, much better IPC and even though it's just two RAM channels, they're offering around 3x bandwidth on what you currently have.

Screw period correct; I wanted a faster system back then. I choose no dropped frames, super fast loading, fully compatible and quiet operation.

Reply 2 of 4, by awgamer

User metadata
Rank Oldbie
Rank
Oldbie

To me processing would be done by your main rig, not server. My idea of a server is extreme low power, fast(very,) reliable, spacious.

cheap used infiniband fdr 4x cards and cable that do 6.82GB/s, cpu performance irrelevant in the transfer, processing handled by the cards.
old system that uses vastly cheaper ddr3 ram, 128/256gb using 32/64gb dimms, caches drives, 90+ hit ratio, nearly always operating out of ram, it's slow for memory nowdays, but still far faster than disk in bandwidth and as always latency. Oh yeah, infiniband is extreme low latency too, so that aspect is retained.
modded to low power(10w idle) https://ssj3gohan.tweakblogs.net/blog/8217/fl … p-computer.html
1/2 tb ssd most frequent cache miss still relatively fast, ssd can be setup to cache or as tiered storage(faster, more space)
8tb hd (not raid, just one drive, security of data is handled by having backups, I gain more from power savings than 24/7 uninterrupted uptime for personal use, why I would also pay the premium to a point for a larger drive than array of the cheapest drives) online/on call data space
ups for brown outs.
offline backup: array of cheapest used 3tb sas drives(current price space sweet spot) that are only powered up and used for weekly/monthly backups, could be dedicated system for backups, power down configuration, usb, whatever as long as no power draw when not in use and its only used for backups.
low power file server allows for 24/7 without killing the power bill.

main rig, diskless, network boots off 6.82GB/s file server.

Reply 3 of 4, by SirNickity

User metadata
Rank Oldbie
Rank
Oldbie

I'm all about having the right tool for the job. I have three Supermicro boards in 1U rack enclosures. One is a low-power Atom that hosts my NAS (software RAID-5), another of the same model for my router / firewall, and a third multi-core board with a bunch of RAM for a VM host.

One big reason to have these separated is, if I lose one or it's just time for a rebuild, I don't lose all the services all at once. In a pinch I can even have the VM host take over temporarily as the router, so I can stay online while I'm rebuilding it. Having all your eggs in one basket makes for stressful maintenance.

Reply 4 of 4, by awgamer

User metadata
Rank Oldbie
Rank
Oldbie

Could add a nearline storage layer, how I would handle the large array dosfreak already has. Working set fitting in cache and ssd and roaming set venturing into the 8tb drive of the setup I mentioned, and then a powered down large array for the rare access. dosfreak could probably save on some power given he probably doesn't constantly touch all 11x 12tb on the regular, pull out a drive, or two for a smaller raid if you must, leaving the larger array for the infrequent access, the drive/array size configurations conforming to your particular usage pattern.