[GTALUG] Which Distro is Best for Running a ZFS-on-Linux Fileserver.

David Mason dmason at ryerson.ca
Thu Aug 30 14:10:49 EDT 2018


If you’re building 4TB, I’t recommend raidz2 which doesn’t give you much storage (4 x 1TB disks would give you <2TB of user storage). I have 5 x 1TB raidz2 which gives me 2.82TB of user storage. This may have changed, but when I went to ZFS you couldn’t add more disks and change the structure, you could only enlarge the disks - so if you went for a 5-disk raidz2, you were stuck with a 5-disk raidz2. I really hope this has changed, but I haven’t researched it.

The reason to go with raidz2 as the pool gets larger is that with raidz if a disk dies, while it rebuilds on the replacement disk, you have no redundancy, so any error on one of the other disks will be replicated.

So I’d recommend you get an additional SATA card, and then you don’t need to do ZFS on /, and you could add another drive to get better cost-effectiveness from your ZFS. I have / on a moderate-sized SSD and 5 hard drives in a ZFS pool.

One reason to use not-whole-disks for ZFS is that you can migrate to larger disks as time goes on.  I started out with 500GB (if I remember correctly) and was able to move to 1TB and copy the partitions.

Scott knows much more about this than I, but it may (now) be that (with a raidz2) you can enlarge by simply removing a xGB disk and replacing it with a 2xGB drive, let it rebuild, then replace the next one. The problem with that is you are replacing the disks, so it’s not very incremental. So if you can enlarge by adding an additional drive, that is substantially better.

../Dave
On Aug 30, 2018, 9:34 AM -0400, Scott Sullivan via talk <talk at gtalug.org>, wrote:
> On 2018-08-29 11:43 PM, Amos H. Weatherill wrote:
> > Scott,
> >
> > My reasoning for / on ZFS is pretty Simple ... the machine that is
> > becoming my first NAS only has 4 SATA Ports, so I can't afford to Waste
> > one on a boot drive.
>
> Recommended best Practice is to use ZFS with whole disks. That said,
> most of the arguments for that are 'because the manual says so',
> 'because zfs datasets are far more flexible then partitions' and
> references to Solairs taking advantage of disk caches. I throw that all
> out the windows in favor of doing at rest encryption, with whole luks
> partitions(*).
>
> My more practical argument is choice of MBR vs GUID partitioning. The
> latter is just cleaner (and the default when ZFS manages the disk), and
> works well with large disks (>2TB).
>
> But if your booting from that disk, you either need to be:
> "BIOS / CSM" + MBR + /boot
> or
> UEFI + GUID + "biosboot (partition)" + /boot
>
> Either of those makes for some lopsided partitioning, compared to the
> remainder of your data disks. A work around is to use a USB drive for
> your /boot. But in general your creating a more complex setup to
> maintain either way.
>
> Not knowing what hardware your using, if you have PCIe slots additional
> sata ports can be had for a low a $10/port.
>
> I've been using the Syba / IOCrest cards for a variety of needs,
> including ZFS arrays without issue.
>
> https://www.newegg.ca/Product/Product.aspx?Item=N82E16816124064
>
> > For Distro, I think I'll go with Fedora, as long as the / on ZFS guide
> > is sufficiently detailed.
>
> Fedora was not one of the ones I listed as having a guide to do rootfs
> on ZFS. If you found one, can you post the link?
>
> I'd also not recommend fedora in general for a NAS. CentOS would be a
> more dependable choice. LTS Ubuntu would be more reasonable as they
> ship(**) ZFS and support rootfs on it.
>
>
> ===
> * Native encryption in ZFS was added after the OpenZFS split from
> Sun/Oracle. So work to re-added it has been happening for a while. We're
> likely to see a stable version in the v0.8.x series.
>
> ** This is due to their adoption of a minority legal opinion about
> compatibility of the CDDL and GPL licenses that has not been tested in
> court.
> https://blog.ubuntu.com/2016/02/18/zfs-licensing-and-linux
> https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/
>
> --
> Scott Sullivan
> ---
> Talk Mailing List
> talk at gtalug.org
> https://gtalug.org/mailman/listinfo/talk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gtalug.org/pipermail/talk/attachments/20180830/0488620f/attachment.html>


More information about the talk mailing list