System Notes - talisker.SGK - File Server

These notes cover the creation of a FreeBSD fileserver serving encrypted ZFS volumes via Samba.

General Info

Hostname: talisker.SGK
Version: FreeBSD 12.1

Motherboard: X8DT3-LN4F (manual saved in hw_support)
Processors: 2x L5630 Xeons (4 cores @ 2.13 GHz, low power)
Memory: 48 GB (12x 4GB R2 Registered ECC)
        Note: Configured in lockstep mode, leaving 32 GB usable
Hard Drives:
    3x 120 GB Intel DC S3500 (3-way boot mirror)
    2x 8.0 TB WD Red (2-way mirror for media)
    3x 3.0 TB WD Red (3-way mirror for personal files)
    2x 2.0 TB used SAS (2-way mirror for scratch space)
    Note: The onboard SAS controller is limited to 2.0 TB max drive size.
          Consequently, one boot drive and the five drives >2.0 TB are on the
          SATA channels and all remaining drives are on SAS, even though this
          splits the boot mirror across controllers.

Installed Ports

    (due to dependency errors, build devel/llvm80 and devel/meson first)

Encrypted ZFS Mirrors

The following example creates a 2-way mirror using ada1 and ada2. First, create the encrypted devices.

geli init -l 256 /dev/ada1
geli init -l 256 /dev/ada2
geli attach /dev/ada1
geli attach /dev/ada2
geli status

In order to be prompted for the passphrase on boot, add the following line to /etc/rc.conf.

geli_devices="ada1 ada2"

Next, create the ZFS mirror. Enable compression by default, using LZ4 since it will abort the compression attempt if the initial results are not significant.

zpool create zfs_mirror_1 mirror /dev/ada1.eli /dev/ada2.eli
zfs set compress=lz4 zfs_mirror_1
zpool status

Automated ZFS Snapshots

Set the com.sun:auto-snapshot property on relevant zpools and verify it is inherited.

zfs set com.sun:auto-snapshot=true zfs_mirror_1

Create /etc/cron.d/zfs-snapshots with something like the following.

15,30,45 * * * * root /usr/local/sbin/zfs-auto-snapshot frequent  4
0        * * * * root /usr/local/sbin/zfs-auto-snapshot hourly   24
7        0 * * * root /usr/local/sbin/zfs-auto-snapshot daily     7
14       0 * * 7 root /usr/local/sbin/zfs-auto-snapshot weekly    4
28       0 1 * * root /usr/local/sbin/zfs-auto-snapshot monthly  12

Note that you can exclude specific snapshot intervals with the following property (e.g. frequent, daily, etc).

zfs set com.sun:auto-snapshot:frequent=false zfs_mirror_1

Automated ZFS Scrubs

Create /etc/cron.d/zfs-scrubs with the following contents.

0 0 0 * * root /sbin/zpool scrub zroot
0 0 0 * * root /sbin/zpool scrub zfs_mirror_1
0 0 0 * * root /sbin/zpool scrub zfs_mirror_2
0 0 0 * * root /sbin/zpool scrub zfs_mirror_3

Samba Notes

Create /usr/local/etc/smb4.conf with the following contents. Add additional entries for each zpool.

workgroup = WORKGROUP
server string = Samba Server
netbios name = Talisker
wins support = Yes
security = user
passdb backend = tdbsam
ntlm auth = yes

path = /zfs_mirror_1
valid users = ataylor
writable  = yes
browsable = yes
read only = no
guest ok = no
public = no
create mask = 0666
directory mask = 0755

Create a Samba user, using a different password than the system account.

pdbedit -a ataylor

Manually start Samba.

service samba_server start

Configure Samba to autostart on boot by adding the following to /etc/rc.conf.


Status Emails

After building, run make replace inside the mail/ssmtp port to automatically disable sendmail/etc and replace with ssmtp.

Create /usr/local/etc/ssmtp/ssmtp.conf with the following contents.

# The person who gets all mail for userids < 1000
# Make this empty to disable rewriting.

# The place where the mail goes. The actual machine name is required
# no MX records are consulted. Commonly mailhosts are named
# The example will fit if you are in and your mailhub is so named.

# Where will the mail seem to come from?

# The full hostname

# Set this to never rewrite the "From:" line (unless not given) and to
# use that address in the "from line" of the envelope.

# Use SSL/TLS to send secure messages to server.

# Credentials accepted by remote SMTP server

Edit /etc/passwd and /etc/master.passwd, changing the name of the root account from Charlie & to something suitable for the FROM: field in emails. After, run /usr/sbin/pwd_mkdb -p /etc/master.passwd.

Create /etc/cron.d/status-emails with suitable contents. For example:

0 0 * * 0 root /sbin/zpool list | /usr/bin/mail -s "talisker.SGK - zpool list"
0 0 * * 0 root /sbin/zpool status | /usr/bin/mail -s "talisker.SGK - zpool status"
0 0 * * 0 root /sbin/zfs list -t snapshot | /usr/bin/mail -s "talisker.SGK - zfs snapshots"
0 0 * * 0 root /sbin/zfs list | /usr/bin/mail -s "talisker.SGK - zfs list"
0 0 * * 0 root /usr/local/bin/zfs-stats -IMAE | /usr/bin/mail -s "talisker.SGK - zfs stats"