[GTALUG] sysadmining is full of stupid details

[D. Hugh Redelmeier]

Context: I had a working system that was stable for a decade (mail server,
DNS, VPN, etc.).  But that was insane.  Regular updates are needed but old 
systems fall off the map.  So rebasing is needed.  And the longer you put 
it off, the worse it gets

So I'm redoing everything, based on CentOS 7.  What a sea of shallow 
lore is needed.  By "shallow", I mean that each thing is easy.  What
makes the problem hard is that there is so much of this crud to get
right.

Today's example: spamassassin's log file shows a lot of messages to do
with not being able to access the DNS server via 127.0.0.1.  Eg:

Nov 03 12:12:09 redeye spamd[1455]: dns: sendto() to [::1]:53 failed: Connection refused, failing over to [127.0.0.1]:53
Nov 03 12:12:09 redeye spamd[1455]: dns: sendto() to [127.0.0.1]:53 failed: Connection refused, failing over to [::1]:53
Nov 03 12:12:09 redeye spamd[1455]: dns: bad dns reply: Connection refused

Why doesn't it use the server specified by /etc/resolv.conf?
Apparently the spamassassin folk think that you'd be better off not
sharing a DNS server since a lot of RBL servers limit the amount of
traffic that a particular client DNS server is allowed to generate.
So they force you to run a local DNS server on the same machine as
spamassassin.

This happens to be a lose for my system.  My spamassassin host was
using a DNS shared by my LAN.  This was not shared with any other
spamassassin, so I had no problem that that limitation solved.  But it
created a problem.  And the diagnostic wasn't rubbed in my face, and the
diagnostic does not point at the problem unless you know more than you
should have to.

I, of course, tried to find where this is explained and failed.  I did
see suggestions for running a local DNS server, but the reason did not
apply.

Simple fix:
	yum install unbound
	systemctl start unbound
	systemctl enable unbound
I hope that the default settings are reasonable.

What a waste of my time.

Irony: once I fixed this, spamassassin spams syslog with a half dozen
lines per mail message processed.

And this is just one of a million trivial impediments.


Notice that that systemd (and the previous init systems) needed
separate commands for starting a service now (start) and starting one
on boot (enable).  Surely the most common case is to do both.


Firewalld's GUI has this same problem.  It is more confusing because
you click away at tick boxes, the ones you have changed are not
distinguished.  You cannot see the "now" settings at the same time as
the "on boot" settings.


Looking at the log (to discover the original problem) is arcane in
itself.  The systemd command to look at logs is journalctl(1) (I'm
surprised it is in section 1).  The flag -b limits the output to
things since the most recent boot.  To look at a particular service, use
_SYSTEMD_UNIT=spamassassin.service.  How awful.  It can be shortened to
"-u spamassassin", but that's hardly mentioned in the enormous journalctl(1).