[GTALUG] Crontab versioning

Fri May 4 15:37:32 EDT 2018

On Fri, May 4, 2018 at 9:10 AM, Alex Beamish via talk <talk at gtalug.org>
wrote:

> On Fri, May 4, 2018 at 8:11 AM, Jamon Camisso via talk <talk at gtalug.org>
> wrote:
>
>> On 2018-05-03 10:44 AM, Alex Beamish via talk wrote:
>> > I'm developing scripts that get run by crontab, so I'm in there making
>> > updates fairly regularly. I would love to be able to document the
>> changes,
>> > so I'm wondering if there a usual and customary technique to version
>> > crontabs?
>>
>> What are you changing in crontabs so much that you need versioning?
>>
>
> I'm developing EDI scripts, and as development progresses, some of the
> older scripts are being replaced by newer scripts. On occasion, there are
> times when I need to disable a crontab job while I solve an issue.
>
> Having version control would be a great back-stop, and would also be
> useful from a historical point of view. (Why did I disable that job last
> Thursday? Where did the such and such script go?)
>
>
>> Shouldn't all the logic be in your scripts? Maybe I'm missing something,
>> but it seems like needless complication to me.
>>
>
> What to do and how to do it is in the scripts. How often to do it is in
> the crontab.
>
>
>>
>> > Ideally there would be some sort of hook around 'crontab -e', but
>> failing
>> > that, I'd have the output of 'crontab -l' (run regularly by cron?) go
>> to a
>> > versioned file. Plan B sounds a bit hokey to me.
>> If you do need to version things, you can just chuck a git repo under
>> /var/spool/cron/crontabs and ignore any files there you don't want
>> tracked.
>>
>
> Under my system, that directory is owned by root, and I prefer not to
> version anything as root; the code I'm developing is application code, not
> system code.
>

I'll share what I shared above again and explain how you don't have to
version anything as root with that scheme. Here are the contents of
roots/cronjobs/init.sls.

cron:
    service.running:
        - name: cron
        - enable: True

root-crontab:
    file.managed:
        - name: /root/root-crontab
        - source: salt://cronjobs/root-crontab
        - template: jinja
    cmd.run:
        - name: cd /root; /usr/bin/crontab -r; /usr/bin/crontab root-crontab

The "source:  salt://cronjobs/root-crontab" line represents where the file
"root-crontab" is stored. Let me back up. I have Git repo called "ss" which
is cloned/updated into a non-privileged directory.

cilkay at jupiter:~/ss$ ls -al
total 32
drwxrwxr-x   6 cilkay cilkay 4096 Mar 29 21:12 .
drwxrwxr-x. 13 cilkay cilkay 4096 Mar 21 13:15 ..
drwxrwxr-x   8 cilkay cilkay 4096 May  4 14:45 .git
-rw-rw-r--   1 cilkay cilkay  203 Mar 29 21:12 .gitignore
-rw-rw-r--   1 cilkay cilkay  303 Mar 29 21:12 minion
drwxrwxr-x   2 cilkay cilkay 4096 Mar 21 12:50 minion.d
drwxrwxr-x   3 cilkay cilkay 4096 Mar 21 12:50 pillar
drwxrwxr-x  19 cilkay cilkay 4096 May  3 16:21 roots

In the roots directory, the relevant bit is this.

cilkay at jupiter:~/ss$ ls -al roots/cronjobs/
total 16
drwxrwxr-x  2 cilkay cilkay 4096 Mar 21 12:50 .
drwxrwxr-x 19 cilkay cilkay 4096 May  3 16:21 ..
-rw-rw-r--  1 cilkay cilkay  303 Mar 29 21:12 init.sls
-rw-rw-r--  1 cilkay cilkay  331 Mar 29 21:12 root-crontab

init.sls is above. The file root-crontab contains this.

{% set current_env =  grains['current_env'] %}
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

# Update the server on every reboot

@reboot /usr/local/bin/update_tcm.sh

{% if current_env == 'dev' or current_env == 'qa' %}
# Update the server every day at 0410

10 4 * * * /usr/local/bin/update_tcm.sh
{% endif %}

The shell script update_tcm.sh contains this.

#! /bin/bash

# Have to invoke the SaltStack state (ss.init) twice in case
# the salt repo has been updated. If it is only updated once
# in highstate, it cannot execute any commands or set states
# that have changed since highstate was invoked.

/usr/bin/python /usr/bin/salt-call --local state.sls ss.init

# Must do this to have any custom modules be available.
/usr/bin/python /usr/bin/salt-call saltutil.sync_modules
/usr/bin/python /usr/bin/salt-call --local state.highstate

Salt is in a masterless minion configuration. In other words, this is a
self-configuring machine. It does not rely on some external controller. The
requirements on this particular project precluded the more conventional
master/minion configuration. All the virtual machines are behind corporate
firewalls. On first boot or reboot, update_tcm.sh is invoked and Salt runs
through all the states to bring the machine to the desired state.

That state file referenced in update_tcm.sh, ss.init, bootstraps the whole
thing. It does a git update on a specific version. If there are any changes
to the root-crontab file, it will be applied by the cronjobs.init state.
cronjobs.init maps to ~/ss/roots/cronjobs/init.sls. I could even leave out
the ".init" part in cronjobs.init because it's implicit that "init.sls"
would be invoked in a state anyway. I like being explicit.

cronjobs.init does a few things. in the "cron" section on top, it ensures
that the cron daemon is enabled and running. In the root-crontab section,
it copies ~/ss/roots/cronjobs/root-crontab to /root/root-crontab, removes
the existing crontab for root, and applies whatever is in
/root/root-crontab to root's crontab. This happens every time the machine
is rebooted or in the case of dev or qa configurations, at 0410 UTC. Of
course it can also be done on-demand.

Even for a relatively simple requirement like yours, using this approach is
perfect because you can start simple and add states incrementally.
Eventually, you'll be able to spin up a new server with very little human
intervention required. The whole point of this is to have your
infrastructure defined as code and kept under revision control.

Regards,

Clifford Ilkay

+1 647-778-8696
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gtalug.org/pipermail/talk/attachments/20180504/21057d09/attachment.html>