Operations task management software?
Lee
ler762 at gmail.com
Thu Jul 28 00:20:29 UTC 2016
On 7/27/16, David Hubbard <dhubbard at dino.hostasaurus.com> wrote:
> Full automation is planned but does not eliminate the need for the software.
> Zero human auditing of fully automated processes and data collection are
> not acceptable to various certifying entities, the relevant auditors, the
> inevitably involved lawyers, and won’t pick up on bad data, like a bad
> thermometer or snmp counter that says a CRAC is 65 degrees when it’s really
> 90. So I’m still going to need a management solution to the issue whether
> it’s to tell someone to do the work or to tell someone to check the
> automated work.
You have a ticketing system - right? Create a cron job that creates a
ticket to check whatever.
Regards,
Lee
>
> David
>
> On 7/27/16, 7:19 PM, "Lee" <ler762 at gmail.com> wrote:
>
> On 7/27/16, David Hubbard <dhubbard at dino.hostasaurus.com> wrote:
> > Hi all, curious if anyone has recommendations on software that helps
> manage
> > routine duties assigned to operations staff?
>
> Have computers do the routine scut work - not people.
>
> > For example, let’s say we have a P&P that says someone from the netops
> group
> > must check that Rancid is successfully backing up all router configs
> > bi-weekly.
>
> You've got the source code for rancid, so change rancid-run to do
> something like
> LOGFILE=$LOGDIR/$GROUP.`date +%Y%m%d.%H%M%S`; export LOGFILE
> change the
> ) >$LOGDIR/$GROUP.`date +%Y%m%d.%H%M%S` 2>&1
> to
> ) >$LOGFILE 2>&1
>
> and then in control_rancid do something like
> grep "clogin error:" $LOGFILE | sort | uniq -c >$TMP.fail
> if [ -s $TMP.fail ]; then
> # got some output, mail the report
> ...
>
> Do the same type thing for checking on
> > backup failures, backup internet circuit status, out of band
> interfaces, etc.
>
> Automate the checks, put the scripts in crontab & mail out an
> "OhNoes!" or "all clear" msg at the end. At which point you're left
> with the problem of making sure the managers are looking at the emails
> & making sure whatever problems are found actually get fixed :)
>
> Regards,
> Lee
>
>
>
More information about the NANOG
mailing list