Templating/automating configuration

Wed Jun 7 18:52:22 UTC 2017

---- On Wed, 07 Jun 2017 04:23:33 -0500 <tim at pelican.org> wrote ----

Hi Brian,

On Tuesday, 6 June, 2017 21:48, "Brian Knight" <ml at knight-networks.com> said:

> Because we had different sources of truth which were written in-house, we wound up

> rolling our own template engine in Python. It took about 3 weeks to write the

> engine and adapt existing templates. Given a circuit ID, it generates the full

> config for copy and paste into a terminal session. It also hooks into a

> configuration parser tool, written in-house, that tracks configured interfaces, so

> it is easy to see whether the template would overwrite an existing interface.

Interesting. I'm going through much the same process at the moment, due to similar requirements - multiple sources of truth, validation that there's no clash with existing configs, but also with a requirement for network-wide atomic operations. The latter has been a strong driver for a custom tool - it's now grabbing an exclusive lock on all the devices, making all the checks, pushing all the config, commit check everywhere, commit everywhere, and only once all the commits succeed, release the locks. If any of those steps fail anywhere, we get to roll back everywhere. (Obviously with appropriate timeouts / back-offs / deadlock prevention, and specific to platforms with sane config management - no vanilla IOS).

Did you find anything to give you a leg-up on config parsing, or did you have to do that completely from scratch? At the moment, I'm working with PyEZ (I know, vendor lock-in, but we're firmly a Juniper shop, and going in eyes-open to the lock-in) to build a limited model of just the parts of the config I'm interested in validating, and it seems to be working.

The import process to the database runs directly on our rancid server, reading the downloaded configs out of the appropriate directory within rancid. Most of our gear is Cisco, so the ciscoconfparse module for Python helps a lot with organizing and querying the config.  From there, the config is parsed for key items like interface name, description, configured bandwidth, etc., and that info is then added or updated as necessary in the database.

Because it's dependent on rancid, there is some lag time between when a change is made and when the database gets updated, so we still strongly encourage running the pre-config checks for new circuits.  But with PyEZ, it looks like you easily could, after grabbing that lock, validate the existing config before pushing down new config.  Lots of possibilities there.  I'm envious that you have a vendor-written Python module as a choice!

> If I had a free hand and unlimited budget, I would find a single app that

> functions as a source of truth for all circuits and products, which includes a

> templating engine that hooks in easily.

Plus the business buy-in and the resource to go back and standardise all the existing configs, so the application can fully parse and understand the network before it starts. That, and a pony :)

Or, at least, rebuild the existing configs based on the new source of truth, so that subsequent config parsing conforms to a single standard.

Regards,

Tim.

Cheers,

-Brian