Managing IOS Configuration Snippets
ryanshea at google.com
Fri Feb 28 15:35:10 UTC 2014
It sounds like your suggestion is to not check version numbers, write smart
audits and react. Thinking of ACLs for example, version checking from a
remark is a limted shortcut (but still very useful way) to validate
distribution of an ACL. These things may change ~continuously. Grabbing a
config and jamming that into some rancid rcs thing is different than intent
version tracking. Our friends at Juniper (or the IOS-XR folks) seem to
think the version control paradigm lends itself well to router configs,
albeit at the whole-config level.
Oh, and I think we're crossing streams here regarding versioning. I am not
suggesting to take the configured state of the network, consider that
proper intent, record it in a revision and then check for it in the future.
I am assuming that the intent versions represent reviewed and tested
configuration states by ops/neteng. Different across the fleet does not
necessarily mean non-deterministic, so intent versioning can still be
useful. I'm sure (not first-hand sure) that some of the other suggested
templating systems could build parts of the expected config based on
attributes of a device.
On Fri, Feb 28, 2014 at 9:49 AM, Keegan Holley <no.spam at comcast.net> wrote:
> On Feb 28, 2014, at 9:11 AM, Ryan Shea <ryanshea at google.com> wrote:
> Keegan, don't get me wrong, I am not suggesting that even if version
> numbers were happily encoded in robust comments that this would be the same
> as actually digesting the configuration. If the function of checking using
> 'fancy versioning' is not an operational best practice, what do you suggest
> (all-knowing/singing/dancing tool which understands the configuration and
> your intent aside)? You say IF NTP or CPP were configured differently -
> with a large enough network there will always be configurations which have
> differences. With that as an operational constant, how do you determine
> which devices have the latest iteration of your line vty configuration.
> That’s what I mean. The things that lend well to to version tracking
> don’t tend to change much. How many ways are there to configure VTY lines,
> or NTP, or CPP, or even OSPF and if we’re talking about an access ACL why
> not just audit the configs to make sure that all the entries are there. Am
> I really going to care that one router has version 1.0 versus another
> router that has version 2.2.12 build9? It’s not source code..
> How often will a change not be rolled out to every router. This is again
> related to the size and churn of the network, but my practical estimation
> is that once you get into thousands of routers there will almost always be
> some that get missed.
> Again, a router that was missed is a reason for audit and remediation not
> versioning. If you find a router with config missing does it really matter
> what version it’s on and when that version was valid? Not in my experience.
> Comprehensive auditing is very important, and arguably more useful than
> version checking - but it requires that you make knowledgeable and complete
> assertions. I assert the my snmp config should look like the snmp snippet
> version 77 is easier to grok than "make sure our community string is not
> set to public" (and repeat hand-crafted audit logic for every segment of
> the config).
> There may be some differences, but those are normally due to equipment
> lifecycle, mergers/consolidations and such. It’s easy to refer to
> something as the config for a particular platform or company than a version
> number. This can be tracked in GIT or SVN. Even then there will not be
> constant changes. I’d lean towards standardization. So the equipment that
> cannot adhere to the defined standards probably won’t evolve much on it’s
> What if some of the configs don't match the defined versions? This is why
> it may make sense to break snippets into functional areas. "Just fix it"
> might be sane for a banner, but squashing an interface mtu change that was
> put there for a reason could end in tears. I consider this bit out of the
> scope of the question, but yes it is another important problem.
> I wasn’t saying just fix it. I was saying that router configs don’t lend
> well to versioning. With software for example, if something is different
> it might be a different version of that application with compatibility
> issues, dependencies, library issues, etc. When it’s a router config
> chances are someone fat-fingered something. Most of the time the best
> thing to do is to fix or at least alert on the error, not to record it as a
> valid config version. Again, this is for things that lend themselves to
> snippets. ACL’s, NTP, SNMP, CPP, even Spanning-tree. Not for things like
> interface IP’s or static routes that may be different across different
> boxes or location. If you’re referring to the latter I may have
> misunderstood your question..
> On Thu, Feb 27, 2014 at 10:03 PM, Christopher Morrow <
> morrowc.lists at gmail.com> wrote:
>> On Thu, Feb 27, 2014 at 8:38 PM, Keegan Holley <no.spam at comcast.net>
>> > Putting aside the fact that snippets aren't a good way to conceptualize
>> deployed router code, my gut still tells me to question the question here.
>> The first is does this stuff change often enough to warrant a fancy
>> versioning solution? I have yet to see NTP deployed in a different way
>> than when I first learned to configure it. Next, when it does change how
>> often is it not rolled out to every router. If NTP or CPP or SNMP or some
>> other administrative option were configured differently across my
>> sure, so you're saying that a large bit (maybe) of the router config
>> is 'one size fits all' and 'never changes' where 'never' is really
>> 'very infrequently'.
>> sure, agreed... but there are parts of the config that do change more
>> frequently (depending on the network perhaps)... how do you go about
>> seeing which version / setup is deployed EXCEPT by building a
>> home-grown 'config parser' and seeing that 'what is deployed matches
>> mostly what I have in my config store for this
>> router/class-of-router/network' ?
>> It's a shame that vendors of network equipment don't have to manage
>> large networks of their own equipment under constrained opex
>> environments (no fair comparing contracted work where you bill for
>> time + materials, that's the wrong incentive set)... I bet that'd get
>> them to fix stuff up right quick.
>> network I would want to audit it and fix not version control. What if
>> some of the configs don't match the defined versions? It may be
>> better to create standard templates and version them in SVN or GIT and
>> then use config backups to track which devices have the standard
>> configs. There are some for pay tools that can search for certain
>> statements on various boxes and either alert or remediate when
>> differences are found.
>> > On Feb 26, 2014, at 4:22 PM, Ryan Shea <ryanshea at google.com> wrote:
>> >> Howdy network operator cognoscenti,
>> >> I'd love to hear your creative and workable solutions for a way to
>> >> in-line the configuration revisions you have on your cisco-like
>> >> Let me clearify/frame:
>> >> You have a set of tested/approved configurations for your routers
>> which use
>> >> IOS style configuration. These configurations of course are always
>> >> and updated. You break these pieces of configuration into logical
>> >> for example a configuration file for NTP configuration, a file for
>> >> plane filter and store these in some revision control system. Put
>> aside for
>> >> the moment whether this is a reasonable way to comprehend deployed
>> >> configurations. What methods do some of you use to know which version
>> of a
>> >> configuration you have deployed to a given router for auditing and
>> >> purposes? Remarks are a convenient way to do this for ACLs - but I
>> >> have similar mechanics for top level configurations. About a decade
>> ago I
>> >> thought I'd be super clever and encode versioning information into the
>> >> location - but that is just awful and there is a much better way
>> >> is using, right? Flexible commenting on other vendors/platforms make
>> this a
>> >> bit easier.
>> >> Assume that this version encoding perfectly captures what is on the
>> >> and that no person is monkeying with the config... version 77 of the
>> >> control plane filter is the same everywhere.
More information about the NANOG