Wednesday, May 6, 2009

Cisco : "this behavior is expected" & Archive Config Logger

The last 2 months i've been busy playing with an ASR1000, finding bugs and "fighting" with the BUs.

Dealing with a lot of issues lately (not necessarily of ASR's), there are a few times that i have come across Cisco's motto "this behavior is expected". One of them was the fact that the very useful functionality of archive config logger (Configuration Logging) is ruined by the inability to filter out per-user configs applied through an aaa server.

The Configuration Change Notification and Logging (Configuration Logging) feature allows the tracking of configuration changes entered on a per-session and per-user basis by implementing a configuration log. The configuration log tracks each configuration command that is applied, who applied the command, the parser return code for the command, and the time the command was applied.

As you all know, a basic configuration like the following will help you keep track of changes in the configuration of a router:

log config
logging enable

In a "normal" router you'll probably see something like the following:

router#sh archive log config all
idx sess user@line Logged command
8 3 user@vty0 | no ipv6 nd suppress-ra
9 3 user@vty0 | ipv6 ospf 100 area 10
10 4 user@vty0 |no ip sla monitor 99
11 5 user@vty0 |interface GigabitEthernet0/2.35211201
12 5 user@vty0 | encapsulation dot1Q 3521 second-dot1q 1201
13 5 user@vty0 | ip address

If you happen to have a router that is acting like a bras, accepting ppp sessions and using radius attributes in order to apply per-user ACLs and/or routes, then you'll find out that the above feature is totally useless, because all these per-user entries are stored into the archive logging buffer, overwriting the actual configuration changes you want to watch (when the configuration log is full, the oldest log entry will be removed every time a new entry is added).

router#sh archive log config all
idx sess user@line Logged command
177 0 console@console |permit ip host any
178 0 console@console |deny ip any any
179 0 console@console |deny ip any any
180 0 console@console |ip route
181 0 console@console |ip route
182 0 console@console |ip access-list extended Virtual-Access2.10785#106240381
183 0 console@console |deny ip any
184 0 console@console |deny ip any
185 0 console@console |deny ip any
186 0 console@console |deny ip any
187 0 console@console |deny ip any
188 0 console@console |deny ip any

Note : Please don't ask me about the double entries...

According to Cisco's cli help, only changes to the running config, not the one stored on nvram, are to be logged.

router(config-archive)#log ?
config Logging changes to the running configuration

If you issue the "show run" command (which is supposed to display the running configuration), you won't see these per-user config changes. That's because all these per-user configs, after being downloaded from the aaa server, have a flag set that they should not be nvgen'ed. So they are not shown.

In the Cisco IOS software configuration model, the configuration state is maintained in a distributed manner, with each component storing its own configuration state. To retrieve configuration information, the software must poll every component to collect the distributed information. This configuration state retrieval operation is performed by a process known as nonvolatile generation (NVGEN), and it is used by command line interpreter (CLI) commands such as show running-configuration, write memory, and copy system:running-configuration to display or copy the running system configuration. When invoked, NVGEN queries each system component and each instance of interface or other configuration objects. A running configuration file is constructed as NVGEN traverses the system performing these queries.

Also, according to Cisco:

Restrictions for Configuration Change Notification and Logging
•Only complete commands input in a configuration mode are logged.
•Commands that are part of a configuration file applied with the copy command are not logged.

The Configuration Change Notification and Logging feature tracks changes made to the Cisco IOS software running configuration by maintaining a configuration log. This configuration log tracks changes initiated only through the command-line interface (CLI) or HTTP.

Why Cisco chose to include these per-user "dynamic" config changes into the archive logger, making it another thrown-away feature, is still a mystery to me.

There is also the possibility that this kind of continuous logging might be causing a little increase in cpu usage too.

Yep, time for another Cisco-should-have-fixed-it-in-the-first-place PER.


  1. I tried to look for the ip accounting stats on the ASR 1000 interface... looks like the command doesn't work.. What's your say?

  2. Spikes, i rarely use this command, so i haven't really tested it on the ASR1k.


Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Greece License.