Saturday, February 28, 2009

RFC 5396, RFC 5398, RFC 5462

Here are 3 interesting RFCs that i have read lately:

RFC 5398
Autonomous System (AS) Number Reservation for Documentation Use

To allow documentation to accurately describe deployment examples, the use of public or private-use AS numbers is inappropriate, and a reserved block of AS numbers is required. This ensures that documentation does not clash with public- or private-use AS numbers in deployed networks, and mitigates the risks to operational integrity of the network through inappropriate use of documentation to perform literal configuration of routing elements on production systems.

To allow for examples relating to the transition to use of 32-bit AS numbers to be correctly described, a reservation of two blocks of AS numbers is proposed in this document. One reserved block of 16 contiguous AS numbers is to lie in the range of numbers that can be expressed as a 16-bit AS number value (i.e., values less than 65536), and a second reserved block of 16 contiguous AS numbers is to lie in the range of numbers that can only be expressed as 32-bit AS numbers (values greater than 65535).

IANA has reserved a contiguous block of 16 Autonomous System numbers from the unallocated number range within the "16-bit" number set for documentation purposes, namely 64496 - 64511, and a contiguous block of 16 Autonomous System numbers from the "32-bit" number set for documentation, namely 65536 - 65551. These reservations have been documented in the IANA AS Number Registry.


So, from now on, you know what AS number you should use in your documentation.

2-byte (16-bit) ASNs : 64496 - 64511
4-byte (32-bit) ASNs : 65536 - 65551


RFC 5396
Textual Representation of Autonomous System (AS) Numbers

A taxonomy of representation for AS numbers is as follows:

asplain refers to a syntax scheme of representing all AS numbers using decimal integer notation. Using asplain notation, an AS number of value 65526 would be represented as the string "65526" and an AS number of value 65546 would be represented as the string "65546".

asdot+ refers to a syntax scheme of representing all AS numbers using a notation of two integer values joined by a period character: .. Using asdot+ notation, an AS number of value 65526 would be represented as the string "0.65526" and an AS number of value 65546 would be represented as the string "1.10".

asdot refers to a syntax scheme of representing AS number values less than 65536 using asplain notation and representing AS number values equal to or greater than 65536 using asdot+ notation. Using asdot notation, an AS number of value 65526 would be represented as the string "65526" and an AS number of value 65546 would be represented as the string "1.10".


Cisco has chosen the asdot notation for the 4-byte ASNs in most mainstream releases (12.0(32)S12, 12.4(24)T), which might cause you a little bit of headache when dealing with regexp. To make it more interesting, in some other releases (12.0(32)SY8; GSR only?) Cisco has given the option (through the "bgp asnotation dot" command) to use asplain or asdot when displaying or matching (through regexp) 4-byte ASNs, while both formats are available when configuring.
As you may already know, asplain is being used widely for the well-known 2-byte ASNs.

Here
you'll find an interesting article regarding 4-byte ASNs by Jeff Doyle.


RFC 5462
Multiprotocol Label Switching (MPLS) Label Stack Entry: "EXP" Field Renamed to "Traffic Class" Field

The format of an MPLS label stack entry is defined by RFC 3032 to include a three-bit field called the "EXP field". The exact use of this field is not defined by RFC 3032, except to state that it is to be "reserved for experimental use".

The EXP field, from the start, was intended to carry "Class of Service" (CoS) information. The field was actually called the "Class of Service field" in early versions of the working group document that was published as RFC 3032. However, at the time that RFC 3032 was published, the exact usage of this "Class of Service" field was not agreed upon and the field was designated as "experimental use"; hence, the name has since been the "EXP field".

The designation "for experimental use" has led other Standards Development Organizations (SDOs) and implementors to assume that it is possible to use the field for other purposes. This document changes the name of the field to clearly indicate its use as a traffic classification field.


The use of the EXP field was first defined in RFC 3270, where one of the longest acronyms ever invented has been used:

E-LSP
   which gets translated into
EXP-Inferred-PSC LSP
   which gets translated into
EXP-Inferred-PHB (Scheduling Class) LSP
   which gets translated into
EXP-Inferred-Per Hop Behavior (Scheduling Class) LSP
   which finally gets translated into
EXP-Inferred-Per Hop Behavior (Scheduling Class) Label Switched Path
   which could also get further translated into
Experimental-Inferred-Per Hop Behavior (Scheduling Class) Label Switched Path
   which should get translated into
EIPHBSCLSP to keep everyone happy :-p

Keep in mind that there is a Traffic Class (TC) field in the IPv6 header too.

Saturday, February 21, 2009

EEM 3.0 – Variable Logic Pt #2

Continuing from my previous "EEM 3.0 Variable Logic Part #1" article, here is Part #2. You'll notice that although EEM begun as a customized approach to event detection and recovery, it has turned into a very powerful framework that not only has the ability to monitor events and act accordingly, but it can also take complex informational or corrective actions when the monitored events occur. Of course there is still TCL that can be used for all kinds of scripting magic, but it's nice to have something completely IOS-configured.

1) Checking traceroute hops automatically
Below you'll find an applet that checks the output of a traceroute everyday at 17:07 and if a specific hop is found, then the ospf process is cleared.

Actions used : "set", "if/else/end", "regexp".


event manager applet CHECK-TRACEROUTE-HOPS-APPLET
! the applet is run daily at 17:07 with a max execution time of 60"
! cron-entry uses the known unix syntax
event timer cron name Daily-17:07 cron-entry "07 17 * * *" maxrun 60
action 1.1 set trace_dest "3.3.3.3"
action 1.3 set hop_found "23.23.23.3"
action 1.5 syslog msg "Searching for hop $hop_found in traceroute to $trace_dest."
action 2.1 cli command "trace $trace_dest" pattern "#"
! we match a regexp of a pattern (which includes the hop we're looking for) on the traceroute result
action 2.3 regexp "(.*) (23\.23\.23\.3) (.*)" "$_cli_result" _match _sub1 _sub2
! we check the regexp result (1=found, 0=not found)
action 2.5 if $_regexp_result eq 1
action 2.5.1 syslog msg "Hop $hop_found found in traceroute to $trace_dest."
action 2.5.3 cli command "enable"
action 2.5.5 cli command "clear ip ospf process" pattern "[no]"
action 2.5.7 cli command "yes" pattern "#"
action 2.5.9 syslog msg "OSPF process cleared."
action 2.7 else
action 2.7.1 syslog msg "Hop $hop_found not found in traceroute to $trace_dest."
action 2.9 end


You can use something like your provider's web server for traceroute destination. If traceroute takes too long to complete, you can increase the applet execution time (default is 20") like below:


event timer cron name Daily-17:07 cron-entry "07 17 * * *" maxrun 120


When the applet is executed automatically, the following is displayed on screen (having "term mon" enabled) :

Feb 21 17:07:00.031: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOPS-APPLET: Searching for hop 23.23.23.3 in traceroute to 3.3.3.3.
Feb 21 17:07:03.303: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOPS-APPLET: Hop 23.23.23.3 found in traceroute to 3.3.3.3.
Feb 21 17:07:03: %OSPF-5-ADJCHG: Process 100, Nbr 2.2.2.2 on FastEthernet0/0 from FULL to DOWN, Neighbor Down: Interface down or detached
Feb 21 17:07:03.847: %HA_EM-6-LOG: CHECK-TRACEROUTE-HOPS-APPLET: OSPF process cleared.
Feb 21 17:07:04: %OSPF-5-ADJCHG: Process 100, Nbr 2.2.2.2 on FastEthernet0/0 from LOADING to FULL, Loading Done


2) Checking traceroute hops automatically until a specific hop is not found
You can also change the cli commands executed; i.e. in a broadband connection, you can have the dialer interface cleared, until you get connected (or not connected) to a specific router.

Actions used : "set", "decrement", "wait", "while/end", "if/else/end", "regexp".


event manager applet CHECK-DIALER-TRACEROUTE-HOPS-APPLET
event timer cron name Daily-21:40 cron-entry "40 21 * * *" maxrun 60
action 1.1 set trace_dest "3.3.3.3"
action 1.3 set hop_to_find "23.23.23.3"
action 1.9 syslog msg "Searching for hop $hop_to_find in traceroute to $trace_dest."
! we do a traceroute 3 times only, in order to avoid having the link continuously flapping
action 2.1 set trace_tries "3"
action 2.9 while $trace_tries gt 0
action 3.1 cli command "trace $trace_dest" pattern "#"
action 3.3 regexp "(.*) (23\.23\.23\.3) (.*)" "$_cli_result"
action 4.1 if $_regexp_result eq 1
action 4.3 syslog msg "Hop $hop_to_find found in traceroute to $trace_dest."
action 4.4 cli command "enable"
action 4.5 cli command "clear int dialer 1"
action 4.6 syslog msg "Interface dialer 1 cleared."
action 4.7 decrement trace_tries 1
action 4.8 syslog msg "$trace_tries tries left."
action 4.9 wait 10
action 5.1 else
action 5.3 syslog msg "Hop $hop_to_find not found in traceroute to $trace_dest."
action 5.5 syslog msg "Keeping connection."
action 5.7 set trace_tries "0"
action 5.9 end
action 6.1 end


When the applet is executed automatically, the following is displayed on screen :

As long as (max 3 tries) the router we connect is the one we want to avoid, we drop the connection.

Feb 21 21:46:00.042: %HA_EM-6-LOG: CHECK-DIALER-TRACEROUTE-HOPS-APPLET: Searching for hop 23.23.23.3 in traceroute to 3.3.3.3.
R1#
Feb 21 21:46:03.338: %HA_EM-6-LOG: CHECK-DIALER-TRACEROUTE-HOPS-APPLET: Hop 23.23.23.3 found in traceroute to 3.3.3.3.
Feb 21 21:46:03.410: %HA_EM-6-LOG: CHECK-DIALER-TRACEROUTE-HOPS-APPLET: Interface dialer 1 cleared.
Feb 21 21:46:03.414: %HA_EM-6-LOG: CHECK-DIALER-TRACEROUTE-HOPS-APPLET: 2 tries left.
R1#
Feb 21 21:46:16.774: %HA_EM-6-LOG: CHECK-DIALER-TRACEROUTE-HOPS-APPLET: Hop 23.23.23.3 found in traceroute to 3.3.3.3.
Feb 21 21:46:16.822: %HA_EM-6-LOG: CHECK-DIALER-TRACEROUTE-HOPS-APPLET: Interface dialer 1 cleared.
Feb 21 21:46:16.834: %HA_EM-6-LOG: CHECK-DIALER-TRACEROUTE-HOPS-APPLET: 1 tries left.
R1#
Feb 21 21:46:30.098: %HA_EM-6-LOG: CHECK-DIALER-TRACEROUTE-HOPS-APPLET: Hop 23.23.23.3 found in traceroute to 3.3.3.3.
Feb 21 21:46:30.142: %HA_EM-6-LOG: CHECK-DIALER-TRACEROUTE-HOPS-APPLET: Interface dialer 1 cleared.
Feb 21 21:46:30.154: %HA_EM-6-LOG: CHECK-DIALER-TRACEROUTE-HOPS-APPLET: 0 tries left.


When we get connected to a router different from the one we want to avoid, we keep the connection.

Feb 21 21:48:00.042: %HA_EM-6-LOG: CHECK-DIALER-TRACEROUTE-HOPS-APPLET: Searching for hop 23.23.23.3 in traceroute to 3.3.3.3.
Feb 21 21:48:03.318: %HA_EM-6-LOG: CHECK-DIALER-TRACEROUTE-HOPS-APPLET: Hop 23.23.23.3 not found in traceroute to 3.3.3.3.
Feb 21 21:48:03.322: %HA_EM-6-LOG: CHECK-DIALER-TRACEROUTE-HOPS-APPLET: Keeping connection.


3) Checking traceroute hops manually providing arguments
If you want to modify the initial traceroute applet, so you can run it manually (passing the ip addresses as arguments), you can use the following code. I have added a little bit of the new cli-interactivity too, by using the new "puts" action.

Actions used : "set", "if/else/end", "regexp", "puts".

event manager applet MANUAL-CHECK-TRACEROUTE-HOPS-APPLET
! we make the applet synchronous so we can get the output on our local tty
event none sync yes maxrun 60
! we check the number of arguments
action 1.1 if $_none_argc eq 2
! we use the 1st argument as the traceroute destination address
action 2.1 set trace_dest "$_none_arg1"
! we use the 2nd argument as the hop we're looking for
action 2.3 set hop_found "$_none_arg2"
action 2.5 puts "Searching for hop $hop_found in traceroute to $trace_dest."
action 3.1 cli command "trace $trace_dest" pattern "#"
action 3.3 regexp "(.*) (23\.23\.23\.3) (.*)" "$_cli_result" _match _sub1 _sub2
action 4.1 if $_regexp_result eq 1
action 4.3 puts "Hop $hop_found found in traceroute to $trace_dest."
action 4.5 cli command "enable"
action 4.6 cli command "clear ip ospf process" pattern "[no]"
action 4.7 cli command "yes" pattern "#"
action 4.9 puts "OSPF process cleared."
action 5.1 else
action 5.3 puts "Hop $hop_found not found in traceroute to $trace_dest."
action 5.5 end
action 6.1 else
action 6.3 puts "Wrong number of arguments supplied."
action 6.5 puts "Please use : applet <dest> <hop>"
action 6.9 end


When the applet is executed manually, the following is displayed on screen :

R1#event manager run MANUAL-CHECK-TRACEROUTE-HOPS-APPLET 3.3.3.3
Wrong number of arguments supplied.
Please use : applet <dest> <hop>

R1#event manager run MANUAL-CHECK-TRACEROUTE-HOPS-APPLET
Wrong number of arguments supplied.
Please use : applet <dest> <hop>

R1#event manager run MANUAL-CHECK-TRACEROUTE-HOPS-APPLET 2.2.2.2 23.23.23.3
Searching for hop 23.23.23.3 in traceroute to 2.2.2.2.
Hop 23.23.23.3 not found in traceroute to 2.2.2.2.

R1#event manager run MANUAL-CHECK-TRACEROUTE-HOPS-APPLET 3.3.3.3 23.23.23.3
Searching for hop 23.23.23.3 in traceroute to 3.3.3.3.
Hop 23.23.23.3 found in traceroute to 3.3.3.3.
OSPF process cleared.

Feb 21 17:26:12: %OSPF-5-ADJCHG: Process 100, Nbr 2.2.2.2 on FastEthernet0/0 from FULL to DOWN, Neighbor Down: Interface down or detached
Feb 21 17:26:18: %OSPF-5-ADJCHG: Process 100, Nbr 2.2.2.2 on FastEthernet0/0 from LOADING to FULL, Loading Done


Notes
a) In all TRACEROUTE applets, i couldn't find a way to include a variable into the regexp (this is why the 2nd argument is not actually used). I someone could shed some light, i would be very grateful.

b) There is an action "string replace" which could be used to replace "." with "\." in the regexp ("." means a single character in regular expressions, so we need to put a "\" in front of it, if we want to treat it like the actual dot character), but it supports only string replacement based on character positions.

c) The "_match _sub1 _sub2" arguments in the regexp action are not actually needed in our case, because we're just searching into a string (the arguments are needed if you want to get the results of the match too). We could use the "match string" action instead of regexp (see below), but it doesn't seem to work. Documentation is probably wrong too, because the syntax description is different from the example.


action 3.3 string match "$hop_found" "$_cli_result"
action 4.1 if $_string_result eq 1


d) The "puts" command displays the output on your local console (tty), when the applet is run as synchronous. This means that the system logger (i.e. syslog) is bypassed. If you change the "sync yes" to "sync no", then you get the output on syslog.

e) The timestamp of syslog output produced by the applets should use the "service timestamps log" format, but it doesn't.

f) I hope EEM 3.0 will soon get included in other releases too, like the SX/SR ones.

Wednesday, February 18, 2009

EEM 3.0 – Variable Logic Pt #1

Latest IOS (12.4(22)T) from Cisco includes EEM 3.0, which is a major improvement over previous versions. Among the various enhancements introduced, the most interesting seem the interactive CLI support (a much requested feature) and the variable logic.

Here is a simple applet that displays a countdown timer before shutting down, using a little bit of variable logic.

Actions used : "set", "while/end", "wait", subtract", "decrement".


event manager applet WHILE-TEST-APPLET
! we run manually the applet, so no event detector is used
event none
! we set the variable $i equal to “10” ($i is a long-integer)
action 1.0 set i "10"
! the while loop starts here and lasts as long as $i is greater than or equal to 0
action 2.0 while $i ge 0
! we print a syslog message showing the variable value
action 2.2 syslog msg "$i secs till shutdown"
! we wait for 1 sec
action 2.3 wait 1
! we subtract 1 from $i (the result is not automatically saved to $i)
action 2.4 subtract $i 1
! we store the result of the previous operation to $i
action 2.6 set i "$_result"
! the while loop ends here
action 2.8 end
! we print the final syslog message
action 3.0 syslog msg "Shutting down now!!!"
! You can put other command (i.e. reload) here


Keep in mind that instead of

action 2.4 subtract $i 1
action 2.6 set i "$_result"

you can use just a single command

action 2.5 decrement i 1


Notes
a) All arithmetic calculations are performed as long integers without any checks for overflow.
b) If you need to break out of a loop you created by mistake, try pressing “Ctrl+Shift+6 + X” many times.
c) You get a warning in case you forget to close a loop. You don't get a warning in case you have used wrong logic inside the loop.
d) You can insert actual (that are saved in the configuration) comments between your actions, but it gets a little bit complicated, because you have to use extra actions ("action x comment xxx") in order to do so! C’mon Cisco, we all know you can do better.

At the end, we run the applet manually:

R1#event manager run WHILE-TEST-APPLET

*Feb 17 00:20:18.635: %HA_EM-6-LOG: WHILE-TEST-APPLET: 10 secs till shutdown
*Feb 17 00:20:19.635: %HA_EM-6-LOG: WHILE-TEST-APPLET: 9 secs till shutdown
*Feb 17 00:20:20.635: %HA_EM-6-LOG: WHILE-TEST-APPLET: 8 secs till shutdown
*Feb 17 00:20:21.635: %HA_EM-6-LOG: WHILE-TEST-APPLET: 7 secs till shutdown
*Feb 17 00:20:22.635: %HA_EM-6-LOG: WHILE-TEST-APPLET: 6 secs till shutdown
*Feb 17 00:20:23.635: %HA_EM-6-LOG: WHILE-TEST-APPLET: 5 secs till shutdown
*Feb 17 00:20:24.635: %HA_EM-6-LOG: WHILE-TEST-APPLET: 4 secs till shutdown
*Feb 17 00:20:25.643: %HA_EM-6-LOG: WHILE-TEST-APPLET: 3 secs till shutdown
*Feb 17 00:20:26.647: %HA_EM-6-LOG: WHILE-TEST-APPLET: 2 secs till shutdown
*Feb 17 00:20:27.651: %HA_EM-6-LOG: WHILE-TEST-APPLET: 1 secs till shutdown
*Feb 17 00:20:28.651: %HA_EM-6-LOG: WHILE-TEST-APPLET: 0 secs till shutdown
R1#
*Feb 17 00:20:29.651: %HA_EM-6-LOG: WHILE-TEST-APPLET: Shutting down now!!!


Here are another 2 applets that create or delete 10 loopbacks with ip addresses based on their number.


event manager applet CREATE-10-LOOPBACKS-APPLET
event none
action 1.1 set i "1"
action 2.1 cli command "enable"
action 2.3 cli command "conf t"
action 3.1 while $i le 10
action 3.3 cli command "interface Loopback $i"
action 3.5 cli command "ip address $i.$i.$i.$i 255.255.255.255"
action 3.7 increment i 1
action 3.9 end
action 4.1 cli command "end"

event manager applet DELETE-10-LOOPBACKS-APPLET
event none
action 1.1 set i "1"
action 2.1 cli command "enable"
action 2.3 cli command "conf t"
action 3.1 while $i le 10
action 3.3 cli command "no interface Loopback $i"
action 3.7 increment i 1
action 3.9 end
action 4.1 cli command "end"


First we create the loopback interfaces :

R1#event manager run CREATE-10-LOOPBACKS-APPLET

*Feb 18 01:23:19.999: %LINK-3-UPDOWN: Interface Loopback1, changed state to up
*Feb 18 01:23:20.315: %LINK-3-UPDOWN: Interface Loopback2, changed state to up
*Feb 18 01:23:20.675: %LINK-3-UPDOWN: Interface Loopback3, changed state to up
*Feb 18 01:23:20.987: %LINK-3-UPDOWN: Interface Loopback4, changed state to up
*Feb 18 01:23:21.027: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback1, changed state to up
*Feb 18 01:23:21.259: %SYS-5-CONFIG_I: Configured from console by eem-user on vty0 (EEM:CREATE-10-LOOPBACKS-APPLET)
*Feb 18 01:23:21.267: %LINK-3-UPDOWN: Interface Loopback5, changed state to up
*Feb 18 01:23:21.315: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback2, changed state to up
*Feb 18 01:23:21.651: %LINK-3-UPDOWN: Interface Loopback6, changed state to up
*Feb 18 01:23:21.675: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback3, changed state to up
*Feb 18 01:23:21.943: %LINK-3-UPDOWN: Interface Loopback7, changed state to up
*Feb 18 01:23:21.987: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback4, changed state to up
*Feb 18 01:23:22.267: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback5, changed state to up
*Feb 18 01:23:22.287: %LINK-3-UPDOWN: Interface Loopback8, changed state to up
*Feb 18 01:23:22.647: %LINK-3-UPDOWN: Interface Loopback9, changed state to up
*Feb 18 01:23:22.655: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback6, changed state to up
*Feb 18 01:23:22.943: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback7, changed state to up
*Feb 18 01:23:22.959: %LINK-3-UPDOWN: Interface Loopback10, changed state to up
*Feb 18 01:23:23.287: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback8, changed state to up
*Feb 18 01:23:23.647: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback9, changed state to up
*Feb 18 01:23:23.959: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback10, changed state to up
R1#
R1#sh ip int br
Interface IP-Address OK? Method Status Protocol
FastEthernet0/0 unassigned YES NVRAM up up
FastEthernet0/1 unassigned YES NVRAM up up
Loopback1 1.1.1.1 YES manual up up
Loopback2 2.2.2.2 YES manual up up
Loopback3 3.3.3.3 YES manual up up
Loopback4 4.4.4.4 YES manual up up
Loopback5 5.5.5.5 YES manual up up
Loopback6 6.6.6.6 YES manual up up
Loopback7 7.7.7.7 YES manual up up
Loopback8 8.8.8.8 YES manual up up
Loopback9 9.9.9.9 YES manual up up
Loopback10 10.10.10.10 YES manual up up


Then we delete them :

R1#event manager run DELETE-10-LOOPBACKS-APPLET

*Feb 18 01:24:30.455: %LINK-5-CHANGED: Interface Loopback1, changed state to administratively down
*Feb 18 01:24:30.499: %LINK-5-CHANGED: Interface Loopback2, changed state to administratively down
*Feb 18 01:24:30.835: %LINK-5-CHANGED: Interface Loopback3, changed state to administratively down
*Feb 18 01:24:31.171: %LINK-5-CHANGED: Interface Loopback4, changed state to administratively down
*Feb 18 01:24:31.195: %SYS-5-CONFIG_I: Configured from console by eem-user on vty0 (EEM:DELETE-10-LOOPBACKS-APPLET)
*Feb 18 01:24:31.307: %LINK-5-CHANGED: Interface Loopback5, changed state to administratively down
*Feb 18 01:24:31.455: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback1, changed state to down
*Feb 18 01:24:31.499: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback2, changed state to down
*Feb 18 01:24:31.591: %LINK-5-CHANGED: Interface Loopback6, changed state to administratively down
*Feb 18 01:24:31.835: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback3, changed state to down
*Feb 18 01:24:31.871: %LINK-5-CHANGED: Interface Loopback7, changed state to administratively down
*Feb 18 01:24:32.167: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback4, changed state to down
*Feb 18 01:24:32.187: %LINK-5-CHANGED: Interface Loopback8, changed state to administratively down
*Feb 18 01:24:32.307: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback5, changed state to down
*Feb 18 01:24:32.543: %LINK-5-CHANGED: Interface Loopback9, changed state to administratively down
*Feb 18 01:24:32.591: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback6, changed state to down
*Feb 18 01:24:32.871: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback7, changed state to down
*Feb 18 01:24:32.919: %LINK-5-CHANGED: Interface Loopback10, changed state to administratively down
*Feb 18 01:24:33.187: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback8, changed state to down
*Feb 18 01:24:33.543: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback9, changed state to down
*Feb 18 01:24:33.919: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback10, changed state to down
R1#
R1#sh ip int br
Interface IP-Address OK? Method Status Protocol
FastEthernet0/0 unassigned YES NVRAM up up
FastEthernet0/1 unassigned YES NVRAM up up


I know i might be dreaming, but the days of TCL are coming to an end (just kidding) :-p
Anyhow, the more applet functionality we get, the less scripting we need.

Wednesday, February 11, 2009

Storm-control differences between 6500/7600 and other switches

3550/3560/3750
The graph in the following figure shows broadcast traffic patterns on an interface over a given period of time. In this example, the broadcast traffic being forwarded exceeded the configured threshold between time intervals T1 and T2 and between T4 and T5. When the amount of specified traffic exceeds the threshold, all traffic of that kind is dropped for the next time period. Therefore, broadcast traffic is blocked during the intervals following T2 and T5. At the next time interval (for example, T3), if broadcast traffic does not exceed the threshold, it is again forwarded.


Conclusion : if storm-control threshold is exceeded during a 1 sec interval, broadcast traffic is dropped for the whole next interval(s).


6500/7600
Traffic storm control monitors the level of each traffic type for which you enable traffic storm control in 1-second traffic storm control intervals. Within an interval, when the ingress traffic for which traffic storm control is enabled reaches the traffic storm control level that is configured on the port, traffic storm control drops the traffic until the traffic storm control interval ends.

The graph in the following figure shows the broadcast traffic patterns on a LAN interface over a given interval. In this example, traffic storm control occurs between times T1 and T2 and between T4 and T5. During those intervals, the amount of broadcast traffic exceeded the configured threshold.



Conclusion : if storm-control threshold is exceeded during a 1 sec interval, broadcast traffic is dropped from the time the threshold was exceeded until the end of the current interval.

That means that on 3550/3560/3750 switches you can have broadcast traffic above the storm-control threshold within a 1 sec interval.

Besides the above inner-workings of storm-control, 3550/3560/3750 switches support rising and falling thresholds, different actions for storm-control and thresholds based on pps/bps levels.

Different architectures, different implementations.

Tuesday, February 10, 2009

ARP spoofing - The easy way

...or why a shut interface is not always shut...

Copying from Wikipedia:
Address Resolution Protocol (ARP) spoofing, also known as ARP poisoning or ARP Poison Routing (APR), is a technique used to attack an Ethernet wired or wireless network which may allow an attacker to sniff data frames on a local area network (LAN), modify the traffic, or stop the traffic altogether (known as a denial of service attack). The attack can obviously only happen on networks that indeed make use of ARP and not another method.

The principle of ARP spoofing is to send fake, or "spoofed", ARP messages to an Ethernet LAN. Generally, the aim is to associate the attacker's MAC address with the IP address of another node (such as the default gateway). Any traffic meant for that IP address would be mistakenly sent to the attacker instead. The attacker could then choose to forward the traffic to the actual default gateway (passive sniffing) or modify the data before forwarding it (man-in-the-middle attack). The attacker could also launch a denial-of-service attack against a victim by associating a nonexistent MAC address to the IP address of the victim's default gateway.

ARP spoofing attacks can be run from a compromised host, a jack box, or a hacker's machine that is connected directly onto the target Ethernet segment.


"...or a backup 7200 with shut interfaces." i should add :-P

Here is the complete story...

Some months ago a fellow engineer had to replace a 7200/G1 with a 7200/G2. He had prepared the 7200/G2 with the same configuration as the 7200/G1, shut all the interfaces and made sure that the new ethernet/optical cables were working ok.

Both the 7200s were connected to a 6500 switch. 7200/G1 was the production router, 7200/G2 was the backup router to be put in production during a maintenance window in 2 days. One day later, we had a power failure on the 7200/G2 which caused the router to reload. Nothing to worry, since its power supplies were connected to a test facility and during the maintenance window they would be moved to the normal facilities. But here comes the fun part that came up during the reload:

6500


Sep 24 13:29:12: %LINK-3-UPDOWN: Interface GigabitEthernet3/21, changed state to up
Sep 24 13:29:12: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet3/21,
changed state to up
Sep 24 13:29:14: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet3/21,
changed state to down
Sep 24 13:29:14: %LINK-3-UPDOWN: Interface GigabitEthernet3/21, changed state to down


7200/G1

Sep 24 13:29:12: %IP-4-DUPADDR: Duplicate address x.x.x.x on GigabitEthernet0/1,
sourced by xxxx.xxxx.xxxx


During the reload, the ethernet interfaces of the 7200/G2 (which were shut in the configuration) moved to the up state for some seconds. And during those seconds, the ip functionality of them was activated! So we had 2 identical ip addresses on the same LAN, sending similar arp messages. This is more than enough to cause havoc in the production router (something that happened in our case), until arp-cache times out or arp-table is updated/cleared.

For everyone interested, the bug opened is CSCsv69222 and it's still being investigated by the developers, who think that it's a very corner case and fixing it might break something else.

It applies to 7200s with either NPE-G1 or NPE-G2 using a Marvell ethernet controller, under these conditions:

GBIC & no autonegotiation (enabled through "no negotiation auto")
RJ-45 & autonegotiation (enabled through "speed/duplex auto")
RJ-45 & no autonegotiation (enabled through "speed/duplex 1000/full")

In the case of

GBIC & autonegotiation (enabled through "negotiation auto")
it doesn't seem to apply.

What worries me most is the fact that a part of the configuration ("ip address") seems to be activated before everything else, even the shut command. Or even better, according to Cisco's answer, the shut command is ignored :

resolution to this problem is having "no ip addr" or to change the IP address statement in the configuration, since this is common to all interfaces and the problem is due to the IOS init code implementation. The actual implementation is that we should have some different "ip addr" related statement in the configuration. Otherwise during reload time, it will consider them as new interface, so to put them in shutdown mode we have to include the "no ip addr".

Conclusion:

shut + ip addr => no shut + ip addr
shut + no ip addr => shut + no ip addr

Surely there are quite a few of workarounds that can be implemented (shut the 6500 ports instead, remove portfast from switch, enable carrier-delay on 7200, etc), but why should i prefer a workaround instead of a bug fix?

Tuesday, February 3, 2009

6500/7600 & storm-control limitations

I do not know about you, but i quite often fall into CCO documents that instead of giving answers, they arise more questions.

Here is an example of CCO documentation cryptology causing even more confusion.

From Configuring Traffic Storm Control in 7600/12.2SR:

Note On these modules, a level value of 0.33 percent or less suppresses all traffic:

—WS-X6704-10GE
—WS-X6748-SFP
—WS-X6724-SFP
—WS-X6748-GE-TX


From Configuring Traffic-Storm Control in 6500/12.2SXF

On these modules, these levels suppress all traffic:

–WS-X6704-10GE: 0.33 percent or less
–WS-X6748-GE-TX 10Mbps ports: 0.33 percent or less
–WS-X6748-GE-TX 100Mbps ports: 0.03 percent or less


Questions risen :

1) "all traffic" means ALL traffic or all traffic referenced in the relevant storm-control config (i.e. broadcast/multicast/unicast)?
You probably guessed it right; It's not very clear, but it's the 2nd.

2) Why are modules WS-X6724-SFP and WS-X6748-SFP not included in the 6500 document?
This is because the 7600 document is somewhat wrong. I tried the WS-X6724-SFP in both platforms and it didn't experience this 0.33 percent limitation. Also, according to CSCsq75731 :

A WS-X6724-SFP doesn't block all broadcast traffic as we may expect it will do per the limitation described in the config guide.

Probably the same applies to WS-X6748-SFP (WS-X6724-SFP x 2).

3) For WS-X6748-GE-TX, is it 0.03% or 0.33% ?
Once again the 7600 document is wrong. WS-X6748-GE-TX ports operating in 10 Mbps use 0.33%, while ports operating in 100 Mbps use 0.03%. 1 Gbps ports do not have any limitation.

4)
Why 0.33% and 0.03% ? What exactly is the problem with these modules?
From what i have have found by asking around, there must be a specific asic which imposes this limitation. You can use the following command in order to find some of the major asics used in your 6500/7600 modules:


7600#sh mod
Mod Ports Card Type Model Serial No.
--- ----- -------------------------------------- ------------------ -----------
3 24 CEF720 24 port 1000mb SFP WS-X6724-SFP XXXXXXXXXXX
4 48 CEF720 48 port 10/100/1000mb Ethernet WS-X6748-GE-TX XXXXXXXXXXX
5 4 CEF720 4 port 10-Gigabit Ethernet WS-X6704-10GE XXXXXXXXXXX

7600#sh asic-version slot 3
Module in slot 3 has 3 type(s) of ASICs
ASIC Name Count Version
JANUS 1 (1.0)
SSA 1 (9.0)
ROHINI 2 (1.5)
7600#sh asic-version slot 4
Module in slot 4 has 3 type(s) of ASICs
ASIC Name Count Version
JANUS 2 (1.0)
SSA 2 (9.0)
ROHINI 4 (1.5)
7600#sh asic-version slot 5
Module in slot 5 has 3 type(s) of ASICs
ASIC Name Count Version
JANUS 2 (1.0)
SSA 2 (9.0)
ROHINI 4 (1.5)


As you can also see, all 3 modules use the same asics, with WS-X6748-GE-TX and WS-X6704-10GE having double the number of each one of them (probably due to double bandwidth = double fabric connections). So i'm still missing some data here. I tried various "remote command module x show platform hardware asicreg" commands, but nothing interesting came up.

Update 4 Feb 10:00 : I had an email from Cisco that this is a limitation of modules that have the ROHINI asic and their ports are operating in 10 Mbps, 100 Mbps or 10 Gbps (CSCee79131). That explains a lot and it's nice to know. They also said, they'll try to update the documentation (but i'm not counting on that).

And now some quite interesting answers:

For these modules what is actually happening is that the level value configured for storm-control is multiplied by 3 and then round down to its closest integer. If 0 is the result, then 0 is actually used. If something >= 1 is the result, then the original configured value is used. Therefore, anything that is 0.33 or less will be rounded down to 0.



























































storm-control configured value multiplication by 3 rounding down storm-control actual value
0.10 0.10 x 3 = 0.30 0.00 0.00
0.20 0.20 x 3 = 0.60 0.00 0.00
0.30 0.30 x 3 = 0.90 0.00 0.00
0.33 0.33 x 3 = 0.99 0.00 0.00
0.34 0.34 x 3 = 1.02 1.00 0.34
0.50 0.50 x 3 = 1.50 1.00 0.50
0.90 0.90 x 3 = 2.70 2.00 0.90
1.00 1.00 x 3 = 3.00 3.00 1.00



Here is a much cleaner summary including the module :































ModulePort speedstorm-control configured valuestorm-control actual value
WS-X6704-10GE -X <= 0.33%0.00%
WS-X6704-10GE -X > 0.33%X
WS-X6748-GE-TX1 GbpsX X
WS-X6748-GE-TX100 MbpsX <= 0.03%0.00%
WS-X6748-GE-TX100 MbpsX > 0.03%X
WS-X6748-GE-TX10 MbpsX <= 0.33%0.00%
WS-X6748-GE-TX10 MbpsX > 0.33%X
OTHER -X X


As an extra note, you might want to ignore the errdisable functionality regarding storm-control on these platforms, since it's not actually supported.


7600#sh errdisable detect
ErrDisable Reason Detection status
----------------- ----------------
...
storm-control Enabled
...

7600#sh errdisable recovery
ErrDisable Reason Timer Status
----------------- --------------
...
storm-control Enabled
...

Maybe Cisco should implement actions for storm-control, like in other (i.e. 3750) platforms, so errdisable can get some meaning.


3750(config-if)#storm-control action ?
shutdown Shutdown this interface if a storm occurs
trap Send SNMP trap if a storm occurs

Sunday, February 1, 2009

Cisco IOS under Solaris

See anything strange below?


%SYS-5-RELOAD: Reload requested by console. Reload Reason: Reload command.
unix_reload()

Restricted Rights Legend

Use, duplication, or disclosure by the Government is
subject to restrictions as set forth in subparagraph
(c) of the Commercial Computer Software - Restricted
Rights clause at FAR sec. 52.227-19 and subparagraph
(c) (1) (ii) of the Rights in Technical Data and Computer
Software clause at DFARS sec. 252.227-7013.

cisco Systems, Inc.
170 West Tasman Drive
San Jose, California 95134-1706



Cisco IOS Software, Solaris Software (UNIX-P-M), Experimental Version 12.2(20080714:162947) [arvinder-ss_auto_nightly 147]
Copyright (c) 1986-2008 by Cisco Systems, Inc.
Compiled Wed 24-Sep-08 12:28 by arvinder
Image text-base: 0x00011D98, data-base: 0x022F2008

Solaris Unix (Sparc) processor with 56561K bytes of memory.
Processor board ID 122882152
4 Ethernet interfaces
16K bytes of NVRAM.



Press RETURN to get started!


*Mar 1 00:00:00.527: Bootstrap Emulator called with code 135
%SYS-5-RESTART: System restarted --
Cisco IOS Software, Solaris Software (UNIX-P-M), Experimental Version 12.2(20080714:162947) [arvinder-ss_auto_nightly 147]
Copyright (c) 1986-2008 by Cisco Systems, Inc.
Compiled Wed 24-Sep-08 12:28 by arvinder


Router#sh ver
Cisco IOS Software, Solaris Software (UNIX-P-M), Experimental Version 12.2(20080714:162947) [arvinder-ss_auto_nightly 147]
Copyright (c) 1986-2008 by Cisco Systems, Inc.
Compiled Wed 24-Sep-08 12:28 by arvinder

ROM: Bootstrap program is Solaris

Router uptime is 43 minutes
System returned to ROM by reload at 0
System restarted at XXX
System image file is "unix:./.image"

Solaris Unix (Sparc) processor with 56561K bytes of memory.
Processor board ID 122882144
4 Ethernet interfaces
16K bytes of NVRAM.

Configuration register is 0x0


Although the Cisco IOU (IOS-on-Unix) simulator is quite old, i hope some time in the future it will be provided to customers too (i guess it started being used internally by tac people to simulate customer scenarios and now it's being used for various "internal" labs too). It also seems a much better alternative than the one that's included in the various written exam simulation labs. You will be surprised of the features that it supports (with a little bit of refinement it could even be used instead of the actual CCIE labs)! It runs special IOS images (depending on what features you need to test), so it's not a hardware emulator like dynamips (this has the advantage that it's less cpu intensive).

I still don't know if there is any relationship with this.

PS : Some years ago, there was a rumor that Cisco employees would get in trouble for even mentioning its existence and certainly if they ever gave access to somebody (only a very small number of Cisco employees had access to it). Nowadays, that it's used widely internally, i hope the above rumor has ceased to exist.

Please do not ask here for information where to get it. Just talk to your AM/SE.

 
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Greece License.