Showing posts with label QoS. Show all posts
Showing posts with label QoS. Show all posts

Sunday, April 18, 2010

How to find queue utilisation on 7600/ES+ cards

Cisco usually provides various technical characteristics about their products, but you never get the details you need. One big mystery are the ES/ES+ cards on the 7600 platform. We've been using the ES+ cards for quite a long time and i was trying to get a comparison with the ES+T ones, which come in lower prices. The most worrying fact (regarding a specific project's needs) was a difference in QoS (especially egress) :

76-ES+T-20G : (16, 8, 4) (Level 4 Queues, Level 3 Shaper, Level 2 Shaper) queues per port
7600-ES+20G3CXL : 64,000 ingress queues & 64,000 egress queues

This seems like a big difference, but it may not be relevant to your case, unless you know how to count them easily.

An easy way to start experimenting is the "sh tech-support" command, which includes a lot of information. A variant of this is "sh hw-module slot x tech-support", if you're interested in a specific module. Beware of bug CSCta88917 if you try this. Bug toolkit doesn't say anything useful, but after we opened a new SR, we had to rma the whole module, because according to the tac engineer the crash was due to a faulty sensor (any idea why GOLD didn't catch it?).

Anyway, the command that displays the queue usage is "show platform hardware qos np x queue resources" and this is the output from a 7600-ES+20G3CXL card in slot 9 of a 7600:


7600#rem com mod 9 show platform hardware qos np 0 queue resources

np tm level groups entity
----------------------------------------------
0 0 L4 4096/15 32768/34
0 0 L3 256/13 4096/15
0 0 L2 8/11 256/13
0 0 L1 32/10 32/10
----------------------------------------------
0 1 L4 4096/14 32768/51
0 1 L3 256/10 4096/14
0 1 L2 8/5 256/6
0 1 L1 32/5 32/5
----------------------------------------------
0 2 L4 4096/6 32768/16
0 2 L3 256/6 4096/6
0 2 L2 8/5 256/5
0 2 L1 32/5 32/5

7600#rem com mod 9 show platform hardware qos np 1 queue resources

np tm level groups entity
----------------------------------------------
1 0 L4 4096/15 32768/34
1 0 L3 256/13 4096/15
1 0 L2 8/11 256/13
1 0 L1 32/10 32/10
----------------------------------------------
1 1 L4 4096/5 32768/10
1 1 L3 256/5 4096/5
1 1 L2 8/5 256/5
1 1 L1 32/5 32/5
----------------------------------------------
1 2 L4 4096/5 32768/10
1 2 L3 256/5 4096/5
1 2 L2 8/5 256/5
1 2 L1 32/5 32/5


You can see the queues per NP and per level. 7600-ES+20G3CXL has two NPs, each one controlling 10 ports. Each NP has 3 tm; tm 0 in each NP doesn't seem to be updated by the configuration changes (any clue?). The ones that interest you are tm 1 and tm 2 entity values, which control 5 ports each.

Summarizing it up :

NP 0, tm 1 : ports 1-5
NP 0, tm 2 : ports 6-10
NP 1, tm 1 : ports 11-15
NP 1, tm 2 : ports 16-20

According to the above outputs, there are 32768+32768 entities per NP, so i don't know if the official number of 64000 queues is totally correct. i.e. can you use all of them on the first 5 ports? 7600/ES+ QoS documentation provides some info, but it's not very clear.

These are the values you start with:


level entity
-------------------
L4 32768/10
L3 4096/5
L2 256/5
L1 32/5


If you have a single level policy, then you only reserve queues from L4 level.
If you have a hierarchical policy with 2 levels, then you reserve queues from L4 and L3 levels.
If you have a hierarchical policy with 3 levels, then you reserve queues from L4, L3 and L2 levels.

Service instance, port-channel service instance, and Layer 3 subinterface support 2-level policy-map:
- parent => class-default
--- child => user defined classes

Main interface supports 3-level policy-map:
- grand-parent => class-default
--- parent => user defined classes
----- child => user defined classes


Examples

If you have the following 2-level egress H-QoS applied on a main interface


7600#sh policy-map int gi9/6 | i Class
Class-map: class-default (match-any)
Class-map: 1A-CLASS (match-all)
Class-map: 1B-CLASS (match-all)
Class-map: 1C-CLASS (match-all)
Class-map: 1D-CLASS (match-all)
Class-map: 1E-CLASS (match-all)
Class-map: class-default (match-any)


then the output becomes...

Before:

level entity
-------------------
L4 32768/10
L3 4096/5
L2 256/5
L1 32/5


After:

level entity
-------------------
L4 32768/16
L3 4096/6
L2 256/5
L1 32/5


which means you're using :

1 L3 queue
6 L4 queues


If you have the following 3-level egress H-QoS applied on a main interface


7600#sh policy-map int gi9/1 | i Class
Class-map: class-default (match-any)
Class-map: 1A-CLASS (match-all)
Class-map: 2A-CLASS (match-all)
Class-map: 2B-CLASS (match-all)
Class-map: 2C-CLASS (match-all)
Class-map: 2D-CLASS (match-all)
Class-map: class-default (match-any)
Class-map: 1B-CLASS (match-all)
Class-map: 2A-CLASS (match-all)
Class-map: 2B-CLASS (match-all)
Class-map: 2C-CLASS (match-all)
Class-map: 2D-CLASS (match-all)
Class-map: class-default (match-any)
Class-map: 1C-CLASS (match-all)
Class-map: 2A-CLASS (match-all)
Class-map: 2B-CLASS (match-all)
Class-map: 2C-CLASS (match-all)
Class-map: 2D-CLASS (match-all)
Class-map: class-default (match-any)
Class-map: 1D-CLASS (match-all)
Class-map: class-default (match-any)


then the output becomes...

Before:

level entity
-------------------
L4 32768/10
L3 4096/5
L2 256/5
L1 32/5


After:

level entity
-------------------
L4 32768/27
L3 4096/10
L2 256/6
L1 32/5


which means you're using :

1 L2 queue
5 L3 queues
17 L4 queues


Notes

1) Does anyone have any explanation about the 2 extra L4 queues from the last example? If you count the leaf classes, you'll find they are 15. I guess the last 2 classes (that have no child) reserve an extra "hidden" child queue, probably in order to make their depth equal to the others'.

2)
In IOS hierarchical levels are represented as follows and current support is up to five levels:

• Physical or main interface
• Subinterface or logical layer
• Grandparent class
• Parent class
• Child class

A policy map with 2 levels has :
• 3 levels of hierarchy when attached on the main interface
• 4 levels of hierarchy when attached on a subinterface

A policy map with 3 levels has :
• 4 levels of hierarchy when attached on the main interface
• 5 levels of hierarchy when attached on a subinterface

Saturday, January 16, 2010

Shaper granularity on ME-3400 demystified

Everyone using ME-3400 switches might have noticed the following error message when trying to configure an output policy-map under an interface:


QoS: Configuration failed. The configured rate 4000000 bps is not
achievable in hw within 1% of configuration.
Closest value(s) are: 11111120 bps, 5882368 bps


Cisco documentation is cryptic (as always) about the details of this, but it has to do with some hardware limitation of this specific platform. In particular the granularity of the hardware for the shaping action is somehow based on pre-configured values.

The ME-3400 (as most lower end switches) has physical memory buffers, which can be used by IOS only in bunches of specific (pre-configured) sizes. In routers and high end switches, QoS is usually implemented in software through the use of memory pools, which allows the IOS to use parts of buffers with variable sizes.

Egress shaping on ME-3400 comes into two categories : Port shaping and Class-based shaping. Port shaping applies to all traffic passing through an interface, while class-based shaping applies to specific classes of traffic leaving an interface. Each one of them is using a different formula in order to give you all the supported values.

Port shaping values are based on the following formula:

(1 - 16

N
) * IfSpeed



Where:
N is a value between 17 and 64000
IfSpeed is the interface speed : 10 Mbps, 100 Mbps, 1 Gbps
The result in then rounded up to a multiple of 16.
An equivalent excel formula would be : CEILING((1-16/N)*IfSpeed;16)

Class-based shaping values are based on the following much simpler formula:

1

N
* IfSpeed



Where:
N is a value between 1 and 15625
IfSpeed is the interface speed : 10 Mbps, 100 Mbps, 1 Gbps

You can see some possible values in the table below:



According to the above table, in the port shaper the low end granularity is very coarse and the high end granularity is very dense, while the opposite happens in the class-based shaper. So there is no possibility you can have a <58 Mbps port shaper for 1 Gbps interfaces, as there is no possibility to have <5,8 Mbps port shaper for 100 Mbps interfaces. Similarly you can't have a 700 Mbps class-based shaper on a 1000 Mbps interface, nor a 80 Mbps class-based shaper on a 100 Mbps interface.

Let's take for example the following policy-map configuration which is applied on a 1 Gbps interface.


policy-map CHILD
class TEST-CLASS
shape average 100000
policy-map PARENT
class class-default
shape average 930000000
service-policy CHILD

If you try to change the shaper of the child class to a value >500 Mbps, you'll get a warning like the following:

3400(config-pmap-c)#shape average 700000000
QoS: Configuration failed. The configured rate 700000000 bps is not achievable in hw within 1% of configuration.
Closest value(s) are: 930000000 bps, 500000000 bps


If you check the formula for the class-based shaper (or have a quick look at the table), 930000000 is not actually a valid value to configure, but it gets printed because this is the limit imposed by the parent class. You just have to ignore it.

If you need specific values you might want to try changing the speed of the interface : choosing between 10/100/1000 should be easy for BaseTX interfaces, while 100/1000 Mbps SFPs exist for the SFP-based ones.

Also, it's strongly recommended that you disable port speed autonegotiation when you attach an output policy map to a 10/100/1000 port, to prevent the port from autonegotiating to a rate that would make the output policy map invalid.

Keep in mind that ME-3400E (the new enhanced version of ME-3400) implements a improved version of Egress Shaping Granularity, which uses a simpler linear formula. It's 64 Kbps for the class-based shaper and 100/500/1000 Kbps for the 10/100/1000 Mbps port-based shaper (100 Kbps for 10 Mbps ports, 500 Kbps for 100 Mbps ports, 1000 Kbps for 1000 Mbps ports).

Saturday, May 24, 2008

QoS classification/marking using PBR

Most of you already know the 2 usual methods of doing QoS classification and marking:

Use MQC:


class-map match-all HOST-CLASS
match access-group name HOST-ACL
!
class-map match-any P2P-CLASS
match protocol fasttrack
!
policy-map TEST-POLICY
class HOST-CLASS
set ip precedence 3
class P2P-CLASS
police 64000 conform-action set-prec-transmit 0 exceed-action drop
!
ip access-list extended HOST-ACL
permit tcp host 10.0.0.1 any eq telnet
!
interface serial 2/1
service-policy input TEST-POLICY


Use legacy QoS, aka CAR:


interface serial 2/1
rate-limit input access-group 100 64000 8000 8000 conform-action set-prec-transmit 3 exceed-action set-prec-transmit 0
!
access-list 100 permit tcp host 10.0.0.1 any eq telnet



If you have already used QoS policy propagation via BGP (QPPB) you might get the idea more easily about the next method :

Use PBR:


route-map QOS-MAP permit 10
match ip address HOST-ACL
set ip precedence 3
!
route-map QOS-MAP permit 20
match ip next-hop 99
set ip precedence 0
!
interface serial 2/1
ip policy route-map QOS-MAP
!
access-list 99 permit 192.168.0.1
ip access-list extended HOST-ACL
permit tcp host 10.0.0.1 any eq telnet


No sign of MQC, no sign of CAR, just simple PBR. So now you know what you can do, if you're told to configure QoS classification/marking without using neither MQC nor CAR. Just keep in mind that PBR happens only in the ingress direction.

 
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Greece License.