How to find queue utilisation on 7600/ES+ cards

Cisco usually provides various technical characteristics about their products, but you never get the details you need. One big mystery are the ES/ES+ cards on the 7600 platform. We've been using the ES+ cards for quite a long time and i was trying to get a comparison with the ES+T ones, which come in lower prices. The most worrying fact (regarding a specific project's needs) was a difference in QoS (especially egress) :

76-ES+T-20G : (16, 8, 4) (Level 4 Queues, Level 3 Shaper, Level 2 Shaper) queues per port
7600-ES+20G3CXL : 64,000 ingress queues & 64,000 egress queues

This seems like a big difference, but it may not be relevant to your case, unless you know how to count them easily.

An easy way to start experimenting is the "sh tech-support" command, which includes a lot of information. A variant of this is "sh hw-module slot x tech-support", if you're interested in a specific module. Beware of bug CSCta88917 if you try this. Bug toolkit doesn't say anything useful, but after we opened a new SR, we had to rma the whole module, because according to the tac engineer the crash was due to a faulty sensor (any idea why GOLD didn't catch it?).

Anyway, the command that displays the queue usage is "show platform hardware qos np x queue resources" and this is the output from a 7600-ES+20G3CXL card in slot 9 of a 7600:

7600#rem com mod 9 show platform hardware qos np 0 queue resources

 np   tm  level    groups     entity
 ----------------------------------------------
  0  0     L4     4096/15    32768/34
  0  0     L3      256/13     4096/15
  0  0     L2        8/11      256/13
  0  0     L1       32/10       32/10
 ----------------------------------------------
  0  1     L4     4096/14    32768/51
  0  1     L3      256/10     4096/14
  0  1     L2        8/5      256/6
  0  1     L1       32/5       32/5
 ----------------------------------------------
  0  2     L4     4096/6    32768/16
  0  2     L3      256/6     4096/6
  0  2     L2        8/5      256/5
  0  2     L1       32/5       32/5

7600#rem com mod 9 show platform hardware qos np 1 queue resources

 np   tm  level    groups     entity
 ----------------------------------------------
  1  0     L4     4096/15    32768/34
  1  0     L3      256/13     4096/15
  1  0     L2        8/11      256/13
  1  0     L1       32/10       32/10
 ----------------------------------------------
  1  1     L4     4096/5    32768/10
  1  1     L3      256/5     4096/5
  1  1     L2        8/5      256/5
  1  1     L1       32/5       32/5
 ----------------------------------------------
  1  2     L4     4096/5    32768/10
  1  2     L3      256/5     4096/5
  1  2     L2        8/5      256/5
  1  2     L1       32/5       32/5

You can see the queues per NP and per level. 7600-ES+20G3CXL has two NPs, each one controlling 10 ports. Each NP has 3 tm; tm 0 in each NP doesn't seem to be updated by the configuration changes (any clue?). The ones that interest you are tm 1 and tm 2 entity values, which control 5 ports each.

Summarizing it up :

NP 0, tm 1 : ports 1-5
NP 0, tm 2 : ports 6-10
NP 1, tm 1 : ports 11-15
NP 1, tm 2 : ports 16-20

According to the above outputs, there are 32768+32768 entities per NP, so i don't know if the official number of 64000 queues is totally correct. i.e. can you use all of them on the first 5 ports? 7600/ES+ QoS documentation provides some info, but it's not very clear.

These are the values you start with:

level      entity
-------------------
 L4       32768/10
 L3        4096/5
 L2         256/5
 L1          32/5

If you have a single level policy, then you only reserve queues from L4 level.
If you have a hierarchical policy with 2 levels, then you reserve queues from L4 and L3 levels.
If you have a hierarchical policy with 3 levels, then you reserve queues from L4, L3 and L2 levels.

Service instance, port-channel service instance, and Layer 3 subinterface support 2-level policy-map:
- parent => class-default
--- child => user defined classes

Main interface supports 3-level policy-map:
- grand-parent => class-default
--- parent => user defined classes
----- child => user defined classes


Examples

If you have the following 2-level egress H-QoS applied on a main interface

7600#sh policy-map int gi9/6 | i Class
    Class-map: class-default (match-any)
        Class-map: 1A-CLASS (match-all)
        Class-map: 1B-CLASS (match-all)
        Class-map: 1C-CLASS (match-all)
        Class-map: 1D-CLASS (match-all)
        Class-map: 1E-CLASS (match-all)
        Class-map: class-default (match-any)

then the output becomes...

Before:

level      entity
-------------------
 L4       32768/10
 L3        4096/5
 L2         256/5
 L1          32/5

After:

level      entity
-------------------
 L4       32768/16
 L3        4096/6
 L2         256/5
 L1          32/5

which means you're using :

1 L3 queue
6 L4 queues


If you have the following 3-level egress H-QoS applied on a main interface

7600#sh policy-map int gi9/1 | i Class
    Class-map: class-default (match-any)
        Class-map: 1A-CLASS (match-all)
            Class-map: 2A-CLASS (match-all)
            Class-map: 2B-CLASS (match-all)
            Class-map: 2C-CLASS (match-all)
            Class-map: 2D-CLASS (match-all)
            Class-map: class-default (match-any)
        Class-map: 1B-CLASS (match-all)
            Class-map: 2A-CLASS (match-all)
            Class-map: 2B-CLASS (match-all)
            Class-map: 2C-CLASS (match-all)
            Class-map: 2D-CLASS (match-all)
            Class-map: class-default (match-any)
        Class-map: 1C-CLASS (match-all)
            Class-map: 2A-CLASS (match-all)
            Class-map: 2B-CLASS (match-all)
            Class-map: 2C-CLASS (match-all)
            Class-map: 2D-CLASS (match-all)
            Class-map: class-default (match-any)
        Class-map: 1D-CLASS (match-all)
        Class-map: class-default (match-any)

then the output becomes...

Before:

level      entity
-------------------
 L4       32768/10
 L3        4096/5
 L2         256/5
 L1          32/5

After:

level      entity
-------------------
 L4       32768/27
 L3        4096/10
 L2         256/6
 L1          32/5

which means you're using :

1 L2 queue
5 L3 queues
17 L4 queues


Notes

1) Does anyone have any explanation about the 2 extra L4 queues from the last example? If you count the leaf classes, you'll find they are 15. I guess the last 2 classes (that have no child) reserve an extra "hidden" child queue, probably in order to make their depth equal to the others'.

2) In IOS hierarchical levels are represented as follows and current support is up to five levels:

• Physical or main interface
• Subinterface or logical layer
• Grandparent class
• Parent class
• Child class

A policy map with 2 levels has :
• 3 levels of hierarchy when attached on the main interface
• 4 levels of hierarchy when attached on a subinterface

A policy map with 3 levels has :
• 4 levels of hierarchy when attached on the main interface
• 5 levels of hierarchy when attached on a subinterface