Tuesday, April 7, 2015

CAC Order of Operation




When multiple CAC mechanisms are configured, the selection process follows these steps:
Step 1.
The gateway checks for the max-conn configuration on the outbound dial peer.

If the limit has been reached, the call is rejected. If no max-conn is configured, or the limit has not been reached, the router goes to Step 2.
Step 2.
The router checks for CAC mechanisms based on local system resources, such as CPU use.

If these resources are not configured, or if the test succeeds, the router goes to Step 3.
Step 3.
If a gatekeeper is used, and it is configured to do CAC based on bandwidth, the gatekeeper is checked.

If this test succeeds, or no gatekeeper CAC is used, the router proceeds to Step 4.
Step 4.
If RSVP is configured, an RSVP reservation is attempted. When RSVP is used, this is the last test.

If the call is allowed, call setup continues. If not, the router rejects the call. Only if RSVP is not configured does the router go to Step 5.
Step 5.
Any CAC mechanism that use probes to measure network availability is now checked.

This is the last test. If the call is allowed, setup proceeds. If not, it is rejected.

If any of these tests rejects the call, the process ends at that point, and no further checking is done.

Gateway-Controlled RSVP CAC (CME or CUBE - no CUCM)


You must configure RSVP on each router that will create reservation using ip rsvp bandwidth [total-kbps] [single-flow-kbps] command. If you do not specify the total bandwidth to reserve, the router reserves 75 percent of the interface bandwidth. If no flow value is specified, any flow can reserve the entire bandwidth.

Synchronization between RSVP and call setup is enabled by default. If it has been disabled, the command to enable it is call rsvp-sync.
To set the DSCP value for RSVP control messages, use the ip rsvp signaling dscp {dscp-value} interface command.
To enable RSVP reservation on dial-peers of originating and terminating gateways, use the commands req-qos & acc-qos. The command req-qos defines the requested QoS level in RESV Request message while the command acc-qos defines the minimum acceptable QoS level in the RESV Response message. The combination of the commands at both gateways defines how the call is treated. This is summarized below in the table.
Call Scenarios
Originating Gateway

Terminating Gateway



Requested QoS
Acceptable QoS
Requested QoS
Acceptable QoS
Results
1
controlled-load or guaranteed-delay
controlled-load or guaranteed-delay
controlled-load or guaranteed-delay
controlled-load or guaranteed-delay
Call proceeds only if both RSVP reservations succeed.
2
controlled-load or guaranteed-delay
controlled-load or guaranteed-delay
controlled-load or guaranteed-delay
best-effort
Call proceeds only if both RSVP reservations succeed.
3
controlled-load or guaranteed-delay
controlled-load or guaranteed-delay
best-effort
best-effort
Call is released.
4
controlled-load or guaranteed-delay
best-effort
controlled-load or guaranteed-delay
controlled-load or guaranteed-delay
Call proceeds only if both RSVP reservations succeed.
5
controlled-load or guaranteed-delay
best-effort
controlled-load or guaranteed-delay
best-effort
Call proceeds regardless of RSVP results. If RSVP reservation fails, call receives best-effort service.
6
controlled-load or guaranteed-delay
best-effort
best-effort
best-effort
Call proceeds with best-effort service.
7
best-effort
best-effort
controlled-load or guaranteed-delay
controlled-load or guaranteed-delay
Call is released.
8
best-effort
best-effort
controlled-load or guaranteed-delay
best-effort
Call proceeds with best-effort service.
9
best-effort
best-effort
best-effort
best-effort
Call proceeds with best-effort service.

There is no clear guidelines in IETF docs for the use of controlled-load versus guaranteed-delay. However, in summary, guaranteed-delay should be used for voip application.
  • The Guaranteed service does not control the minimum or average delay of datagrams, merely the maximal queuing delay. The service guarantees that datagrams will arrive within the requested delivery time and will not be discarded due to queue overflows, provided the flow's traffic stays within its specified traffic parameters.
  • Controlled load Service approximates the end-to-end behavior provided to best effort service under unloaded conditions. The assumption here is that under unloaded conditions, a very high percentage of the transmitted packets are successfully delivered to the end-nodes, and the transmission delay experienced by a very high percentage of the delivered packets will not vary much from the minimum transit delay

!!...Configure RSVP on the outgoing interface
Shanghai-GW(config)#int s2/0
Shanghai-GW(config-if)#bandwidth 1544
Shanghai-GW(config-if)#ip rsvp bandwidth 400 40
Shanghai-GW(config-if)#ip rsvp signalling dscp 31

!!...Configure QoS on the outgoing dial peer
Shanghai-GW(config-if)#dial-peer voice 3400 voip
Shanghai-GW(config-dial-peer)#req-qos guaranteed-delay
Shanghai-GW(config-dial-peer)#acc-qos guaranteed-delay


  1. FIFO SHOULDN’T be used as queueing mechanism (usually WFQ) when enabling RSVP. This should be noted while having Ethernet interfaces for WAN since default is FIFO. For serial interfaces default is WFQ.
  2. In case sub-interfaces are used, WFQ and RSVP should be enabled on parent interface, and RSVP should be enabled on sub-interface (once its enabled on sub it will reflect automatically on parent).
  3. In case Loopback interface is used as signaling source (in this diagram H323), still RSVP should be applied in sub interface and NOT Loopback interface.

BR1#sh ip rsvp interface       
interface    rsvp  allocated  i/f max  flow max sub max
Fa0/0        ena   80K        75M      75M      0  
Fa0/0.1      ena   80K        1M       100K     0  
Lo0          ena   0          1M       100K     0  

Sunday, April 5, 2015

MultiCast MoH Bugs

During my testing I faced the following bugs and would like to share them.


Multicast MOH reversing G729 codec to G711 codecs - media resource issue
CSCum06291 - CUCM 9.1(2.11002.1)

Symptom: Multicast MOH is using G729 codec instead of G711 codec even though the region mapping between the 2 regions is set to 64kbps

c2951-router-02#show call active voice br calling-number 0561776201

Telephony call-legs: 1
SIP call-legs: 1
H323 call-legs: 0
Call agent controlled call-legs: 0
SCCP call-legs: 0
Multicast call-legs: 0
Total call-legs: 2
21F2 : 21969 1863809730ms.1 (09:19:49.616 UAE Sun Apr 5 2015) +1760 pid:4 Answer 0561776201 active
 dur 00:02:15 tx:3905/655128 rx:4302/326280 dscp:0 media:0 audio tos:0x0 video tos:0x0
 Tele 0/0/0:15 (21969) [0/0/0.2] tx:57380/51720/0ms g729r8 noise:-46 acom:3  i/0:-42/-64 dBm

21F2 : 21970 1863809730ms.2 (09:19:49.616 UAE Sun Apr 5 2015) +1750 pid:3401 Originate 5814474 connected
 dur 00:02:15 tx:1716/274560 rx:3905/623888 dscp:0 media:0 audio tos:0xB8 video tos:0x0
 IP 239.1.1.12:16384 SRTP: off rtt:1ms pl:77530/40ms lost:0/0/1 delay:55/55/65ms g729r8 TextRelay: off Transcoded: No
 media inactive detected:n media contrl rcvd:n/a timestamp:n/a
 long duration call detected:n long duration call duration:n/a timestamp:n/a
 LostPacketRate:0.00 OutOfOrderRate:0.00

Cisco Bug: CSCur40004 - Need to support Multicast Music on Hold for Jabber clients

Cisco Jabber for Windows

Symptom: Jabber clients do not support Multi cast MOH (MMOH)

Friday, April 3, 2015

Multicast MoH to PSTN combined with ASA , IGMP Snooping, VRFs

Configuring MMoH for PSTN in L3 networks require some work to be done at network level as well as IPT level.


  • CUCM Configuration

  1. Enable IP Voice Media Streaming (IPVMS) service
  2. Configure Location, DP, Region to be used by MoH servers. Note: MMoH isn't deducted from locations BW.
  3. By default, G711ulaw is the only supported codec by MoH servers. Navigate to System > Service Parameters > #Server# > IPVMS and enable other codecs based on your regions configuration.
  4. By default MMoH DSCP is EF, you can change this value based on your network QoS (e,g, CS3)
  5. Navigate to Media Resource > Music On Hold Source > #File# and make sure to enable Allow Multicasting checkbox.
  6. Navigate to Media Resource > Music On Hold (MoH) Server and configure the following:
    1. Max Hops for each Source File (you need to make sure that this value is enough for the MMoH stream to reach PSTN. For above diagram should be mini 3)
    2. Max multicast connections
    3. Assign DP, Location, and enable Run Flag
    4. Enable Multi-cast Audio Sources on this MOH Server
    5. Configure Base multicast IP and Port
    6. Make sure to configure Increment Multicast on IP Address. This is used when multiple codecs are enabled for MoH server. Each codec will use a unique multicast IP as below



Inc. Multicast on IP Address

Audio Stream
Codec
Dst. IP Address
Dst. Port
1
G.711 ulaw
239.1.1.10
16384
1
G.711 Alaw
239.1.1.11
16384
1
G.729
239.1.1.12
16384
1
Wideband
239.1.1.13
16384

Increment on Port will cause the CUCM to use same IP for all codecs with different ports for each. Since multicast routing on the network level is based on IP and not ports, all the endpoints on hold or PSTN will receive all mulitcast streams sharing the same IP which will cause multicast flooding in the network (especially when multiple endpoints are on hold at same time) .
  1. Create Media Resource Group (MRG) and enable Use Multi-cast for MOH Audio (If at least one multi-cast MOH resource is available)
  2. Create MRGL, add the MRG, and assign the MRGL to device pool, endpoint, or common device configuration.
  • Voice Gateway Configuration
On the voice gateway it is very important to enable the command ccm-manager music-on-hold. Without this command, the voice gateway will connect to the MoH server using call forwarding message but won't send IGMP message to join the multicast group for MMoH.
  • Multicasting Network Configuration
    • On the voice gateway the following configuration will be applied:
ip multicast-routing vrf DMVPN
!
interface Loopback170
 ip vrf forwarding DMVPN
 ip address 10.170.170.2 255.255.255.255
 ip pim sparse-mode
!
interface GigabitEthernet0/1.111
 encapsulation dot1Q 111
 ip vrf forwarding DMVPN
 ip address 10.170.200.84 255.255.255.240
 ip pim sparse-mode
!
ip pim vrf DMVPN send-rp-announce Loopback170 scope 100
ip pim vrf DMVPN send-rp-discovery scope 100

  • On ASA the following configuration will be applied
object-group network CUCM-SERVERS
 network-object host 10.170.4.10
 network-object host 10.170.4.11
 network-object host 10.170.4.12
!
object-group network MMoH-SERVERS
 network-object 239.1.1.10 255.255.255.255
 network-object 239.1.1.11 255.255.255.255
 network-object 239.1.1.12 255.255.255.255
 network-object 239.1.1.22 255.255.255.255
 network-object 239.1.1.21 255.255.255.255
 network-object 239.1.1.20 255.255.255.255
!
access-list ACL-VOICE-VIDEO extended permit ip object-group CUCM-SERVERS object-group MMoH-SERVERS

  • On 4500 the following configuration will be applied
ip multicast-routing vrf VOICE/VIDEO
!
ip igmp snooping querier
!
interface Vlan51
 ip vrf forwarding VOICE/VIDEO
 ip address 10.170.200.20 255.255.255.240
 ip pim sparse-mode
!
interface Vlan104
 description VoIP Subnet
 ip vrf forwarding VOICE/VIDEO
 ip address 10.170.4.2 255.255.254.0
 ip pim sparse-mode

Note: ASA supports sparse-mode only. Therefore, sparse-mode is used across the network.

During MoH condition, here are some outputs from different devices:

On Voice Gateway

C2951-router-02#sh voip rtp connections
VoIP RTP Port Usage Information:
Max Ports Available: 8091, Ports Reserved: 101, Ports in Use: 1
Port range not configured, Min: 16384, Max: 32767

                                                Ports       Ports       Ports    
Media-Address Range                             Available   Reserved    In-use   

Default Address-Range                           8091        101         1        

VoIP RTP active connections :
No. CallId     dstCallId  LocalRTP RmtRTP LocalIP                                       RemoteIP                              
1     21751      21752      21540    16384  10.170.170.2                                239.1.1.12                            
Found 1 active RTP connections

C2951-router-02#show ip mroute vrf DMVPN 239.1.1.12
IP Multicast Routing Table
Flags: D - Dense, S - Sparse, B - Bidir Group, s - SSM Group, C - Connected,
       L - Local, P - Pruned, R - RP-bit set, F - Register flag,
       T - SPT-bit set, J - Join SPT, M - MSDP created entry, E - Extranet,
       X - Proxy Join Timer Running, A - Candidate for MSDP Advertisement,
       U - URD, I - Received Source Specific Host Report,
       Z - Multicast Tunnel, z - MDT-data group sender,
       Y - Joined MDT-data group, y - Sending to MDT-data group,
       G - Received BGP C-Mroute, g - Sent BGP C-Mroute,
       N - Received BGP Shared-Tree Prune, n - BGP C-Mroute suppressed,
       Q - Received BGP S-A Route, q - Sent BGP S-A Route,
       V - RD & Vector, v - Vector, p - PIM Joins on route,
       x - VxLAN group
Outgoing interface flags: H - Hardware switched, A - Assert winner, p - PIM Join
 Timers: Uptime/Expires
 Interface state: Interface, Next-Hop or VCD, State/Mode

(*, 239.1.1.12), 08:09:57/stopped, RP 10.170.170.2, flags: SJCL
  Incoming interface: Null, RPF nbr 0.0.0.0
  Outgoing interface list:
    GigabitEthernet0/1.111, Forward/Sparse, 00:02:00/00:02:18

(10.170.4.10, 239.1.1.12), 08:09:57/00:02:48, flags: PLTX
  Incoming interface: GigabitEthernet0/1.111, RPF nbr 10.170.200.81
  Outgoing interface list: Null

C2951-router-02#show ccm-manager music-on-hold
Current active multicast sessions : 1
 Multicast       RTP port   Packets       Call   Codec    Incoming
 Address         number     in/out        id              Interface
===================================================================
239.1.1.12        16384   9305/9305        21752   g729r8  Gi0/1.111

On ASA Firewall

ASA5545X# show conn address 239.1.1.12
6952 in use, 14310 most used

UDP DMVPN  239.1.1.12:16384 VOICE-VIDEO  10.170.4.10:16384, idle 0:00:00, bytes 390880, flags -

ASA5545X# show mroute 239.1.1.12

Multicast Routing Table
Flags: D - Dense, S - Sparse, B - Bidir Group, s - SSM Group,
       C - Connected, L - Local, I - Received Source Specific Host Report,
       P - Pruned, R - RP-bit set, F - Register flag, T - SPT-bit set,
       J - Join SPT
Timers: Uptime/Expires
Interface state: Interface, State

(*, 239.1.1.12), 00:05:21/never, RP 0.0.0.0, flags: SCJ
  Incoming interface: Null
  RPF nbr: 0.0.0.0
  Immediate Outgoing interface list:
    DMVPN, Null, 00:05:21/never
    OUTSIDE, Forward, 00:05:21/never
    MPLS, Forward, 00:05:21/never
             
(10.170.4.10, 239.1.1.12), 00:05:21/00:03:30, flags: SJT
  Incoming interface: VOICE-VIDEO
  RPF nbr: 10.170.200.21
  Immediate Outgoing interface list:
    DMVPN, Forward, 00:05:21/00:03:25
  Inherited Outgoing interface list:
    OUTSIDE, Forward, 00:05:21/never
    MPLS, Forward, 00:05:21/never

On 4500

C4507-switch#show ip mroute vrf VOICE/VIDEO 239.1.1.12
IP Multicast Routing Table
Flags: D - Dense, S - Sparse, B - Bidir Group, s - SSM Group, C - Connected,
       L - Local, P - Pruned, R - RP-bit set, F - Register flag,
       T - SPT-bit set, J - Join SPT, M - MSDP created entry, E - Extranet,
       X - Proxy Join Timer Running, A - Candidate for MSDP Advertisement,
       U - URD, I - Received Source Specific Host Report,
       Z - Multicast Tunnel, z - MDT-data group sender,
       Y - Joined MDT-data group, y - Sending to MDT-data group,
       G - Received BGP C-Mroute, g - Sent BGP C-Mroute,
       N - Received BGP Shared-Tree Prune, n - BGP C-Mroute suppressed,
       Q - Received BGP S-A Route, q - Sent BGP S-A Route,
       V - RD & Vector, v - Vector, p - PIM Joins on route,
       x - VxLAN group
Outgoing interface flags: H - Hardware switched, A - Assert winner, p - PIM Join
 Timers: Uptime/Expires
 Interface state: Interface, Next-Hop or VCD, State/Mode

(*, 239.1.1.12), 08:18:47/stopped, RP 10.170.170.2, flags: SP
  Incoming interface: Vlan51, RPF nbr 10.170.200.18
  Outgoing interface list: Null

(10.170.4.10, 239.1.1.12), 08:18:47/00:02:04, flags: PT
  Incoming interface: Vlan104, RPF nbr 0.0.0.0
  Outgoing interface list: Null

C4507-switch#sh ip igmp snooping groups vlan 111
Vlan      Group                    Version     Port List
---------------------------------------------------------
111       224.0.1.39               v2          Te3/1
111       239.1.1.12               v2          Te3/1

DNS Performance Troubleshooting

When you are troubleshooting internet performance, there are different parts of the connection should be verified:   ·         DNS Pe...