Expert Reference Series of White Papers
Troubleshooting
Signaling with Traces
in Cisco Unified
Communications
Manager
1-800-COURSESwww.globalknowledge.com
Troubleshooting Signaling with Traces
in Cisco UC Manager
Chris Olsen; Global Knowledge Cisco, Microsoft and VMware Trainer,
Author and Independent IT Consultant
Introduction
For those familiar with Cisco IOS-based devices, the real time troubleshooting feature is an IOS command called
debug. A comparable tool in Cisco Unified Communications Manager (CUCM) is called trace. The main difference is that CUCM can store traces on server storage for quite some time, whereas IOS devices are not designed
to be storage devices, and IOS debug is only in real-time, or redirected to another storage media.
Also, just like the quantity of different debug commands are quite immense on a router, the same holds true
with trace on CUCM. There are a lot of different CUCM traces that can exist. This white paper is intended to
enlighten you on the powerful troubleshooting capabilities of trace in CUCM.
This white paper provides an introduction to troubleshooting Cisco telephony utilizing the CUCM tracing
feature. The steps included here are based on CUCM v8.x, although the procedure will be very similar if you
are running version 5.x, 6.x or 7.x of CUCM. This paper is also written for direct application so you can connect
directly into your CUCM and follow step by step.
Initial Setup
After logging in to CUCM navigate in the upper right side of your screen to Cisco Unified Serviceability.
Press Go.
Navigate to Trace > Configuration. Select your Server, and select CM Services. (While outside the scope of
this white paper, you’ll see other options that exist for traces.) Press Go then select the service Cisco CallManager (Active) and press Go again.
Copyright ©2011 Global Knowledge Training LLC. All rights reserved.
2
Note the many items that can be enabled. For the purpose of this white paper, enable at least the following,
(you can enable more if you like, but these will get you started on some good basic trace analysis):
• Enable SIP Call Processing Trace
• Enable Media Resource Manager Trace
• Enable All Phone Device Trace
• Enable H.225 and gatekeeper Trace
If you are in a lab environment, select the Debug Trace Level of Detailed then press Save. Be cautious with
the Detailed debug level in a production environment as the files will grow quickly.
If you are in a production environment with lots of telephony calls occurring, you can skip directly to the next
step on initiating traces. If you are performing these steps in a CUCM lab environment, try some of these basic
actions to generate some trace content:
1. Make a call from one IP phone to another. Try with a SIP phone
2. Make a call from an outside PSTN phone to an IP phone
Copyright ©2011 Global Knowledge Training LLC. All rights reserved.
3
3. Make a call from an IP phone to an outside PSTN phone
4. Set up a conference call
5. Call from one CUCM cluster to another over an Intercluster trunk (H.323)
You can hang up shortly after each call is completed as tracing is focused on the signaling of setting up and
tearing down a call. After the telephony events have completed, then you can capture the trace files as follows
from the RTMT tool.
Real Time Monitoring Tool (RTMT)
The main tool for collecting traces in CUCM is the Real Time Monitoring Tool (RTMT). This can be installed from
CUCM as a free plugin onto your computer as follows:
Navigate in your CUCM to Cisco Unified CM Administration. Go to Application > Plugins. Press Find.
Look down the list of plugins until you see Cisco Unified Real-Time Monitoring Tool. For your Linux or
Windows OS that you are running on your computer, click Download on the left column.
Click Run, and follow the steps of the installation.
Copyright ©2011 Global Knowledge Training LLC. All rights reserved.
4
After the installation is complete, start the tool by logging into your CUCM. To initiate a trace, on the left side of
RTMT, click on System, then Trace & Log Central. Double click on Collect Files.
Copyright ©2011 Global Knowledge Training LLC. All rights reserved.
5
Check the All Servers box for Cisco CallManager and if you want to trace media for conferencing, then
select Cisco IP Voice Media Streaming App. Press Next. Don’t check the box for Select all Services on all
Servers and don’t select other items. Press Next again.
Change the Download File Directory to a familiar location on your computer. Select the Relative Range within
the timeframe of the phone event.
Recent versions of CUCM now default to compress the trace files in a .gz format. Be sure to check Uncompress
Log Files, or you will need to download a tool to decompress these files with a program such as BitZipper.
Copyright ©2011 Global Knowledge Training LLC. All rights reserved.
6
Press Finish. Watch on the bottom right on of the RTMT tool to see the spinning circle progress meter.
Under the location you choose to save the trace files from RTMT on your computer hard drive, navigate to
“yourdirectoryname” > “yourservername” > “yourdate” > cm > trace > ccm > sdi. Note the time
and date of the files.
Triple Combo Tool
While you can open the files and examine their content with Wordpad, finding specific events in a trace file in
a production environment often looks like a “needle in a hay stack” search. It’s much easier to use the Triple
Combo Parsing tool an excellent free parsing tool for analyzing trace files. You can get this tool from Cisco TAC,
or download free from here: http://www.employees.org/~tiryaki/tc
With both the trace files in the directory and the Triple Combo tool visible on your desktop, drag and drop the
trace file in question (ccm…txt) into the open pane on top of the Triple Combo tool.
Copyright ©2011 Global Knowledge Training LLC. All rights reserved.
7
Press Apply on the bottom right of the tool after you select the following different options and then observe
the parsed results in the lower left side of Triple Combo.
Select one or several of the buttons below CCM Trace as follows:
1. Q
.931/H.225. Since these are both protocols from the International Telecommunication Union (ITU.ORG), it
is no surprise that the outputs of these displays are very similar, which is why they are grouped together in
the Triple Combo tool. However, telephony actions that use these signals are different. Q.931 is the signaling
for ISDN on the D channel for call setup and teardown, which shows caller ID and called numbers for ISDN
PRI calls. H.225 is the call setup and teardown protocol for the larger umbrella protocol H.323. H.225 would
appear for calls thru an Inter-Cluster Trunk (ICT) or an H.323 gateway to or from the CUCM cluster.
2. MGCP. Media Gateway Control Protocol can be used in CUCM for signaling between CUCM and a Cisco
ISR gateway. While MGCP is an internet standard, MGCP is only used from CUCM to a Cisco gateway. Note
that Triple Combo will show many MGCP keepalives between CUCM and the gateway. These keepalives are
normal, but don’t say much else.
3. S
CCP. Skinny has been the traditional Cisco IP phone client server signaling protocol for all events on Cisco
Phones. Note that if you’ve already migrated your Cisco IP phones to a firmware version that supports SIP,
Copyright ©2011 Global Knowledge Training LLC. All rights reserved.
8
then you will not see any SCCP messages. However SCCP remains the only protocol that CUCM supports
between CUCM and an ISR to control DSP resources.
4. H
.245. This protocol is for capabilities exchange during an H.323 call. H.245 will show negotiations for
elements such as voice, video, codec selection and conferencing. SCCP, SIP or MGCP will not generate H.245
traffic, only an H.323 call will.
5. S
IP. Session Initiation Protocol is an Internet standard and is rapidly becoming the signaling protocol of
choice in the industry, and Cisco continues to add SIP support. You can see SIP traffic if you make a call to or
from a SIP phone, make a call thru a SIP trunk, use presence from a CUCM SIP trunk to Cisco Unified Presence Server (CUPS), or for voice mail if SIP is configured for integration. CUCM can be integrated with Microsoft’s telephony products OCS 2007, OCS 2007 R2 or Lync 2010 with a Microsoft Mediation Server. SIP is the
only option for this integration as it is the only signaling protocol that Microsoft currently supports. Other 3rd
party SIP phones can also register to CUCM, and calling to or from them will generated SIP traces.
6. D
A. This is not a separate protocol but simply shows the digit analysis which displays the Calling or ANI
numbers called from and the Called or DNIS numbers dialed to. DA will display the phone numbers for all
signaling protocols used.
Note that for one call multiple different protocols could be used, for instance:
1. A
CUCM IP phone calls an outside PSTN call. This will show SIP or SCCP for the IP phone, and then
MGCP or H.323 for the gateway, and Q.391 for ISDN. You will also see DA for the digits dialed.
2. A
SCCP phone calls a SIP phone in the same CUCM cluster. You will see SCCP and SIP, as well as DA
for the dialed digits.
While gateway debug analysis is beyond the scope of this paper, note the IOS Trace option in the middle of
Triple Combo.
You can turn on voice related debug commands such as debug isdn q931 on a Cisco IOS gateway known as
an Integrated Services Router (ISR). Then cut and paste the debug output from the router console into a Notepad file, and then drag and drop the entire saved Notepad file into the top of Triple Combo. You can analyze its
content as well.
Copyright ©2011 Global Knowledge Training LLC. All rights reserved.
9
An effective approach to troubleshooting is to educate yourself to the very specifics that traces will show when
everything is functioning normally in your environment. Spend some time and really study the details you see.
Then, when something goes wrong, perform these steps for tracing again, and you will be better educated to
identify the differences of the trace outputs, which will better enable you to identify the origin of the problem.
Conclusion
There is far more to troubleshooting all possible telephony events on CUCM than meets the eye. However, by
practicing these basic skills of creating, parsing and studying trace files with Triple Combo, you’ll be well on your
way to become an excellent CUCM Unified Communications troubleshooter.
Learn More
Learn more about how you can improve productivity, enhance efficiency, and sharpen your competitive edge.
Check out the following Global Knowledge course:
TVOICE - Troubleshooting UC Systems v8.0
For more information or to register, visit www.globalknowledge.com or call 1-800-COURSES (1-800-2687737) to speak with a sales representative.
Our courses and enhanced, hands-on labs and exercises offer practical skills and tips that you can immediately
put to use. Our expert instructors draw upon their experiences to help you understand key concepts and how to
apply them to your specific work situation. Choose from our more than 1,200 courses, delivered through Classrooms, e-Learning, and On-site sessions, to meet your IT and business training needs.
About the Author
Chris Olsen has been an IT trainer since 1993 and an independent consultant and technical writer since 1996.
He has taught over 60 different IT and telephony classes to over 15,000 students. He is a technical editor for
Global Knowledge’s lab manuals and he has published two books with Cisco Press, CIPT part 2 and CCNA Voice
Flash Cards. He is an author and technical editor on both Microsoft OCS 2007 and 2007 R2 certification exams.
He is a technical author for Cisco certified Courses. He is also currently working with his publisher, CiscoPress,
finalizing CIPT part 2 v8.
Copyright ©2011 Global Knowledge Training LLC. All rights reserved.
10