HMS Developer Guide
HVP Release 5.1
Document number:
Version:
Issue date:
hvp-hms-0014
1-0
22 December 2009
Confidential
Copyright
© Copyright 2009 West Corporation. This document is confidential. No part of this document may be
reproduced, published or disclosed in whole or part, by any means: mechanical, electronic,
photocopying, recording or otherwise without the prior written permission of West Corporation or Holly
Australia Pty Ltd. T
The information contained in this document is strictly commercial in confidence and can only be
provided to persons who have signed a non-disclosure agreement. This document is not to be copied
without prior written consent.
Control
History
Version
Date
Change Notes
Author
1-0
22 Dec 2009
Approved for release
R Teale
Related Documents
Title
Doc No
Holly Management System User Guide – HVP Release 5.1
hvp-hms-0013
Holly Voice Platform Operations Guide – HVP Release 5.1
hvp-0028
Final
HMS Developer Guide, v1-0, 22 December 2009
2/39
Confidential
Preface
This document provides a brief description of the Holly Management System (HMS) and a functional
overview and user guide for the HMS Developer (HMS Dev) configuration tools, reports and utilities
available to Developers.
•
Chapter 1 contains an introduction to HMS and HMS Developer.
•
Chapter 2 describes the layout of the HMS and how to get started.
•
Chapter 3 outlines the interface for configuring developer applications.
•
Chapter 4 describes the pre-defined HMS reports and HMS Report Generator.
•
Chapter 5 describes additional configuration available to developers.
•
Appendix A describes the log events that can be accessed via the Report Generator.
Audience
This document contains information for technical staff and systems administrators responsible for the
installation and day-to-day management of a Holly deployment.
Glossary
Term
Definition
Application
An application as set up with the HMS Applications page. An application can have
multiple URLs set or none at all.
CLID
The calling party Telephone number
CSV
Comma Separated Values. The comma delimited file format. The Holly VAD uses files
with any text delimiter.
CVAD
Corporate Voice Activated Dialer. The system to allow calls to be routed to contacts
listed in a large central directory such as a corporate staff list by speaking the contact
names.
Dialog
Dialogs define how Holly interacts with a caller at any point in time. They define the
phrases Holly will speak to the user and how to interpret phrases (user utterances)
spoken back to Holly. Dialogs define what response to make if the user says something
unexpected or unintelligible, and Dialogs contain references to the functions which will
process the user’s answers. Dialogs are the building blocks of a Holly application and
applications are defined by linking Dialogs together.
Dictionaries
Configuration files for Recognition servers which contain words and phonetic
translations.
DNIS
The telephone number dialed to access the system
Grammars
Configuration instructions for Recognition servers which define which phrases can be
spoken and how they are interpreted.
HMS
Holly Management System.
Holly
Holly Australia Pty Ltd
Holly Configuration
Manager
The component of the Holly Voice Platform which maintains all configuration
parameters for the other components. See the Holly Administrators Guide for more
information.
Final
HMS Developer Guide, v1-0, 22 December 2009
3/39
Confidential
Term
Definition
HVP
Holly Voice Platform.
Prompts
The phrases spoken by Holly to the caller
Text to Speech
An application which converts text into synthesized speech
Utterances
The phrases spoken by the caller to Holly.
VoiceXML
application
The initial document referred to by an application URL, this could be a VoiceXML
document or perhaps an Audium application which generates VoiceXML documents
dynamically.
VoiceXML document
A single VoiceXML document as executed by the Holly VoiceXML Browser.
Final
HMS Developer Guide, v1-0, 22 December 2009
4/39
Confidential
Table of Contents
1.
About the Holly Management System ............................................................ 6
1.1
2.
3.
Navigating the Holly Management System ...................................................... 8
2.1
The Context Chooser .........................................................................8
2.2
Navigating the HMS ...........................................................................8
2.3
Logging In as a Developer ....................................................................9
Administration ...................................................................................... 11
3.1
4.
5.
Developer Applications ..................................................................... 11
Reports ............................................................................................... 13
4.1
Introduction ................................................................................. 13
4.2
The Reporting Subsystem .................................................................. 13
4.3
The Main Reports Menu .................................................................... 14
4.4
The Report Layout .......................................................................... 15
4.5
Call Details Display ......................................................................... 21
4.6
My Reports ................................................................................... 23
4.7
Additional Reports .......................................................................... 24
Configuration ....................................................................................... 35
5.1
A.
HMS Developer (HMS Dev) ...................................................................6
Change Password ............................................................................ 35
Appendix: LOG_EVENTS table ................................................................... 36
Final
HMS Developer Guide, v1-0, 22 December 2009
5/39
Confidential
1.
About the Holly Management System
The Holly Management System (HMS) provides Administrators and System Owners of the Holly Voice
Platform with a dashboard to access configuration web pages and usage reports. It acts as a centralized
entry point to the tools and utilities of the Holly Voice Platform, and provides a consolidated view
across all components in the deployment. The utilities and reports of the HMS are web-based to
enable users to access them remotely. Entry to the system is securely controlled by a group/role-based
authorization model.
The HMS system configuration tools allow authorized administrators to easily alter system behavior and
resource usage policies and ensure that configuration information is consistent and correct across all
instances in very large deployments. In addition the HMS provides several graphical reports to
facilitate the monitoring of key trends such as system usage, customer behavior, task completion rates
etc. The HMS reports allow administrators and support staff to drill down to examine the detail of a
single call, and even to hear the actual utterance spoken by the caller.
The Holly Voice Platform allows many applications owned by numerous virtual tenants to be executed
on a single deployment. The utilities and reports of the HMS securely partition the system data and
configuration information to allow tenants to have free access to reports and self-service application
configurations for their own purposes, without compromising the security of the data and the
applications belonging to other tenants.
Browser Requirements
The Holly Management System is qualified for use with Internet Explorer version 7.
Please note that a minimum of QuickTime 6.5 is required for playback of recorded audio files.
1.1
HMS Developer (HMS Dev)
HMS Dev is a set of web pages available separately through the HMS where developers can create, test
and maintain their VoiceXML applications for remote execution on the Holly Voice Platform. Configured
by the System Owner, HMS Dev provides developers with the following capabilities:
1.1.1
•
Configuration of URL and login PIN for pre-assigned inbound phone numbers
•
Ability to view execution logs for calls to HVP including the ability to view the VoiceXML
documents that were executed along with other call events
•
Ability to view other HMS reports including call statistics and Application Performance reports
•
Access to speech recognition logs to facilitate development and tuning for speech applications
executing on a remote hosted instance of the Holly Voice Platform.
Account Management
Creating and deleting a developer account requires system owner or service provider access to a full
HMS Operations installation. For security reasons developer accounts do not have these permissions.
1.1.2
Inbound Numbers and PIN Protection
A single inbound phone number corresponds to a single application and, according to the Holly IVR
Virtualization model, a single affiliate which is the level a developer has access to. In this model the
developer has a choice between:
Final
HMS Developer Guide, v1-0, 22 December 2009
6/39
Confidential
•
HVP performing a PIN verification before a “goto” to the developer’s VoiceXML application
•
HVP executing the developer’s VoiceXML application immediately upon answering the call. The
developer may choose to incorporate security checks in their application.
The following steps are performed when calling an application:
1.1.3
•
The developer calls the HMS Dev phone number
•
If a PIN is required, the developer enters the PIN. A correct PIN results in continuation of the call
with the developer’s application. An incorrect PIN results in an appropriate error message and
hang-up
•
If a PIN is not required, HVP immediately executes the developer’s application.
Viewing Call Logs
During execution of a VoiceXML application, both the browser and the recognition engine generate log
data. HMS Dev provides the means to view browser logs online, any documents that were executed and
download log files from the recognizer to be used with the developer tools supplied by the developers
of the particular recognition engine (e.g. ScanSoft or Nuance).
HMS Dev supports HTTPS (secure HTTP) for secure communication between a developer and the HMS /
HVP installation. This permits use of HMS Dev over the Internet. Further, the use of HTTP or HTTPS
allows HMS Dev to reside behind a firewall, VPN or other security mechanism to protect the HMS Dev
and HVP installation.
Final
HMS Developer Guide, v1-0, 22 December 2009
7/39
Confidential
2.
Navigating the Holly Management System
The web pages for the Holly Management System are displayed as a single page. On the top-left is a
navigation menu of all the tools and reports available to the user, and on the bottom the detail for
each particular tool or report.
Figure 1 HMS pages
2.1
The Context Chooser
The top-right of the web page also includes a context chooser. This enables the user to change the
scope for the HMS system, and use tools or view reports for a specific application. The contents of
these dropdown lists depend on the user’s access level. The context chooser also allows the option to
choose “All Applications”.
2.2
Navigating the HMS
Click on a link in the navigation menu to view the relevant page.
Each tool has a number of fields to configure the report. Enter the configuration data in the fields, or
use the dropdown lists to choose from available options.
Information displayed in reports is hyperlinked to related information. Click on a link to examine in
more detail. For example, in the System Usage report, each date is linked to a separate page displaying
all calls handled for that date. Similarly, each call ID is linked to a Customer Usage report.
Note:
Some pages may require you to maximize your web browser and scroll to see all the
information.
Final
HMS Developer Guide, v1-0, 22 December 2009
8/39
Confidential
2.3
Logging In as a Developer
To use the Holly Management System you must first supply a username and password. This is necessary
for security reasons, and ensures that sensitive information about the system statistics remains
confidential.
Figure 2 HMS Login page
1. From a web browser, browse to http://<domainname>:2020/.
The Holly Management System Login screen displays.
Note:
Browse to https://<domainname>:2021/ for SSL-enabled communication.
2. Enter your username and password and click Login.
The syntax is <username>@<affiliate name>; e.g. developer@companyx.
If the login process fails you are notified of the error and given further chances to retry.
Up to three attempts to login in are permitted. If you fail three times in a row, the account will
become locked and must be enabled by someone with access to the System Users, Service Provider
Users or Affiliate Users page for that account. If the disabled account is the administrator user then the
account needs to be enabled through the database.
Final
HMS Developer Guide, v1-0, 22 December 2009
9/39
Confidential
When logged in correctly, the following screen displays:
Figure 3 HMS welcome page
It is recommended to change the initial password after logging in for the first time, refer to Change
Password.
Final
HMS Developer Guide, v1-0, 22 December 2009
10/39
Confidential
3.
Administration
The Administration menu provides access to the Developer Applications page. Developers can use this
page to modify selected properties of pre-defined applications.
3.1
Developer Applications
Figure 4 Developer Applications
To modify a developer application:
1. Click Developer Applications in the Administration menu.
2. Select a Service Provider from the dropdown list (if not already selected).
3. Select an Affiliate from the dropdown list (if not already selected).
4. Select the appropriate radio button under Available Applications.
5. Change the Description if desired (used in other pages and dropdown boxes to identify the
application).
6. Change the application Name if desired (used in report details where a full description would be to
long).
7. Enter the URL for the application. This will modify the main URL for the application however
failover URLs specified by an Administrator on the Applications page will be left unchanged.
Note:
Application URLs may be entered without escaping URL- or XML-reserved characters; such
URLs are stored in a URL-encoded form. If an invalid URL is entered an error message will
occur when trying to load the application.
8. Enter a PIN to set restricted access to the application.
Final
HMS Developer Guide, v1-0, 22 December 2009
11/39
Confidential
9. Select Check PIN to enable the use of the PIN.
Note:
Enabling PIN checking modifies the first application URL used by the Holly Licenses Manager
so that a PIN checking application is called first, the result of this modification can be seen by
an Administrator on the Applications page.
10. Click Update.
3.1.1
PIN Checking
PIN checking occurs by passing the developer’s URL along with the application PIN to a PIN checking
VoiceXML document which is set in the configuration web page for Developer Applications. This occurs
by changing the first URL within the applications list of URLs, for example:
http://hostname/vxml/sandpit_intro.vxml?url=http://hostname2/test.vxml&ap
plication=1234
Where http://hostname/vxml/sandpit_intro.vxml is the PIN checking URL as set within the
Developer Applications configuration web page, http://hostname2/test.vxml is the developers
VoiceXML application, and 1234 is the application PIN.
The true URL can be seen on the Applications page, if there is more than one URL specified then the
others are left unchanged.
Final
HMS Developer Guide, v1-0, 22 December 2009
12/39
Confidential
4.
Reports
4.1
Introduction
The Holly5 platform continually collects important information about the characteristics, performance
and behaviour of every call as it is received and serviced, and stores this information securely in the
Holly database. Each call generates a single Call Detail Record containing more than 70 different
metrics and attributes of the call including important information about the call origins, application
instructions and execution environment; numerous performance metrics including latencies, cpu and
resource usage; and application specific information written to the Call Detail Record by the VoiceXML
application. In addition the Holly5 platform can be configured to collect Call Events which details
each step in the Call execution.
The Holly Management System includes a wide range of pre-packaged and custom reports to allow
business managers, application owners and operations personnel to interrogate and present this
important information in a clear, meaningful and appropriate manner to satisfy their specific
requirements. The many pre-packaged reports are designed to satisfy most user requests and
requirements, and present extensive and clear, graphical and tabular information displays quickly
within a few mouse-clicks. These reports provide comprehensive system trend analysis of the CDR
metrics, and allow side-by-side comparisons of the various different data sets. The reports also enable
users to quickly drill-down to the specific details of each individual call if required.
In addition the HMS includes features to allow users to easily customise these pre-package reports
through the HMS web interface. Using these features, the information and attributes of the reports can
be augmented and modified to suit specific requirements, and the new report configurations saved for
future use.
Finally all report data sets can be exported to the local file system for import into third-party reporting
tools, spreadsheets and databases.
4.2
The Reporting Subsystem
Most reports in the Holly Management System belong to the Reporting Subsystem introduced in version
5.1. Although each report is configured to display related data in specific categories, the layout,
“look-and-feel”, behaviour and functionality of each report is the same, and presented in the same
framework or subsystem. Additionally the factory configuration of each report which determines the
presented data sets and filter criteria, can be modified to suit any specific requirement.
The remainder of this section describes the features and functionality of this Reporting Subsystem.
Note:
The reports of the older versions of the Holly Voice Platform are also available and are
described in section 4.7 Additional Reports. Although the functionality of the Reporting
Subsystem largely supersedes these reports they remain very functional and useful.
Final
HMS Developer Guide, v1-0, 22 December 2009
13/39
Confidential
4.3
The Main Reports Menu
All Reports are accessed from the main “Reports” menu of the Holly Management System. There are
15 main reports listed below:
•
Call Volumes: key statistics and trends on use of system capacity
•
Call Information: core data on calls such as start time, duration, ANI, DNIS etc.
•
Virtual IVRs: compare behaviours of each Virtual IVR (application, Affiliate or SP)
•
Telephony: monitor a range of telephony-related call statistics
•
VoiceXML: key statistics on page executions, errors, fail-over, logging, performance
•
Error Analysis: more detailed view of VoiceXML error including first-error detail
•
Transfers: detailed statistics on transfer types, destinations and success/failure
•
Input - ASR & DTMF: detailed stats on input attempts, caller behaviours, responses
•
Output – TTS & Prompts: detailed stats on prompt usage, performance, TTS etc.
•
Content Fetching: detailed stats on the application web fetching activity
•
Logging & Recording: view all the usage of logging and recording resources
•
System Resources: view the use of critical shared system resources
•
Web Server Performance: detail on the performance of every web/app server
•
ASR Logging: get access to speech recognition logs for Nuance ASR engines
•
Port Usage : show trends relating to the number of concurrent calls serviced by the system
Figure 5 The main Reports menu
Final
HMS Developer Guide, v1-0, 22 December 2009
14/39
Confidential
4.4
The Report Layout
Each report is presented with the same screen layout and functionality with standard buttons, three
presentation types and a configuration page as shown and described below.
Figure 6 The tabs and buttons of the standard report layout (showing the “Call Report” tab of the
“Call Volumes” report.)
4.4.1
Call Report Tab
The Call Report tab shows a tabular presentation of all calls which are included in the base data set
for the report, with summaries and totals for each of the metrics in the report.
The sort order of the table can be modified by clicking one of the column headers.
By default the first column of the Call Report Tab is the call identifier which is also a hyperlink to the
Call Detail Report described in section 4.5 Call Details Display.
The summary totals at the bottom of the page are for the whole report (not just the displayed
records). The number of records displayed may be changed in the Configure Reports Tab described in
section 4.4.4 The Configure Reports Tab. Applying Filtering and Modifying the Reports.
The [Export Data] button creates a CSV version of the table for import to Excel and other programs.
The [Refresh] button re-executes the report with the current settings.
Final
HMS Developer Guide, v1-0, 22 December 2009
© 2009 Holly Connects
15/39
hvp-hms-0014
Holly 5-1 HMS Developer Guide v1-0.doc
Confidential
Figure 7 The Call Report Tab display
Note:
4.4.2
The “ASR Logging” and “Port Usage” reports are not based on a data set derived from calls to
the system, but rather from the number of ASR logs and the number of “ports” (ie concurrent
calls) recorded respectively. For this reason, for these reports the Call Report Tab is replaced
with a tab to show the appropriate base data set for each report.
The Trending Report Tab
The Trending Report Tab is included to allow the report metrics to be graphed showing growth trends
over time, and each report contains several tabs to graph each of the various data items in the report.
Related attributes are graphed together and distinguished by different colours. For example the
VoiceXML Errors Trend graph includes data for critical, severe and warning type errors as well as total
errors.
The sample time periods for the graphs can be defined by year, month, day, hour or minute. The
period can be changed by the drop-down list. The [Refresh] button must be pressed to re-execute the
graph with the new period.
The graph is drawn as either a line diagram or a block diagram depending on the number of data
points.
Final
HMS Developer Guide, v1-0, 22 December 2009
© 2009 Holly Connects
16/39
hvp-hms-0014
Holly 5-1 HMS Developer Guide v1-0.doc
Confidential
The data set is shown in tabular form below the graph.
The [Export Data] button creates a CSV version of the data table for import to Excel and other
programs.
Figure 8 The Trend Report Tab display
4.4.3
The Comparison Report Tab
The Comparison Report Tab is included to allow the report metrics to be compared across various
groupings. These groupings are:
•
•
•
•
•
•
•
Service Provider/Affiliate/Application
Hang Up Party – The party which terminated the call.
Call Outcome – which is a VoiceXML application filled value indicating success of fail.
Web Server Host – the host which served the VoiceXML documents.
Call Description 1 - which is a VoiceXML application filled value. The VoiceXML application
developer can write any string to this field to enable application specific reporting. For
example it may hold a specific caller property (type, nationality, gender etc) which can then
be used in these comparison reports. Please refer to the Holly VoiceXML Application
Developers Guide for more information.
Call Description 2 – As above. A second VoiceXML application filled value which may be used
Call Host Name – the client host which fetched the VoiceXML documents.
Final
HMS Developer Guide, v1-0, 22 December 2009
© 2009 Holly Connects
17/39
hvp-hms-0014
Holly 5-1 HMS Developer Guide v1-0.doc
Confidential
The grouping defaults to “Application” but can be changed using the drop-down list. The [Refresh]
button must be pressed to re-execute the graph with the new grouping.
Each comparison report contains several tabs to show each data item. As with the Trend Report,
related attributes are graphed together and distinguished by different colours.
The data set is shown in tabular form below the graph.
The [Export Data] button creates a CSV version of the table for import to Excel and other programs.
Figure 9 The Comparison Report Tab display
Final
HMS Developer Guide, v1-0, 22 December 2009
© 2009 Holly Connects
18/39
hvp-hms-0014
Holly 5-1 HMS Developer Guide v1-0.doc
Confidential
4.4.4
The Configure Reports Tab. Applying Filtering and Modifying the Reports.
The Configure Reports tab includes powerful features to allow users to modify both the search/filter
criteria of each report and the displayed data set. Using these features users can customise the
reports to suit specific requirements and save the new report configuration for future use.
The [Run Report] button re-executes the report after a configuration change.
The [Save as New Report Button] prompts for a report name and saves the report to a user specific
repository. These reports can be accessed from the My Reports option in the main Reports menu.
The [Reset] button sets all configurations back to the factory defaults for that report.
Search Criteria.
The Configure Reports tab allows users to change the default search criteria for the report.
Note:
As with all HMS pages, the reports are always filtered by the Service
Provider/Affiliate/Application combination displayed in the top right of the HMS dashboard.
The Range: drop down boxes allow users to quickly change the date range for the report. By default
all reports show data from the last 7 days. The Range: drop down lists allow this operator on this
range to be change (“from”, “between”, “current” and “last”), followed by a second drop down box
with time periods ranging from 60 minutes to 28 days. More complex date searching can be
implemented by defining conditions based on the “Start Time” or “End Time” attributes using the
Search Box described below.
The Search: Box allows users to add and remove any available attribute to the filter criteria of the
report. Attributes can be added by selecting from the drop down box and clicking the [+] button to
add to the filter criteria.
A second drop down box presents a selection of relevant operators for the data type of the attribute,
and entry field is provided to enter the input value. For enumerated types a drop down box of the
options is presented and date/time attributes are presented with a calendar to facilitate the entry of
input values. Attributes which have no operators and input values are not included in the search.
The current filter definition is displayed in the Search Box header.
Attributes can be removed by clicking the [-] button, and the default attributes for the report can be
reset using the [Reset] button. The modified report can be run (using the [Run Report] button)
and/or saved (using the [Save as New Report Button] button).
Displayed Data.
The Show: box allows users to change the attributes shown in the reports. Using the [+] and [-]
buttons as described above, attributes and be included or removed from the report. Because the
reports present grouped summary information, a grouping summary function must be selected for each
attribute. The drop-down box next to the attribute presents appropriate summarising options for the
attribute data type.
Final
HMS Developer Guide, v1-0, 22 December 2009
© 2009 Holly Connects
19/39
hvp-hms-0014
Holly 5-1 HMS Developer Guide v1-0.doc
Confidential
Rows per Page
A drop-down box at the bottom of the “Configure Reports display allows the number of records
included in the tabular displays to be configured. This value defaults to 20 but may be increased up to
1000 rows.
Figure 10 The Configure Reports Tab display
Final
HMS Developer Guide, v1-0, 22 December 2009
© 2009 Holly Connects
20/39
hvp-hms-0014
Holly 5-1 HMS Developer Guide v1-0.doc
Confidential
4.5
Call Details Display
The Call Details is a separate report showing all detailed information about a single call. It can be
accessed from the Call ID hyperlinks in the Call Report Tab of each report (described in section 4.4 The
Report Layout above) or by using the Find Call option on the main report menu, which allows a single
call to be retrieved based on the Holly allocated Call ID or SIP ID values.
The Call Details display has header information including the Call Id, an object to play the call
recording (if it has been collected), the Start Time of the call and the owning application. The header
also includes the [Show Latest Call] button which will display the last know call for the application,
and a [Export Call Events] button which will export all information about the call into a comma
separated format.
Note:
The Export Call Events information and format is particularly valuable and should be
included in any fault raised to Holly Operations.
Below the header information are four tabs described below.
The Events Tab shows all collected events for the call. Events show the granular detail of the
progress of the call and are typically used for fault finding and debugging.
Note:
Events are only collected if the application parameters have been set to collect this
information.
Event Types can be filtered using the Search: field which will return records with an occurrence of the
input string in either the Event Type or parameter field.
Each displayed Event Type is a hyperlink which when selected will display all information for all Events
of that type in this call.
Each Event Type has an associated [-] button which expands the parameter list to show all parameters
for the event.
The Document Dump Event parameter is a URL to display the actual executed VoiceXML document in a
marked-up report. This display also allows the document to be downloaded to the file system.
The Call Information Tab shows general and commonly queried attributes of the Call.
The Input and Output Tab shows attributes related to call input, prompt playing and various system
responsiveness metrics with regard to processing caller input.
The Performance and Resource Tab shows attributes related to system and application performance
including metrics concerning information logging in the database, access to speech resources, cpu
usage and fetch performance.
Final
HMS Developer Guide, v1-0, 22 December 2009
© 2009 Holly Connects
21/39
hvp-hms-0014
Holly 5-1 HMS Developer Guide v1-0.doc
Confidential
Figure 11 The Call Details report showing the Events tab
Figure 12 The Call Details report showing the Performance and Resources tab
Final
HMS Developer Guide, v1-0, 22 December 2009
© 2009 Holly Connects
22/39
hvp-hms-0014
Holly 5-1 HMS Developer Guide v1-0.doc
Confidential
4.6
My Reports
My Reports is the new page for managing custom reports and is available from the main Reports menu.
This option displays a listing of all saved report configurations for the report owner (Adminstrator,
Service Provider or Affiliate), with the option to edit (using the hyperlink of the report name), delete
(using the [-] button) or create a new report (using the [New Report] button.)
The [New Report] button will create a default report with a single Call Id presentation attribute and
the default search criterion of the last 7 days. This report can be changed using the “Configure
Reports” tab in as described in section 4.4.4 The Configure Reports Tab. Applying Filtering and
Modifying the Reports.
Final
HMS Developer Guide, v1-0, 22 December 2009
© 2009 Holly Connects
23/39
hvp-hms-0014
Holly 5-1 HMS Developer Guide v1-0.doc
Confidential
4.7
Additional Reports
The Holly Management System contains several reports which were available in earlier versions of the
Holly Voice Platform are included not only for backward compatibility but also because they remain
useful and very functional. These reports are described in this section.
Final
HMS Developer Guide, v1-0, 22 December 2009
© 2009 Holly Connects
24/39
hvp-hms-0014
Holly 5-1 HMS Developer Guide v1-0.doc
Confidential
4.7.1
The System Usage Report
The System Usage report provides daily call statistics in both graphical and tabular formats. This report
shows the call activity of the whole system over time including:
•
the total number of calls
•
the number of distinct callers
•
the average duration of calls
•
the total number of call minutes
•
the call volumes, showing high water usage of the system ports (i.e. the number of simultaneous
calls across the system).
The System Usage graphs this information, to clearly show trends in system usage and customer
behavior, and also presents a daily breakdown in tabular form. The Date column of the daily
breakdown table contains a hyperlink which displays all the calls for that day in the Customer Usage
report.
Figure 13 System Usage report
Final
HMS Developer Guide, v1-0, 22 December 2009
© 2009 Holly Connects
25/39
hvp-hms-0014
Holly 5-1 HMS Developer Guide v1-0.doc
Confidential
The report period can be changed in the From and To fields.
The Show checkbox must be selected to see a graphical representation of the call data, followed by
the [Display Usage] button.
The Fit to Data checkbox will modify the display to removed areas with no data
Either a line or a bar graph will display depending on the time interval displayed.
The Date column is a hyperlink to the Customer Usage report showing all calls made to the system on
that date.
4.7.2
Customer Usage
The Customer Usage report displays a list of calls matching a given search criteria that fall within a
specified usage period. This report is used to examine an individual call made to the system, perhaps
to determine its origin or the caller’s identity for customer support purposes.
Calls may be filtered by any of the call attributes including:
•
Call ID
•
Application Name
•
Duration
•
CLID (the “a” party or caller’s phone number – Calling Line ID)
•
DNIS (the “b” party or host’s phone number – Dialed Number Information Service)
•
CLID P/R (whether full CLID is Present or Restricted or if the P/R status of the CLID is Unknown)
•
Clear CLID
•
Event ID
•
Parameter.
The report presents the call’s details in tabular format. The column values for Service Provider,
Affiliate, Application, CLID, DNIS, CLIDP/R and Clear CLID are also hyperlinks which will display a
Customer Usage report restricted by the selected column value. For example, clicking on a CLID will
restrict the report to show only calls made from that particular phone for the defined period.
The Call ID value is also a hyperlink, which will display the Event Details of that call in the Customer
Usage call events report.
Customer Usage reports may be used to monitor user behavior and generate billing information.
Final
HMS Developer Guide, v1-0, 22 December 2009
© 2009 Holly Connects
26/39
hvp-hms-0014
Holly 5-1 HMS Developer Guide v1-0.doc
Confidential
Figure 14 Customer Usage report
4.7.3
Search Options
Wildcard search
Wildcards are symbols that replace a single character or range of characters in a search string to assist
in searching for either a wider range or specific range of call data. For instance, to search for all DNIS
that begin with 555 enter “555%” into the DNIS field, to search for calls with a Param ID of
result=success enter “%success”.
NOT
Final
HMS Developer Guide, v1-0, 22 December 2009
© 2009 Holly Connects
27/39
hvp-hms-0014
Holly 5-1 HMS Developer Guide v1-0.doc
Confidential
A number of fields permit exclusion of search criteria by selecting the associated Not checkbox. For
instance entering a DNIS of “5558207” will not return any calls matching this value.
Total number of calls
By default the total number of calls and the number of pages of results is not shown as this calculation
results in slower performance for large date ranges when executing reports. If required, selecting the
Calculate Total Number of Calls Returned checkbox will ensure these values are calculated.
CSV files
The output of a search on the Customer Usage page can be saved to CSV (an Excel-compatible file
format) by clicking the [CSV File] button.
4.7.4
Call Events
Each call made in the Holly system is logged as a sequence of call events, the Customer Usage call
events report shows the event path the customer has taken through each dialog on a call. To view a
list of events and a description of each event refer to the LOG_EVENTS table in the Holly Voice
Platform Reference Manual.
Clicking on a hyperlink in the Call ID column of the Customer Usage report displays the Customer
Usage Call Events report and allows a user to reproduce the call.
Note:
Events are not logged by default. The logging of an event can be configured by an
administrator.
Final
HMS Developer Guide, v1-0, 22 December 2009
© 2009 Holly Connects
28/39
hvp-hms-0014
Holly 5-1 HMS Developer Guide v1-0.doc
Confidential
Figure 15 Customer Usage call events
The Call Utterance displayed above the Call Log table (if Call Utterance logging is turned on) contains
the sound file for the entire call, including both application and caller utterances.
The time at which each Event occurred is listed in the Event Time column of the table, Parameter
values associated to each event are listed in the Param ID column of the table.
To display only a given type of call Event, click on any Event ID value in the table. Only call events
matching the selected Event will be displayed.
Final
HMS Developer Guide, v1-0, 22 December 2009
© 2009 Holly Connects
29/39
hvp-hms-0014
Holly 5-1 HMS Developer Guide v1-0.doc
Confidential
The call logs can also be searched using the Event Search fields Event ID and Parameter fields located
above the table. Queries can be entered using % as a wildcard.
In the Utterance column of the table are the wav files for each caller utterance logged during the call.
The utterances listed against each event are the utterances spoken by the user after hearing the
corresponding dialog prompt. In some cases there will be more than one utterance per event – if Holly
doesn’t understand the utterance the user is re-asked the same question.
If Document dumping is turned on, links are provided to URLs in the Param ID column.
Clicking the URL shown in the Param ID field will display the VoiceXML document web page. This page
shows the application URL for the document, the VoiceXML code, and any invalid VoiceXML code.
The Received Document URL shows the URL that was used to fetch the individual VoiceXML document,
selecting from this drop down menu lets you choose from all available VoiceXML document dumps
within the call without having to go back to the Log Events page.
Figure 16 HMS Customer Usage: Display URL
In some HMS setups the ASR logs can also be viewed from the Customer Usage report. If multiple ASR
engines and hosts were used during the call you can choose to view logs for a selected ASR engine on a
particular host.
4.7.5
ASR Logging
The ASR Logging page provides access to daily ASR log data for use in debugging or tuning the ASR
engine. To simplify searching for desired ASR daily log data it is possible to restrict which ASR entries
are listed by searching on date range, Host machine, application name (as set in Applications page or
the Developer Applications page) and by a particular ASR engine.
Final
HMS Developer Guide, v1-0, 22 December 2009
© 2009 Holly Connects
30/39
hvp-hms-0014
Holly 5-1 HMS Developer Guide v1-0.doc
Confidential
This page permits download of a compressed file containing log data in a form determined by the
particular ASR engine provider. The data can then be used with the development tools supplied by the
ASR engine provider.
Daily ASR logs are only present for applications which have the application parameter sr.asrlogmode
set to a value containing daily, for example “daily” or “daily,call”.
The following parameter must be set under Holly Configuration for ASR Logging to be available.
Administrators should refer to Operations Guide for details on configuration parameters.
Component
Parameter
Value
Web Page – Customer Usage
showasrlogselection
Set this value to “true” to enable viewing of ASR call log
information.
Figure 17 ASR Daily Logs page
4.7.6
Metrics
The Metrics report page displays call metrics such as call duration, total CPU, CPU Percentage, number
of recognition events, number of TTS events, TTS duration, TTS density, number of bytes logged, and
number of log events. These metrics can be grouped on different levels, that is, for a given Service
Provider, Affiliate, Application or for each individual call. The call metrics can be displayed in both
tabular and graphical formats.
Call duration for the metrics report is calculated in microseconds and rounded to milliseconds,
therefore the value is close to (but not necessarily exactly the same as) the call duration value
displayed in Customer Usage report.
Final
HMS Developer Guide, v1-0, 22 December 2009
© 2009 Holly Connects
31/39
hvp-hms-0014
Holly 5-1 HMS Developer Guide v1-0.doc
Confidential
Figure 18 Metrics report
The display of call data can be restricted according to whether a given call metric is Greater Than or
Less Than a specified value. For example to view metrics for calls with a minimum call duration of 20s,
type 20 into the Call Duration field, select Greater Than and click Display Usage.
To change the grouping of metric data, choose from Select Metric Type.
To see a graphical representation of the metrics data, select the Show checkbox.
The Fit to Data checkbox will modify the display to removed areas with no data..
The drop down menu below the graph enables the specific metric to be chosen: Call Duration, Total
CPU, CPU Percentage, # Of Recognition Events, # of TTS Events, TTS Duration, TTS Density, Bytes
Logged and # of Log Events.
Each graph is only generated as needed and it may take time for a graph to be calculated the first time
it is displayed.
4.7.7
Fetch Latencies
The Fetch Latencies report page displays latency metrics for the fetching of VoiceXML documents,
audio files, grammars and other objects. These metrics include:
•
Number Of Fetches
•
Percent Not Cached
•
Average Duration
Final
HMS Developer Guide, v1-0, 22 December 2009
© 2009 Holly Connects
32/39
hvp-hms-0014
Holly 5-1 HMS Developer Guide v1-0.doc
Confidential
•
Maximum Duration
•
Total Number of Bytes
•
Average Number of Bytes
•
Number of Fetch Errors
•
Fetch Type.
These fetch latencies can be grouped on different levels, that is, for a given Service Provider, Affiliate,
Application, Host or for each individual call. The call fetch latencies can be displayed in both tabular
and graphical formats.
Figure 19 Fetch Latencies
The display of call data can be restricted according to whether a given value is Greater Than or Less
Than a specified value. For example to view only the fetch latencies for calls with a maximum duration
greater than 2 seconds, enter 2 into the Max. Duration field and select Greater Than in the drop down
menu.
Select Fetch Latency Type drop down box changes the Latency grouping.
The Fetch Latencies report displays for the date range and criteria selected ordered by CPU.
The drop down menu below the graph allows the metric to be changed. Options are # of Fetches, %
Not Cached, Avg. Duration, Max. Duration, Total # of Bytes, Avg. # of Bytes, # Of Fetch Errors, and
Fetch Type.
Each graph is only generated as needed and it may take time for a graph to be calculated the first time
it is displayed.
Final
HMS Developer Guide, v1-0, 22 December 2009
© 2009 Holly Connects
33/39
hvp-hms-0014
Holly 5-1 HMS Developer Guide v1-0.doc
Confidential
4.7.8
Response Latencies
The Response Latencies report page displays response latencies, such as Response Events, Average
Latency and Maximum Latency. These response latencies can be displayed on different levels, that is,
for a given Service Provider, Affiliate, Application or for each individual call. The call response
latencies can be displayed in both tabular and graphical formats.
Figure 20 Response Latencies
The display of call data can be restricted according to whether a given value is Greater Than or Less
Than a specified value.
The Select Response Latencies Type drop down box allows to grouping to be changed.
The drop down menu below the graph enables the metric to be changed. Options are Number of
Response Events, Average Latency and Max. Latency.
Each graph is only generated as needed and it may take time for a graph to be calculated the first time
it is displayed.
Final
HMS Developer Guide, v1-0, 22 December 2009
© 2009 Holly Connects
34/39
hvp-hms-0014
Holly 5-1 HMS Developer Guide v1-0.doc
Confidential
5.
Configuration
Under the Configuration menu is the option to change your password.
5.1
Change Password
Figure 22 Change Password page
1. Click Change Password in the navigation menu.
2. Enter your old password and then your new password twice for verification. Note that all passwords
will be stored in the database, so avoid entering sensitive information such as existing system
passwords.
3. Click Change.
Final
HMS Developer Guide, v1-0, 22 December 2009
© 2009 Holly Connects
35/39
hvp-hms-0014
Holly 5-1 HMS Developer Guide v1-0.doc
Confidential
A.
Appendix: LOG_EVENTS table
The Holly Voice Platform inserts records into the LOG_EVENTS table during a call. These records
represent significant events which may be reported on for various reasons.
The LOG_EVENTS table includes the attribute EVENTID which contains the type of Event and the
attribute PARAM the value of which will vary dependant on the EVENT. The following table lists the
EVENTID and PARAM for all Events which may be logged and reported on via the Holly Report Generator.
Table structure
EVENTID
PARAM
Description
Answer
result=%s
Indicates the call has been answered or not. The
parameter “result” may be success or failure.
ASR Session
asrengine=%S| sessionid=%S|
address=<host:port>|
server=<server>|
endpoint=<address>
Logs the ASR session event. The endpoint address of the
recognizer is also included.
Call end
cpu=%.3f| normalcpu=%.3f|
callduration=%.3f| reccount=%d|
ttscount=%d| ttsduration=%.3f|
logcount=%d| logbytes=%d
Summary call statistics recorded irrespective of the
level of logging.
Call start
ANI=%s| DNIS=%s| VURL=%s|
follow on,reason=%s
At the start of a call logs the Caller Line Identifier,
DNIS, and the initial applications document identifier.
Disconnect
List of variables and
corresponding values specified
with VXML attribute ‘namelist’.
Records a hang-up event.
Document dump
<a reference to the VoiceXML
document>
Logs the whole VoiceXML document as it is fetched.
Document
transition
uri=%s| cpu=%.3f|
normalcpu=%.3f
Written when document scope changes.
Enrolment start
attempts=%d| minattempts=%d|
maxattempts=%d| timeout=.3f
Start of voice enrolment.
Enrolment end
Result=(success| |repeat| |no
input| |no match| |disconnect|
|error)
End of voice enrolment.
Error Critical
error=%d| msg=%s
Logged when a critical error occurs.
Error Severe
error=%d| msg=%s
Logged when a severe error occurs.
Error Warning
error=%d| msg=%s
Logged when a warning error occurs.
Exit
result=%s
Logs exit event
Fetch
uri=%s| fetchtype=(VXML|
|audio| |grammar| |other)|
incache=(true| |false)|
latency=%.3f| documentsize=%d|
outcome=(success| |no response|
|error| |timeout)|
failover=(true| |false)|
Logged for each document fetch. When external scripts
are fetched they are logged with a fetch type of 'vxml'.
Final
HMS Developer Guide, v1-0, 22 December 2009
© 2009 Holly Connects
36/39
hvp-hms-0014
Holly 5-1 HMS Developer Guide v1-0.doc
Confidential
EVENTID
PARAM
Description
localport=%s| hostname=%s
Grammar
activation
URI=%s
Fetch and/or activation of a new grammar.
Grammar
deactivation
URI=%s
Logged when a grammar is deactivated.
License
mode=(acquire| |resolve)|
key=nnn| outcome=(success|
|maximum licence number
reached or exceeded| |license
key not found| |socket error|
|message encode error|
|message decode error|
|application not found| |affiliate
not found| |service provider not
found| |licences do not
reconcile| |licence manager not
seeded| |connect failure)|
service=<service provider
ID>:<affiliate ID>:<application ID>
The first event for a session, preceding the “call start”
event representing the license lookup. Values are:
mode=acquire or resolve.
The value 'acquire' means that this license is being
requested for the duration of the call; this is the license
that authorizes all sessions within the call. The value
'resolve' means that this license is being requested
solely to obtain application data for a new session -- the
Holly License Manager will not increase the license
allocation for the application.
key=The key for the license lookup.
outcome= outcome of the request
If the key lookup is successful, a fourth field will be
present:
service=<service provider ID>:<affiliate ID>:<application
ID>
Note: by default this event is not included in the Holly
Voice Browser callevents parameter which determines
which events are logged.
Log Element
EVNT=<event id>| Label=<label in
log tab>| expr=<expression in log
tag>| content=<user defined
parameter
Logged as a result of a <LOG> tag included in the
VoiceXML document. . The VoiceXML log tag format
includes the attributes label and expr in addition to the
log tag content. The format and contents of the PARAM
field are under the control of the application builder.
The VoiceXML log tag format includes the attributes
label and expr.
Note: The browser treats <log> content differently if it
is of the form 'EVNT=<txt>|...'. This content will be
logged to the ASR engine with the event name <txt> and
content all the text after the '|'. If <txt> begins with
'SWI' or 'calllog:?' the event will NOT be logged to HMS;
otherwise the event will also be logged to HMS with the
event name <txt>.
This format is supported by the
SpeechWorks/ScanSoft/Nuance OSDM (OpenSpeech
DialogModule) products which log a lot of useful dialog
state information with events that start with
EVNT=SWI. This information can then by used by the
OpenSpeech Insight reporting tool to do some powerful
ASR analysis.
Placecall start
remote=<SIP URI>| local=<app
number>
Logs the SIP URI and application number.
Placecall end
result=(no reason supplied| |user
disconnect| |silent| |maximum
Logs the result of the outbound call.
Final
HMS Developer Guide, v1-0, 22 December 2009
© 2009 Holly Connects
37/39
hvp-hms-0014
Holly 5-1 HMS Developer Guide v1-0.doc
Confidential
EVENTID
PARAM
Description
duration| |special information
tones| |fax| |busy| |cti| |no
answer| |error| |bad
destination| |bad format|
|answering machine)
Prompt (external)
type=external| URI=%s
Logged when an external prompt is played. URI is the uri
of the audio file
Prompt (SSML)
type=SSML| content=%s
Logged when a TTS prompt is played. Content is the
TTS string.
Prompt
(disconnect )
status=disconnect
Logged when the Holly Voice Gateway attempts to play
a prompt but the call has already disconnected. This is
common in normal operation for most applications.
Recognition start
inputmodes=(dtmf| |voice|
|dtmf,voice)| threshold=%d|
timeout=%.3f|
bargeintype=(speech| |hotword)
Start of recognition. This event contains information
such as inputmodes, timeout, threshold and bargein
type.
Recognition end
(fail)
result=(no input| |disconnect|
|no match| |error)|
bargein=(true| |false)|
inputmode=unknown|
dtmfinput=%s
End of recognition when the recognition fails. The
PARAM value indicates the failure reason. Failed DTMF
input (if applicable) is also logged.
Recognition end
(success)
result=success| utterance=%s|
confidence=%d| bargein=(true|
|false)| inputmode=(dtmf|
|speech)| utterance=%s|
confidence=%d
End of a successful recognition.
Recording start
maxtime=%.3f| dtmfterm=(true|
|false)| type=%5
Start of voice recording.
Recording end
(fail)
result=(maxtime exceeded||no
input||disconnect||max speech
timeout||error)
End of voice recording when the recording has failed.
The PARAM value indicates the failure reason.
Recording end
(success)
result=success| duration=%d
End of a voice recording when successful.
SIP session
callid=<SIP called>| remotertp=%s| local-rtp=%s
System response
latency=%.3f
Logged whenever a recognition event occurs, in seconds
to millisecond resolution.
Transfer start
mode=(network| |blind| |bridge|
|conditional)| {URI=%s|
|destination=%s}
Start of a call transfer. Given either the URI or the
destination.
Transfer end(fail)
result=(bad destination|
|disconnect| |error| |remote
busy| |timeout| |network busy|
|maxtime exceeded)
End of a call transfer. PARAM value indicates the failure
reason.
Transfer
end(success)
result=success| duration=%.3f
End of a call transfer when successful. Duration is only
present for a successful bridge-transfer, value in
seconds to millisecond resolution.
Final
HMS Developer Guide, v1-0, 22 December 2009
© 2009 Holly Connects
38/39
hvp-hms-0014
Holly 5-1 HMS Developer Guide v1-0.doc
Confidential
EVENTID
PARAM
Description
VXML Element
event=%s
or event=%s| message=%s
Logs events thrown by VXML application
*** End of Document ***
Final
HMS Developer Guide, v1-0, 22 December 2009
© 2009 Holly Connects
39/39
hvp-hms-0014
Holly 5-1 HMS Developer Guide v1-0.doc