Denodo
Platform 4.6
Summary of New and Enhanced Features
Denodo Confidential
denodo.com
Denodo Platform 4.6. Summary of New and Enhanced Features
Executive Summary
Get ready to experience the benefits of Denodo Platform 4.6. It delivers unmatched flexibility, ease
of use, and performance to enable Enterprise Data Services to serve your application needs
Some of the key new features in Denodo Platform 4.6 are driven
by Your Needs and Our Innovation. They include:
Easier Access to Data Sources
Improved point and click access to data connections for SAP
R/3 and ECC, Oracle EBS, and other packaged applications,
SAP BW and other OLAP engines, complex multi-tab Microsoft
Excel content, encrypted files, etc.
Advanced Integration and Data Quality
New advanced transformation functions to modify the schema
of the data service exactly as you want it; Custom functions to
create your own specific transformations; Custom functions
that encapsulate any external process or tool in any step of
the data integration; New task dependence capabilities in
Scheduler for more control of batch export tasks.
New Data Governance Tools
New Data Lineage and Change Impact Analysis options so
that you can see how granular changes in the schemas or
views affect your overall data services; Enhanced Tree views,
Related Views, Attribute trace features from source to target
views.
More Flexible Data Services Publishing Options
Additional authentication mechanisms for SOAP/REST Web
Services with HTTP and WSS authentication; Data Services
publisher also supports HTML, RSS, JSON, ATOM etc. New
Portlet publisher allows one-click publishing of JSR-168,
JSR-286, Open Ajax and Microsoft WebParts portlets to your
favorite portal servers.
Increased Performance and Scalability
Denodo Engineers have worked hard to
dramatically improve performance in accessing
all types of data sources resulting in an average
improvement of more than 10% in most types
of queries when compared to 4.5. Denodo
continues to relentlessly focus on enterprise-class
performance to make Data Virtualization a viable
replacement to physical data integration for most
use case scenarios.
Usability Improvements
Denodo GUI modeler / editor is now integrated
with Eclipse IDE for easier development of custom
functions, connectors and stored procedures;
The GUI adds many new graphical and
auto-complete features that enable you to type
complex conditions and expressions quickly;
Web Integration wizard has been completely
enhanced and include templates for most
common scenarios.
Other Enterprise Features
New JMX and WS-Management monitoring
agents allow you to control and monitor Denodo
from your favorite console; New Denodo Monitor
tool for in-depth systems analysis, that allows
you to inspect the whole systems environment
more deeply (executed views, threads, memory,
sockets and process, etc.)
Denodo Confidential
denodo.com
Denodo Platform 4.6. Summary of New and Enhanced Features
Virtual DataPort (VDP)
Data Virtualization and Data Services
Data Access
Data Combination
Packaged Applications
Custom Functions
Lets users create new custom functions or encapsulate
external tools as functions and add them to the existing
Denodo function library. This allows these functions to
be visualized and invoked in the Denodo graphical UI
for use in any execution node.
New wrappers and enhancements to existing
ones that enable easy access to packaged
applications such as SAP R/3, SAP ECC or
Oracle eBusiness Suite.
Multi-dimensional Data Warehouses
New wrappers and enhancements that enable
access to MDX and/or XMLA-compliant
multidimensional data warehouses. The
wrappers enable direct MDX query execution,
or transformation from SQL to MDX. The
output will also be relational, enabling further
combination among the rest of the data
sources.
Relational Database Multi-table Import
This features lets users import several
tables out of a single schema of a relational
database, in a graphical way without having
to do them one by one.
Performance Enhancement in Delimited File
wrappers
Allows the delegation of processing from the view to
the wrapper.
General Performance Enhancement
Overall, greater than 10% performance improvement
in all types of data sources.
New Functions
Denodo provides new useful functions related to schema
transformation and matching.
• XSLT: XSLT transformations of XML datatypes.
• CAST: improvements on type casting.
• NEST: ability to create arrays from flat structures.
• ROW: ability to create records from flat structures.
Denodo Confidential
These additional functions make it easy to
re-create arrays and records even if they did
not exist or were flattened before. This helps
in preparing bottom-up data views to be in
alignment with pre-existing, top-down data
models and structures (for example existing
XSDs or Web Service definitions) for ease of
mapping.
Data Services Publishing
Exporting Authenticated Web Services
SOAP and REST Web Services published by
Virtual DataPort can be authenticated. The
supported authentication formats available
are:
• HTTP Basic authentication.
• HTTP Digest authentication.
• WSS basic token profile.
Support for Web Services using HTTPS
Virtual DataPort can import SOAP and REST
Web Services accessible through HTTPS
certificates. The process for allowing the
use of HTTPS in Denodo embedded web
services container is documented.
denodo.com
Denodo Platform 4.6. Summary of New and Enhanced Features
Exporting Portlets, Web parts and Open Ajax Widgets
This feature allows exporting a data view in Virtual DataPort
as a visual web widget. The widget then queries the dataview
using a web form and will show the results in a table
supporting pagination and sorting. The widget HTML could be
customized through stylesheets.
The widgets will also support inter-operation with other
independently developed widgets (see example below).
The widgets can be exported to use one or all of the
following widget standards:
• Java Portlets. The exported widgets can be compliant
to the JSR-168 and JSR-286 portlets specifications.
Those are the standards supported by virtually every
JAVA portal server, such as JBoss or IBM Websphere.
• Microsoft Web Parts. The Microsoft standard for visual
widgets that allows them to be used in tools such as
Microsoft Sharepoint.
• OpenAjax (http://www.openajax.org). The emerging
standard for AJAX-based widgets, used by tools such
as SAP Enterprise Mashup Application Builder (a
research prototype by SAP).
Widget Inter-operability
Suppose you use the Denodo Platform to publish a portlet which
allows you to query in a unified form the data from your clients,
and you load the portlet in your JSR-286 compliant Intranet
web portal (e.g. BEA Portal or IBM Websphere portal). Let
us also suppose that your Intranet web portal also includes
a portlet (external to the Denodo platform) able to show an
address in a map (e.g. Google Maps). This feature would allow
the portal administrator to configure the Portal system in such
a way that, when the user clicks on a client obtained in the
result listing shown by the Denodo portlet, the map portlet
automatically locates the client address in the map.
Data Services Modeling, Governance and Lineage
Management
The following is a list of new features and usability improvements that were added to the
VDP admin tool related to Data Services Modeling, Data Governance and Data Lineage
Management. Details on these improvements with screen shots are available in a separate
document - below is a summary:
• Improved browsing of the Tree View of a Denodo data view.
• Related Views. Given a View-V, see all the views that View-V participates in and
browse their related tree views.
• Full support for editing views and stored procedures without need to recreate them.
• Source refresh. Ability to synchronize base views with the changes made on the
sources since the base view was created. The changes are shown to the user who
can decide how to deal with them.
• Change Impact analysis. Before a change is made on a view, the user can preview it
and customize the way the change should be propagated to the views depending on
the modified view.
• Attribute Trace. Ability to trace the origin of any attribute on a view back to its source.
For any attribute, Virtual DataPort is able to show a tree view version of that derived
view, but highlighting the attributes that take part of the creation of that attribute,
and showing the expressions involved in the creation of their values.
• New editors for conditions and expressions:
• Graphical editor for building simple conditions.
• Auto-complete features for typing complex conditions and expressions.
• Ability to include functions in JOIN Conditions. This reduces the number of views
needed for some join conditions.
• Ability to include selection conditions in any kind of view, avoiding the need to create
a separate selection view.
• Some metadata such as a description and a “last-modified” date were added to the
catalog. This helps with data assurance.
• Ability to search views by name and/or metadata.
• Ctrl+Z and Ctrl+Y allows the user to undo and redo changes in any text field like a
view name field or an expression definition.
Denodo Confidential
denodo.com
Denodo Platform 4.6. Summary of New and Enhanced Features
Security & Management
• Memory information.
Improvements in JMX Monitoring
• Information about sockets state in the machine where the server
runs. Information about processes state in the machine where the
server runs.
Some new parameters were added to the auditing and monitoring
information exported via JMX such as:
• Number of wrappers / queries currently executing per datasource
type (the number per individual data source is already available).
• Rows returned up to this moment by a query/wrapper.
• Time elapsed since a query/wrapper started its execution.
Test Integration with Systems Monitoring Packages
The Virtual DataPort/ ITPilot server exposes monitoring information
through JMX. Denodo has analyzed and provided best-practices and
extensions (where required) to use some popular System monitoring
packages to monitor Virtual DataPort / ITPilot management information
exported through JMX.
Monitoring packages tested include the following:
Denodo Monitor tool
Denodo monitor is a command-line based utility for monitoring and auditing
purposes. Denodo Monitor periodically checks the state of the server and
logs the following information:
• Executed statements. All the queries/statements executed by the
server are logged. The exact time of execution and the user that
executed it are also logged.
• Threads information. Allows seeing the state of the threads, including
the estimated CPU usage. It is possible to know what query is
associated to each thread.
• HP Openview - a leading commercial packages for system
monitoring.
• Nagios, the most popular open-source monitoring tool.
• Osmius, an open-source offering.
Encrypted Files
• Support for accessing encrypted files in XML and data federation
wrappers.
Denodo Confidential
denodo.com
Denodo Platform 4.6. Summary of New and Enhanced Features
ITPilot
Web integration
Data Extraction
Excel Converter
An Excel to HTML converter is included. This way,
ITPilot can be used to parse Excel information
from spreadsheets that do not follow the tabular
structure assumed by the ODBC access.
Example-based Generation Improvements
Significant
improvements
in
the
example-based generation of DEXTL programs
in the Extractor component:
• Strong improvement of disambiguation
features.
• Improvements in tagset auto-generation
to reduce the number of tagsets
automatically created.
• Support for example-based extraction of
OPTION values in selection lists.
• Significant improvements when dealing
with records containing compound types
(sub-levels).
New Extractor Wizard
The wizard of the Extractor component has been
totally reworked. Some of the main changes are
highlighted below:
• The structure can now be created in the
browser while providing examples. If
manual creation is used, it is automatically
inferred from DEXTL specification.
• Recording ExtractorSequences do
not require selecting navigation
steps when providing examples. The
user can simply record the sequence
on one example.
• Examples provided in previous
sessions can now be revised / edited
in the browser.
• Creating
specifications
without
providing examples can now be done
through a mostly graphical process
that involves providing one single
example and then editing the DEXTL
specification as needed.
• Revised, easier to use support for
testing specifications in HTTP Browser
and Firefox.
• Revised, easier to use support for
testing extractor sequences.
Data Processing
Custom Functions
Lets users create new custom functions or
encapsulate external tools as functions and
add them to the existing Denodo function
library. This allows these functions to be
visualized and invoked in the Denodo
graphical UI for use in any execution
node.
Denodo Confidential
User Experience
Usability Improvements in ITPilot Wrapper
Generation Tool
The following improvements have been made on
ITP Wrapper generation tool:
• NSEQL editor with support for commands
auto-complete features and error detection
as the user types commands.
• Editors for conditions and expressions:
• Simple graphical editor for building
simple conditions.
• Auto-complete features for typing
complex conditions and expressions.
• Copy-paste components between different
wrappers.
• Insert Before / Insert After
components’ contextual menus.
options
in
• Improvements in Sequence Debugger:
• Auto-complete features.
• Ability to
together.
add
several
commands
• Ability to set breakpoints.
Support for recording drag and drop operations
in Navigation Sequence Recording Toolbar
Support is included for recording and executing
sequences including drag and drop operations.
denodo.com
Denodo Platform 4.6. Summary of New and Enhanced Features
Systems Management
JMX Improvements related to ITPilot
Several new parameters related to ITPilot were added to the JMX-accessible
information.
Aracne
Content Integration
Persistent Crawling with IECrawler
The parameters include:
• Total number of browsers .
• Total number of browsers per type (IE, Firefox, Denodo HTTP).
• HTTP Browser threads are correlated with the wrapper/query using it (using
name conventions in threads). The estimated CPU usage of each thread is
calculated. The time elapsed for each thread is also available.
In addition, naming conventions for IE and Firefox browsers have been used, so
they can also be correlated with wrappers/queries when monitored using standard
OS-dependent management tools (e.g. Windows task manager).
For long crawling processes, this feature allows IECrawler to
periodically serialize the state of a crawling process to disk, so it
can be resumed in the event of interruption or failure.
Scheduler
Batch-mode Integration Tasks
Support of Exporting Encrypted Files
Other Features
The CSV exporter is able to store the exported data in
encrypted form. The encryption used is compatible with the one
supported by the Virtual DataPort wrappers, so the exported
data can be directly consumed by Virtual DataPort.
i18n
Full support for non-western character sets.
Browser Support added
Chain of Tasks
• Firefox 3.0, 3.5 support.
• Internet Explorer 8 support.
• Ability to export / import wrappers between different instances of the ITPilot
Wrapper Generation Tool.
It is now possible to configure that a task should begin its
execution when another task finishes. This way, scenarios with
simple dependencies between tasks can be managed out of the
box.
Other Features
• New “append-based” CSV exporter.
• Allow to specify server paths for CSV export /import.
• Support for cloning tasks.
• Support for writing custom handlers.
• Minor usability improvements in web administration tool.
Denodo Confidential
denodo.com
Denodo Platform 4.6. Summary of New and Enhanced Features
Denodo Platform - Other
This section describes the features of the general
platform, including the Denodo Platform Control Center and
Installer which are not linked to any specific module.
Eclipse Plug-in for Development
Integration with Eclipse IDE for developing custom
functions and custom wrappers. This plug-in allows users
to launch Denodo servers and applications attached to
selected Eclipse projects containing extensions for those
applications. This way, users are able to debug their
extensions (VDP custom stored procedures, Scheduler
exporters, ITP custom functions, etc.) from Eclipse.
Upgrade Now to Denodo Platform 4.6
Please visit http://support.denodo.com
and fill in the form available in the Upgrade to 4.6 section.
Visit www.denodo.com
Email: [email protected]
Twitter.com/denodo
Unified Update Management
The Denodo Platform updates are now general to the
platform across all components such as Virtual DataPort,
ITPilot, Aracne, and Scheduler and synchronized.
Supported Environments Enhanced /Added
• Internet Explorer 8
Palo Alto, CA United States
Phone: (+1) 877 556-2531
London, United Kingdom
Phone: (+44) (0) 20 7869 8053
Madrid, Spain
Phone: (+34) 912 77 58 55
• Firefox 3.0, 3.5
• Windows 7
Denodo Confidential
denodo.com