Practical IT Research that Drives Measurable Results
Move from Chaos to a Realistic Data
Management Strategy
Un-complicate and integrate.
Present
Future
100101101010101101
+ organization + tools + strategy
101100101010011000100
0110010110001
100101110100100101101
101011010101010100100
10101101010111
•
Data architectures evolve as an organization grows. There comes a point when simple
integrations do not support business functions, yet 84% of organizations are handcoding point-to-point integrations.
If the present is a chaotic mess of integrations, learn how an understanding of the
basics, organization, a few key tools, and a realistic strategy can stop the chaos and
move towards a controlled state.
010111001110110011101
011001011000101101010
10110010101001100010101001101110100100
0101110011101100111010010100101010101111
101101110100110101010
10010101101010001111
•
01100101110100101000101101010
Executive Summary
•
There are no quick fixes to a chaotic data architecture. Getting to an optimized state is a
strategic journey. Getting to a controlled state is tactical.
•
76% of organizations don’t do data architecture documentation. This increases the
complexity and risk of each subsequent integration project exponentially and leads to a
poorly planned, organic data architecture.
•
Gaps in a data integration strategy, such as lack of executive buy-in or standardization in
approach, tools and processes, lead to roadblocks in the path to the desired architecture.
•
There are four data architecture approaches that can guide the long term journey: pointto-point, hub and spoke, federated, and distributed. Determining which to use is a matter
of who you are as an organization, who’s behind your initiative, and some key
requirements.
•
82% of organizations agree that data integration projects will not be approved as standalone projects. Leverage tack-on projects to demonstrate your strategy, instead of letting
your architecture become a victim to them.
•
While the journey to optimization is longer than any one project, there are tactics that
can be employed now to move towards a controlled data architecture.
Get the
Picture
Understand the
Options
Map the Madness
Pick a Path
Plan the
Attack
Assess the Gaps
Pick a Tool
Standardize an
approach
Make the
Case
Put Pen to Paper
Be Transparent
Get to Work
The integration space consists of data, application and process
integration. This solution set focuses on data integration.
Data Integration
• Reflects the convergence of EAI, DI and ETL
vendors.
• Gaining momentum as organizations move from
hand-coding to tools.
Integration
Application Integration
• Generally part of an application
implementation project.
• Application integration projects typically handcoded or a point solution used.
Process Integration
• Workflow automation for processes such as
online ordering to eliminate lag time.
• Creates opportunity for coordinated escalation.
• Solid architecture and process is paramount.
SOA and Middleware
• Creates access to data in distributed
architectures.
• Advanced architecture is critical.
Data architecture is a map showing how your integrations fit
together. If yours is chaos, it affects the business and IT jointly.
Moderate
pain point
for % of
organizations
Major pain
point for % of
organizations
Data manually assembled and reconciled from various
sources for enterprise reporting
32%
55%
Poor data architecture documentation increasing
integration complexity and project timelines
32%
54%
Data not synchronized across applications, so data
conflicts across systems
34%
51%
Duplicate records in the same source system creating
integration and reporting issues
37%
48%
Manually re-keying data is causing data quality issues
40%
45%
Manually re-keying data is wasting resource time
38%
45%
Paying for extra application licenses so users can view
key data
44%
34%
Performing multiple replications, cleansing and
transformation of the same data
44%
34%
46%
29%
Pain Point – is this your organization?
Poor access to data residing in applications with a
limited number of licenses
IT
Feels it
Business
Feels it
N= 114 Source: Info-Tech Research Group
The first step to stopping chaos is to level the playing field.
Don’t assume your integration developers know the basics.
Having a common understanding of the basics of data integration is the
first step in assessing which tools, approaches and architecture fit best with
current goals.
The Basics of Data Integration
• Data integration is the process of combining information stored in various sources to
provide a unified view of the data, increase accessibility and decrease inconsistencies.
• Inaccurate data makes operating a business a challenge. All business units are affected
by bad data.
• Data architectures should be able to accommodate business needs.
• Master Data Management coordinates disparate data sources; one method of creating a
single version of the truth.
• Data integration enables strategies such as business intelligence and establishing a
service-oriented architecture.
• Refer to Info-Tech’s: “Data Integration 101” for a more in-depth explanation.
Know your drivers. For 23% of mid-sized organizations, sharing data
between internal and external organizations is a main driver.
Organizations experience different drivers of
data integration.
Determine what
drives integration
Sharing (Internal/External)
23%
Cleansing
22%
Data Acquisition
21%
Migrations/Conversions
Synchronization between Systems
20%
13%
N=116 Source: Info-Tech Research Group
• Advanced business
functions require
more sophisticated
integration methods.
• Consider the
following
architectures:
− Point-to-Point
− Hub and Spoke
− Federated
− Distributed
• See slides 10 and 11
for more detail.
Document your architecture. 76% of organizations don’t
document data architecture; most report increased complexity
and risk to integration projects.
Map the madness: Every poorly documented integration added to your
architecture increases complexity and the need for rework in subsequent
projects.
•
Clients cited the following as causes of poor architecture:
−
−
−
−
−
−
Poor documentation, including lack of data dictionary and data model.
Lack of vision, model to work against.
Poor project planning and scoping.
Lack of standard integration approaches, tools.
Poor developer skill sets.
Application redundancy.
“Systems upon systems were built with[out] the thought of integration.
Now we are trying to connect and transform all that data.”.
– VP, Technology Services Company.
Know features and benefits of each of the 4 data architecture
models that organizations can use.
1
Point-to-Point
• Tightly coupled connections between
software applications.
• Quick method to solving problems.
• Increases complexity of architecture.
• Challenging to map and adapt as
needed.
• Does not accommodate efficient
change to architecture.
• Should be used sparingly.
3
2
Hub and Spoke
4
Distributed
• Centralized architecture.
• Uses a data store (Hub) to populate
systems (Spokes).
• Single version of the truth resides in
the Hub.
• Reduces complexity.
• Allows for incremental connections.
• Building two way communication
between systems is challenging.
Federated
• Similar to Hub and Spoke.
• The data store does not actually
store any data – only rules to direct
and coordinate data between
systems.
• Allows for many transactions.
• Updates occur in real-time.
011001010010
101010011110
101010010100
10100010110
• Similar to Hub and Spoke.
• Uses ‘agent computers’ positioned
between the system and data store.
• Agent Computers reduce processing
load on the system they are
connected to.
• Provides scalability.
• Adapts to changes in business flow.
• Challenging when dealing with
operating systems and platforms from
multiple vendors.
Use key factors to decide which model provides the best vision
for your organization.
0110010
1001010
0010110
Point-to-Point
• Immature data
integration methods.
• Little need for data
consistency.
• Weak executive
support.
• Little need for
enterprise reporting.
• Cheapest in the
short run, but will
cost more to fix
later.
• If not fixed, will
inhibit
organizational
growth.
Hub and Spoke
• Streamlined
propagation of
master data, means
less redundant
transactions across
systems, lowering
bandwidth and
number of required
integrations.
• Cheaper to swap out
a source system
future, but more
expensive to get
started.
• Most mature
architecture
approach.
Federated
• Several data marts
already built.
• Can be a good
stepping stone
model on the path
to Hub and Spoke .
• Expensive
equipment costs to
facilitate
simultaneous
integration and
transaction
processing on
source systems.
• Ideal when only
some real time
integration needed.
Distributed
• Sophisticated data
integration
methods.
• Less heavy
processing load on
source systems, but
still more so than
Hub and Spoke.
• More additional up
front cost for agent
computers.
• Best for heavy real
time integration
needs.
“Identify the most important integration and choose a model that
best suits this need”. - Data Architect, Advisory Firm.
Info-Tech Helps Professionals To:
Sign up for free trial membership to get practical
Solutions for your IT challenges
“Info-Tech helps me to be proactive instead of reactive - a cardinal rule in
stable and leading edge IT environment.”
- ARCS Commercial Mortgage Co., LP