How Does Data-Driven Decision-Making
Affect Firm Productivity and CEO Pay?
Erik Brynjolfsson, MIT Sloan
Lorin Hitt, Wharton, School, University of Pennsylvania
Heekyung Kim, MIT Sloan
September 15, 2010
Preliminary and Incomplete Draft
For consideration at WISE 2010
Comments welcome. Please do not quote.
Abstract
While there is a great deal of anecdotal evidence that firms can boost their performance by
adopting a data driven decision-making (DDD) approach, there is little data or systematic
analysis of these claims themselves. We gather detailed information on the business practices
and information technology investments of 165 large publicly traded firms and find that DDD
can explain a 3-5% increase in their output and productivity, beyond what can be explained by
traditional inputs and IT usage. Furthermore, firms with more DDD tend to have a higher degree
of consistency in business practices across business units and geography and stronger linkages
between business and technology. In addition, we find that DDD is correlated with a significant
increase in CEO pay even after controlling for the average worker’s wage, suggesting that the
data-driven decision-making may further widen the gap between top managers and average
workers. Intriguingly, the quadratic effect of DDD is positive for CEO compensation, but not for
firm productivity, echoing Rosen’s (1982) model of increasing rewards to “superstars”.
1. Introduction
Each revolution in science is typically preceded by a revolution in measurement and data
gathering. The invention of microscope, for example, allowed the exploration of an entire new
world that had been invisible, transforming biology and medicine. Today’s data revolution
promises to transform management in an equally fundamental way. Businesses now can
measure consumer behavior, business operations and myriad other aspects of their environment
with unprecedented precision. For instance, clickstream data provides an unprecedented digital
trace of customer actions and ERP systems generate terabytes of data as byproducts of their
normal activities, while email, RFID, mobile phones, and many other information technologies
are all contributing to a doubling of the quantity of business data roughly every 1.2 years.
Data by itself does not change anything. It must be analyzed and used to change decisions to
have any value. The ability to collect and analyze the enormous amount of data is, therefore,
becoming of critical importance for firms to stay competitive and gain market share. Ironically,
although there is a great deal of anecdotal evidence of firms’ using data to gain a competitive
edge in the business press in popular books, there has been virtually no systematic data analysis
of the productivity effects of data-driven decision making using statistical methods. We seek to
address this gap by examining detailed business practices and information technology investment
of publicly traded large 165 firms in the US. We find that approximately 3-5% increase in the
productivity can be explained by data-driven decision-making (DDD) and that the top executives
in data driven firms, particular CEOs, have reaped disproportionately large rewards in terms of
personal compensation. Our results can help explain not only the relatively higher productivity
growth rates that IT-intensive firms have recently experienced, but also the growing gap between
CEO pay and the compensation of ordinary workers.
As information technology has penetrated every step of business practices, business decisions are
increasingly relying on data and experiments. Consumer-facing companies rich in transaction
data are routinely testing innovations and randomized testing is standard procedure in online
firms such as Amazon, eBay, and Google. For example, Amazon.com continually experiments
with their websites for the best appearance. “We have been implementing changes on <online
site> based on opinion, gut feeling or perceived belief. It was clear that this was no way to run a
successful business…Now we can release modifications to the page based on purely on
statistical data” (Kohavi, Longbotham et al., 2009). Kohavi et al. (2009) illustrates how data can
overrule the traditional approach of relying on intuition and “the highest paid person’s opinion”
(HiPPO) for corporate decision-making. “Greg Linden at Amazon created a prototype to show
personalized recommendations based on items in the shopping cart. Linden notes that while the
prototype looked promising, a marketing senior vice-president was dead set against it, claiming it
will distract people from checking out. Greg was forbidden to work on this any further.
Nonetheless, Greg ran a controlled experiment” to prove that the new feature would bring in a lot
more sales at a statistically significant level and the HiPPO was wrong.
This suggests that information technology, once thought of a major driver in widening income
gap between college and high school graduates by replacing workers of routine and simple
clerical jobs (i.e., Autor, Katz et al., 1998) is now poised to replace HiPPOs. On the contrary,
however, we have been witnessing that a further widening income gap in every sector and every
income percentile of the population in recent years (i.e., Piketty and Saez, 2003). In this study
we report that the data-driven decision-making is correlated with the increase in CEO’s pay even
after controlling for the average worker’s wage, suggesting that DDD is further widening the
income gap between the top manager and other workers. Our interpretation of this finding is that
information technology is eroding the high-level managers’ work as it has replaced the workers
of routine jobs (Autor, Katz et al., 1998), shrunken the ranks of middle managers (Pinsonneault
and Kraemer, 1997), and flattened hierarchies (Rajan and Wulf, 2006). At the same time,
however, it is complementing the CEO’s job. A CEO’s vision for the future and steering her
firm to the future cannot be experimented and replaced by data. Rather, talented CEOs are even
more in demand to make sound decisions based on the ever-increasing amount of information, to
shape her firm to be data-driven and encourage her employees to experiment and collect and
analyze data, to exploit the capability of information technology to gain a competitive advantage.
2. Theory
The starting point for economic theories of the value of information typically begins with the
seminal work of Blackwell (1953). Blackwell’s theorem (Blackwell, 1953) states that decision
makers observe signals correlated to the state of nature prior to their choice of action and hence
“update” the probability distribution before the “optimal” action is chosen. In this framework
the value of information (in terms of expected utility) is always positive. Since the advent of
computers in the 1980s, firms have deployed enormous resources to computerize work and
collect information. Many researchers, however, showed that only organizations with certain
characteristics such as high endowments of human capital and decentralized work practices tend
to reap the benefit of their information technology investment (i.e., Bresnahan, Brynjolfsson et
al., 2002). Our study is related to the stream of literature, uncovering another organizational
characteristic that may be associated with this increase in productivity. Controlling for firms’
human capital and information technology investments, we show that data-driven decisionmaking practices (DDD) significantly affect firm productivity, and are also highly correlated
with the consistency of business practices across business units.
The increase in the amount of available information may not be always valuable when acquiring
and processing the information is too costly. “The real issue is not the lack of data, but our
ability to properly process and distribute the information quickly. If we cannot properly process
and distribute the information quickly, we run the risk of destroying the value of the information
we are striving so hard to attain.” (Collins, 2010). CIOs have expressed concerns that after
investing in data collection, they lack the capability and personnel to analyze these data.1 Our
study explores the question by empirically examining whether there is an optimal level of DDD.
While the answer to the question is inconclusive due to our limited sample size, a striking result
emerged – the reward to CEOs shows increasing returns to investments in DDD. For instance,
we find that firms at two standard deviations above the mean of our DDD scale have CEO
compensation four times larger than the average firm.
CEO compensation might be related to DDD because it affects the ability of organizations to
transfer and process knowledge. In particular, information technology makes some types of
information alienable from workers or mid-level managers (see e.g. Brynjolfsson, 1993) but not
necessarily the most senior managers. This increases the marginal value of decisions made by
CEOs.
1
CIO forum, MIT center for digital business, 2010.
Numerous researchers have noted that some information is not easily transferable because it
“tacit” (i.e., Polanyi, 1958; Rosenberg, 1982) or “specific” (i.e., Jensen and Meckling, 1976) or
“sticky”(i.e., Von Hippel, 1994). Advancement in information technology has, however, brought
in a wave of codification of knowledge. Balconi, 2002) presented an illustrative example of the
changes in the progressive codification of technological knowledge occurred over the last 30
years formerly embodied in skilled workers in the steel industry. Until the end of the 1960s
instruments to measure the temperature and chemical composition of liquid steel were not widely
available so measurements were carried out based on some physical characteristics observable by
sight. For example, “in order to know the temperature of liquid steel, a sample was taken out of
the furnace, poured upon an iron plate and the temperature was deduced by observing the
forming of the spot, its shape and the way it solidified and attached itself to the plate.” The
ability to recognize the temperature by sight was acquired with 5 years of experience or more.
By the 1970s the content and the temperature of liquid steel was routinely conducted by
automated measurement tools but the process was still controlled by workers and line managers.
By the end of 1980s the whole processing cycle was automated. Since the mid-90s, workers are
principally involved in monitoring automated data collection and processing equipment, and
approving process changes recommended by the processing algorithms. Over time, the
knowledge once embodied in experienced workers has been supplanted by machine monitoring
and largely automated decision making. A consequence is that the specialized knowledge only
possessed by the most skilled individuals can now be embedded in computer instruments and
available to distant operators in the operating rooms throughout the entire company. More
broadly, firms now gather and propogate knowledge not only from production workers, but from
their consumers, suppliers, alliance partners, and competitors much faster and more cheaply with
the aid of information technology, make information available for distant decision makers.
Knowledge generation does not stop at passive analysis of data. New and broadly available
software has enabled managers to conduct active experiments with their new business ideas and
base their decisions on scientifically valid data. From banks such as PNC, Toronto-Dominion,
and Wells Fargo to retailers such as CKE Restaurants, Famous Footwear, Food Lion, Sears, and
Subway to online firms such as Amazon, eBay, and Google, firms test many business ideas
through a randomized test before launch, called as “information-based strategy” (Davenport,
2009). This information-based strategy alienates the high-level manager’s tacit knowledge – the
knowledge of what was or was not likely to be a successful business innovation. Just as the
process of controlling steel processes was transformed by improved measurement of temperature,
the innovative process in an online business is now being transformed by the improved
measurement of customer preferences. However, this process automation cannot substitute for
strategic decision making, although it does allow strategic decisions to be moved from managers
at the level of a task or a business unit, to a manager with responsibility for the entire firm.
Consequently, a small difference in the talent of top manager who has an enormous amount of
information to process may make a huge difference to the fortune of the firm.
Our study is also related to the effects of IT on command, control, coordination, and organization
of firms (see Leavitt and Whisler, 1958;Rule and Attewell, 1989;Gurbaxani and Whang,
1991;Malone, Yates et al., 1987;Brynjolfsson and Mendelson, 1993;Brynjolfsson, Malone et al.,
1994;Brynjolfsson and Hitt, 2000 and the literature reviews cited therein). In theory, IT could
shift power either toward the center or away from it, leading to centralization or decentralization
of a firm. If IT can transfer knowledge to central managers, a firm can become more centralized
which would suggest higher CEO relative pay. On the other hand, if IT facilitates lateral
communication of local knowledge between employees, employees can coordinate tasks among
themselves more easily with the need for senior management involvement, suggesting lower
CEO relative pay. Researchers have found evidence of decentralization at task level, but the
increase in the scope of CEO as well (Rajan and Wulf, 2006). Our finding is consistent with the
increase in the scope of CEO, but not as a substitute for decentralized work practices. Data
driven firms generally invest in both decentralized work practices, but with greater overall
centralized coordination of these activities, making them consistent across the firm.
Our study is also related to the literature on IT and wage inequality (i.e., Autor, Katz et al., 1998).
Our view on the role of IT in increasing CEO pay is close to those of Garicano (Garicano and
Rossi-Hansberg, 2006;Garicano, 2000). Their model suggests a hierarchy of knowledge. Some
knowledge resides in lower level employees and is used to solve routine problems, while other
knowledge resides in higher-level employees that is primary directed at solving non-routine
problems. As the cost of communication among agents decreases, the complicated problems that
lower-ranked employees cannot solve can be easily passed to their superiors. Once solved, the
solution can be disseminated throughout the firm and become part of routine work practice. This
can lead to the dependency of the problem-solving on a few “superstars” (Rosen, 1981) and thus
a higher wage for the superstars. Their explanation is consistent with our view that the
availability of data allows a distant few problem solvers to provide a solution to non-trivial
problems and increase their marginal productivity disproportionately. Therefore, data driven
firms become dependent on a small number of superstars which should be associated with
greater pay. CEOs may or may not be one of these problem solvers, but they are nonetheless
responsible for coordinating, managing, and rewarding their activities, which also increases the
marginal value of their effort.
Finally, our study is related to the literature on CEO compensation, especially studies that
document or explain the observed increase in CEO pay. Following Gabaix and Landier’s (2008)
summary of the literature on the topic, we recapitulate the literature in four views. The agency
view (Jensen and Murphy, 1990;Dow and Raposo, 2005;Holmstrom and Kaplan, 2001) suggests
that increased expected compensation is necessary to provide incentives in an increasingly
volatile business environment and to reward difficult to measure activities like innovation. The
market failure view (Yermack, 1997;Bertrand and Mullainathan, 2001;Bebchuk and Fried,
2005;Bebchuk, Fried et al., 2002;Hall and Murphy, 2003) suggests that the rise in CEO pay is a
consequence of failure of corporate governance mechanisms. A third possibility is that the CEO
job has changed (Frydman, 2005) to become more general which has increased their outside
options and put upward pressure on pay. Finally, the market equilibrium view (Gabaix and
Landier, 2008;Tervio, 2008) suggests that CEO pay increases simply reflect an increase in firms
size. In equilibrium, the most talented CEO is correctly matched to the largest firm and paid the
highest because the marginal value of their decisions is greatest there.
Our paper is most closely linked to the market equilibrium view; the increased marginal
productivity of CEOs resulted from the increase of firm size is a major determinant of the recent
rise of CEO pay. We articulate that what is relevant to the marginal productivity of CEO is an
effective size, of which data-driven firm is greater than others of the same nominal size. The
more data-driven a firm is, the easier the flow and access of knowledge and thus the larger the
effective size of the firm that distant managers can control. As a result, the marginal productivity
of CEO increases and this drives an increase in CEO pay.
Our argument is related to the study by Baker and Hall (Baker and Hall, 2004), who attributed to
a specific mechanism of increased marginal productivity. In particular, they argue that when
CEOs perform tasks that affect the entire firm, optimal compensation should be connected to the
proportional change in value rather than the absolute change in value. As firms become larger,
this implies larger CEO pay. Since modern investments in DDD tend to lead to firm wide
changes or standardization of processes across the firm, the Baker and Hall argument would
suggest that DDD leads to greater CEO compensation.
One challenge to investigating these theories is that different firms, in different vary in the
marginal productivity and therefore wages of all workers (Katz and Summers, 1989;Krueger and
Summers, 1987;Krueger and Summers, 1988;Gibbons and Katz, 1992). For example, the
median workers’ wage in the industry of computer systems and related services, $65,000, was
over 3-times higher than that in the industry of food services, $18,000, in 2005. This variance is
at least partly due to the fact that DDD can raise the productivity of all workers. However, our
theory predicts not just increases in wages, but disproportionate increases in the wages of top
executives. Therefore, to isolate this effect, we will control for wages of other workers in our
analysis.
Data
Our business practice and information system measures are estimated from a survey
administered to senior human resource (HR) managers and chief information officers (CIO) from
large publicly traded firms in 2008. We received matched responses (both HR and CIO) from
127 firms, HR only responses from 122 firms, and only CIOs-only responses from 81 firms. The
survey asks over 80 questions on business practices as well as organization and usage of
information systems. The questions extend a previous wave of surveys on IT usage and
workplace organization administered in 1995-1996 and 2001 (Hitt and Brynjolfsson, 1997
Tambe, Hitt et al., ), but adds additional questions on organizational structure, the usage of
information for decision making, and the consistency of their organizational practices. To test
our hypothesis, we used the survey response to construct measures of firms’ organizational
practices. We combine these measures with publicly financial data and CEO compensation.
This yielded 164 firms with complete data for an analysis of firm productivity, and 165 firms
during the same period with the variables needed to analyze CEO compensation. Our sample
spans manufacturing, retail/wholesale trade, information, and finance/insurance industries over
the period from 2005 to 2009.
Business Practices
We constructed our key independent variable, data-driven decision-making (DDD), from three
questions of the survey: 1) the usage of data for the creation of a new product or service, 2) the
usage of data for business decision-making in the entire company, and 3) the existence of data
for decision-making in the entire company. Two other measures from the survey were also
constructed to indicate the general employee’s human capital; 1) the importance of typical
employee’s education and 2) the average of % of employees using e-mail and % of employees
using PCs, workstation, or terminal.
capture in 5-point Likert scales.
All measures except the ones based on percentages were
We created DDD by first standardizing each factor with mean of zero and standard deviation of 1
and then standardizing the sum of each factor for each composite measure:
DDD = STD(STD(use of data for creation of a new product/service) + STD(use of data for
business decision in the entire company) + STD(existence of data for such a decision))
To examine the factors which may be correlated with DDD, we constructed a number of other
organizational measures listed in Table 1. “Consistency” refers to the consistency of business
practices in the entire company. The consistency measure was constructed from a composite of
responses to six survey questions on consistency of business practices across operating units,
within business units, across functions, and across geographies (4 questions); the effectiveness
of IT for supporting consistent practices; and consistency of prioritization of projects (Table 1).
“BT-link” captures the integration of business strategy with information technology and is
derived from size survey questions focusing on the interaction of employees from both groups,
understanding of each other, and the importance of technology in strategy. “Dissemination” was
based on two questions focusing on embedding processes in technology and the ability of the
firm to disseminate business practices across the firm.. “IT-governance” is a measure on how IT
project is governed in terms of prioritization, involvement with business, and managing demand
and benefit.
“Find” is the effectiveness of employees’ finding knowledge or a colleague with a
particular expertise or broadcasting their expertise. “Exploration” is the capability of innovating
and creating a new business capability, product or service, a composite from 7 survey questions.
“Exploitation” is the ability of improving their existing capability, a composite from 15 survey
questions.
The number of instances of IT systems on ERP/SCM, CRM, HR, and Financial
Systems and the level of centralization of IT resources on application development and
maintenance was used to construct a composite designated “Centralized IT system”.
We
capture employee human capital a question about the importance of typical employee’s
education and the percentage of employees using emails or pc were included.
Financial data
Financial measures were derived from Compustat. Measures of physical assets, employees, sales
and operating income were taken directly from the Industrial annual file from 2005 to 2009.
Materials were estimated by subtracting operating income before tax and labor expense from
sales. In the case that labor expense was not available, the industry-average was used with the
industry average based on average labor expense for all firms that report labor expense and
employees in the same industry at the most detailed industry breakdown available.
CEO compensation
We used Compustat database for Executive pay for the period from 2005 to 2009 which provides
data on compensation of as many as 13 senior executives from each company. The Executives
database is compiled from proxy statements filed by the companies in compliance with
Securities and Exchange Commission (SEC).
The two measures of CEO compensation were used; one is the variable, tdc1, from Compustat
Executives data set and the other is the sum of salary and bonus. The tdc1 includes salary, bonus,
other annual, restricted stock grants, LITP payouts, all other, and value of option grants valued at
the day of grants.
For a variable for firm’s nominal size, we used three variables; total employee number, sales, or
market capitalization. Market capitalization was estimated in the same way as Gabaix and
Landier (2008) as the market value of equity at the end of the fiscal year, plus debt less deferred
taxes.
IT-employee data
The survey included the questions about IT budgets, outsourcing, change of IT budgets from
2008 to 2009, and full time IT employment. The number of full-time IT employees for the year
2008 was asked in the survey, but for the year 2009 it was estimated from the questions on IT
budget. Using the change of IT budget from 2008 to 2009, the percentage of outsourcing, and IT
FTE for 2008, we were able to estimate the IT FTE for the year 2009. The year from 2005 and
2006, we used data collected in a previous study (Tambe and Hitt, ). For the year 2007, a
moving average from 2005, 2006, 2008 and 2009 was used. The number of non-IT employees is
equal to the number of employees reported on Compustat less our computed IT employment
measure.
3. Methods
Productivity Tests
We use the Cobb-Douglas specification, the most commonly used model in information
technology and productivity literature (e.g., Brynjolfsson and Hitt, 1993, 1996; Dewan and Min,
1997). Our primary regression model can be written as the following:
ln (‫)ݏ݈݁ܽݏ‬௜௧ = ߚ଴ + ߚଵ ln(݇)୧୲ + ߚଶ ln(‫ ܶܫ‬− ‫)݁݁ݕ݋݈݌݉ܧ‬୧୲ + ߚଷ ln(ܰ‫ ݊݋‬− ‫)݁݁ݕ݋݈݌݉ܧ ܶܫ‬୧୲ + ߚସ (DDD)୧୲
+ ߚହ (DDDxDDD)୧୲
Where k is physical capital, IT-Employee is the number of IT employees, Non-IT Employee is
the number of Non-IT employees, DDD is our data-driven decision-making variable, and
DDDxDDD is the square term of DDD. The last term, DDDxDDD, was included to examine the
potential for increasing marginal productivity effects of DDD. In some specifications, we
included variables indicating the firm’s human capital, such as importance of typical employee’s
education, to rule out some alternative explanations for our results.
Although our data on IT and other production inputs and outputs are longitudinal, our main
independent variable, DDD, is based on a single survey conducted in 2008. We constructed a 5year panel (2005-2009) by making the assumption that DDD was the same for the years from
2005 to 2009. Similar assumptions have been used in previous studies (Bresnahan, Brynjolfsson
et al., 2002), and many researchers have reported that organizational practices change very
slowly, especially for large firms (i.e., Milgrom and Roberts, 1990 Hannan and Freeman, 1984).
For a robustness check, we categorized the firms into 5 groups based on their response to one of
the survey questions about the change in the consistency of their business practices. The
productivity tests were conducted controlling for the group, examining the effect of DDD within
the same group which is likely to have a similar degree of change in the consistency of business
practices. The rationale for this test was based on the observation that the consistency of
business practices was highly correlated with DDD in our sample firms. Therefore, the change
in the consistency of business practices is likely to be correlated with the change in DDD.
Estimating the effect of DDD within the same group mitigates the bias that our assumption may
cause.
CEO Compensation Tests
A robust and high correlation between the nominal size of a firm and its CEO compensation has
been supported by theoretical and empirical models in numerous studies (i.e., Baker and Hall,
2004; Gabaix and Landier, 2008) . We extend the model in prior literature by including our
organizational variable, DDD and the square term of DDD to capture nonlinear effects. Our
empirical model specification is therefore:
ln(ܿ݁‫݋‬௜௧ ) = ߚଵ + ߚଶ ln(݂݅‫)݁ݖ݅ݏ ݉ݎ‬୧୲ + ߚଷ (DDD) + ߚସ (DDDxDDD)
where ceoit is CEO pay of firm i and year t. We used output as the firm size in most analysis but
results are similar when other measures of size are used (employees, assets and market
capitalization). In some specifications, we include other variables to control for firm’s human
capital to address other potential omitted variables explaining the result.
4. Results and Discussion
Productivity Tests
The key result for the productivity was shown in Table 2; the first column is the productivity test
only with capital, employee, and materials. The coefficients associated with IT employee and
non-IT employee are consistent with previous studies (i.e., Tambe, Hitt et al., ). The second
column shows the model including our key independent variable, DDD. These coefficients
suggest that firms with a one standard deviation higher in on our measure of data-driven decision
making (DDD) have a 5% greater productivity than the mean firm. In the third column we add
the square of DDD. We find it to be negative but not significant, suggesting the possibility of
declining marginal benefits. Our results are largely unchanged when we include a variety of
human capital controls.
In table 3, we examine the relationship between our DDD measure and other organizational
practices. One surprising result was that individual IT use (employees using PCs or e-mail) was
not correlated with DDD at all. It seems that the widespread usage of computers and e-mail do
not separate high IT-usage firms from low-IT usage firms by this measure. The consistency of
business practices across business units was positively correlated with the output in some
industries at a statistically significant level (results not shown). The complementarity between
DDD and the consistency measure was not, however, statistically significant (results not shown).
We assumed that DDD was the same for the tested period (2005-2009). To check the robustness
of this assumption, we categorized our sample firms in 5 groups based on their answers on the
change of the consistency of business practices over the last 3 years because the change of the
consistency of business practices is likely to be correlated with the change of DDD. The DDD
coefficient was qualitatively the same for both models; one controlling for the group, the other
not controlling for the group (Table 4).
Overall, our results are consistent with a positive relationship between DDD and productivity
that is robust to a variety of controls and checks.
CEO Compensation and DDD
Table 5 reports our analysis of the relationship between CEO pay and firm size. The first 3
columns use the total compensation as the dependent variable including options and the next 3
models only the sum of salary and bonus. This result is robust to using physical capital as a size
control, which leads to small change reduction in the DDD coefficient. When market
capitalization was used as firm size variable, the coefficient of DDD is reduced by half. By
construction, the market capitalization value includes the stock price and reflects the expected
value of future sales. Firms with high DDD are likely to have a potential to grow faster and their
stock price may reflect the growth potential . Indeed, DDD is correlated with the market
capitalization at a statistically significant level. When the growth potential of a firm is captured
through its market capitalization, DDD coefficient may become smaller.
Following the norm in the literature of CEO compensation where sales are most commonly used
as size variable, we selected sales as size variable in the next model specifications (Table 6). In
the models in Table 6 the nonlinearity of CEO pay on DDD was first examined. Although the
coefficient of the square term of DDD in the first model in Table 4 is not statistically significant
at 90% confidence level, it becomes statistically significant when more controls are included. It
is a striking result that CEO pay increases not only with DDD but the square term of DDD. This
result echoes Rosen’s result (1981) that top talent’s earing increases with the quadratic term of
her talent as both price and the size of market increases with her talent. Further study will
formulate a model to link our result to Rosen’s economics of superstars (1981). DDD coefficient
was still statistically significant after controllling for firms’ human capital measured in the
importance of education and percentage of employees.
Conclusion
Overall, a variety of literature suggests a potential connection between data driven decision
making, productivity and executive compensation. Using survey data, our initial analysis
suggests that DDD is associated with productivity. Moreover, we find a positive and increasing
effect of DDD on executive compensation, consistent with the idea that DDD increases the value
of central decision makers and “superstar” individual decision makers. More analysis will be
conducted to strengthen this result by considering more complex models that control for firm
heterogeneity and potential reverse causality.
Table 1. Construction of Measure of Organizational Practices
Measure 1: Data-Driven Decision-making (DDD)
Typical basis for the decision on the creation of a new
product or service
(HR survey q13a)
We depend on data to support our decision making
(the work practices and environment of the entire
company)
(HR survey q16j)
We have the data we need to make decisions
(HR survey q16p)
Measure 2: Consistency
Looking across your entire company, please rate the
level of consistency in behaviors and business
processes across operating units
(HR survey q1)
Regarding the first core activity of your company, the
consistency within business unit
(HR survey q9a)
Regarding the first core activity of your company, the
consistency across functions (e.g., sales, finance, etc)
(HR survey 9b)
Regarding the first core activity of your company, the
consistency across geographies
(HR survey q9c)
Effectiveness of IT in building consistent systems and
processes for each operating unit
(IT survey q13b)
Consistency of project prioritization and approval
processes
(IT survey q15a)
Measure 3: Business-Technology Link (BT Link)
Linkage of Business and IT strategy (IT survey q10)
Business Domain Knowledge of Existing IT Staff (IT
survey q14b)
Business Unit Support for IT projects (IT survey
q14e)
Employees from both business and IT working as
peers (HR survey q16h)
Business executives understand IT (HR survey q16r)
IT executives understand the business (HR survey
q16s)
Measure 4: Dissemination
Embedding many processes in technology
(HR survey q16d)
Strong ability to disseminate changes to business
processes (HR survey q16m)
Measure 5: IT Governance
Consistency of IT project prioritization and approval
process (IT survey 15a)
Involvement of Business with IT project
Range
N
Mean
Std.
Dev.
1-5
(Experience/e
xpertise =1
Data = 5)
1-5
160
3.01
1.10
163
3.83
0.81
1-5
161
3.42
0.86
1-5
162
3.02
0.75
1-5
154
3.73
0.97
1-5
137
3.32
1.02
1-5
150
3.46
1.00
1-5
101
3.50
0.89
1-3
102
2.55
0.68
1-4
1-5
118
120
2.70
3.73
0.91
1.14
1-5
120
3.88
1.09
1-5
180
3.57
0.98
1-5
1-5
178
179
3.10
3.64
1.02
0.93
1-5
177
3.25
.094
1-5
175
2.99
0.93
1-3
120
2.53
0.70
1-3
119
2.33
0.61
Chronbach’
Alpha
0.59
0.75
0.57
(IT survey 15b)
Managing demand (IT survey 15c)
Tracking and managing the capture of IT project
benefits (IT survey 15e)
Measure 6: Find
Effectiveness of 1) broadcasting employees’ work; 2)
describing experience/expertise; 3) collaboration; 4)
finding a colleague; 5) finding information/knowledge
(HR survey 17a/b/c/d)
Measure 7: Exploration (EXPR)
IT facilitates to create new products (IT survey 11a)
IT facilitates to enter new markets (IT survey 11b)
IT supports growth ambitions by delivering services
or products that set us apart from competitors (IT
survey 12c/HR survey 15c)
IT plays a leading role in transforming our business
(IT survey 12d/HR survey 15d)
IT partnering with BIZ to develop new business
capabilities supported by technology (IT survey
13f/HR survey 14e)
Strong ability to make substantial/disruptive changes
to business processes (HR survey 16l)
Measure 8: Exploit (EXPT)
Deliver year-over-year productivity (IT survey 11d)
Deliver basic technology services to the business at
the lowest cost (IT survey 12a/HR survey 15a)
IT improves the efficiency and cost of business
operations (IT survey 12b/HR survey 15b)
Providing basic services cost-effectively (IT survey
13a/HR survey 14a)
Delivering new projects or enhancements on time and
within budget (IT survey 13c/HR survey 14b)
Reactively responding to business needs to improve
existing systems or functions (IT survey 13d/HR
survey 14c)
Proactively engaging with business leaders to refine
existing processes and systems (IT survey 13e/HR
survey 14d)
Stress operational excellence over innovation (HR
survey 16f)
Strong ability to make incremental changes or
improvements to business processes (HR survey 16k)
Measure 9: Centralized IT system
How many instances of the following systems are you
running globally? (Number of Instances)
ERP/SCM (IT survey q8a)
CRM (IT survey q8b)
1(no
real
demand
management)
–
3
(chargeback
based
on
usage)
1-3
120
2.55
0.52
120
1.64
0.68
1-5
178;
178;
176;
177
2.86;
2.81;
3.31;
2.89
0.91;
0.97;
1.01;
1.08
1-5
118
118
119;
172
3.76
3.66
2.49;
2.56
1.17
1.06
1.08;
1.00
119;
172
120;
177
2.88;
3.00
3.25;
3.01
1.16;
1.13
0.95;
1.06
1-5
174
2.87
1.05
1-4
1-4
117
119;
177
120;
171
119;
179
119;
178
120;
177
4.07
2.75;
2.49
1.88;
1.87
4.24;
3.91
3.66;
3.17
3.65;
3.15
1.06
1.17;
1.25
0.75;
0.77
0.68;
0.80
0.87;
0.97
0.93;
0.86
1-5
120;
176
3.4;
3.03
0.86;
0.99
1-5
177
3.28
1.11
1-5
177
3.56
0.91
1 (1); 2 (2-3);
3(4+)
1 (1); 2 (2-3);
3(4+)
100
1.47
0.88
100
1.26
0.81
1-4
1-4
1-5
1-4
1-5
1-5
1-5
HR (IT survey q8c)
Financial System (IT survey q8d)
Level of centralization of IT resources on application
development and maintenance
Measure 10: Importance of Typical Employee’s
Education
The importance of educational background in making
hiring decisions for the “typical” job (HR survey q4)
Measure 11: % of employees using IT
% of employees using PC/terminals/workstations (HR
survey q7a)
% of employees using e-mails (HR survey q7b)
Measure 11: IT investment
Ln(IT employee/total employee)
Year = 2005
Year = 2006
Year = 2007
Year = 2008
Year = 2009
1 (1); 2 (2-3);
3(4+)
1 (1); 2 (2-3);
3(4+)
1-3
100
1.33
0.68
100
1.52
0.77
120
1.28
0.54
1-5
179
3.39
0.98
%
179
77.4
26.4
%
179
73.9
28.6
158;
158;
165;
116;
107
-4.10;
-4.04;
-3.78;
-3.59;
-2.87
0.81;
0.81;
0.91;
1.09;
1.10
Table 2. Correlation between Output and Data-Driven Decision-Making (DDD). The industry control is at 2digit NAICS level for manufacturing industries and 1-digit NAICS level for other industries. The years are
from 2005 to 2009. The number of firms was 165. Standard errors were clustered around firms.
Dependent Variable = Ln(Sales)
Ln(Material)
Ln(Capital)
Ln(IT Employee)
Ln(Non-IT Employee)
Data-driven decision-making (DDD)
DDD x DDD
Importance of Typical Employee's Education
Ln(% of Employees using PC/e-mail)
Industry and Year Control
Constant
Observations
R-squared
Robust standard errors in parentheses
*** p<0.01, ** p<0.05, * p<0.1
0.507*** 0.497*** 0.499*** 0.501***
(0.0449) (0.0446) (0.0446) (0.0454)
0.120*** 0.120*** 0.120*** 0.121***
(0.0280) (0.0273) (0.0270) (0.0267)
0.0823*** 0.0814*** 0.0809*** 0.0643***
(0.0225) (0.0231) (0.0228) (0.0233)
0.199*** 0.206*** 0.204*** 0.224***
(0.0316) (0.0322) (0.0325) (0.0328)
0.0464** 0.0445** 0.0434**
(0.0205) (0.0204) (0.0204)
-0.0123
(0.0159)
0.0374
(0.0247)
0.0421
(0.0319)
Yes
Yes
Yes
Yes
1.079*** 1.088*** 1.113*** 0.840***
(0.1910) (0.1840) (0.1860) (0.2530)
628
628
628
628
0.918
0.92
0.92
0.922
Table 3. Correlation between Data and other survey variables
BT
DISS
IT_gov
Find
EXPR
EXPT
NI
EDU
DDD
CON
Consistency
(CON)
0.53***
1
BIZ-IT Link (BT)
0.39***
0.35***
1
Dissemination
(DISS)
0.37***
0.40***
0.49***
1
Exploit (EXPT)
0.30***
0.39***
0.64***
0.51***
1
IT governance
(IT_gov)
0.28**
0.36***
0.30***
0.17*
0.27***
1
Find
0.27***
0.17*
0.28***
0.41***
0.23***
0.079
1
Centralized IT
0.21***
0.32***
0.17***
0.23***
0.11
0.22**
0.22**
1
Exploration
(EXPR)
0.17**
0.24***
0.59***
0.41***
0.62***
0.16*
0.26***
0.02
1
Importance of
Typical
Employee's
Education (EDU)
0.21***
0.03
0.02
0.12
-0.007
0.054
0.20**
-0.09
0.165*
1
Ln(% of
employees using
e-mail or PC) (PC)
-0.07
-0.19**
-0.02
-0.06
-0.09
-0.03
-0.08
0.005
0.029
0.27**
*
Ln(ITEmployee/Total
Employee)
0.19**
-0.004
0.11
0.05
-0.03
0.07
-0.09
-0.07
0.11
0.28**
*
Partial correlation, controlling industry at 2-digit NAICS. ***p<0.001, **p<0.05, *p<0.1. Test is against the null hypothesis
that the correlation is zero.
Table 4. Robust check on the assumption that data-driven decision-making (DDD) is quasi-fixed over the last
5 years. Firms were categorized into 5 groups based on their response to a question on the change in the
consistency of business practices. The question was “To what extent do the following statements describe the
work practices and environment of your entire company; our business processes have become more
consistent over the past 3 years.” The choices were from 1 as “Describes not all” to 5 to “Completely
describes”. The first column is a model controlling for the group and the second not controlling for the
group.
Dependent Variable: Ln(Sales)
Ln(Material)
Ln(Capital)
Ln(IT Employee)
Ln(Non-IT Employee)
Data-driven decision-making (DDD)
Importance of Typical Employee's Education
Ln(% of Employees using PC/e-mail)
Constant
Group Control
Industry and Year Control
Observations
R-squared
Robust standard errors in parentheses
*** p<0.01, ** p<0.05, * p<0.1
0.491***
(0.0460)
0.131***
(0.0260)
0.0603***
(0.0223)
0.228***
(0.0313)
0.0411**
(0.0198)
0.0365
(0.0243)
0.0541**
(0.0256)
0.683***
(0.2250)
0.491***
(0.0460)
0.130***
(0.0258)
0.0612***
(0.0230)
0.229***
(0.0325)
0.0371*
(0.0199)
0.0407*
(0.0244)
0.0627**
(0.0258)
0.655***
(0.2300)
Yes
Yes
617
0.926
No
Yes
617
0.924
Table 5: The correlation of Ln(CEO pay) with Data-driven decision-making (DDD). CEO pay in the first
three models was the total compensation including salary, bonus and value of option granted at the date of
grant as listed as tdc1 in the Compustat database. The standard errors were clustered around each firm.
CEO pay in the last three models is the sum of only salary and bonus. The period is from 2005 to 2009. The
survey was conducted in 2008 and the same value for “Data” constructed from the survey was applied to all 5
years.
Dependent Variable: Ln(CEO Pay)
Data-driven decision-making (DDD)
Ln(Employee)
Ln(Sales)
Ln(Market Capitalization)
Constant
Observations
R-squared
Robust standard errors in parentheses
*** p<0.01, ** p<0.05, * p<0.1
Ln(Salary+Bonus+Option)
Ln(Salary+Bonus)
0.157*** 0.118*** 0.0671* 0.0917*** 0.0710** 0.0603*
(0.0484) (0.0438) (0.0350) (0.0328) (0.0277) (0.0307)
0.256***
0.122***
(0.0447)
(0.0254)
0.454***
0.232***
(0.0419)
(0.0306)
0.477***
0.223***
(0.0545)
(0.0295)
7.453*** 4.645*** 4.026*** 6.989*** 5.540*** 5.383***
(0.3200) (0.3760) (0.4800) (0.2550) (0.2950) (0.3030)
683
0.247
683
0.371
613
0.474
683
0.148
683
0.2
613
0.217
Table 6: CEO pay and DDD with control for firms’ human capital. The average worker’s wage is at firm
level when data available and industry-average if not available.
Dependent Variable: Ln(CEO Pay)
Data-driven decision-making (DDD)
DDD x DDD
Ln(Sales)
0.121***
(0.0455)
0.0533
(0.0352)
0.454***
(0.0445)
0.116**
(0.0452)
0.0590*
(0.0330)
0.465***
(0.0464)
0.128***
(0.0490)
-0.0242
(0.1110)
4.189***
(0.4030)
4.189***
(0.6940)
0.115**
(0.0450)
0.0608*
(0.0330)
0.467***
(0.0472)
0.122**
(0.0486)
-0.0274
(0.1110)
0.0527
(0.0503)
3.603***
(0.8630)
683
0.337
663
0.357
663
0.358
Importance of Typical Employee's Education
Ln(% of Employees using PC/e-mail)
Ln(Average Worker's Wage)
Constant
Observations
R-squared
Robust standard errors in parentheses
*** p<0.01, ** p<0.05, * p<0.1
5. References
Autor, D., L. Katz, et al. (1998). "Computing Inequality: Have Computers Changed the Labor
Market?*." Quarterly Journal of Economics 113(4): 1169-1213.
Autor, D. H., L. F. Katz, et al. (1998). "Computing Inequality: Have Computers Changed the
Labor Market?" Quarterly Journal of Economics 113(4): 1169-1213.
Baker, G. and B. Hall (2004). "CEO incentives and firm size." Journal of Labor Economics
22(4): 767-798.
Baker, G. P. and B. J. Hall (2004). "CEO Incentives and Firm Size." Journal of Labor Economics
22(4): 767-798.
Balconi, M. (2002). "Tacitness, codification of technological knowledge and the organisation of
industry* 1." Research Policy 31(3): 357-379.
Bebchuk, L. and J. Fried (2005). "Pay Without Performance: Overview of the Issues." Journal of
Applied Corporate Finance 17(4): 8-23.
Bebchuk, L., J. Fried, et al. (2002). "Managerial power and rent extraction in the design of
executive compensation." University of Chicago Law Review 69: 751-846.
Bertrand, M. and S. Mullainathan (2001). "Are CEOS Rewarded for Luck? The Ones without
Principals Are*." Quarterly Journal of Economics 116(3): 901-932.
Blackwell, D. (1953). "Equivalent comparison of experiments." Ann. Math. Statist. 24: 265-272.
Bresnahan, T., E. Brynjolfsson, et al. (2002). "Information Technology, Workplace Organization
and the Demand for Skilled Labor: Firm-level Evidence." Quarterly Journal of Economics
111(1): 339-376.
Brynjolfsson, E. and L. Hitt (2000). "Beyond computation: Information technology,
organizational transformation and business performance." The Journal of Economic
Perspectives: 23-48.
Brynjolfsson, E., T. Malone, et al. (1994). "Does information technology lead to smaller firms?"
Management Science: 1628-1644.
Brynjolfsson, E. and H. Mendelson (1993). "Information systems and the organization of modern
enterprise." Journal of Organizational Computing 3(3): 245-255.
Collins, D. (2010). "Destroying the value of information." Journal of Management Excellence:
The Value of Information(9).
Davenport, T. (2009). "How to design smart business experiments." Harvard Business Review.
Dow, J. and C. Raposo (2005). "CEO compensation, change, and corporate strategy." Journal of
Finance: 2701-2727.
Frydman, C. (2005). Rising through the ranks: The evolution of the market for corporate
executives, 1936-2003. Working Paper, Harvard University.
Gabaix, X. and A. Landier (2008). "Why has CEO pay increased so much?" Quarterly Journal of
Economics: 49-99.
Garicano, L. (2000). "Hierarchies and the Organization of Knowledge in Production." Journal of
Political Economy 108(5): 874-904.
Garicano, L. and E. Rossi-Hansberg (2006). "Organization and Inequality in a Knowledge
Economy*." The Quarterly Journal of Economics 121(4): 1383-1435.
Gibbons, R. and L. Katz (1992). "Does Unmeasured Ability Explain Inter-Industry Wage
Differentials?" Review of Economic Studies 59(July): 515-35.
Gurbaxani, V. and S. Whang (1991). "The impact of information systems on organizations and
markets." Communications of the ACM 34(1): 59-73.
Hall, B. J. and K. J. Murphy (2003). "The Trouble with Stock Options." Journal of Economic
Perspectives 17(3): 49-70.
Hannan, M. and J. Freeman (1984). "Structural inertia and organizational change." American
sociological review 49(2): 149-164.
Hitt, L. and E. Brynjolfsson (1997). "Information technology and internal firm organization: an
exploratory analysis." Journal of Management Information Systems 14(2): 101.
Holmstrom, B. and S. Kaplan (2001). "Corporate Governance and Merger Activity in the United
States: Making Sense of the 1980s and 1990s (Digest Summary)." Journal of Economic
Perspectives 15(2): 121-144.
Jensen, M. and K. Murphy (1990). "Performance pay and top-management incentives." Journal
of Political Economy 98(2): 225.
Jensen, M. C. and W. H. Meckling (1976). "Theory of the Firm: Managerial Behavior, Agency
Costs and Ownership Structure." Journal of Financial Economics 3(4): 305-60.
Katz, L. and L. Summers (1989). "Industry Rents: Evidence and Implications." Brookings Papers
on Economic Activity: Microeconomics: 209-75.
Kohavi, R., R. Longbotham, et al. (2009). "Controlled experiments on the web: survey and
practical guide." Data mining and knowledge discovery 18(1): 140-181.
Krueger, A. and L. Summers (1987). Reflections on the Inter-Industry Wage Structure.
Unemployment and the Structure of Labor Markets. K. Lang and J. Leonard. Oxford, Basil
Blackwell.
Krueger, A. and L. Summers (1988). "Efficiency Wages and the Inter-Industry Wage Structure."
Econometrica 56(March): 259-93.
Leavitt, H. and T. Whisler (1958). "Management in the 1980's." Harvard Business Review
November-December: 41-48.
Malone, T., J. Yates, et al. (1987). "Electronic markets and electronic hierarchies."
Communications of the ACM 30(6): 455.
Milgrom, P. and J. Roberts (1990). "The economics of modern manufacturing: technology,
strategy, and organization." The American Economic Review: 511-528.
Piketty, T. and E. Saez (2003). "Income Inequality in The United States, 1913-1998*." Quarterly
Journal of Economics 118(1): 1-39.
Pinsonneault, A. and K. Kraemer (1997). "Middle management downsizing: an empirical
investigation of the impact of information technology." Management Science 43(5): 659-679.
Polanyi, M. (1958). Personal knowledge, Routledge London.
Rajan, R. and J. Wulf (2006). "The flattening firm: Evidence from panel data on the changing
nature of corporate hierarchies." The Review of Economics and Statistics 88(4): 759-773.
Rosen, S. (1981). "The Economics of Superstars." American Economic Review 71(5): 845-857.
Rosenberg, N. (1982). Inside the black box: technology and economics, Cambridge Univ Pr.
Rule, J. and P. Attewell (1989). "What do computers do?" Social Problems: 225-241.
Tambe, P. and L. Hitt (2008). "Job Hopping, Knowledge Spillovers, and Regional Returns to
Information Technology Investments." Working paper.
Tambe, P., L. Hitt, et al. (2009). "The Extroverted Firm: How External Information Practices
Affect Productivity." working paper.
Tervio, M. (2008). "The difference that CEOs make: an assignment model approach." American
Economic Review 98(3): 642-668.
Von Hippel, E. (1994). "" Sticky information" and the locus of problem solving: Implications for
innovation." Management Science 40(4): 429-439.
Yermack, D. (1997). "Good timing: CEO stock option awards and company news
announcements." Journal of Finance: 449-476.