Lifetime Analysis of Automotive Batteries using
Random Forests and Cox Regression
JOHANNA ROSENVINGE
Degree project in
Computer Science
Second cycle
Stockholm, Sweden 2013
Lifetime Analysis of Automotive Batteries using
Random Forests and Cox Regression
JOHANNA ROSENVINGE
February 2013
Master’s Thesis in Computer Science at CSC
Supervisor: Örjan Ekeberg
Examiner: Anders Lansner
Project provider: Scania CV AB
TRITA xxx yyyy-nn
Abstract
Worn out batteries is a frequent cause of unplanned immobilization of
trucks, causing disrupted operations for haulage contractors. To avoid
unplanned maintenance, it is desirable to accurately estimate the battery lifetime to perform preventive replacements before the components
fail. This master’s thesis has investigated how technical features and
operational conditions influence the lifetime of truck batteries and how
the risk of failure can be modeled.
A support vector machine classifier has been used to examine how
well the available data discriminate the vehicles with battery failure
from those without. The performance of the classifier, according to the
area under the receiver operating characteristic curve, was 70.54% and
76.95% for haulage and distribution vehicles respectively. Maximum
likelihood estimation was applied to censored failure time data showing that, if failures that occurred within 100 days after delivery were
omitted, both failure data sets were normal distribution on a 95% significance level.
To investigate how different features influence the lifetime, random forests
and Cox regression were applied on two different models, one intended
to be applied for new vehicles and one for vehicles that have been operating for a time, hence having an age covariate. The results from
the first model were satisfying, having significant Cox coefficients and
low Brier scores for both random forests and Cox. The second model
however did not give credible results, having non-significant regression
coefficients.
Sammanfattning
Livslängdsanalys av bilbatterier med random forests
och Coxregression
Utslitna batterier är en vanligt förekommande orsak till oplanerade
stopp för lastbilar, vilket leder till störningar i åkeriernas planering. För
att undvika oplanerat underhåll är det önskvärt att kunna uppskatta batteriets livslängd, för att utifrån den uppskattningen kunna utföra
förebyggande byten innan komponenten går sönder. Det här examensarbetet har undersökt hur olika tekniska egenskaper och driftsförhållanden
påverkar livslängden på lastbilsbatterier och hur risken över tid för att
batteriet går sönder kan modelleras.
En supportvektormaskin har använts för att studera hur väl tillgänglig
data utskiljer de fordon med batteriproblem från dem utan. Klassificerarens prestanda, enligt arean under receiver operating characteristickurvan, var 70.53% för fjärrtransportsfordon och 76.95% för distributionsfordon. Maximum likelihood-estimering tillämpades på censurerad
data över tidpunkter för inträffade fel. Denna analys visade att datamängderna över båda fordonstyperna var normalfördelade på 95% signifikansnivå om de hundra första dagarna efter att fordonet levererades
till kund utelämnades.
För att undersöka hur olika egenskaper inverkar på livslängden tillämpades metoderna random forests och Coxregression på två olika modeller.
Den första modellen är avsedd att tillämpas på nya fordon och den andra
för fordon som har varit i bruk under en tid, och därmed har en variabel
som beskriver fordonets ålder. Resultatet från den första modellen var
tillfredsställande. Coxkoefficienterna var signifikanta och Brierpoängen
var låga både för random forests och för Cox. Den andra modellen gav
däremot inte tillförlitliga resultat, då dess regressionskoefficienter ej var
signifikanta.
Acknowledgements
First, I would like to express my gratitude to my supervisor at Scania,
Thomas Claesson, whose expertise, understanding and patience have
been invaluable to me. I would also like to thank Ann Lindqvist and
Björn Y Andersson at Scania for helping me in procuring the data for
this thesis. For their technical support and valuable contributions, I
would like to thank Erik Frisk and Mattias Krysander at Linköping
University and Gunnar Ledfelt at Scania. I would also like to thank
Örjan Ekeberg, my supervisor at KTH, for his academic experience, as
well as my examiner Anders Lansner.
Abbreviations
Ah
Ampere-hour
ANN Artificial neural networks
AUC Area under curve
CBM Condition-based maintenance
MLE Maximum likelihood estimation
MLR Multiple logistic regression
PDF
Probability density function
RBF
Radial basis function
RF
Random forests
ROC Receiver operating characteristic
RSF
Random survival forests
RUL
Remaining useful life
SOC
State of charge
SOH
State of health
SVM Support vector machine
Contents
1 Introduction
1.1 Problem description . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 Objectives with the thesis . . . . . . . . . . . . . . . . . . . . . . . .
1.3 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2 Background
2.1 Battery characteristics and degeneration . . . . . . . . . .
2.1.1 The lead-acid battery . . . . . . . . . . . . . . . .
2.1.2 Degeneration processes and stress factors . . . . .
2.2 Component maintenance policies and remaining useful life
2.3 Lifetime estimations of batteries . . . . . . . . . . . . . .
2.4 Failure time analysis . . . . . . . . . . . . . . . . . . . . .
2.4.1 Reliability function . . . . . . . . . . . . . . . . . .
2.4.2 Lifetime distribution and lifetime density . . . . .
2.4.3 Hazard function . . . . . . . . . . . . . . . . . . .
2.4.4 Censoring . . . . . . . . . . . . . . . . . . . . . . .
1
1
2
2
.
.
.
.
.
.
.
.
.
.
4
4
4
6
8
10
11
12
12
12
12
3 Data
3.1 Failure data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2 Technical specification . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3 Operational data . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14
14
15
15
4 Classification and feature selection
4.1 Support vector machine . . . . . .
4.2 Data preprocessing . . . . . . . . .
4.3 Feature selection . . . . . . . . . .
4.4 Parameter selection . . . . . . . . .
4.5 Performance evaluation . . . . . .
4.6 Implementation . . . . . . . . . . .
4.7 Data . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
17
17
20
20
20
21
21
22
5 Lifetime distribution estimations
5.1 Maximum likelihood estimation . . . . . . . . . . . . . . . . . . . . .
5.2 Common distributions . . . . . . . . . . . . . . . . . . . . . . . . . .
23
23
24
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
5.3
5.4
5.5
5.2.1 Normal distribution
5.2.2 Weibull distribution
5.2.3 Gamma distribution
Performance evaluation . .
Implementation . . . . . . .
Data . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
24
24
25
25
26
26
6 Reliability models
6.1 Random forests . . . . . . . . . . . .
6.1.1 Splitting rules . . . . . . . . .
6.2 Cox regression model . . . . . . . . .
6.2.1 Assumption of the Cox model
6.2.2 Estimation of baseline hazard
6.3 Performance evaluation . . . . . . .
6.4 Implementation . . . . . . . . . . . .
6.5 Data . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
27
27
29
29
30
30
31
31
31
7 Results
7.1 Classification and feature selection
7.1.1 Discriminating features . .
7.1.2 Classification results . . . .
7.2 Lifetime distribution estimations .
7.2.1 Haulage vehicles . . . . . .
7.2.2 Distribution vehicles . . . .
7.2.3 Goodness of fit . . . . . . .
7.3 Reliability model I . . . . . . . . .
7.3.1 Haulage vehicles . . . . . .
7.3.2 Distribution vehicles . . . .
7.4 Reliability model II . . . . . . . . .
7.4.1 Haulage vehicles . . . . . .
7.4.2 Distribution vehicles . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
33
33
33
33
38
38
40
42
42
42
49
54
54
58
.
.
.
.
.
.
.
.
.
.
.
.
.
8 Discussion
62
9 Conclusion
65
Bibliography
67
Chapter 1
Introduction
Effective transportation of goods and people requires that vehicles are available
when they are planned to operate. For safety reasons, it is crucial to avoid unplanned breakdowns that can cause traffic casualties and tailbacks. Disrupted operations can also entail fees for late delivery and destroyed goods for the haulage
contractor. Accurate lifetime predictions for components enable preventive replacements to be performed more efficiently to avoid unplanned maintenance. If the
component lifetime can be estimated, the reliability can be improved while avoiding
unnecessary maintenance and enabling maintenance to be scheduled more efficiently.
This thesis focuses on predictive replacements of automotive batteries, one of the
more complex components in a vehicle to monitor and predict. A fixed, conservative battery replacement interval that covers operating situations can be used to
reduce the risk of changing battery too late. However, by using information about
operating conditions, the battery replacement interval could be flexible, reducing
the maintenance cost for the haulage contractors.
1.1
Problem description
A discharged battery is the main cause of unplanned immobilization of long haulage
trucks produced by Scania. Total power demands are constantly increasing, due to
a larger number of electrically powered systems in modern vehicles. More vehicles
are now equipped with features like windscreen heating, seat heating, kitchen equipment and communication systems. If a truck driver consumes too much electrical
energy with the engine inoperative, the battery will not be able to deliver enough
power needed to start the engine. Battery related problems are more likely to occur
in winter as the battery’s ability to accept charge drops in cold temperatures, which
increases the time it takes to recharge the battery. Also, the load on the battery
usually increases in colder weather. In most cases it is enough to charge the battery to restore acceptable functionality. In many situations of failure, however, the
battery is worn out and must be replaced.
1
CHAPTER 1. INTRODUCTION
Battery performance and useful lifetime are affected by various factors, such as
operating temperature and discharging/charging cycles. The useful life is in this
context defined as the period during which the battery is expected to be usable for
the purpose it was acquired. The purpose of the automotive battery is to start the
vehicle in all kinds of weather, but also to cover the electrical power needs of the
vehicle when the alternator is switched off or cannot generate enough power.
The aging of batteries is a complex process where several parallel degeneration
processes are involved. As the battery becomes older, it loses performance and
becomes more susceptible to failure. Today, satisfactory methods to estimate the
health of the battery in a Scania vehicle are missing. One way to prevent starting
problems due to worn out batteries is to change the battery more often. However,
this can be unnecessary expensive for the haulage contractor. For this reason a
better method to determine the risk of battery failure is desired.
1.2
Objectives with the thesis
The ambition of the master’s thesis is to develop a model that can predict the risk
of failure of an automotive battery. The actual lifetime of a battery is typically random and unknown, and therefore it must be statistically estimated from available
sources of information.
This master’s thesis attempts to:
• analyze which variables discriminate the vehicles with battery failure within
a distinct period of time from those without,
• examine if a classifier can separate these two groups of vehicles from the
available data,
• model the distribution of lifetimes of automotive batteries,
• model the impact of explanatory variables on the lifetime distribution.
1.3
Outline
The thesis starts off with a background chapter which initially describes theory of
lead-acid battery characteristics and the dominating degeneration processes, which
aims to give the reader basic understanding of how different conditions and usage
affect the battery lifetime. Next, the chapter discusses different maintenance policies for components and the research position of lifetime estimation and failure time
analysis of components in general and batteries in particular.
The background is followed by a chapter describing the data available in the project,
2
CHAPTER 1. INTRODUCTION
and the form the variables are given on.
The description of the methods used in the project is divided into three chapters,
each covering one or more of the objectives described in section 1.2. The first of
these chapters contains theory and implementation of the classification and feature
selection that is used to analyze the first two thesis objectives. The second chapter contains theory and implementation of maximum likelihood estimation, which
partly attempts to analyze the third objective. The third and last method chapter
describes the theory and implementation of the reliability models that are used to
model the lifetime distributions and the impact of different variables on the lifetime.
This analysis aims to cover the third and forth objectives.
The method chapters are followed by results, discussion and conclusion.
3
Chapter 2
Background
2.1
Battery characteristics and degeneration
In a modern road vehicle, the electrical system is vital for the mobility, safety and
comfort. Since the electrical system in an automotive vehicle is required to be active
before the alternator can function, some kind of battery is necessary. The original
function of the battery was to support the starting, lighting and ignition system.
In a modern vehicle however, the list of electrical equipment is constantly growing
which imposes heavier loads on the battery.
The electrical system of a vehicle consists of a battery, an alternator, voltage control
and protective devices and the electrical loads. Once the engine is operating, the
alternator will generate electrical power and distribute it to the electrical energy
consuming devices in the vehicle. The surplus energy is used to charge the battery.
The battery and the alternator are placed in parallel across the system and the
battery is kept at the alternator output voltage, which is also applied directly to
the loads.
2.1.1
The lead-acid battery
In automobiles, the lead-acid battery is by far the most common. There are several
advantages with the lead-acid battery that makes it suitable in the broad spectrum
of automotive duties, and although new technologies are constantly being introduced
on the market, it appears as if it will remain on its dominant position. Some
advantages with the lead-acid battery can be listed as (Reasbeck and Smith, 1997):
• The ability to deliver very large currents over a short period of time, which is
required to start the vehicle.
• The high cell voltage, which results in fewer cells per battery for a given
voltage.
• The availability and low cost of the materials in the component.
4
CHAPTER 2. BACKGROUND
• The high chemical stability over the range of temperatures automobiles normally operate in.
The lead-acid battery, invented by Gaston Planté in 1859, was the first battery that
could be recharged by passing a reverse current through it. Although the battery
has developed and improved a great deal since then, the basic principles are still
the same.
A battery is made up by several power producing cells connected in series or in
parallel to achieve the desired voltage and capacity. The cells are normally identical
and contribute with the same amount of voltage. The voltage from each cell depends on the chemical reactions within the cell. Each cell consists of two electronic
conduction plates, the electrodes, in contact with an ionically conducting phase,
the electrolyte. At the surfaces of the plates, reactions occur. In these reactions,
electrons are exchanged between the electrodes and the ions in the electrolyte. The
circuit of charge flow is completed through an electric circuit between the electrodes.
When the lead-acid battery is fully charged, the positive electrode consists of lead
dioxide (P bO2 ) and the negative electrode of metallic lead (P b). The third active material is the sulfuric acid (H2 SO4 ), which forms the conductive electrolyte
between the electrodes. The electrolyte also consists of water. On discharge, the
following chemical reactions, on the positive plate, the negative plate and the overall
reaction respectively, occur:
P bO2 + 4H + + SO42− + 2e → P bSO4 + 2H2 O
(2.1)
P b + SO42− − 2e → P bSO4
(2.2)
P bO2 + P b + 2H2 SO4 → 2P bSO4 + 2H2 O
(2.3)
During discharge, the concentration of sulfuric acid in the electrolyte decreases.
This fact has frequently been used to indicate the state of charge of the battery
by measuring the density of the electrolyte. As the concentration of sulfuric acid
decreases, the conductivity reduces which contributes to lower power-outputs at low
states of charge. At the electrodes, lead sulfate is produced during discharge. The
layers of sulfate blocks the active materials in the electrodes. As more lead sulfate is
built up, less current can be discharged and the power output lowers. The charging
process is the reversed process, where electrons are forced from the positive plate
to the negative plate.
The lead-acid cell consists of the following functioning parts:
5
CHAPTER 2. BACKGROUND
Active materials
The porous lead dioxide on the positive plate and the porous metallic lead on the
negative plate are the active materials that provide the electrode reactions.
Support grids
The purpose of the grids is to provide mechanical support for the porous active
materials and to provide a low-resistance current path to the cell terminals.
Separators
The purpose of the separators is to prevent short-circuit through physical contact
between positive and negative plates.
Electrolyte
The electrolyte provides an ionic path between the plates. In the lead-acid cell the
electrolyte is also one of the active materials as it provides the reaction with sulfuric
acid.
Cell terminals
The cell terminals provide a low-resistance electrical connection to the outer circuit.
Usually, all positive plates are linked to one terminal and all negative plates to
another terminal.
Cell container
The cell container encloses the cell components. It is made of plastic, which is
chemically resistant and insulating.
2.1.2
Degeneration processes and stress factors
As the battery becomes older, the inner structure of the components and the materials are changing. This leads to degradation in the performance and eventually to
end of life of the battery. The following aging mechanisms dominate the lead-acid
battery damaging process (Svoboda, 2004):
Corrosion of the positive grid
As parts of the grid corrode, the connection between the active material and the
terminals is reduced. This causes a reduction in capacity. Grid corrosion also
causes an increase in the internal resistance. Corrosion of the positive grid is a
natural aging mechanism, due to the fact that the lead on the positive electrode is
thermodynamically unstable. According to Ruetschi (2004), positive grid corrosion
is probably the most frequent cause of failure in lead-acid automotive batteries.
6
CHAPTER 2. BACKGROUND
Sulfation
During discharge, sulfate is created at both electrodes. When charged, the sulfate is
dissolved and converted to lead dioxide on the positive electrode and metallic lead
on the negative electrode. These are the fundamental reactions of the lead-acid
battery. Under some conditions though, the sulfate is built up in large crystals that
cannot be dissolved during the charging process. As a consequence, the amount of
active material is decreased which leads to a loss of capacity.
Shedding
Sulfation can cause active material to detach from the electrodes, since sulfate
crystals have a larger volume than lead oxide. This process is called shedding.
Overcharging can also cause shedding as gassing bubbles can force active material
to detach from the electrode.
Active mass degradation
Active mass degradation is a process where the mechanical structure at the boundary between the electrolyte and the electrodes is changed. This leads to a decrease
in the surface area, which reduces the capacity.
Water loss/drying out
The lead-acid batteries used in Scania vehicles require maintenance in form of water
addition. If the battery is dried out, the battery can be damaged.
Electrolyte stratification
When the electrolyte stratifies, the acid content with higher density sinks to the
bottom. Due to this, the chemical reactions are concentrated to the lower parts
of the electrodes, which reduces the capacity. This can also cause corrosion at the
parts of the electrodes where the acid concentration is lower.
The damage mechanisms discussed above are highly affected by certain operating
conditions and usage patterns. The major factors, called stress factors, affecting
the aging processes are identified by Svoboda (2004):
• Time at low state of charge (SOC)
• Ah-throughput (defined as the cumulative ampere-hour discharge in a oneyear period normalized in units of the battery nominal capacity)
• Charge factor (defined as the Ah charged divided by the Ah discharged over
a period of time)
• Time between full charge
7
CHAPTER 2. BACKGROUND
• Temperature
Table 2.1 shows the relationships between some stress factors and aging processes.
Although the relationship matrix is a simplification, it shows clearly that a stress
factor can have positive correlation with some aging processes while negative correlation with other aging processes. Another complicating factor is that the various
aging processes interact in complex manners (Sauer and Wenzl, 2007).
Table 2.1: Relationships between stress factors and aging processes (Svoboda, 2004).
Long
time
at
low SOC
Ahthoughput
Charge
factor
Time between full
charge
Temperature
2.2
Corrosion
Sulphation Shedding
Water
loss
Indirect
through
low
acid
concentration
and
low
potentials.
None
Strong
positive
correlation.
No direct
impact.
None
No direct
impact.
No direct
impact.
Impact.
Strong
indirect
impact
through
high polarization of
electrodes.
Strong
negative
correlation.
Strong
positive
correlation.
Negative
correlation.
Impact
through
mechanical
stress
Strong
impact
through
gassing.
Strong impact.
No direct
impact.
Negative
correlation.
Negative
correlation.
No direct
impact.
No direct
impact.
Positive
correlation.
Low
pact.
Strong
positive
correlation.
Correlation
can
be
both negative and
positive.
AM
degradation
None
im-
Electrolyte
stratification
Indirect
effect.
Longer
time
through
higher
sulfation.
Strong
positive
correlation.
Strong
positive
correlation.
Strong
positive
correlation.
No direct
impact.
Component maintenance policies and remaining useful
life
With optimal replacement intervals system reliability can be improved, system failures prevented and maintenance costs reduced. Under some rare circumstances,
the optimal replacement strategy is to replace the component after a failure has
occurred, so called reactive maintenance. The benefit with this strategy is that the
8
CHAPTER 2. BACKGROUND
lifetime of the component will be maximally used, which reduces waste. However,
in many situations a breakdown is costly and can cause a dangerous situation and
must be avoided.
A broadly practiced replacement technique is to utilize a predetermined maintenance schedule. This strategy reduces the probability of failure and the risk of
unplanned downtime. Failure can still occur and under certain operation conditions the prescribed interval might be too extensive or too narrow.
One way to make more accurate estimations of the component lifetime is to use
information about individual usage and conditions of the system or component.
Condition-based maintenance (CBM) is a maintenance strategy that bases the maintenance decisions on information collected by monitoring the condition of the system. Two important aspects of CBM are diagnostics and prognostics. Diagnostics
attempts to detect faults that have occurred in the system while prognostics deals
with prediction of faults before they occur.
The most common form of prognostics is to predict how much time is left before failure. By evaluating component condition and data from past operations
the remaining useful life (RUL) of the component can be estimated. The RUL is
the length from the particular time to the end of the useful life of a component or
system. The RUL is a random variable and it depends on the current age of the
component, the environment and the observed information of the health of the component. When continuously estimating RUL of the component, fluctuating usage is
taken in account, which makes the estimate more accurate.
If Xt is defined as the random variable of the remaining useful life at time t and
Yt as the operating history up to t, the probability density function (PDF) of Xt
conditional on Yt will be denoted as f (xt |Yt ). The estimation of RUL can be formulated as estimating f (xt |Yt ) or E(Xt |Yt ). If there is no information about Yt , then
the estimation of f (xt |Yt ) becomes:
f (xt |Yt ) = f (xt ) =
f (t + xt )
R(t)
(2.4)
where f (t + xt ) is the value of the PDF at time t + xt and R(t) is the reliability
function at t. If Yt is available, this will provide more information to make the
estimation of RUL more accurate (Si, 2011).
Prognosis approaches for estimating lifetime could either be physics-based models or data driven. Algorithms that use the data driven approach for predicting
lifetime construct models directly from data, rather than relying on any physics
or engineering principles. Dorner et al. (2005) investigates the latter approach by
constructing a physico-chemical aging model of the battery. The model is based on
Ohm’s law, fundamental equations of chemical reactions and the diffusion processes
9
CHAPTER 2. BACKGROUND
of reactants. For any point in the battery at any time, the model provides state
variables like potential, current density, state of charge, temperature, acid concentration etc. This information is used to quantify degradation processes and how
these processes impact the battery performance. This model requires data from
laboratory experiments on the aging mechanisms, which is usually very difficult
to achieve. Due to the complex and non-linear behavior of the battery, authentic
physics-based models that can be applied in varying operating conditions are difficult to achieve and rarely suitable (Sauer and Wenzl, 2007).
With data driven methods, the RUL model is fitted to the available data. These
methods are suitable for complex systems like batteries where the chemical and
physical processes, and their interactions, are difficult to represent analytically.
The data are generally of two main types: past recorded failure data and operational data. Here operational data incorporate any data which may have an impact
on the RUL, such as environmental information, performance information and information about the condition of the system or asset. The data driven approaches
can be machine learning approaches or statistical approaches, or a mix of them both.
Some research has focused on applying machine learning techniques to RUL estimation. Tian (2009) introduces a method to predict remaining useful life of equipment
using artificial neural networks (ANN). The model takes the age and condition
monitoring measurement values at discrete inspection points as input and the life
percentage as output. To reduce the effects of noise the measurement series are
fitted with a generalized Weibull distribution failure function. The ANN method
is validated using data from condition monitoring of pump bearings. Yang and
Widodo (2008) proposes a regression based support vector machine (SVM) method
for machine prognosis. Based on previous state data it attempts to predict the
future state condition.
For risk analyses, it is essential to provide a probability distribution rather than
a mean estimate of the time to failure (Wang and Christer, 2000). There is a large
body of literature on estimation of f (xt |Yt ) using statistical methods. The statistical methods for estimating the RUL are however usually based on times series data
about the state of health of the component, either directly or indirectly observed.
2.3
Lifetime estimations of batteries
In the past several decades, different approaches for health management of batteries have been extensively studied in the literature. Traditional approaches have
mostly focused on estimating the state of charge (SOC) rather than the state of
health (SOH) or RUL. While SOH mainly considers diagnostics, SOC and RUL are
prognostic concerns. For applications where the variation of operating conditions is
large, lifetime prediction is still at an early stage. There is also limited experience
10
CHAPTER 2. BACKGROUND
in lifetime estimations derived from real operating condition data.
Techniques based on statistical methods have been considered by some researchers.
Jaworski (1999) applied statistical parametric models to predict time to failure
of batteries exposed to varying temperatures. Saha et al. (2007) introduced a
Bayesian learning framework to predict remaining useful life of lithium-ion batteries. The approach combines relevance vector machine (RVM) and particle filter
(PF) to generate a probability density function (PDF) for the end-of-life to estimate RUL. RVM is a type of SVM constructed under Bayesian framework and thus
has probabilistic outputs. PF is a technique for implementing a recursive Bayesian
filter using Monte Carlo simulations to approximate the PDF. The model was built
using internal parameters such as charge transfer resistance and electrolyte resistance under the assumption that the parameters gradually change as the battery
ages.
In a more recent paper, Saha et al. (2009) compared autoregressive integrated moving average (ARIMA), extended Kalman filtering (EKF) and RVM-PF approach to
estimate RUL from experimental data from lithium-ion batteries. ARIMA is a
model that is used to fit time series data to predict future points in the series.
ARIMA models are used for observable non-stationary processes that have some
clearly identifiable trends. EKF uses a series of measurements observed over time
and produces statistically optimal estimates of the underlying system state. Compared to the traditional Kalman filter, the EKF can handle non-linear systems. In
the first step of the EKF algorithm, a state transition model is used to propagate
the state vector into the next time step. In the second step of the algorithm, a
measurement of the system is used to correct the prediction. Their result showed
considerable differences in performance of the three approaches where the Bayesian
statistical approach outperformed the ARIMA and the EKF methods.
2.4
Failure time analysis
Survival analysis is a field in statistics for modeling data that describe time to an
event. In short, it is the study of lifetimes and their distributions. Survival analysis is traditionally used in the area of biostatistics, where an event usually means
death. The methods have however spread to other disciplines and are often used to
analyze events such as unemployment in economy, divorce in sociology and failure
in mechanical systems in engineering. In engineering, the field of survival analysis
is usually called failure time analysis.
The time to failure T may be thought of as a random variable. There are several representations of the distribution of T .
11
CHAPTER 2. BACKGROUND
2.4.1
Reliability function
The reliability function, R(t), represent the probability that the time of failure is
later than some time t:
R(t) = P r(T > t)
(2.5)
It is usually assumed that R(0) = 1 and that R(t) → 0 as t → ∞.
2.4.2
Lifetime distribution and lifetime density
The cumulative distribution function, F (t), is the complement of the reliability
function:
F (t) = P r(T ≤ t) = 1 − R(t)
(2.6)
This function is also called the failure function as it represents the proportion of
units that have failed up until time t. The derivative of the cumulative distribution
function is the density function of the lifetime distribution:
f (t) =
d
F (t)
dt
(2.7)
where f (t) represents the instantaneous failure rate.
2.4.3
Hazard function
The probability that failure occur in the next short period of time given survival
up to that time is called the hazard function h(t):
h(t) = lim
∆t→0+
P r(t ≤ T ≤ t + ∆t|T ≥ t)
f (t)
=
∆t
R(t)
(2.8)
The hazard function equals the proportion of the population that fail per unit time
t, among those still functioning at this point in time.
The cumulative hazard function H(t) is the area under the hazard curve up until time t:
! t
H(t) =
h(u)du
(2.9)
0
A plot of H(t) shows the cumulative probability that failure has occurred at any
point in time.
2.4.4
Censoring
Often the data used in failure time analysis are censored. Censoring occurs when
each observation result either in knowing the exact value or in knowing that the
value lies within an interval. When data are right censored, a censored data point
is above a certain known value but the exact value is unknown. In a failure analysis
test, let n be the number of units. During time C, r failures are observed, where
12
CHAPTER 2. BACKGROUND
0 ≤ r ≤ n. Since the times of failure for n − r units are unknown, but it is known
that these times to failure are larger than C, the data are right censored.
In reliability analysis it is commonly assumed that the variables C and T are independent. This is called non-informative censoring. The distribution of survival
times of units that are censored at a particular time is no different from that of units
that are still observed at this time. One common type of independent censoring is
the simple type I censoring, where all subjects in the study are censored at the
same, fixed time.
13
Chapter 3
Data
In this section, the available data used in the analyses in this report are described.
The possible analyses of battery lifetimes are restricted heavily due to the limitations in the data. First, there exist no database over complete lifetimes of truck
batteries used in vehicles produced by Scania. Another problematic aspect is that
truck batteries are relatively easy to replace, so batteries might be exchanged without Scania’s knowledge, either by the haulage contractor or at a workshop not
connected to Scania.
Another limitation is the variables available in the data. No direct measurement of
the health of the battery is available. Instead, the analyses must be based on information about technical specification and operating conditions that are assumed
to influence the lifetime. This information is however not available for all vehicles,
which restricts the number of vehicles to base the analyses on.
The operational data that are recorded and stored in the trucks are stored accumulated and is not time stamped. Therefore, a regular time series analysis is
difficult to carry out.
3.1
Failure data
The analyses in the project are based on data from trucks delivered to customers in
England between 2007 and 2010. Each truck has a repair contract that is valid for
two years from the delivery date. The data contain information about which trucks
have replaced their batteries during this period of time and when this replacement
occurred. When using this data, two assumptions are made. It is assumed that no
other battery replacements are made during these two years, so it is assumed that
a haulage contractor utilizes the repair contract it has paid for. It is also assumed
that there actually is a problem with the battery replaced, so each replacement is
considered to be a battery failure.
14
CHAPTER 3. DATA
The choice of using data from England was made due to its relatively long and
uniform contract time. The English truck drivers are also more frequent customers
of the Scania Assistance service, due to England’s strict regulations when a driver
is obligated to contact road assistance services.
The assistance data for battery components include the date of the assistance errand, as well as the action made, for example a replacement or a jump start. The
assistance errands of interest for a vehicle are those that occurred before the time of
failure or before the contract expired. The data used in the analysis are the number
of assistance errands for a vehicle that required a jump start during this period of
time.
The data set consists of approximately 10 000 haulage vehicles and 3 500 distribution vehicles. A subset of this complete set, here denoted the operational set,
consists of all vehicles for which complete information about technical specification
and collected operational data are available. This set consists of about 1 400 haulage
vehicles and 450 distribution vehicles.
3.2
Technical specification
For each vehicle, the size of the battery and the alternator is known. The battery size
is measured in ampere-hours (Ah) and the alternator in amperes (A). For haulage
vehicles it is also known if some sort of kitchen equipment, such as a microwave or
a coffee maker, is installed. These features together with their possible values are
presented in table 3.1.
Table 3.1: Technical specification data and possible values.
Battery
Alternator
Kitchen
3.3
Values
140/180/225
80/100/150
Yes/No
Unit
Ah
A
-
Operational data
Operation data from trucks are automatically recorded and stored throughout the
life of the vehicle. This data source provides information about the use and performance of the truck. The data are accumulated in bins, either as a scalar, a vector
or a matrix depending on the nature of the variable and stored each time when the
vehicle visits a workshop.
The operational variables used in the analysis in this report are presented in table
3.2. The vectors for voltage and ambient temperature are ten elements long. For
15
CHAPTER 3. DATA
voltage, the first element contains the percentage of time with a system voltage less
than 26 V, the second element 26-26.5 V, the third 26.5-27 V etc. The last element
contains the percentage of time with voltage above 30 V. For temperature, the first
element contains the percentage of time with a measured ambient temperature below -30 ◦ C. The following elements cover an interval of 10 ◦ C, hence the second
element contains percentage of time between -30 and -20 ◦ C and the last element
the percentage of time above 50 ◦ C.
Each of the three method chapters that now follow contains a data section describing which data are used in that analysis.
Table 3.2: Operational data.
Form
Number of kilometers driven
Scalar
Time in drive
Scalar
Time idle with PTO*
Scalar
Time idle without PTO*
Scalar
Start time
Scalar
Number of starts
Scalar
Voltage
Vector
Ambient temperature
Vector
* PTO (power take-off) means that power is taken
from the truck engine to an attached application,
for example a dump trailer or a crane.
16
Chapter 4
Classification and feature selection
In this section the methods used for classification and feature selection are described.
The purpose of the classification is to investigate if the available data are sufficient
to distinguish the vehicles with battery failure. The feature selection is used to
identify properties that discriminates this group of vehicles. The result from the
feature selection can also be used to improve the result of the classifier by omitting
less relevant attributes.
4.1
Support vector machine
Support vector machine (SVM) is a supervised machine learning technique for data
classification. Like other classification methods, the aim is to construct a model
from training samples of which the class is known, and use this to predict the class
label of unseen samples. The SVM algorithm was introduced by Vapnik in 1992
and has gained popularity due to its several advantages when compared to other
machine learning techniques. The classification performance is usually high and the
method is considered to have a high generalization performance. Also, the training
of the model when using SVM is guaranteed to result in the best classifier, compared with ANN, which can result in a local minimum. This motivates the use of
SVM in this thesis.
The basic idea is to map the training vectors into a high dimensional space where
the SVM finds a separating hyperplane with the maximal margin. In principle, it
is possible to transform any data set so that the classes can be separated linearly
(Vapnik, 1998). In Figure 4.1 the maximum margin hyperplane is a line that separates two classes in two dimensions. The data points in each class that lie on the
margins are called support vectors. The maximum margin hyperplane, and therefore the classification problem, is only a function of the support vectors, and not all
data points in the training set.
17
CHAPTER 4. CLASSIFICATION AND FEATURE SELECTION
Figure 4.1: The solid line is the maximum margin hyperplane that separates the two classes. The samples
on the dashed lines are the support vectors.
The hyperplane can be expressed as:
&w, x' + b = 0
w ∈ H, b ∈ R
(4.1)
where w is the normal vector of the plane, b is a parameter that determines the
offset of the hyperplane from the origin and H is some dot product space.
For any testing instance x, the decision function is:
f (x) = sgn(&w, x' + b)
(4.2)
The task to find a maximum margin hyperplane can be formulated into a Lagrangian
problem:
m
"
1
L(w, b, α) = )w)2 −
αi (yi (&w, xi ' + b − 1))
(4.3)
2
i=1
Using this method, the Lagrangian is minimized with respect to the variables w
and b and maximized with respect to the Lagrange multipliers αi . The multipliers
reflect the weight given to each training sample.
The problem can be expressed in a dual form optimization problem. This requires
that w is eliminated from 4.3. This can be accomplished by using the KarushKuhn-Tucker condition, which implies that the solution can be expressed as a linear
18
CHAPTER 4. CLASSIFICATION AND FEATURE SELECTION
combination of the training vectors:
w=
m
"
αi yi xi
(4.4)
i=1
Doing so, the dual optimization problem takes the following form:
max
α∈Rm
m
"
i=1
αi −
m
1 "
αi αj yi yj &xi , xj '
2 i,j=1
subject to αi ≥ 0, i = 1, ..., m
m
"
(4.5)
αi yi = 0
i=1
The Lagrangian formulation above may not be solvable if the data cannot be separated by a hyperplane. To solve this problem, SVMs use the kernel trick and
soft margin classifiers. By using the kernel trick, the SVM can perform nonlinear
classification. This can be done by replacing every dot product with a nonlinear
kernel function. A soft margin classifier permits mislabeling of some of the training
samples if no hyperplane that can split the two classes exists. This is done by introducing an upper bound C on the Lagrange multipliers, thus limiting the influence
of a single support vector. With kernel function and soft margin, the optimization
becomes:
m
m
"
1 "
αi αj yi yj k(xi , xj )
maxm
αi −
α∈R
2 i,j=1
i=1
(4.6)
subject to 0 ≤ αi ≤ C, i = 1, ..., m
m
"
αi yi = 0
i=1
The decision function can then be expressed as:
f (x) = sgn
#m
"
αi yi k(x, xi ) + b
i=1
$
(4.7)
There are three basic kernel functions that are commonly used and that have shown
good performance: polynomial, radial basis and sigmoid kernels. In the applications
of SVM in this paper the linear kernel and the radial basis kernel have been used and
compared. The Gaussian radial basis function (RBF) has good general performance.
It can be expressed as:
2
k(xi , xj ) = e−γ%xi −xj %
(4.8)
The parameter γ is a measure of how similar samples are required to be.
The overall effectiveness of SVM depends on the selection of kernel, the size of
the parameters of the kernel and the soft margin parameter C.
19
CHAPTER 4. CLASSIFICATION AND FEATURE SELECTION
4.2
Data preprocessing
SVM requires each data point to be represented as a vector of real numbers. Boolean
attributes need to be converted into numeric data. To represent a histogram of
length m, m elements in the feature vector are used.
To avoid attributes with large numeric ranges to dominate, attributes are scaled.
The attributes are scaled to the range [-1, 1] by:
x& = 2
x − mi
−1
Mi − mi
(4.9)
where Mi and mi are respectively the maximum and the minimum value of the ith
attribute, and x& is the scaled value of x.
The training and testing sets are scaled using the same scaling factors.
4.3
Feature selection
In machine learning, feature selection is the technique for selecting the subset of
most relevant features that are used in the model. A simple and effective approach
for extracting features is to calculate the F-score for each feature and drop features
below a certain threshold. The F-score is a measure of the discrimination of the
positive and negative data sets. For the ith feature, the F-score is defined as (Chen
and Lin, 2006):
F (i) =
n
%
(+)
x̄i
− x̄i
&2
%
(−)
+ x̄i
− x̄i
n
&2
+ %
− %
&
&
"
"
1
1
(+)
(+) 2
(−)
(−) 2
xk,i − x̄i
+
xk,i − x̄i
n+ − 1 k=1
n− − 1 k=1
(4.10)
Here, n+ and n− are the number of positive and negative instances respectively,
(+)
(−)
x̄i , x̄i and x̄i are the mean values of the whole, positive and negative data sets
(+)
(−)
and xk,i and xk,i are the kth instance of the positive and negative sets. The larger
F-score, the more discriminative is the feature.
A drawback with F-score is that it does not take shared information among features
into account. Despite this, it is however a simple and effective method for ranking
and measuring the importance of features, and this motivates the use of the F-score
feature selector in this thesis.
4.4
Parameter selection
The optimization of the parameters C and γ is done using ν-fold cross-validation
and grid search. In ν-fold cross validation, the training set is divided into ν subsets
20
CHAPTER 4. CLASSIFICATION AND FEATURE SELECTION
of equal size. All of the subsets are used as the test set once to evaluate a classifier
that is trained on the remaining ν − 1 subsets. The cross-validation accuracy is
the percentage of data points correctly classified. The motive of the procedure of
cross-validation is to prevent overfitting.
In grid search, various values of the parameters C and γ are tried out and the
pair with the best cross-validation accuracy is picked.
4.5
Performance evaluation
To assess the performance of the model, the data were split into training and testing
sets. The sets were split randomly 60:40. The training set was used in parameter
selection and in the construction of the SVM model, while the testing set only was
used at the last stage to evaluate the performance.
To illustrate the performance of a binary classifier, a receiver operating characteristic (ROC) curve can be used. It is a plot with the true positive rate (TPR) on
the y-axis and the false positive rate (FPR) on the x-axis. The TPR is also known
as sensitivity and the FPR is one minus the specificity or true negative rate (TNR).
Sensitivity =
TP
TP + FN
(4.11)
TN
(4.12)
TN + FP
The area under the ROC curve (AUC) has shown to be a useful measure of the
performance of the classifier. A high value of the AUC implies that the model efficiently discriminates between the classes. The AUC varies between 0.5 (random
guess) and 1.0 (a perfect classifier). Barakat and Bradley (2006) showed that ROC
curves and AUC could provide a more reliable measure of quality than the accuracy.
Specificity =
To evaluate the performance of the SVM model, multiple logistic regression (MLR)
analysis was performed with the same selected features. The modeling was performed on the training data set and the estimated coefficients were applied to the
test data set to calculate the AUC values.
4.6
Implementation
To generate the SVM models, the SVM library LIBSVM was used. LIBSVM is a
freely available software library implemented by Chang and Lin (2011).
For every feature, the F-score is calculated using a script in the LIBSVM library.
For a set of thresholds, the features with F-scores above every threshold are used to
train an SVM with the RBF kernel. The threshold with the lowest validation error
21
CHAPTER 4. CLASSIFICATION AND FEATURE SELECTION
Table 4.1: Variables used in the feature selection and SVM analysis.
Variable
Voltage histogram
Temperature histogram
Battery size
Kilometers per day
Start time per start
Fraction of time idle with PTO
Fraction of time idle without PTO
Kilometers per start
Number of assistance errands
Kitchen equipment
Alternator size
Number of features
10
5
1
1
1
1
1
1
1
1
1
is used, and the features below this threshold are dropped.
The package also included a utility (grid.py) that was used to find the optimal
values of the parameters C and γ. Five-fold cross validation was used, with C
spanned between 2−5 and 215 and γ between 2−15 and 23 . It also included functions
for plotting ROC curves and to calculate the AUC.
The logistic regression analysis was done in the statistical program R (R Development Core Team, 2008).
4.7
Data
A list of the features used in the feature selection and the SVM analysis is presented
in table 4.1, along with the number of features it corresponds to in the feature vector.
Each element in the voltage and the temperature histograms becomes a feature in
the feature vector. The two first and the three last elements in the temperature
vector are omitted since the corresponding values are almost exclusively zero at
these extreme temperatures. The kitchen feature is set to one if kitchen equipment
is installed, otherwise it is zero. This feature is omitted for distribution vehicles
since the corresponding value is zero in every case. The rest of the features are set
to its numerical value. Finally, the feature values are normalized as described in
section 4.2.
22
Chapter 5
Lifetime distribution estimations
A simple approach to lifetime distribution modeling is to analyze if the times of
failure follow some defined probability distribution. The model is simple in the
sense that it does not investigate how different features influence the time to failure,
it only includes the times to failure or, if the subject is right censored, the known
time of survival.
5.1
Maximum likelihood estimation
Maximum likelihood estimation (MLE) is a widely used approach to parameter estimation and inference in statistics. Given a chosen probability distribution model
and the observed set of data, the MLE method estimates the unknown model parameters such that the probability of obtaining the particular set of data is maximized.
Let f (x|θ) represent the PDF for a random variable x conditioned on a set of
parameters θ. For n independent and identically distributed observations the joint
density function is:
f (x1 , ..., xn |θ) =
n
'
i=1
f (xi |θ) = L(θ|x1 , ..., xn )
(5.1)
where L(θ|x1 , ..., xn ) is called the likelihood function. A maximum likelihood estimator is a value of the parameter θ such that the likelihood function is a maximum.
The maximum likelihood estimate for parameter θ is denoted θ̂. It is usually more
convenient to work with the logarithm of the likelihood function, called the loglikelihood:
n
ln L(θ|x1 , ..., xn ) =
"
i=1
ln f (xi |θ)
(5.2)
Since the logarithm function is a monotonically increasing function, maximizing
ln L(θ|x1 , ..., xn ) is equivalent to maximizing L(θ|x1 , ..., xn ). If the log-likelihood
function is differentiable and if a maximum likelihood estimate exists, θ̂ must satisfy
23
CHAPTER 5. LIFETIME DISTRIBUTION ESTIMATIONS
the likelihood equation:
∂ ln L(θ|x1 , ..., xn )
=0
(5.3)
∂θ
For a sample containing both exact and right censored observations, the likelihood
can be written as:
L(θ|x1 , ..., xn ) =
n
'
i=1
f (xi |θ)δi [1 − F (xi |θ)]1−δi
(5.4)
where δi = 1 for an exact observation and δi = 0 for a right censored observation
and F (xi |θ) is the cumulative distribution function.
5.2
Common distributions
This section considers probability distributions which are most often used when
modeling likelihood of failure: the normal, the Weibull and the gamma distributions.
5.2.1
Normal distribution
The normal (or Gaussian) distribution is a continuous probability distribution with
a probability density function known as the Gaussian function:
1 x−µ 2
1
f (x|µ, σ 2 ) = √ e− 2 ( σ )
σ 2π
(5.5)
The parameters µ and σ 2 are the mean and the variance of the distribution respectively. The cumulative distribution function for the normal distribution is:
(
)
1
x−µ
√
F (x|µ, σ ) =
1 + erf
2
σ 2
2
*+
(5.6)
where erf(x) is the error function. The normal distribution describes an increasing
failure rate.
5.2.2
Weibull distribution
The probability density function of the Weibull distribution is:
β
f (x|α, β) =
α
) *β−1
x
α
e−(x/α)
β
(5.7)
The parameter α > 0 is the scale parameter and parameter β > 0 is the shape
parameter. The cumulative distribution function for the Weibull distribution is:
F (x|α, β) = 1 − e−(x/α)
β
(5.8)
The mean and the variance of the Weibull distribution are respectively:
%
&
α Γ(1 + β −1 )
24
(5.9)
CHAPTER 5. LIFETIME DISTRIBUTION ESTIMATIONS
%
α2 Γ(1 + 2β −1 ) − Γ(1 + β −1 )2
where Γ(x) is the Gamma function.
&
(5.10)
The Weibull distribution is often used to model the distribution of lifetimes of
objects. The value of the parameter β is an indicator of the failure rate. If β < 1,
the failure rate decreases over time, if β = 1 the failure rate is constant over time
and if β > 1 the failure rate increases over time.
5.2.3
Gamma distribution
The gamma distribution with shape parameter α and scale parameter β has the
probability density function:
1
f (x|α, β) =
β −α xα−1 e−x/β
(5.11)
Γ(α)
The cumulative distribution function for the gamma distribution is:
1
F (x|α, β) =
Γ(α, x/β)
Γ(α)
(5.12)
where Γ(x) is a complete Gamma function and Γ(a, x) an incomplete Gamma function.
The mean and the variance of the gamma distribution are respectively:
(5.13)
αβ
αβ
2
(5.14)
The value of the parameter α determines the failure rate. With α > 1 the failure
rate increases over time.
5.3
Performance evaluation
To evaluate the goodness of fit of the three distributions to the data, Pearson’s
chi-squared test is applied. The value of the chi-squared test statistic is:
χ2 =
k
"
(Oi − Ei )2
Ei
i=1
(5.15)
where k is the number of subintervals, Oi is the observed number of data points in
interval i and Ei is the expected number of data points in interval i. The statistic
value is compared to a chi-squared distribution with d = k−1−n degrees of freedom,
where n equals the number of estimated parameters, and if:
χ2 < χ21−α,d
(5.16)
it can be assumed at the significant level α that the data follow the assumed theoretical distribution.
25
CHAPTER 5. LIFETIME DISTRIBUTION ESTIMATIONS
5.4
Implementation
To fit the probability distributions to failure data using maximum likelihood estimation, the Statistics Toolbox in Matlab is used.
5.5
Data
In this analysis technical specification and operational data are unnecessary. Therefore the complete data set can be used where the only requirement is that the time
of failure is known. The data set for haulage vehicles consists of 2107 observed
failures and 7919 censored for which failure was not observed after 730 days. For
distribution vehicles, 469 failures were observed and 2987 were censored.
26
Chapter 6
Reliability models
In this chapter, the two methods used for constructing reliability models are explained. For each method, two models are developed. The aim of both models is to
describe the influence of features on the reliability probability of the battery in the
vehicle. They differ in the following sense:
Model I
The aim of model I is to describe the reliability as a function of time for a vehicle
that is new. It therefore only includes features that are known or can be estimated
when the vehicle has yet not been in use.
Model II
When a vehicle has been operating for a time, more information about events and
operating conditions is available, for example how many times a certain vehicle has
called assistance service due to battery problems. Some data collected from the vehicle are also likely to be more accurate than the assumptions used in model I, such
as the ambient temperatures or the average kilometers driven per day. Treating the
time since the vehicle began operating as a feature, model II aims to describe the
remaining lifetime for a vehicle in use.
The two methods used for modeling are random forests and Cox regression. Both of
the methods can handle censored data, which most other regression methods, such
as the support vector regression, are not designed for. This is the main reason for
using these two methods.
6.1
Random forests
Random forests (RF) is an ensemble machine learning method developed by Breiman
(2001). Ensemble predictors are built from multiple base learners which can significantly improve learning performance. The RF classifier is constructed of many
27
CHAPTER 6. RELIABILITY MODELS
decision trees and it outputs the class that is the most popular among the trees.
For building each tree, a randomly drawn bootstrap sample constitutes the training
set. At each node, a subset of features is randomly chosen and a rule for splitting
is used to calculate the best split of the training set at that node.
Ishwaran et al. (2008) extended the RF algorithm for analysis of right censored
survival data. Their resulting random survival forests (RSF) algorithm is outlined
as followed:
1. Draw B bootstrap samples from the complete data set. The excluded data
set is called out-of-bag data.
2. For each sample, a survival tree is grown. Randomly select p variables at
each node. Split the node using the variable that maximizes the difference in
survival between the child nodes.
3. Each tree is grown to full size so that every leaf has at least one unique failure.
4. Calculate the cumulative hazard function for each tree. The ensemble cumulative hazard is achieved by averaging.
5. Calculate the prediction error using the out-of-bag data.
As the tree grows, each node will be populated by more similar cases since unlike
cases will be pushed apart. When the cases at each node cannot be divided without
violating the constraint that each leaf must have at least one failure, the growing
of the tree saturates.
The cumulative hazard function at a node h in a tree is estimated using the NelsonAalen estimator:
" dl,h
Ĥh (t) =
(6.1)
Y
t ≤t l,h
l,h
Above, dl,h is the number of failures and Yl,h are the number of cases at risk at
time tl,h . For a case i with feature vector xi the cumulative hazard function is the
estimation for the leaf node that xi falls into:
H(t|xi ) = Ĥh (t), if xi ∈ h
(6.2)
The ensemble cumulative hazard function is computed by averaging over all survival
trees. Let Ii,b = 0 if i belongs to the training sample for constructing tree b and
Ii,b = 1 otherwise and let Hb (t|xi ) be the cumulative hazard function for this tree.
The ensemble cumulative hazard function is computed as:
He (t|xi ) =
,B
b=1 Ii,b Hb (t|xi )
,B
b=1 Ii,b
28
(6.3)
CHAPTER 6. RELIABILITY MODELS
6.1.1
Splitting rules
At each node, one wants to find the predictor x and the split value c that maximizes
the survival difference between the two child nodes. The splitting rule if formed in
such way that if x ≤ c the case belongs to one child node and if x > c it belongs to
the other.
There is a number of splitting rules that can be used to search for the best x
and c. The most commonly used is the logrank test:
L(x, c) = -
,N %
i=1
,N
Yi,1
i=1 Yi
%
di,1 − Yi,1 Ydii
1−
Yi,1
Yi
&%
&
Yi −di
Yi −1
&
(6.4)
di
A larger value of |L(x, c)| implies a larger difference between the two groups.
Other splitting rules are the Conservation of events splitting and the Logrank score
splitting and an Approximate logrank splitting which can reduce the often heavy
computations. However, the logrank test is well established and is known for its
robustness, so further explanations of these three splitting rules are left out.
6.2
Cox regression model
Cox proportional hazard model is a technique to model the hazard function and
to analyze how different variables influence the risk of failure. The Cox model fits
survival data with features x to a hazard function of the form:
h(t, x) = h0 (t) exp(x& β)
(6.5)
where β is an unknown vector and h0 (t) is the baseline hazard. Cox (1972) proposed
the method of partial likelihood with which the Cox model can be estimated without
specifying the baseline hazard. Assume that only one unit can fail at a distinct point
in time and let
t1 < t2 < ... < tm
(6.6)
denote the observed times of failure. The risk set at time ti , Ri , is the set of indices
of the units that have not yet failed just before time ti . The conditional probability
that a particular unit j with features xj fails at time ti given the risk set Ri and
the fact that one and only one unit fails at that time is:
h(ti , xj )dt
h(ti , xk )dt
,
(6.7)
k∈Ri
or using 6.5:
h0 (ti ) exp(x&j β)
exp(x&j β)
,
=
h0 (ti ) exp(x&k β)
exp(x&k β)
,
k∈Ri
k∈Ri
29
(6.8)
CHAPTER 6. RELIABILITY MODELS
which is independent of the baseline hazard. If the conditional probabilities for all
distinct failure times are multiplied, the result is the partial likelihood proposed by
Cox:
m
'
exp(x&j β)
,
L=
(6.9)
exp(x&k β)
i=1
k∈Ri
If the failure times are discrete, the assumption that only one failure can occur
at each point in time does not hold. Cox’s model (Cox, 1972) for discrete data
formulated that the hazard is the conditional probability of failure given survival
up to that point in time:
h(t) = Pr{T = t|T ≥ t}
(6.10)
Cox’s discrete model assumes that the conditional survival odds and some baseline
odds are proportional in such way that:
h0 (t) x" β
h(t, x)
=
e
1 − h(t, x)
1 − h0 (t)
(6.11)
Cox (1972) showed that the partial likelihood of this model is identical to the partial
likelihood of the continuous model above.
6.2.1
Assumption of the Cox model
It can easily be seen from 6.5 that the hazard ratio for two observations are independent of time. Due to this, the Cox model is often referred to as a proportional-hazards
model. The assumption of proportional hazards states that if a vehicle has twice as
high risk for battery failure compared to another vehicle, it will still have so a year
later (if failure has not occurred during that year). The test if proportional hazards
hold can easily be checked using the function cox.zph in R. The mathematics behind
this function is beyond the scope of this thesis.
Another condition that must be fulfilled is the assumption of non-informative censoring, meaning that the time to failure is independent of censoring time. If this
condition is violated, the result will be biased.
6.2.2
Estimation of baseline hazard
The partial likelihood is used to estimate the regression coefficients β. To estimate
the baseline hazard h0 (t) the Kaplan-Meier method is most often used. If the
number of failures at time t(i) is denoted d(i) and Ri is the risk set at that time, the
Kaplan-Meier estimator of the baseline hazard becomes:
ĥ0 (t(i) ) = ,
k∈Ri
30
d(i)
exp(x&k β̂)
(6.12)
CHAPTER 6. RELIABILITY MODELS
The cumulative hazard function then becomes:
Ĥ0 (t(i) ) =
"
ĥ0 (t(j) )
(6.13)
j≤i
From 6.12 the baseline reliability function can be estimated as:

6.3
R̂0 (t(i) ) = exp −
"
j≤i

2
3
ĥ0 (t(j) ) = exp −Ĥ0 (t)
(6.14)
Performance evaluation
The performance evaluations of the random forests and the Cox regression models
are performed using prediction error curves. The prediction error curves are constructed by plotting the average Brier score as a function of time. The Brier score
at time t for a single subject is defined as:
BS(t, R̂) = (Yi (t) − R̂(t|Xi ))2
(6.15)
Here, Yi (t) is the observed reliability status of subject i (0 = failed at time t, otherwise 1) and R̂(t|Xi ) is the predicted reliability probability at time t. Bootstrap
cross-validation is used to compute the estimates, by splitting the data into many
training and test samples, construct reliability models from the training sets and
finally calculate the prediction error by averaging over the Brier scores from the test
data sets.
To compare the prediction error curves, the area under the curve is calculated.
This measure of the performance of the model is called the integrated Brier score.
6.4
Implementation
The random forests analysis is implemented using the RandomSurvivalForest package in R. Here, the number of trees is set to 1000. In model I, two variables are
tried at each split and in model II three variables are tried at each slit. The Cox
regression is also implemented in R using the Survival package and the prediction
error curves are produced using the PEC package in R.
6.5
Data
Model I is based on features that are either known or can be estimated before the
vehicle has started to operate. In addition to these variables, model II also includes
variables that have been stored during operation that are harder to estimate. Table
6.1 and table 6.2 show the features used in model I and II respectively.
31
CHAPTER 6. RELIABILITY MODELS
Table 6.1: Features used in model I.
Variable
Kilometers per start
Kilometers per day
Battery size
Temperature < 0 ◦ C
Kitchen equipment
Table 6.2: Features used in model II.
Variable
Days since delivery
Kilometers
Battery size
Temperature < 0 ◦ C
Kitchen equipment
Voltage 26-26.5 V
Number of starts
The censoring time is fixed and known beforehand to be two years, a so-called
type I censoring. This type of censoring is always non-informative since the censoring time does not provide any information regarding the future reliability beyond
this point in time. The Cox assumption of non-informative censoring therefore
holds. The test of the second assumption, of proportional hazards, is presented in
the result chapter.
32
Chapter 7
Results
7.1
7.1.1
Classification and feature selection
Discriminating features
Haulage vehicles
For haulage vehicles, the most discriminating feature was the number of driven
kilometers per start of the vehicle. Figure 7.1 shows the F-scores for all features for
haulage vehicles. The voltage feature denotes the fraction of time with a system
voltage within the specific interval and the temperature feature denotes the fraction
of time with ambient temperature within the specific interval.
Distribution vehicles
For distribution vehicles, the most discriminating features were the voltage and the
temperature histograms. Figure 7.2 shows how discriminative the features are for
distribution vehicles. The voltage features and the temperature features denote the
fraction of time with a system voltage respectively an ambient temperature within
the specific interval.
7.1.2
Classification results
Feature selection was applied to improve the classification results, by omitting features with F-scores less than a threshold value. This was done iteratively and the
subset of features that gave the best classification result with the RBF kernel was
selected for the SVM model. For haulage vehicles, the 18 features with highest
F-scores were selected and for distribution vehicles, the 17 features with highest
F-scores were selected.
For the RBF and the linear kernels, parameter selection with five-fold cross validation was applied to choose appropriate values of C and γ. The AUC and the
33
CHAPTER 7. RESULTS
Figure 7.1: F-scores for features for haulage vehicles.
Figure 7.2: F-scores for features for distribution vehicles.
34
CHAPTER 7. RESULTS
accuracy results together with the results from the MLR classifier are presented
below.
Haulage vehicles
For haulage vehicles, the optimal parameters for the RBF kernel were C = 2.0 and
γ = 8.0. For the linear kernel, C = 0.5 was the value giving the largest cross validation rate. The accuracy and the AUC for the haulage vehicles are shown in table
7.1.
Table 7.1: Classification performance of SVM with two kernels and the MLR for haulage vehicles.
Accuracy
AUC
RBF
0.7472
0.7054
Linear
0.7361
0.6743
MLR
0.7355
0.6663
The ROC curves for the RBF-kernel and the linear kernel are shown in figures
7.3 and 7.4 respectively.
Distribution vehicles
For distribution vehicles, the optimal parameters for the RBF kernel were C = 512
and γ = 0.0078125. Classification performance for the kernels and the MLR is presented in table 7.2.
Table 7.2: Classification performance of SVM with two kernels and the MLR for distribution vehicles.
Accuracy
AUC
RBF
0.8901
0.7695
Linear
0.8736
0.6524
MLR
0.8791
0.6333
The ROC curves for the RBF-kernel and the linear kernel are shown in figures
7.5 and 7.6 respectively.
35
CHAPTER 7. RESULTS
Figure 7.3: ROC curve for RBF-kernel for haulage vehicle. AUC=0.7054.
Figure 7.4: ROC curve for linear kernel for haulage vehicle. AUC=0.6743.
36
CHAPTER 7. RESULTS
Figure 7.5: ROC curve for RBF-kernel for distribution vehicle. AUC=0.7695.
Figure 7.6: ROC curve for linear kernel for distribution vehicle. AUC=0.6524.
37
CHAPTER 7. RESULTS
7.2
Lifetime distribution estimations
The lifetime distributions with the best fit parameters are plotted together with the
histogram of observed failures. The area under the curve from day 730 and forward
should correspond to the fraction of vehicles that are censored.
7.2.1
Haulage vehicles
Normal distribution
For the normal distribution, the parameters that best fit the data were µ = 1078.05
and σ = 434.81. The distribution is plotted together with the lifetime histogram in
figure 7.7.
Figure 7.7: Normal distribution, haulage vehicles.
Weibull distribution
For the Weibull distribution, the parameters that best fit the data were α = 1693.90
and β = 1.73. The distribution is plotted together with the lifetime histogram in
figure 7.8.
Gamma distribution
For the gamma distribution, the parameters that best fit the data were α = 1.84
and β = 1007.05. The distribution is plotted together with the lifetime histogram
in figure 7.9.
38
CHAPTER 7. RESULTS
Figure 7.8: Weibull distribution, haulage vehicles.
Figure 7.9: Gamma distribution, haulage vehicles.
Table 7.3 shows the expected lifetimes and standard deviations for the three distributions.
39
CHAPTER 7. RESULTS
Table 7.3: Expected values and standard deviations of the lifetime of batteries for haulage vehicles.
Normal
Weibull
Gamma
7.2.2
Expected value (days)
1078.05
1509.61
1841.76
Standard deviation (days)
434.81
899.22
1361.66
Distribution vehicles
Normal distribution
For the normal distribution, the parameters that best fit the data were µ = 1308.29
and σ = 527.71. The distribution is plotted together with the lifetime histogram in
figure 7.10.
Figure 7.10: Normal distribution, distribution vehicles.
Weibull distribution
For the Weibull distribution, the parameters that best fit the data were α = 2644.48
and β = 1.50. The distribution is plotted together with the lifetime histogram in
figure 7.11.
Gamma distribution
For the gamma distribution, the parameters that best fit the data were α = 1.55
and β = 1869.16. The distribution is plotted together with the lifetime histogram
in figure 7.12.
40
CHAPTER 7. RESULTS
Figure 7.11: Weibull distribution, distribution vehicles.
Figure 7.12: Gamma distribution, distribution vehicles.
Table 7.4 shows the expected lifetimes and standard deviations for the three distributions.
41
CHAPTER 7. RESULTS
Table 7.4: Expected values and standard deviations of the lifetime of batteries for distribution vehicles.
Normal
Weibull
Gamma
7.2.3
Expected value (days)
1308.29
2386.65
2899.44
Standard deviation (days)
527.71
1617.14
2328.89
Goodness of fit
The Pearson’s chi-squared test was applied to evaluate the goodness of fit of the
distributions until 730 days in use. The test was performed at the significance level
α = 0.05 with 57 degrees of freedom. The values of the chi-squared test statistic
should then be compared with χ20.95,57 = 75.62. The chi-squared values are collected
in table 7.5. From this it is clear that the inequality in expression 5.16 does not
hold for any of the values.
Another test is performed where failures the 100 first days after delivery where
omitted. This gives a degree of freedom of 49 and χ20.95,49 = 66.34. The results from
this test are shown in table 7.6. In this test, 5.16 holds for the normal distribution
for both haulage and distribution vehicles.
Table 7.5: Chi-squared test statistics calculated using 5.15.
Normal
Weibull
Gamma
Haulage
224.91
494.02
585.01
Distribution
115.65
97.44
104.08
Table 7.6: Chi-squared test statistics calculated using 5.15 with the first 100 days omitted.
Normal
Weibull
Gamma
7.3
7.3.1
Haulage
66.28
168.00
236.72
Distribution
50.11
71.13
78.91
Reliability model I
Haulage vehicles
Random forests
1000 trees are constructed with two variables tried at each split. The average number of terminal nodes in the trees is 61.41.
Figure 7.13 shows the predicted reliability for a vehicle with feature values that
42
CHAPTER 7. RESULTS
are the mean of the whole population of vehicles. Figure 7.14 show reliability functions for varying feature values, with all other variables kept constant at their mean
values.
Figure 7.13: Predicted reliability for mean haulage vehicle with RSF for model I.
Cox model
The estimated coefficients from the Cox regression along with standard errors (SE)
and confidence intervals (CI) are presented in table 7.7. The predicted reliability
for a mean vehicle is shown in figure 7.15. The baseline hazard is plotted in figure
7.16. To illustrate these results, figure 7.17 show the effects of the variables on the
reliability function.
The resulting p-values from testing any violations of the assumption of proportional hazards are presented in table 7.8. A variable that is significant in this test
(p-value < 0.05) violates this assumption. It is clear that none of the variables
violate the proportionality assumption.
Evaluation
The integrated Brier scores for RSF and Cox respectively are 0.041 and 0.048.
Figure 7.18 show the predicted error curve.
43
CHAPTER 7. RESULTS
(a) Km per start
(b) Km per day
(c) Battery
(d) Temperature
(e) Kitchen
Figure 7.14: Reliability functions from RSF model I, haulage vehicles
44
CHAPTER 7. RESULTS
Table 7.7: Estimated Cox regression coefficients for model I, haulage vehicles.
Km per start
Km per day
Battery
Temperature < 0 ◦ C
Kitchen
β̂
-0.012
0.002
0.010
0.056
0.427
exp(β̂)
0.989
1.002
1.010
1.058
1.533
SE
0.004
0.001
0.003
0.029
0.171
95% (CI)
0.981-0.996
1.001-1.003
1.004-1.016
1.000-1.120
1.097-2.141
p-value
0.002
4.871 · 10−7
0.001
0.052
0.012
Table 7.8: Result from test of proportional hazards, haulage vehicles model I.
Km per start
Km per day
Battery
Temperature < 0 ◦ C
Kitchen
p-value
0.240
0.159
0.229
0.083
0.124
Figure 7.15: Predicted reliability with confidence intervals for mean haulage vehicle with Cox for model I.
45
CHAPTER 7. RESULTS
Figure 7.16: Baseline hazard, Cox for haulage vehicle model I.
46
CHAPTER 7. RESULTS
(a) Km per start
(b) Km per day
(c) Battery
(d) Temperature
(e) Kitchen
Figure 7.17: Reliability functions from Cox model I, haulage vehicles.
47
CHAPTER 7. RESULTS
Figure 7.18: Prediction error for model I, haulage vehicles.
48
CHAPTER 7. RESULTS
7.3.2
Distribution vehicles
Random forests
1000 trees are constructed with two variables tried at each split. The average number of terminal nodes in the trees is 10.30.
Figure 7.19 shows the predicted reliability for a vehicle with feature values that
are the mean of the whole population of vehicles. Figure 7.20 show the predicted
reliability functions for different feature values.
Figure 7.19: Predicted reliability for mean distribution vehicle with RSF for model I.
Cox model
The results from the Cox regression analysis are presented in table 7.9. Figure 7.21
show the predicted reliability for a mean distribution vehicle and figure 7.22 show
the baseline hazard function. These results are illustrated in figure 7.23 where the
reliability functions are plotted for different feature values. Table 7.10 presents the
results from the proportional-hazards test, indicating that the assumption holds for
all variables.
Evaluation
The integrated Brier scores for RSF and Cox respectively are 0.025 and 0.026.
Figure 7.24 show the predicted error curve.
49
CHAPTER 7. RESULTS
(a) Km per start
(b) Km per day
(c) Battery
(d) Temperature
Figure 7.20: Reliability functions from RSF model I, distribution vehicles.
Table 7.9: Estimated Cox regression coefficients for model I, distribution vehicles.
Km per start
Km per day
Battery
Temperature < 0 ◦ C
β̂
-0.054
0.005
-0.031
0.147
exp(β̂)
0.948
1.005
0.969
1.158
SE
0.013
0.002
0.010
0.048
95% (CI)
0.924-0.972
1.002-1.009
0.950-0.989
1.054-1.272
p-value
3.511 · 10−5
0.003
0.002
0.002
Table 7.10: Result from test of proportional hazards, distribution vehicles model I.
Km per start
Km per day
Battery
Temperature < 0 ◦ C
50
p-value
0.549
0.684
0.383
0.084
CHAPTER 7. RESULTS
Figure 7.21: Predicted reliability with confidence intervals for mean distribution vehicle with Cox for model
I.
Figure 7.22: Baseline hazard, Cox for distribution vehicle model I.
51
CHAPTER 7. RESULTS
(a) Km per start
(b) Km per day
(c) Battery
(d) Temperature
Figure 7.23: Reliability functions from Cox model I, distribution vehicles.
52
CHAPTER 7. RESULTS
Figure 7.24: Prediction error for model I, distribution vehicles.
53
CHAPTER 7. RESULTS
7.4
7.4.1
Reliability model II
Haulage vehicles
Random forests
1000 trees are constructed with three variables tried at each split. The average
number of terminal nodes in the trees is 63.64. Figure 7.25 shows the predicted
reliability for vehicles that have been operating for one, one and a half and two years
respectively at t = 0, with the same value of kilometers driven per day assumed and
with the rest of the variables kept constant at mean.
Figure 7.25: Predicted reliability for three haulage vehicles using RSF model II.
Cox model
The results from the Cox regression are presented in table 7.11. Figure 7.26 show
the predicted reliability for a mean vehicle and figure 7.27 show the baseline hazard
from this model. Figure 7.28 illustrates how the reliability probability depends on
the number of days the vehicle has been in use at t = 0. From table 7.12 it is clear
that the assumption of proportional hazards holds.
Evaluation
The Brier scores for the RSF and the Cox model are 0.191 and 0.209 respectively.
Figure 7.29 shows the prediction error curves.
54
CHAPTER 7. RESULTS
Table 7.11: Estimated Cox regression coefficients for model II, haulage vehicles.
Days since delivery
Assistance
Km
Battery
Kitchen
Temperature < 0 ◦ C
Voltage 26-26.5 V
Number of starts
β̂
−1.699 · 10−3
0.517
4.634 · 10−6
5.778 · 10−3
0.291
2.490 · 10−2
2.165
1.006 · 10−3
exp(β̂)
0.998
1.677
1.000
1.006
1.339
1.025
8.719
0.999
SE
5.324 · 10−4
8.090 · 10−2
8.487 · 10−7
2.912 · 10−3
0.169
2.395 · 10−2
0.449
1.156 · 10−3
95% (CI)
0.997-0.999
1.431-1.966
1.000-1.000
1.000-1.011
0.960-1.866
0.978-1.074
3.618-21.008
0.997-1.001
p-value
0.0014
1.623 · 10−10
4.764 · 10−8
0.0472
0.085
0.299
1.391 · 10−6
0.385
Table 7.12: Result from test of proportional hazards, haulage vehicles model II.
Days since delivery
Assistance
Km
Battery
Kitchen
Temperature < 0 ◦ C
Voltage 26-26.5 V
Number of starts
p-value
0.658
0.989
0.205
0.206
0.793
0.903
0.437
0.282
Figure 7.26: Predicted reliability with confidence intervals for mean haulage vehicle with Cox for model II.
55
CHAPTER 7. RESULTS
Figure 7.27: Baseline hazard, Cox for haulage vehicle model II.
Figure 7.28: Predicted reliability for three haulage vechiles using Cox model II.
56
CHAPTER 7. RESULTS
Figure 7.29: Prediction error for model II, haulage vehicles.
57
CHAPTER 7. RESULTS
7.4.2
Distribution vehicles
Random forests
1000 trees are constructed with three variables tried at each split. The average
number of terminal nodes in the trees is 10.13. Figure 7.30 shows the predicted
reliability for vehicles that have been operating for one, one and a half and two years
respectively at t = 0, with the same value of kilometers driven per day assumed and
with the rest of the variables kept constant at mean.
Figure 7.30: Predicted reliability for three distribution vehicles using RSF model II.
Cox model
The results from the Cox regression are presented in table 7.13. Figure 7.31 show
the predicted reliability for a mean vehicle and figure 7.32 show the baseline hazard
from this model. Figure 7.33 illustrates how the reliability probability depends on
the number of days the vehicle has been in use. As before, the same value of the
kilometers per day variable is assumed for all three vehicles. Table 7.14 show that
the assumption of proportional hazards holds.
Evaluation
The Brier scores for the RSF and the Cox model are 0.093 and 0.085 respectively.
Figure 7.34 shows the prediction error curves.
58
CHAPTER 7. RESULTS
Table 7.13: Estimated Cox regression coefficients for model II, distribution vehicles.
Days since delivery
Assistance
Km
Battery
Temperature < 0 ◦ C
Voltage 26-26.5
Number of starts
β̂
4.391 · 10−4
−2.233 · 10−2
4.677 · 10−6
−2.475 · 10−2
1.178
9.774
−8.158 · 10−3
exp(β̂)
1.000
0.978
1.000
0.976
3.246
1.757 · 104
0.992
SE
1.457 · 10−3
0.398
3.436 · 10−6
1.073 · 10−2
0.787
1.873
2.760 · 10−3
95% (CI)
0.998-1.003
0.448-2.134
1.000-1.000
0.955-0.996
0.694-1.519
446.990-6.904 · 105
0.987-0.997
p-value
0.763
0.955
0.174
0.021
0.135
1.811 · 10−7
0.003
Table 7.14: Result from test of proportional hazards, distribution vehicles model II.
Days since delivery
Assistance
Km
Battery
Temperature < 0 ◦ C
Voltage 26-26.5 V
Number of starts
p-value
0.130
0.889
0.518
0.346
0.481
0.998
0.228
Figure 7.31: Predicted reliability with confidence intervals for mean distribution vehicle with Cox for model
II.
59
CHAPTER 7. RESULTS
Figure 7.32: Baseline hazard, Cox for distribution vehicle model II.
Figure 7.33: Predicted reliability for three distribution vechiles using Cox model II.
60
CHAPTER 7. RESULTS
Figure 7.34: Prediction error for model II, distribution vehicles.
61
Chapter 8
Discussion
In the classification analysis, the RBF kernel function performed best both for
haulage and distribution vehicles (70.54% and 76.95% respectively). For the MLR,
the AUC values were 66.63% and 63.33%. The SVM approach with the RBF kernel
appears to perform slightly better than the traditional regression method. Applying
the linear kernel function, the performance was slightly better than for the MLR,
although the differences were not significant (67.43% and 65.24% for haulage and
distribution respectively).
Although the AUC and the accuracy were higher for distribution vehicles, these
ROC curves have a more jagged appearance than for haulage vehicles. This is due
to the fact that the data set for distribution vehicles is smaller than for haulage
vehicles. To make the ROC curves smoother, it is possible to construct ROC curves
from several training sets and averaging.
The classification results are however a bit biased when compared since the AUC
are based on the features that give best performance for the RBF kernel. The linear
kernel SVM and the MLR might perform better when applied to another subset of
the available features. Since the total number of features is relatively small the
analysis might benefit from using the whole set of features. This would worsen the
RBF kernel performance but the comparison with the linear kernel and the MLR
method would be more accurate.
From figure 7.1 and figure 7.2 it is clear that some features are considerably more
discriminative when comparing the population of vehicles with battery failure from
those without. However, the analysis does not reveal mutual information among the
features. Two features with low F-scores might be discriminative when combined.
For example, the size of the alternator in the vehicle seems not to be a discriminative feature, but some combinations of alternator size and battery size might
be. However, the SVM with the RBF kernel performed better when omitting the
alternator variable, indicating that this is not the case for this specific example.
62
CHAPTER 8. DISCUSSION
For the maximum likelihood estimation, the results from the goodness of fit tests
implied that the hypothesis that the data followed any of the tested distributions
should be rejected. The plots from the normal distributions (figures 7.7 and 7.10)
indicated however some resemblance between the data histograms and the fitted
curves, when cutting off the lowest values. When omitting the first 100 days, the
goodness of fit test could not reject the hypothesis that the data followed the normal
distribution, neither for haulage nor distribution vehicles.
The fact that the first bars in the data histograms are relatively high might indicate that the batteries are of different quality when delivered to customer. The
batteries might also have been discharged during transportation and are replaced to
satisfy the customer. A better fit might be achieved using the t-distribution, which
is bell shaped like the normal distribution but has a probability density function
with heavier tails. However, this analysis would probably strongly benefit if the
censoring times were later than two years after delivery. Since all failure times after
two years are censored, it is unknown how the failure rate change after this point
in time. This makes the MLE analyses somewhat hypothetical. They can however
give a hint about the failure probability and the lifetime of a battery.
For the reliability models, the prediction error curves for model I (figures 7.18
and 7.24) show that RSF performed slightly better than Cox, although the differences are very small. An advantage with the Cox model is that it is much easier
to interpret. The RSF models consist of 1000 decision trees each, which makes the
models very hard to visualize and present. The Cox model, on the other hand, is
easy to visualize. It is a function (equation 6.5) that consists of a baseline function
(figures 7.16 and 7.22) and the regression coefficients (tables 7.7 and 7.9). Different
covariate values give different slope on the hazard and the reliability function.
The sign of a Cox regression coefficient indicates whether an increase in the corresponding feature value leads to an increase or a decrease in hazard, if all other
covariates are kept constant. A positive value of the coefficient leads to an increase
in hazard for if the feature value increases. The exponentiated coefficients from the
Cox regression can be interpreted as multiplicative effects on the hazard. For example, for a haulage vehicle, if all other covariates are kept constant, an additional
kilometer per day driven increases the hazard of battery failure with a factor of 0.2%.
The p-values in tables 7.7 and 7.9 show that all covariates have statistically significant coefficients except the temperature covariate for haulage vehicles, which is
slightly above the margin value (0.05). The confidence interval for this covariate
(1.000-1.120) includes 1, suggesting no difference in hazard for this value of the
exponentiated coefficients. However, the confidence interval include only positive
values, so there is reason to believe that the temperature feature has an impact on
the hazard even for haulage vehicles.
63
CHAPTER 8. DISCUSSION
For model II, most p-values in the Cox regression are large, which means that
these coefficients are not significant. This suggests that the coefficients are not important to the model. The confidence intervals in figures 7.26 and especially 7.31
also indicate that the results from Cox regressions are not credible for model II. It is
difficult to explain this result since there is no prior theory describing the expected
effects of the features on the remaining lifetime of the battery.
For model I, the impacts of the features, both for Cox and RSF, are exemplified
by plotting the reliability probability for different feature values (figures 7.14, 7.17,
7.20 and 7.23). In general, the results from Cox and RSF show the same trend
in how the features influence the reliability probability. The influence of the number of kilometers driven per start of the vehicle on the reliability has an expected
behavior. A battery in a vehicle that drives a longer distance per start seems to
survive longer, which might be due to the fact that the battery then is charged for
a longer time and gets a better recovery from the stress a start poses on the battery.
For Cox, impacts of the battery size on the survival are opposite for haulage and
distribution vehicles. For distribution vehicle, the result implies that a larger battery has a positive effect on survival, while for haulage the result can wrongly be
interpreted that a larger battery has a negative effect on survival if one wants to
avoid battery failure. This is of course not the case. A better interpretation might
be that those who choose a larger battery might be in need of an even larger one,
while those who choose a smaller battery do so because they don’t need larger.
According to the Cox models, kitchen equipment increases the hazard. In reality, the kitchen itself might not be the largest risk for battery failure. Instead, an
explanation could be that those who install kitchen equipment are more likely to
sleep in the vehicle, and keep lights and heating equipment on, hence exploiting the
battery harder than those who do not sleep in the vehicle. For RSF, neither battery
size nor kitchen equipment seems to have an influence on the reliability. Although
these features earlier were shown to be important, the RSF approach was unable to
identify them as important covariates.
64
Chapter 9
Conclusion
Aging truck batteries is a common cause of immobilization and disrupted operations, leading to fees and destroyed goods for haulage contractors. To reduce the
risk of unplanned breakdowns the replacements of batteries should be performed
preventive. Since different technical specifications and operational conditions can
affect the risk of battery failure, these individual characteristics must be included
to make an accurate estimation of when the battery is likely to fail.
In this master’s thesis, the effect of different features on failure time probability
is investigated. To evaluate whether the available data were incorporating differences between groups of vehicles with or without a history of battery failure, an
SVM classifier was used. The results from the classification tests showed that the
data could distinguish the groups to some extent, although far from perfectly. The
SVM performed at least as good as a traditional statistical MLR approach.
The lack of data was the biggest challenge in this project. The data were not
on a format that motivated a standard time series analysis, like a Markov model.
Since there were only useful data covering the first two years of a vehicle’s lifetime,
it was necessary to apply a method that could handle censored data, and other
methods, like support vector regression, were disqualified. Cox regression is the
most common method for failure analysis of censored data. For comparison, the
machine learning method random forests was applied.
The first model attempted to estimate the probability for the battery to not fail as
function of time for a new battery. The results from this analysis were satisfying.
The values of the Cox regression coefficients give a good insight of how different
features affect the expected lifetime. The second model attempted to estimate the
future probability for the battery to not fail as function of time for the remaining
time in use, treating age of the battery as a covariate. The second model did not
give credible results, having non-significant regression coefficients.
65
CHAPTER 9. CONCLUSION
To construct a model that could accurately estimate the remaining lifetime of a
battery with a statistical or a machine learning approach, more data are needed.
First, data should cover whole lifetimes of batteries, or the censor time should at
least be longer in relation to the expected lifetime. Second, the estimations would
benefit if the data were time stamped and not accumulated and stored only when
the vehicle visits a workshop. Third, the features collected should be more relevant
in explaining the condition and the usage of the battery. For example, it would be
of use to have a measure of the consumption of electrical energy in the vehicle, both
during drive and when the vehicle is standing still. For future work, it is suggested
to look over the data collected and stored before further analyses in the topic are
made.
66
Bibliography
Barakat, N., Bradley, A.P. 2006. Rule extraction from support vector machines:
Measuring the explanation capability Using the area under the ROC curve. Proceedings of the 18th International Conference on Pattern Recognition 2:812-815.
Breiman, L. 2001. Random forests. Machine Learning 45:5-32.
Chang, C.C., Lin, C.J. 2011. LIBSVM: a library for support vector machines,
software available at http://www.csie.ntu.edu.tw/ cjlin/libsvm.
Chen, Y.W., Lin, C.J. 2006. Combining SVMs with various feature selection strategies. In Isabelle Guyon, Steve Gunn, Masoud Nikravesh, and Lofti Zadeh, editors,
Feature extraction, foundations and applications. Springer.
Cox, D.R. 1972. Regression models and life tables (with discussion). Journal of
the Royal Statistical Society 34:187—220.
Dorner, R., et al. 2005. Specific accelerated test procedure for pv batteries with
easy transfer to various kinds of systems and for quality control. European Project
ACTUS.
Ishwaran, H., et al. 2008. Random survival forests. The Annals of Applied Statistics 2(3):841-860.
Jaworski, R.K. 1999. Statistical parameters model for predicting time to failure
of telecommunications batteries. The 21st International Telecommunication Energy Conference, INTELEC.
R Development Core Team. 2008. R: A language and environment for statistical
computing, http://www.R-project.org. R Foundation for Statistical Computing.
Reasbeck, P., Smith, J.G. 1997. Batteries for automotive use. Research studies
press LTD.
Ruetschi, P. 2004. Aging mechanisms and service life of lead-acid batteries. Journal
of Power Sources 127:33-44.
67
BIBLIOGRAPHY
Tian, Z. 2009. An artificial neural network method for remaining useful life prediction of equipment subject to condition monitoring. Journal of Intelligent Manufacturing 23:227-237.
Saha, B., et al. 2007. An integrated approach to battery health monitoring using Bayesian regression and state estimation. Autotestcon, 2007 IEEE, 646-653.
Saha, B., et al. 2009. Comparison of prognostic algorithms for estimating remaining useful life of batteries. Transactions of the Institute of Measurement and
Control, 31(3-4):293-308.
Sauer, D.U., Wenzl, H. 2007. Comparison of different approaches for lifetime prediction of electrochemical systems - Using lead-acid batteries as example. Journal
of Power Sources 176:534-546.
Si, X., et al. 2011. Remaining useful life estimation - A review on the statistical data driven approaches. European Journal of Operational Research 213:1-14.
Svoboda, V. 2004. Definition of performance requirements for energy storage systems in each category of use, deliverable D 3.1 of the Benchmarking project (project
funded in part by the European Union).
Vapnik, V. 1998. Statistical Learning Theory. John Wiley and Sons.
Wang, W., Christer, A.H. 2000. Towards a general condition based maintenance
model for a stochastic dynamic system. Journal of the Operational Research Society 51:145-155.
Yang, B.S., Widodo, A. 2008. Support vector machine for machine fault diagnosis and prognosis. Journal of System Design and Dynamics 2:12-23.
68
www.kth.se
Powered by TCPDF (www.tcpdf.org)