Fundamentals of Applied Probability
and Random Process
Oliver C. Ibe
Elsevier
Academic press
목차
1.
2.
3.
4.
5.
6.
7.
8.
기본적인 확률 개념
랜덤변수
랜덤변수의 모멘트
특별한 확률분포 함수들
다중 랜덤 변수
랜덤 변수의 함수
변환방법
랜덤과정
1. 기본적인 확률 개념
1.1 개요
 1.1 Introduction
o Probability(확률) 이란 unpredictability(불확실성) 과 randomness(임의성)을 가지고 있음.
o Probability theory(확률이론) : the branch of mathematics that is concerned with the study of
random phenomena(임의의 현상).
 Under repeated observation, outcomes are not deterministically predictable.
 However, outcomes obey certain conditions of statistical regularity.
( Relative frequency(상대빈도) of occurrence of the possible outcomes is predictable.)
o Example
- the
- the
- the
- the
#
#
#
#
of
of
of
of
e-mail messages received by all employees of a company in one day
phone calls arriving at the university’s switchboard over a given period
components of a system that fail within a given interval
A’s that a student can receive in one academic year
o probability of the event(사건) = the frequency of occurrence of the event in a long sequence
 real number between zero and one,
 the sum of the probabilities should sum to one.
Basic Probability Concepts
1.2 Sample Space and Events (표본공간과 사건)
 1.2 Sample Space(표본공간) and Events(사건)
o experiments (실험) : any process of trial and observation(관찰 및 시행과정)
 random experiments(랜덤 실험) : an experiment whose outcome is uncertain
o sample space (표본공간) : the collection of possible of elementary outcomes
S  w1 , w2 ,, wn 
 Sample space
 Sample points (표본점)
o event (사건) : any one of a number of possible outcomes of an experiment
 a subset of the sample space
Elementary outcomes
o example
- if we toss a die, the sample space is  S  1,2,3,4,5,6
- the outcome of the toss of a die is an even number 
- three coin-tossing experiment S
E  2,4,6
 HHH , HHT , HTH , HTT , THH , THT , TTH , TTT 
- the event “one head and two tails” 
E  HTT , THT , TTH 
Basic Probability Concepts
1.2 Sample Space and Events
 1.2 Sample Space and Events - 예시
o In a single coin toss experiment,
- sample space (표본공간)  S  H , T 
- the event (사건) that a head appears on the toss 
- the event that a tail appears on the toss  E  T 
E  H 
o If we toss a coin twice
- sample space  S  HH , HT , TH , TT 
- the event that a tail appears on the 2nd toss  E  HT , TT 
o If we measure the lifetime of an electronic component
- Sample space 
S  x | 0  x  
- the event that the lifetime is not more than 7 hours  E  x | 0  x  7
o If we toss a die twice
- sample space  S  (1,1)(1,2)(1,3)(1,4)(1,5)(1,6)(2,1)(2,2)  (6,5)(6,6)
- the event that the sum of the two tosses is 8  E  (2,6)(3,5)(4,4)(5,3)(6,2)
Basic Probability Concepts
1.2 Sample Space and Events
 1.2 Sample Space and Events – algebra of events
o union(합) of events A and B
C  A B
o difference(차) of events A and B  C  A  B
o 사건의 계산과정은 set theory(집합이론)과 유사
intersection of events A and B
D  A B
Basic Probability Concepts
1.3 Definition of Probability (확률의 정의)
 3가지로 정의하는 방법이 있음 : axiomatic, relative-frequency, classical definitions
 공리적 정의
 상대빈도에 의한 정의
 고전적 정의
 Probability란?
- 표본공간 S 에서 정의된 각각의 사건에 대하여 음수가 아닌 숫자를 부여.
- Probability is a function : 정의된 사건의 함수
- P(A) : 사건 A의 확률
 1.3.1 Axiomatic Definition (공리적인 정의)
Axiom 1
0  P( A)  1
; work with nonnegative numbers
Axiom 2
P( S )  1
; sample space itself is an event,
it should have the highest probability
N  N
PU An    P( An )
if Am  An  
 n 1  n 1
For all m  n = 1, 2, …, N with N possibly infinite
; the probability of the event equal to the union of any number of
mutually exclusive events is equal to the sum of the individual event probabilities.
Axiom 3
Basic Probability Concepts
Review
 함수란 무엇인가?
◈ 함수에 대한 일반적인 그림
A
정의역
(domain)
•
x
B
치역
(range)
•
y
A의 원소  독립변수
B의 원소  종속변수
공역
(codomain)
f : A → B : 집합 A에서 B로 가는 대응이 있을 대 A의 원소에 B의 원소가 하나씩만 대응하면
이 대응을 집합 A에서 B로 가는 함수(function)이라 함
◈ Question: 정의역의 원소가 하나씩만 대응되지 않는 경우를 고려하고, 이 경우 함수인가 아닌가를 판별하시오.
◈ 일대일(one-to-one) 대응이란?
Basic Probability Concepts
1.3 Definition of Probability
 Definition of probability – Example
 unbiased
◈ Obtaining a number x by spinning the pointer on a “fair” wheel of chance(회전판) that is
labeled from 0 to 100 points.
S  {0  x  100}
- Sample space
- The probability of the pointer falling between any two numbers
x2  x1
( x2  x1 ) / 100
- Consider events
Axiom 1
0  P( A)  1 A  {x1  x  x2 }
Axiom 2
P( S )  1
Axiom 3
Break the wheel’s periphery into N continuous segments, n=1,2,…N with x0=0
for any
; for all x1, x2
x2  100 and x1  0
An  {xn1  x  xn }, xn  (n)100 / N
P( An )  1 / N
N
1
N  N
PU An    P( An )    1  P( S )
 n 1  n 1
n 1 N
Basic Probability Concepts
1.3 Definition of Probability
 1.3.2 relative frequency(상대빈도) definition
◈ Probability as a relative frequency : 직관을 이용하여 정의하는 방법
- Flip a coin : heads show up nA times out of the n flips
- Probability of the event “heads”
n 
P ( A)  lim  A 
n  n 
- Statistical regularity : relative frequency approach to a fixed value (a probability) as n
becomes large.
 1.3.3 Classical Definition
o Probability as a classical definition
P ( A) 
NA
N
 This probability is determined a priori without actually performing the experiment.
주사위 던지기 (Die or Dice Tossing)
▣ Dice:
A die generally has six sides. To determine the amount of possible
outcomes, take 6 to the power of the amount of rolls.
Your ODDS of getting a number you want is defined as:
(success)/(failure)
If you wanted one six tossing one die:
success: 6 ~> one chance
failure: 1, 2, 3, 4, 5 ~> five chances
1:5 or 1/5
Remember, this is not your probability. This is yours ODDS.
Probability is defined as: (success)/(success + failure)
If you wanted one six tossing one die:
success: 6 ~> one chance
success + failure: 1, 2, 3, 4, 5, 6 ~> six chances
1/6 or about 0.1666 or about 17%
Now, find the ODDS of getting a number less than four tossing one
die:
success: 1, 2, 3 ~> three chances
failure: 4, 5, 6 ~> three chances
3:3 or 3/3 *You can not take 3/3 to equal 1 because you do not
have a 100% of getting what you want.
주사위 던지기 (Die or Dice Tossing)
▣ Dice:
Tossing a die twice: 6^2 = 36 possible outcomes
What is the probability of getting a six in a row?
Success: (6,6) there is only one possible way of getting a six in a
row
Success + Failure: 36
1/36 or about 0.0277 or about 3%
Another way:
Chances of getting a six ~> (1/6)
아무 숫자든 관계없음
Chances of getting another six ~> (1/6)
(1/6)(1/6) = 1/36
What is the probability of getting a number twice?
Chances of getting a number ~> (6/6)
Chances of getting that number ~> (1/6)
(6/6)(1/6) = 1/6
What is the probability of getting a toss of two dice with the sum of
3?
Number of possible outcomes = 6^2 = 36
Success: (1,2) and (2,1) ~> two chances
2/36 = 1/18 or or about 0.055555 or about 6%
주사위 던지기 (Die or Dice Tossing)
▣ Dice:
Here is an easy made list for tossing two dice:
SUM[2] = 1
SUM[3] = 2
SUM[4] = 3
SUM[5] = 4
SUM[6] = 5
SUM[7] = 6
SUM[8] = 5
SUM[9] = 4
SUM[10] = 3
SUM[11] = 2
SUM[12] = 1
Notice that the best possible sum is: (highest sum + lowest sum)/2
Rolling two die: (12 + 2)/2 = 14/2 = 7 The best sum is seven.
This is featured as: f(x) = -|x - 7| + 6 where ‘x’ is the sum of the
dice and ‘y’ is the chances of getting that sum
주사위 던지기 (Die or Dice Tossing)
The number of possible outcomes tossing three dice: 6^3 = 216
What is the probability of getting a sum of 5?
Success: (1,1,3) ; (1,3,1) ; (3,1,1) ; (2,2,1) ; (2,1,2) ; (1,2,2) ~> 6 chances
6/216 = 1/36 or about 0.027777 or about 3%
Best chances of getting a sum: (18 + 3)/2 = 21/2 = 10.5
Since you can not get a sum of 10.5, you would choose the integers on both sides.
The best chances would be either a sum of 10 or a sum of 11.
Here is an easy made list for tossing three dice:
SUM[3] = 1
SUM[4] = 3
SUM[5] = 6
SUM[6] = 10
SUM[7] = 15
SUM[8] = 21
SUM[9] = 28
SUM[10] = 36
SUM[11] = 36
SUM[12] = 28
SUM[13] = 21
SUM[14] = 15
SUM[15] = 10
SUM[16] = 6
SUM[17] = 3
SUM[18] = 1
Equation: f(x) = 0.5( (10.5 - |x - 10.5|)^2 - 3(10.5 - |x - 10.5|) + 2)
~ where x is the sum of die
주사위 던지기 (Die or Dice Tossing)
Chances of getting a sum of 11 by tossing three dice?
36/216 = 1/6 or about 17%
*By reading what you did, try solving these:
5개의 주사위를 던져 모두 같은 숫자가 나올 확률
What is the chances of getting a Yahtzee on the first roll?
.
(6/6)(1/6)(1/6)(1/6)(1/6) = 1/1296 or about 0.0007716049 or about
0.077%
=6^4
What is the chances of getting a sum of 6 by tossing two dice?
.
success: 5 chances
total possibilities: 36
5/36 or about 0.1388 or about 14%
두개의 주사위도 종류가 다를 수 있다.
주사위를 한번 던지면 이전의 데이터가 reset 된다면?
이전 data와의 동질성과 이질성은?
Basic Probability Concepts
1.3 Definition of Probability
 예제1.1 – 주사위 두 개 던지기
Figure 1.1 Sample Space for Example 1.1
A1
Second Die
6
C
(1,6)
(2,6)
(3,6)
(4,6)
(5,6)
(6,6)
5
(1,5)
(2,5)
(3,5)
(4,5)
(5,5)
(6,5)
4
(1,4)
(2,4)
(3,4)
(4,4)
(5,4)
(6,4)
(1,3)
(2,3)
(3,3)
(4,3)
(5,3)
(6,3)
(1,2)
(2,2)
(3,2)
(4,2)
(5,2)
(6,2)
(5,1)
(6,1)
3
2
1
(1,1)
(2,1)
1
2
(3,1)
(4,1)
3
4
5
A2
- Sample space : 62=36 points
- Each possible outcome
-> a sum having values from 2 to 12
A1  {sum  7}, A2  {sum  11}
B  {sum  7}or{sum  11}
C  {2 nd die  1st die},
D  {both dice  even}
6
First Die
 1  1
P( A1 )  6   ,
 36  6
P(C ) 
15 5
 ,
36 12
 1  1
P( A2 )  2  
 36  18
P( D) 
9 1

36 4
P( B)  P( A1 )  P( A2 ) 
2
,
9
Basic Probability Concepts
1.3 Definition of Probability
 Summary - Mathematical model of Experiments (실험의 수학적 모델)
◈ A real experiment is defined mathematically by three thing
1. Assignment of a sample space
2. Definition of events of interest
3. Making probability assignment to the events such that the axioms are satisfied
◈ 올바른 모델을 세우는 것은 쉽지 않음
 끝이 살짝 뭉개진 주사위를 생각해 보자.
Basic Probability Concepts
1.3 Definition of Probability
 Exercise – conditional probability
◈ Example
- 80 resistors in a box : 10W -18, 22W -12, 27W -33, 47W -17, draw out one resistor,
equally likely
P(draw 10W)  18 / 80 P(draw 22W)  12 / 80
P(draw 27W)  33 / 80 P(draw 47W)  17 / 80
- Suppose a 22W is drawn and not replaced. What are not the probabilities of drawing a
resistor of any one of four values?
P(draw 10W | 22W)  18 / 79
P(draw 22W | 22W)  11 / 79
P(draw 27W | 22W)  33 / 79
P(draw 47W | 22W)  17 / 79
The concept of conditional probability is needed.
Basic Probability Concepts
1.4 Applications of Probability
 1.4.1 Reliability Engineering
 time to failure of the system
 1.4.2 Quality Control (QC)
 한 개의 표본을 임의 선정 후 표본에 대한 여러 항목을 테스트
 불량 여부의 판정은 선택된 표본의 테스트 항목의 결과에 의해 좌우됨.
Basic Probability Concepts
1.4 Applications of Probability
 1.4.3 Channel Noise
Noise
Message
Source
Channel
Sink
 1.4.4 System simulation
 random # generation that can be used to represent events – such as arrival of customers
at a bank – in the system being modeled
 Simulation
- 시스템 내 개별소자의 개별 사건에 의해 묘사
- 모델은 여러 개별 소자간의 상호관계를 포함
- 서로간의 상호작용의 효과는 dynamic process 에 포함
Basic Probability Concepts
1.5 기초집합이론
 Definitions
◈ Set (집합) : a collection of objects – A ( 대문자 사용)
◈ Objects : Elements (원소) of the set – a (소문자 사용)
◈ If a is an element of set A :
a A
◈ If a is not an element of set A :
a A
◈ Methods for specifying a set
1. Tabular method (열거법)
o Ex) {6, 7, 8, 9}
2. Rule method (규칙법)
o Ex) {integers between 5 and 10}, {I | 5 < I < 10, I an integer}
◈ Set
- Countable, uncountable : 가산집합, 비가산집합
- Finite, infinite : 유한집합, 무한집합
- Null set(=empty) : Ø  a subset of all other sets : 공집합
- countably infinite : 셀 수 있는 원소들을 가진 무한 집합
Ex) -
a
a
a
a
set
set
set
set
of voltages
of airplanes
of chairs
of sets (집합의 집합)
Basic Probability Concepts
Set Definitions
 Definitions
◈ A is a subset (부분 집합) of B
- If every element of a set A is also an element in another set B, A is said to be contained in B.
A B
◈ A is a proper subset (진부분 집합) of B
- If at least one element exists in B which is not in A
A B
◈ Two sets, A and B are called disjoint (배반) or mutually exclusive (상호배타적) if they have no common
elements
- disjoint : 공통요소를 갖지 않는 집합 A  B  
Basic Probability Concepts
Set Definitions
 Example
◈
◈
◈
◈
◈
◈
◈
A  {1,3,5,7}
D  {0.0}
B  {1,2,3, }
E  {2,4,6,8,10,12,14}
C  {0.5  c  8.5}
F  {5.0  f  12.0}
A : Tabularly specified, countable, and finite
B : Tabularly specified, countable, and infinite
C : Rule-specified, uncountable, and infinite
D and E : Countably finite
F : Uncountably infinite
D is the null set? 
A is contained in B, C, and F
C  F , D  F and E  B
◈ B and F are not subsets of any of the other sets or of each other
◈ A, D, and E are mutually exclusive of each other
Basic Probability Concepts
Set Definitions
 Definitions
Sample space in probability
◈ Universal set (전체집합) : S
- The largest set or all-encompassing set of objects under discussion in a given situation
◈ Example 1.1-2
- Rolling a die (주사위 던지기)
o S={1,2,3,4,5,6}
o A person wins if the number comes up odd : A={1,3,5}
o Another person wins if the number shows four or less : B={1,2,3,4}
o Both A and B are subset of S
- For any universal set with N elements, there are 2N possible subsets of S
o Example : Token
S = {T, H}
{}, {T}, {H}, {T,H}
o Example : Token 2번 던지기
S = {TT, HT, TH, HH}
24=16개의 subsets이 존재
o Example : 주사위 던지기
S = {1, 2, 3, 4, 5, 6}
26=64개의 subsets이 존재
Basic Probability Concepts
Set Definitions - 문제풀이
 Problems
◈ Specify the following sets by the rule method.
A={1,2,3}
->
A={0 < integers < 4}
B={8,10,12,14} ->
B={6 < even integers <16}
C={1,3,5,7,…} ->
C={0 < odd integers}
◈ State every possible subset of the set of letters {a,b,c,d}
{}, {a}, {b}, {c}, {d}, {a,b}, {a,c}, {a,d}, {b,c}, {b,d}, {c,d}, {a,b,c}, {a,b,d}, {a,c,d}. {b,c,d},
{a,b,c,d} -> 총 16개의 부분집합
◈ A random noise voltage at a given time may have any value from -10 to 10V.
(a) What is the universal set describing noise voltage?
-> S={-10≤s≤10}
(b) Find a set to describe the voltages available from a half-rectifier for positive voltages that has a
linear output-input characteristic.
-> V={0≤s≤10}
(c) Repeat parts (a) and (b) if a DC voltage of -3V is added to the random noise.
-> S={-13≤s≤7}, V={0≤s≤7}
Basic Probability Concepts
Set Operations
 Venn Diagram (벤 다이어그램)
Sample space(확률에서)
C is disjoint from both A and B
B is a subset of A
 Equality (등가) : A=B
◈ Two sets are equal if all elements in A are present in B and all elements in B are in A
◈ That is, if
A  B and
B A
AB
 Difference (차) : A-B
◈ The difference of two sets A and B is the set containing all elements of A that are not present in B
◈ Example
A  {0.6  a  1.6}, B  {1.0  b  2.5}
A  B  {0.6  a  1.0}, B  A  {1.6  b  2.5}
A B  B  A
Basic Probability Concepts
Set Operations
 Union(합집합) and intersection(교집합)
◈ Union (Sum) : C  A  B
- The union (call it C) of two sets A and B
- The set of all elements of A or B or both
◈ Intersection (Product) : D  A  B
- The intersection (call it D) of two sets A or B
- The set of all elements common to both A and B
- For mutually exclusive (M.E.) sets A and B,
A B  Ø
◈ The union and intersection of N sets An, n=1,2,…,N
C  A1  A2 
AN 
N
An ,
D  A1  A2 
n 1
AN 
N
An
n 1
 Complement(여집합)
◈ The complement of the set A is the set of all elements not in A
A S A
  S , S  , A  A  S , and A  A  
Basic Probability Concepts
Set Definitions
 Example
S  {1  integers  12}
A  {1,3,5,12}
B  {2,6,7,8,9,10,11}
C  {1,3,4,6,7,8}
◈ Union (Sum) and Intersection (Product)
A  B  {1,2,3,5,6,7,8,9,10,11,12}
A B  
A  C  {1,3,4,5,6,7,8,12}
A  C  {1,3}
B  C  {1,2,3,4,6,7,8,9,10,11}
B  C  {6,7,8}
◈ Complement (보집합)
A  {2,4,6,7,8,9,10,11}
B  {1,3,4,5,12}
C  {2,5,9,10,11,12}
Basic Probability Concepts
Set Operations
 Duality Principle (쌍대성 원리)
◈ If in an identity we replace unions by intersections, intersections by unions,
A  ( B  C )  ( A  B)  ( A  C )
A  ( B  C )  ( A  B)  ( A  C )
◈ Example
A  {1,2,4,6}
B  C  {2, 3  c  4, 6,8,10}
B  {2,6,8,10}
A  B  {2,6}
C  {3  c  4}
A  C  {4}
A  ( B  C )  {2,4,6}
( A  B)  ( A  C )  {2,4,6}
 A  ( B  C )  ( A  B)  ( A  C )
Basic Probability Concepts
Set Operations
 Algebra of Sets
◈ Commutative law (교환법칙)
A B  B  A
A B  B  A
◈ Distributive law (분배법칙)
A  ( B  C )  ( A  B)  ( A  C )
A  ( B  C )  ( A  B)  ( A  C )
◈ Associative law (결합법칙)
( A  B)  C  A  ( B  C )  A  B  C
( A  B)  C  A  ( B  C )  A  B  C
Basic Probability Concepts
Set Operations
 De Morgan’s law
◈ The complement of a union (intersection) of two sets A and B equals the intersection (union) of the
complements A and B
( A  B)  A  B
( A  B)  A  B
◈ Example
S  {2  s  24}
A  {2  a  16}, B  {5  b  22}
C  A  B 
A  B  {5  c  16}
C  A  B  {2  c  5, 16  c  24}
A  S  A  {16  a  24},
B  S  B  {2  a  5, 22  a  24}
C  A  B  {2  c  5, 16  c  24}
 ( A  B)  A  B
Basic Probability Concepts
Set Operations - 문제풀이
 Problems
◈ Show that C ⊂ A if C ⊂ B and B ⊂ A.
-> Ven diagram으로 설명할 것
◈ Two sets are given by A={-6, -4, -0.5, 0, 1.6, 8} and B={-0.5,0,1,2,4}. Find:
(a) A-B
->
{-6, -4, 1.6, 8}
(b) B-A
->
{1, 2, 4}
(c) A∪B
->
{-6, -4, -0.5, 0, 1, 1.6, 2, 4, 8}
(d) A∩B
->
{-0.5, 0}
◈ 1.2-4. Using Venn diagrams for three sets A,B,C, shade the areas corresponding to the sets:
(a) (A∪B)-C
(b) A-B
(d) C-(AＵB)
Basic Probability Concepts
Set Operations - 문제풀이
 Problems
◈ Sketch a Venn diagram for three events where A∩B≠0, B∩C≠0, C∩A≠0, but A∩B∩C=0.
◈ Show the equations of Venn diagrams
C-(AＵB)
(AＵBＵC)-(A∩B∩C)
( A  B  C)
◈ Sets A={1≤s≤14}, B={3,6,14}, and C={2<s≤9} are defined on a sample space S. State if each of the
following conditions is true or false.
(a) C⊂B False
(b) C⊂A  True
(c) B∩C=0  False
(d) CUB=S  False
(e) S    True
(f) A  S    True
(g) C  A  B  False
Basic Probability Concepts
1.6 Properties of Probability (확률의 특성)
 Properties of Probability
1. The probability of the complement of A is one minus the probability of A.
2. The null event has probability zero.
P( A )  1  P( A)
P()  0
3. If A is a subset of B, the probability of A is at most the probability of B.
A  B  P( A)  P( B)
4. P(A)≤1  the probability of event A is at most 1.
5.
6.
N  N
PU An    P( An )
if Am  An  
 n 1  n 1
A  ( A  B)  ( A  B )  P( A)  P( A  B)  P( A  B )
7. P( A  B)  P( A)  P( B)  P( A  B)
 결합확률(Joint Probability) P( A  B)
8. Property 7의 일반화
9. P( A  B)  P( A)  P( B)  P( A  B)  P( A)  P( B)
The probability of the union of two events never exceeds the sum of the event probabilities.
(두 사건의 합집합의 확률은 각 사건 확률의 합을 절대로 넘지 않는다.)
Mutually exclusive한 두개의 확률은 결합확률은 zero임
End of Slide