What is utility? The Utility Function and Random Neural Networks Quality assesment and . . . Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . Martı́n Varela Discussion [email protected] PRIXNC – T Workshop – March 2003 Home Page Title Page JJ II J I Page 1 of 16 Go Back Full Screen Close Quit Outline What is utility? • What is utility? Quality assesment and . . . Real–time multimedia . . . • Quality assesment and utility • Real–time multimedia quality assesment: a quick primer Real–time multimedia . . . A first approach to . . . Discussion • Real–time multimedia quality assesment: the neural approach • A first approach to utility evaluation Home Page • Discussion Title Page JJ II J I Page 2 of 16 Go Back Full Screen Close Quit Outline What is utility? • What is utility? Quality assesment and . . . Real–time multimedia . . . • Quality assesment and utility • Real–time multimedia quality assesment: a quick primer Real–time multimedia . . . A first approach to . . . Discussion • Real–time multimedia quality assesment: the neural approach • A first approach to utility evaluation Home Page • Discussion Title Page JJ II J I Page 2 of 16 Go Back Full Screen Close Quit Outline What is utility? • What is utility? Quality assesment and . . . Real–time multimedia . . . • Quality assesment and utility • Real–time multimedia quality assesment: a quick primer Real–time multimedia . . . A first approach to . . . Discussion • Real–time multimedia quality assesment: the neural approach • A first approach to utility evaluation Home Page • Discussion Title Page JJ II J I Page 2 of 16 Go Back Full Screen Close Quit Outline What is utility? • What is utility? Quality assesment and . . . Real–time multimedia . . . • Quality assesment and utility • Real–time multimedia quality assesment: a quick primer Real–time multimedia . . . A first approach to . . . Discussion • Real–time multimedia quality assesment: the neural approach • A first approach to utility evaluation Home Page • Discussion Title Page JJ II J I Page 2 of 16 Go Back Full Screen Close Quit Outline What is utility? • What is utility? Quality assesment and . . . Real–time multimedia . . . • Quality assesment and utility • Real–time multimedia quality assesment: a quick primer Real–time multimedia . . . A first approach to . . . Discussion • Real–time multimedia quality assesment: the neural approach • A first approach to utility evaluation Home Page • Discussion Title Page JJ II J I Page 2 of 16 Go Back Full Screen Close Quit Outline What is utility? • What is utility? Quality assesment and . . . Real–time multimedia . . . • Quality assesment and utility • Real–time multimedia quality assesment: a quick primer Real–time multimedia . . . A first approach to . . . Discussion • Real–time multimedia quality assesment: the neural approach • A first approach to utility evaluation Home Page • Discussion Title Page JJ II J I Page 2 of 16 Go Back Full Screen Close Quit 1. What is utility? We can define it as a monetary evaluation of the perceived service quality. What is utility? Quality assesment and . . . Real–time multimedia . . . Real–time multimedia . . . U : (Q) −→ –C such that U(q) = xC – A first approach to . . . Discussion Home Page Title Page JJ II J I Page 3 of 16 Go Back Full Screen Close Quit 1. What is utility? We can define it as a monetary evaluation of the perceived service quality. What is utility? Quality assesment and . . . Real–time multimedia . . . Real–time multimedia . . . U : (Q) −→ –C such that U(q) = xC – We’ll start with the following premises A first approach to . . . Discussion Home Page Title Page • the potential customer will only buy the service if he feels its utility to be higher than its price. JJ II • U is unknown and subjective. J I Page 3 of 16 Go Back Full Screen Close Quit 1. What is utility? We can define it as a monetary evaluation of the perceived service quality. What is utility? Quality assesment and . . . Real–time multimedia . . . Real–time multimedia . . . U : (Q) −→ –C such that U(q) = xC – We’ll start with the following premises A first approach to . . . Discussion Home Page • the potential customer will only buy the service if he feels its utility to be higher than its price. JJ II • U is unknown and subjective. J I Title Page Page 3 of 16 Go Back Full Screen Close Quit 2. Quality assesment and utility Very roughly, we could define U like What is utility? Quality assesment and . . . U(Q) = αQβ Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . Discussion Home Page Title Page JJ II J I Page 4 of 16 Go Back Full Screen Close Quit 2. Quality assesment and utility Very roughly, we could define U like What is utility? Quality assesment and . . . U(Q) = αQβ We could then conduct some tests, and try to fit U to the results. Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . There are a couple of serious problems with this approach • we don’t know Q, and we know it depends on several parameters • we don’t know exactly how to conduct the tests for a general case Discussion Home Page Title Page JJ II J I Page 4 of 16 Go Back Full Screen Close Quit 2. Quality assesment and utility Very roughly, we could define U like What is utility? Quality assesment and . . . U(Q) = αQβ We could then conduct some tests, and try to fit U to the results. Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . There are a couple of serious problems with this approach • we don’t know Q, and we know it depends on several parameters • we don’t know exactly how to conduct the tests for a general case Discussion Home Page Title Page JJ II J I Page 4 of 16 Go Back Full Screen Close Quit 2. Quality assesment and utility Very roughly, we could define U like What is utility? Quality assesment and . . . U(Q) = αQβ We could then conduct some tests, and try to fit U to the results. Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . There are a couple of serious problems with this approach • we don’t know Q, and we know it depends on several parameters • we don’t know exactly how to conduct the tests for a general case Discussion Home Page Title Page JJ II J I Page 4 of 16 Go Back Full Screen Close Quit There are two main problems associated with measuring the perceived quality What is utility? Quality assesment and . . . • the perceived quality is, by definition, subjective, and thus it’s hard to measure • it depends on the application considered (think bulk transfers vs. real–time data) Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . Discussion Home Page Solution −→ we’ll consider only multimedia applications for the time being. Why? −→ because we already know how to measure the perceived quality in this case. Title Page JJ II J I Page 5 of 16 Go Back Full Screen Close Quit There are two main problems associated with measuring the perceived quality What is utility? Quality assesment and . . . • the perceived quality is, by definition, subjective, and thus it’s hard to measure • it depends on the application considered (think bulk transfers vs. real–time data) Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . Discussion Home Page Solution −→ we’ll consider only multimedia applications for the time being. Why? −→ because we already know how to measure the perceived quality in this case. Title Page JJ II J I Page 5 of 16 Go Back Full Screen Close Quit There are two main problems associated with measuring the perceived quality What is utility? Quality assesment and . . . • the perceived quality is, by definition, subjective, and thus it’s hard to measure • it depends on the application considered (think bulk transfers vs. real–time data) Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . Discussion Home Page Solution −→ we’ll consider only multimedia applications for the time being. Why? −→ because we already know how to measure the perceived quality in this case. Title Page JJ II J I Page 5 of 16 Go Back Full Screen Close Quit There are two main problems associated with measuring the perceived quality What is utility? Quality assesment and . . . • the perceived quality is, by definition, subjective, and thus it’s hard to measure • it depends on the application considered (think bulk transfers vs. real–time data) Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . Discussion Home Page Solution −→ we’ll consider only multimedia applications for the time being. Why? −→ because we already know how to measure the perceived quality in this case. Title Page JJ II J I Page 5 of 16 Go Back Full Screen Close Quit 3. Real–time multimedia Quality assesment: a quick primer What is our context? What is utility? Quality assesment and . . . Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . Discussion Home Page Title Page JJ II J I Page 6 of 16 Go Back Full Screen Close Quit 3. Real–time multimedia Quality assesment: a quick primer What is our context? What is utility? Quality assesment and . . . Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . Discussion Home Page Title Page Our concept of the one true quality is the Mean Opinion Score (MOS). (we might need to modify it for our purposes, though) JJ II J I Page 6 of 16 Go Back Full Screen Close Quit The perceived quality is a subjective concept: What is utility? → subjective tests are needed to measure it. Quality assesment and . . . Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . Discussion Home Page Title Page JJ II J I Page 7 of 16 Go Back Full Screen Close Quit The perceived quality is a subjective concept: What is utility? → subjective tests are needed to measure it. Quality assesment and . . . Real–time multimedia . . . There are standarized methods for measuring it (ITU P.800, P.500, . . . ), but: • they are difficult to set up Real–time multimedia . . . A first approach to . . . Discussion • they are expensive Home Page • they take a lot of time Title Page All this makes them practically unusable for our purposes (large– scale tests) JJ II J I Page 7 of 16 Go Back Full Screen Close Quit The perceived quality is a subjective concept: What is utility? → subjective tests are needed to measure it. Quality assesment and . . . Real–time multimedia . . . There are standarized methods for measuring it (ITU P.800, P.500, . . . ), but: • they are difficult to set up • they are expensive • they take a lot of time All this makes them practically unusable for our purposes (large– scale tests) Real–time multimedia . . . A first approach to . . . Discussion Home Page Title Page JJ II J I Page 7 of 16 Go Back Full Screen Close Quit The perceived quality is a subjective concept: What is utility? → subjective tests are needed to measure it. Quality assesment and . . . Real–time multimedia . . . There are standarized methods for measuring it (ITU P.800, P.500, . . . ), but: • they are difficult to set up • they are expensive • they take a lot of time All this makes them practically unusable for our purposes (large– scale tests) Real–time multimedia . . . A first approach to . . . Discussion Home Page Title Page JJ II J I Page 7 of 16 Go Back Full Screen Close Quit The perceived quality is a subjective concept: What is utility? → subjective tests are needed to measure it. Quality assesment and . . . Real–time multimedia . . . There are standarized methods for measuring it (ITU P.800, P.500, . . . ), but: • they are difficult to set up • they are expensive • they take a lot of time All this makes them practically unusable for our purposes (large– scale tests) Real–time multimedia . . . A first approach to . . . Discussion Home Page Title Page JJ II J I Page 7 of 16 Go Back Full Screen Close Quit Having more or less discarded subjective tests, we turn to: What is utility? → objective tests, which normally come in the form of: • an algorithm that measures a distance between the signal sent and the signal receieved, • a formula that relates some quality–affecting parameters (such as the codec used, and the network loss rate) to the perceived quality. Quality assesment and . . . Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . Discussion Home Page Title Page JJ II J I Page 8 of 16 Go Back Full Screen Close Quit Having more or less discarded subjective tests, we turn to: What is utility? → objective tests, which normally come in the form of: • an algorithm that measures a distance between the signal sent and the signal receieved, • a formula that relates some quality–affecting parameters (such as the codec used, and the network loss rate) to the perceived quality. Quality assesment and . . . Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . Discussion Home Page Unfortunately, • in many cases they don’t give very good results, • they normally take only encoding impairments into account, Title Page JJ II J I Page 8 of 16 • in most cases they can’t be used in real–time. Go Back Full Screen Close Quit Having more or less discarded subjective tests, we turn to: What is utility? → objective tests, which normally come in the form of: • an algorithm that measures a distance between the signal sent and the signal receieved, • a formula that relates some quality–affecting parameters (such as the codec used, and the network loss rate) to the perceived quality. Quality assesment and . . . Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . Discussion Home Page Unfortunately, • in many cases they don’t give very good results, • they normally take only encoding impairments into account, Title Page JJ II J I Page 8 of 16 • in most cases they can’t be used in real–time. Go Back Full Screen Close Quit Having more or less discarded subjective tests, we turn to: What is utility? → objective tests, which normally come in the form of: • an algorithm that measures a distance between the signal sent and the signal receieved, • a formula that relates some quality–affecting parameters (such as the codec used, and the network loss rate) to the perceived quality. Quality assesment and . . . Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . Discussion Home Page Unfortunately, • in many cases they don’t give very good results, • they normally take only encoding impairments into account, Title Page JJ II J I Page 8 of 16 • in most cases they can’t be used in real–time. Go Back Full Screen Close Quit 4. Real–time multimedia Quality assesment: the neural approach We use a Neural Network to estimate the quality of the streams. What is utility? Quality assesment and . . . Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . Discussion Home Page Title Page → it estimates quality as perceived by an average human observer → as a function of several parameters (both network and encoding), → in real–time if necessary JJ II J I Page 9 of 16 Go Back Full Screen Close Quit 4. Real–time multimedia Quality assesment: the neural approach We use a Neural Network to estimate the quality of the streams. What is utility? Quality assesment and . . . Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . Discussion Home Page Title Page → it estimates quality as perceived by an average human observer → as a function of several parameters (both network and encoding), → in real–time if necessary JJ II J I Page 9 of 16 Go Back Full Screen Close Quit 4. Real–time multimedia Quality assesment: the neural approach We use a Neural Network to estimate the quality of the streams. What is utility? Quality assesment and . . . Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . Discussion Home Page Title Page → it estimates quality as perceived by an average human observer → as a function of several parameters (both network and encoding), → in real–time if necessary JJ II J I Page 9 of 16 Go Back Full Screen Close Quit 4. Real–time multimedia Quality assesment: the neural approach We use a Neural Network to estimate the quality of the streams. What is utility? Quality assesment and . . . Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . Discussion Home Page Title Page → it estimates quality as perceived by an average human observer → as a function of several parameters (both network and encoding), → in real–time if necessary JJ II J I Page 9 of 16 Go Back Full Screen Close Quit We do some subjective evaluations. What is utility? Quality assesment and . . . Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . Discussion Home Page Title Page JJ II J I Page 10 of 16 Go Back Normally, we consider the MOS as the quality measure, but we could use other measures as well. Full Screen Close Quit The Neural Net is trained to estimate the quality. What is utility? Quality assesment and . . . Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . Discussion Home Page Title Page JJ II J I Page 11 of 16 Go Back Full Screen Close Quit What is utility? Quality assesment and . . . Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . Discussion Where: λ+ l + + %h wh,l P • %l = − rl + λ− l + h %h wh,l P h Home Page is the probability of neuron l being excited. Title Page • λ+ and λ− l l are the excitatory and inhibitory signals arriving to neuron l from the environment. • rl is the output rate of neuron l • dl is the probability of neuron l emiting a signal to the environment (in FF RNNs, it’s usually 1 for the output neurons, 0 otherwise) X + − • the following holds: (1 − dl )rl = wl,m + wl,m JJ II J I Page 12 of 16 Go Back Full Screen m Close Quit What is utility? Quality assesment and . . . Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . Discussion Where: λ+ l + + %h wh,l P • %l = − rl + λ− l + h %h wh,l • λ+ l P h is the probability of neuron l being excited. Home Page Title Page λ− l and are the excitatory and inhibitory signals arriving to neuron l from the environment. • rl is the output rate of neuron l • dl is the probability of neuron l emiting a signal to the environment (in FF RNNs, it’s usually 1 for the output neurons, 0 otherwise) X + − • the following holds: (1 − dl )rl = wl,m + wl,m JJ II J I Page 12 of 16 Go Back Full Screen m Close Quit What is utility? Quality assesment and . . . Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . Discussion Where: λ+ l + + %h wh,l P • %l = − rl + λ− l + h %h wh,l • λ+ l P h is the probability of neuron l being excited. Home Page Title Page λ− l and are the excitatory and inhibitory signals arriving to neuron l from the environment. • rl is the output rate of neuron l • dl is the probability of neuron l emiting a signal to the environment (in FF RNNs, it’s usually 1 for the output neurons, 0 otherwise) X + − • the following holds: (1 − dl )rl = wl,m + wl,m JJ II J I Page 12 of 16 Go Back Full Screen m Close Quit What is utility? Quality assesment and . . . Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . Discussion Where: λ+ l + + %h wh,l P • %l = − rl + λ− l + h %h wh,l • λ+ l P h is the probability of neuron l being excited. Home Page Title Page λ− l and are the excitatory and inhibitory signals arriving to neuron l from the environment. • rl is the output rate of neuron l • dl is the probability of neuron l emiting a signal to the environment (in FF RNNs, it’s usually 1 for the output neurons, 0 otherwise) X + − • the following holds: (1 − dl )rl = wl,m + wl,m JJ II J I Page 12 of 16 Go Back Full Screen m Close Quit What is utility? Quality assesment and . . . Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . Discussion Where: λ+ l + + %h wh,l P • %l = − r l + λ− l + h %h wh,l • λ+ l P h is the probability of neuron l being excited. Home Page Title Page λ− l and are the excitatory and inhibitory signals arriving to neuron l from the environment. • rl is the output rate of neuron l • dl is the probability of neuron l emiting a signal to the environment (in FF RNNs, it’s usually 1 for the output neurons, 0 otherwise) X + − • the following holds: (1 − dl )rl = wl,m + wl,m JJ II J I Page 12 of 16 Go Back Full Screen m Close Quit 5. A first approach to utility evaluation Back to our notion of utility. . . What is utility? Quality assesment and . . . • U : Q −→ –C Real–time multimedia . . . • U > P must hold for the customer to buy the service Real–time multimedia . . . A first approach to . . . Discussion We are interested in knowing what price–quality combinations will keep U above P . Thus, we may redefine our utility as a random variable U = Pr(U > P ) Then, we may use the neural approach to find out the distribution Home Page Title Page JJ II J I Page 13 of 16 for U Go Back Full Screen Close Quit 5. A first approach to utility evaluation Back to our notion of utility. . . What is utility? Quality assesment and . . . • U : Q −→ –C Real–time multimedia . . . • U > P must hold for the customer to buy the service Real–time multimedia . . . A first approach to . . . Discussion We are interested in knowing what price–quality combinations will keep U above P . Thus, we may redefine our utility as a random variable U = Pr(U > P ) Home Page Title Page JJ II J I Then, we may use the neural approach to find out the distribution Page 13 of 16 for U Go Back Full Screen Close Quit 5. A first approach to utility evaluation Back to our notion of utility. . . What is utility? Quality assesment and . . . • U : Q −→ –C Real–time multimedia . . . • U > P must hold for the customer to buy the service Real–time multimedia . . . A first approach to . . . Discussion We are interested in knowing what price–quality combinations will keep U above P . Thus, we may redefine our utility as a random variable U = Pr(U > P ) Home Page Title Page JJ II J I Then, we may use the neural approach to find out the distribution Page 13 of 16 for U Go Back Full Screen Close Quit 5. A first approach to utility evaluation Back to our notion of utility. . . What is utility? Quality assesment and . . . • U : Q −→ –C Real–time multimedia . . . • U > P must hold for the customer to buy the service Real–time multimedia . . . A first approach to . . . Discussion We are interested in knowing what price–quality combinations will keep U above P . Thus, we may redefine our utility as a random variable U = Pr(U > P ) Home Page Title Page JJ II J I Then, we may use the neural approach to find out the distribution Page 13 of 16 for U Go Back Full Screen Close Quit We are still working on how to conduct the tests, but the main idea is to provide the subjects with • samples of varying quality • a pricing context What is utility? Quality assesment and . . . Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . And ask them to assign each combination a probability of acceptance from a predefined discrete scale, e.g. Discussion • Certain Home Page • Very likely Title Page • Likely • Not likely JJ II J I Page 14 of 16 • Certainly not Go Back Full Screen Close Quit We are still working on how to conduct the tests, but the main idea is to provide the subjects with • samples of varying quality • a pricing context What is utility? Quality assesment and . . . Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . And ask them to assign each combination a probability of acceptance from a predefined discrete scale, e.g. Discussion • Certain Home Page • Very likely Title Page • Likely • Not likely JJ II J I Page 14 of 16 • Certainly not Go Back Full Screen Close Quit We are still working on how to conduct the tests, but the main idea is to provide the subjects with • samples of varying quality • a pricing context What is utility? Quality assesment and . . . Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . And ask them to assign each combination a probability of acceptance from a predefined discrete scale, e.g. Discussion • Certain Home Page • Very likely Title Page • Likely • Not likely JJ II J I Page 14 of 16 • Certainly not Go Back Full Screen Close Quit We are still working on how to conduct the tests, but the main idea is to provide the subjects with • samples of varying quality • a pricing context What is utility? Quality assesment and . . . Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . And ask them to assign each combination a probability of acceptance from a predefined discrete scale, e.g. Discussion • Certain Home Page • Very likely Title Page • Likely • Not likely JJ II J I Page 14 of 16 • Certainly not Go Back Full Screen Close Quit The neural network would then be trained to output, for each price–quality combination, a histogram of U . What is utility? Quality assesment and . . . Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . Discussion Home Page Title Page JJ II J I Page 15 of 16 Go Back Full Screen Close Quit 6. Discussion What is utility? Quality assesment and . . . Real–time multimedia . . . Real–time multimedia . . . A first approach to . . . Discussion Home Page Title Page JJ II J I Page 16 of 16 Go Back Full Screen Close Quit
© Copyright 2025 Paperzz