©TheUniversityofHongKong
2013‐2‐27CCST9015E.Lam2
Greatest Engineering Achievements of the 20th Century
DIGITAL IMAGES
• AstudybytheUSNationalAcademyofEngineering
• http://www.greatachievements.org/
• G.ConstableandB.Somerville,ACenturyofInnovation:Twenty
EngineeringAchievementsthatTransformedourLives.JosephHenry
Press,2003. http://books.nap.edu/catalog.php?record_id=10726
CCST9015ElectronicTechnologiesinEverydayLife
• Ourlist?
• Thelistfor21st century?
• Thelistfor20th centurypredicted
Dr EdmundLam
ElectricalandElectronicEngineering
27Feb2013
in1900?
Acknowledgment:SomeslidesarebycourtesyofDrHaydenSo.
©TheUniversityofHongKong
2013‐2‐27CCST9015E.Lam3
Greatest Engineering Achievements of the 20th Century
1.Electrification
6.Radioand
6.Radioand
Television
Television
11.Highways
16.Health
16.Health
Technologies
Technologies
2.Automobile
7.Agricultural
Mechanization
12.Spacecraft
17.Petroleumand
Petrochemical
Technologies
3.Airplane
8.Computers
8.Computers
13.Internet
13.Internet
18.Laserand
18.Laserand
FiberOptics
FiberOptics
4.WaterSupply
andDistribution
9.Telephone
14.Imaging
14.Imaging
19.Nuclear
Technologies
5.Electronics
5.Electronics
10.Air
Conditioningand
Refrigeration
15.Household
Appliances
20.High‐
performance
Materials
©TheUniversityofHongKong
2013‐2‐27CCST9015E.Lam4
Digital Images
Digitalimages:Morethanmeetstheeye!
©TheUniversityofHongKong
2013‐2‐27CCST9015E.Lam5
©TheUniversityofHongKong
2013‐2‐27CCST9015E.Lam9
Quiz!
Some Questions for Us
• Howmanyphotoshavebeentakenever?
A. Howdiddigitalimagesandcamerascomeabout?
• history+business+technical+marketing
• Oldestknownpicturedatestoaround1826
• “Moore’sLaw”(exponentialgrowth)
• Howmanyphotoshaveyoutaken?
B. Thefutureofdigitalimagesandcameras(anddisplays)
• Thenextgreatthing?
• Whatpercentageofthesephotoswastakenfromthepastyear?
• Worldpopulation:~7billion(China~1.3billion,India~1.2billion,
US~315million)
©TheUniversityofHongKong
2013‐2‐27CCST9015E.Lam10
The Making of Digital Images
©TheUniversityofHongKong
2013‐2‐27CCST9015E.Lam11
Photography
• Cameraobscura /pinholecamera:MoZi 墨子 ~400BC
• thenAristotle,Euclid
• 1812WilliamWollaston:firstphotographiclens
• ~1817Nicéphore Niépce:bitumen(recordingmaterial)
• 1839LouisDaguerre:daguerreotype(silverhalide)
OnJanuary19,2012,EastmanKodakCompanyanditsU.S.subsidiariesfiled
voluntarypetitionsforChapter11businessreorganizationintheU.S.Bankruptcy
CourtfortheSouthernDistrictofNewYork.…
ROCHESTER,N.Y.,February09‐‐ EastmanKodakCompany(the“Company”)
(OTB:EKDKQ.PK)announcedtodaythat,asaresultofitsongoingstrategic
reviewprocessandcommitmenttodrivesustainableprofitabilitythroughits
mostvaluablebusinesslines,itplanstophaseoutitsdedicatedcapturedevices
business– comprisingdigitalcameras,pocketvideocamerasanddigitalpicture
frames– inthefirsthalfof2012.
• 1888GeorgeEastmansoldthefirstcameracalled“Kodak”
http://en.wikipedia.org/wiki/History_of_the_camera
©TheUniversityofHongKong
2013‐2‐27CCST9015E.Lam12
©TheUniversityofHongKong
2013‐2‐27CCST9015E.Lam13
Digital Photography
Digital Photography
• 1969(AT&TBellLabs):WillardBoyle andGeorgeSmith
• Thecameraweighed8pounds(3.6kg),recordedblackand
inventionofthecharge‐coupleddevice(CCD)
• 2009NobelPrizeinPhysicsfor“theinventionofanimaging
semiconductorcircuit—theCCDsensor”
whiteimagestoacassettetape,hadaresolutionof0.01
megapixels(10,000pixels),andtook23secondstocapture
itsfirstimageinDecember1975.
• 1975:StevenSasson inventedthefirstdigitalcamera
• U.S.Patent4,131,919:ElectronicStillCamera
1976:Kodakcommanded90%offilmsalesand85%ofcamerasalesintheU.S.,according
toa2005casestudyforHarvardBusinessSchool.
©TheUniversityofHongKong
2013‐2‐27CCST9015E.Lam14
©TheUniversityofHongKong
2013‐2‐27CCST9015E.Lam15
Digital Photography
Digital Photography
• 1990:LogitechFotoman/DycamModel1
• Thefirsttrulydigitalcameratobesoldonthemarket.
• Ithadonly320×240andlater376×284resolution;
• Itproducedonlyblack/whitephotoswith256graylevels;
• Ithadonly1MB(later4MB)internalRAM;
• Ithadfixedfocusf/4.5lens;
• ItcostUS$980!
• 1991:KodakDCS100
• Thefirstcommerciallyavailabledigitalsingle‐lensreflex(DSLR)camera.
• MountedonaNikonF3(filmcamera)body.
• Aresolutionof1.3megapixels.
• AseparateshouldercarriedDigitalStorageUnit
(DSU)tostore+visualizetheimages.
• TheDSUcontainedaWinchester200MBhard
diskdrivethatcouldstoreupto156images
withoutcompression,orupto600images
usingaJPEGcompatiblecompressionboard
thatwasofferedlaterasanoptionalextra.
• Atotalof987unitsweresold,at~US$13,000.
• Kodak'sDCS100appearedhalfayearlater.
©TheUniversityofHongKong
2013‐2‐27CCST9015E.Lam16
©TheUniversityofHongKong
2013‐2‐27CCST9015E.Lam17
Digital Photography
A “Kodak Moment”
• 1999:NikonD1
• “TheEndofTheBeginning(ofthedigitalera)― TheBeginningofThe
End(ofthefilmera)”
• A2.7megapixelimagesensor(2000×1312)
• 4.5frames/scontinuousshooting
• AcceptedthefullrangeofNikonF‐mountlenses
• Price:<US$6,000!
• WhydidKodakfail?Whatcouldhavebeendone?
• Whatwouldyouthinkofdigitalphotographyin1975?
• Whatwouldyouthinkofdigitalphotographyin1990?
• Whatwouldyouthinkofdigitalphotographyin1999?
• Whatwouldyouthinkofdigitalphotographyin2013?
• Whatwouldyouthinkofdigitalphotographyin2047?
• KodakDCS460:~6megapixel,>US$30,000.
“Wealwaysoverestimatethechangethatwilloccurinthenexttwoyears
andunderestimatethechangethatwilloccurinthenextten.Don’tlet
yourselfbelulledintoinaction.”
― Bill Gates
©TheUniversityofHongKong
2013‐2‐27CCST9015E.Lam18
A “Kodak Moment”
©TheUniversityofHongKong
2013‐2‐27CCST9015E.Lam19
The Technology of Digital Images
• Mycamerastoresmoreimagesofqualitybetterthanyours!
Appleis~4800× Kodak
BloombergBusinessweek: Jan.19(Bloomberg)‐‐ EastmanKodakCo.,thephotographypioneerthatintroduced
theBrownieCameramorethanacenturyago,filedforbankruptcyafterconsumersembraceddigitalcameras,
atechnologyKodakinventedandfailedtocommercialize.…“Theywereacompanystuckintime,”saidRobert
Burley,anassociateprofessoratToronto’sRyersonUniversitywhohasphotographedshutteredKodak
facilitiesintheU.S.,CanadaandFrancesince2005.“Theirhistorywassoimportanttothem,thisrichcentury‐
oldhistorywhentheymadealotofamazingthingsandalotofmoneyalongtheway.Nowtheirhistoryhas
becomealiability.”…Thecompany’screditdeterioratedasrevenuetumbledfromtraditionalfilm,andthe
inventoroftheInstamaticcameraswasslowduringthepastdecadetocompetewithCanonInc.andHewlett‐
PackardCo.indigitalcamerasandprinters.
Fact?
Legacy?
Marketing?
Howmany?
Howgood?
2013‐2‐27CCST9015E.Lam20
©TheUniversityofHongKong
©TheUniversityofHongKong
The Technical Side
The Technical Side
R
Bitmap
(Apixel‐by‐pixelrepresentationofanimage)
G
B
• ImageSize
• ThenumberofpixelinX‐Ydirection
• Sometimesquotedusingthetotalnumberofpixelsina
picture(N megapixels)
14
pixel
Animageisbrokendownintosmall
regionscalledpictureelements
(pixels)
Lena:Themost(in)famouspictureindigitalimageprocessing…
http://en.wikipedia.org/wiki/Lenna
2013‐2‐27CCST9015E.Lam22
©TheUniversityofHongKong
2013‐2‐27CCST9015E.Lam21
The Technical Side
15
• e.g.:a15×14image,or210pixels,or0.21Kpixels,
or0.00021Mpixels
©TheUniversityofHongKong
2013‐2‐27CCST9015E.Lam23
The Technical Side
• CCDSensorrespondsto“amountoflight”
• Doesnotdistinguishcolor!
Aformofdigitalimage
processing
• Onemethod:colorfilterarray
• “Guessthevalues”:demosaicing (demosaicking)
Black‐and‐white
0or1
“1bit”
Grayscale
0‐255
“8bit”
16‐color
(colorpalette)
“4bit”
256‐color
16million‐color
(colorpalette) 0‐255,0‐255,0‐255
“24bit”
“8bit”
1byte=8bits
RGBcolor(red,green,blue)
CMYcolor(cyan, magenta, yellow)
InventedbyBryceE.BayerofEastman
Kodakin1976(U.S.Patent#3,971,065).
©TheUniversityofHongKong
2013‐2‐27CCST9015E.Lam24
©TheUniversityofHongKong
2013‐2‐27CCST9015E.Lam25
The Technical Side
The Technical Side
• Howlargeisonedigitalimage?
• NikonD1:2000×1312
• Colorimage(RGBBayerpattern)
• 8‐bitpercolor
• Imagecompression(anotherdigitalimageprocessingtopic)
• Idea(I):Dataredundancy
• Givemeapixel,Ihaveagoodchanceto“guess”itsneighbor’svalues
• Icanbewrong…butonaverage,it’sagood“bet”
• HowgoodIamdependsontheimagecontent
• RAW:2000×1312×8/8=2,624,000Bytes ≈2.6MB
• IMAGE:2000×1312×8×3/8=7,872,000Bytes ≈7.9MB
• Idea(II):Perceptualredundancy
• Ifyouchangean8‐bitpixelbyoneintensitylevel,Icannottell
• It’s“ok”tomake(small)changestoyourimage…youwon’tnotice!
• AtacertainpointI’llbegintonotice
• ACTUAL(myphoto):1.29MB(“Compressedbitsperpixel:4”)
• Alsofluctuateswithdifferentimages!
• Witha8GBcard,howmanypicturescanItake?
• Engineersfigureouthowtoturntheseideasintoalgorithms.
• The“JointPhotographicExpertsGroup”(JPEG)
©TheUniversityofHongKong
2013‐2‐27CCST9015E.Lam26
©TheUniversityofHongKong
2013‐2‐27CCST9015E.Lam27
The Technical Side
Quality
• ISO/IECJointTechnicalCommittee1,Subcommittee29,Working
• Howwouldyoudefine“imagequality”?
Group1(ISO/IECJTC1/SC29/WG1)
• ISO=InternationalOrganizationforStandardization
• IEC=InternationalElectrotechnical Commission
• Organizedin1986,firstJPEGstandardin1992
• Approvedin1992asITU‐TRecommendationT.81
• Approvedin1994asISO/IEC10918‐1
Knowthese?
• MPEG‐4
• MP3
• MP4
• H.264
• JPEG2000
• Lossy compression
15:1
23:1
46:1
144:1
Ansel Adams,
“MoonandHalfDome”
(1960)
2013‐2‐27CCST9015E.Lam28
©TheUniversityofHongKong
©TheUniversityofHongKong
2013‐2‐27CCST9015E.Lam29
Resolution
The Technical Side
• Isresolutionacompetitionontechnology?
• Many thingsarewrongwiththismeasure.
• (I)Wecanarbitrarilyscaleupanimage.
• Do“resize”inanyimageeditingsoftware!
14
• (II)Wedidnotspecifyhowtododemosaicing.
• Imageprocessingalgorithmsmakeadifference!
• (III)Weignoredcompression.
15
• Morepixelsonthesensor!
• 1975firstcamera:0.01megapixels
• 1990Fotoman:0.076megapixels
• 1991DCS100:1.3megapixels
• 1999D1:2.7megapixels
• 2013yourphone:5/8/12megapixels?
©TheUniversityofHongKong
887,276×306,908pixels
(stitchedfrom12000images)
FujiSuperCCD
http://gigapan.com/gigapans/66626/
Foveon’s threelayers
2013‐2‐27CCST9015E.Lam30
©TheUniversityofHongKong
2013‐2‐27CCST9015E.Lam31
The Technical Side
The Technical Side
• Whatisright?
• Stillnotquite…
Ideally,a“point”object
resultsina“point”
image.
• (I)Imagepost‐processing
Inreality,aphenomenon
called“diffraction”results
inawideningoftheimage.
Rayleighresolution
criterion:Canyoutell
thereweretwopointsin
theobject?
• (II)Signal‐to‐noiseratio(SNR):Withthesamesensorsize,
morepixels=>smallerpixels=>lesslight=>more“grainy”
©TheUniversityofHongKong
2013‐2‐27CCST9015E.Lam32
©TheUniversityofHongKong
2013‐2‐27CCST9015E.Lam33
The Race for Better Cameras and Images
Hype, or the Next Big Thing?
• Why“resolution”?
• Asimplenumberor“accurate”technicaldetails?
• Lightfield camera(Lytro,Raytrix,…)
• Takepicturenow,focuslater
• Capturethe“lightfield”
• Howdodigitalcameramanufacturerscompetein2013?
• Algorithms?
• Example:http://www.htc.com/www/zoe/
©TheUniversityofHongKong
2013‐2‐27CCST9015E.Lam34
Hype, or the Next Big Thing?
• ACMDoctoralDissertationAwardtoRen NgofStanfordUniversity:He
•
•
•
•
•
wasnamedforhisstudyofDigitalLightFieldPhotography.TheDoctoral
DissertationAwardispresentedannuallytotheauthorofthebest
doctoraldissertation incomputerscienceandengineering.
SiliconValleyBusinessJournalaward:ConsumerTechnology:Lytro light
cameraletsusersfocuslongafterphotosareshot.
Priortohisdeathin2011,AppleCEOSteveJobspurportedlymetwith
Lytro CEORen NgtodiscussimprovementsfortheiPhonecamera.
Lytro LaunchestoTransformPhotographywith$50MinVentureFunds.
DPReview:we’renotconvincedthattheLytro eithersolvesanyexisting
problemorpresentsanycompellingraisond’etre ofitsown.Ifitwere
higherresolutionorallowedgreaterseparationorcouldproducesingle
lens3Dvideoitmightgeneratealotmoreexcitement.Asitis,itfeelslike
aproductarrivingbeforetheunderlyingtechnologyisreallyready.
NewYorkTimes:TheLytro hassomedrawbacks— andnot
inconsiderableones.There’sadifferencebetweenagreattechnology
andagreatproduct.Fornow,ItipmyhattotheinnovatorsatLytro,but
I’mnotopeningmywallet.