EMu Technical Requirements Document Version 3.2 EMu version 5.1 Contents SECTION 1 SECTION 2 ArchitectureandConnectivity 1 Server Workstations Web server Network Open Systems Licensing Multimedia 2 5 5 6 6 7 8 Examplesystemconfigurations 11 Index 17 Architecture and Connectivity SECTION1 ArchitectureandConnectivity EMuisacollectionsmanagementsystemthatprovidesbothclient‐serverandweb browseraccesstoacollection. AtypicalhardwareconnectivitysetupforanEMuinstallationisillustratedbelow.In thissetupamulti‐userservermachineprovidesdatabaseserverfacilitiestoboth: LocalandremoteEMuclientworkstations and Localandremotebrowser‐basedInternet/Intranetusers EMuConnectivityOverview LargeEMusitesinparticularmayhaveadedicatedwebserverlocatedoutsidethe corporatefirewall.InthiscaseEMuwebservicesresideontheexternalwebserver and gain access totheEMu server via a secure connection through the corporate firewall. EMu Technical Requirements 1 Architecture and Connectivity Server Operatingsystem TheEMuservermustrunoneofthefollowingoperatingsystems: Linux/Unix AllofthemostpopularUnixandLinuximplementationscanbeused,including: CentOS RedHatES Ubuntu Solaris FreeBSD WindowsServer2008and2012aresupportedviaaLinuxvirtualmachinerunning throughahypervisor. WerecommendUnixorLinuxforlargeEMuimplementations(over500,000records ormorethan10concurrentusers).Itisestimatedthatthereshouldbeoneserver CPUcoreforevery2‐3concurrentusersforheavyusage,orevery4‐5concurrent users for light usage. Either Unix or Windows is suitable for smaller implementations. Wecanprovideadviceonthebestservertomeetyourrequirements. Authentication EachEMuusermusthavearegistereduseraccountontheEMuserver.Thefollowing userdatabasesaresupported: 2 Unixpasswordfile NIS(NetworkInformationServices) WindowsActiveDirectory LDAP(LightweightDirectoryAccessProtocol) EMu Technical Requirements Architecture and Connectivity Installedsoftware ThefollowingsoftwarepackagesarerequiredbytheEMuServer: Perl5.8.8orgreater‐perlscriptinglanguage Thefollowingperlpackagesarerequired: XML::Parser::Expat File::FcntlLock ThefollowingextrasoftwarepackagesarerequiredbytheEMuwebserver(IMu)if installed: ImageMagick‐imagemanipulationpackage PerlMagick‐perlinterfacetoImageMagickpackage Thefollowingperlpackagesarerequired: HTML::Parser Image::ExifTool PDF::APi2 Backup EMu requires no special backup software and does not use raw partitions. Any standardbackupandrestoresoftwareprovidedwiththeoperatingsystemorathird partyproduct(e.g.ARCserve)maybeused. Storage The disk space requirement for the EMu back‐end (without record data or multimedia)islessthan2GB(thisdoesnotincludespacerequiredbytheoperating system). Other storage requirements depend heavily on the anticipated amount of multimedia which is to be stored within the system, and recommendations vary accordingly. Memory Abasememorylevelof3GBshouldbeallocatedtotheserver. Theadditionalmemoryrequirementfortheserverisdependentonthenumberand roles of the connected users. Generally, the server memory requirement per concurrent user varies between 150 and 300 MB for art and cultural history organisations, and 256 and 512 MB for large, multidisciplinary natural history museums. EMu Technical Requirements 3 Architecture and Connectivity Virtualisation The EMu server may be virtualised. There is a performance overhead in running virtualisationhoweverandifperformanceiscritical,itisrecommendedthatEMuis runonrawhardwareifatallpossible.Theoverheadwithvirtualisationistypically 5to10%butcanbeashighas15%. Inanenvironmentrunningvirtualisationdirectlyonrawhardware,VMWareESXi orCitrixXenServerisrecommended. IfWindowsbasedvirtualisationisrequired,werecommendVMWareoverHyperV. Inallinstances,werecommendeitherRedHatLinuxorCentOSLinuxastheguest operatingsystem. 4 EMu Technical Requirements Architecture and Connectivity Workstations EMuworkstationsmustrunWindows10,8,7,Vista,2012,2008or2003.Thedisk space requirement on the client computer is less than 2 GB (without caching of multimedia and reports), although at least 10 GB of available free disk space is recommended. Minimumrequirementsincludeavideocardcapableofatleast800x600resolution, withatleast16bitcolour.Otherminimum(andrecommended)requirementsare: Pentium2.0GHzDualCoreCPU 2048MBRAM(4096for64bitWindows) AppleMacintosh ItispossibletoruntheEMuWindowsclientonAppleworkstationsthroughtheuse ofWindowsemulationtools.Sometoolsavailableinclude: Parallels(http://www.parallels.com/products/desktop/) VMware(http://www.vmwar.com/products/fusion/) Pleasecontactusforfurtherdetails. TerminalServices Windows users may also use Terminal Services or Citrix, which can be useful for providinghighspeedEMuaccessoverlowspeedconnectionstoremotesitesorfor teleworkers. Clientinstallation TheEMuclientcanbeinstalledinoneoftwoways: Local ThefullEMuclientisinstalledoneachPC.EachPCwillneed tobeupdatedforEMuclientupgrades. Network (recommended) ThemajorportionoftheEMuclientisinstalledonanetwork drivetowhichallEMuusershaveaccess.OnlyasmallEMu clientneedstobeinstalledoneachPC,andinmostinstances onlythenetworkdriveneedstobeupdatedforEMuclient upgrades. Webserver ThewebservermustrunPHP5.3orlater. EMu Technical Requirements 5 Architecture and Connectivity Network AlldataandmultimediaresideontheEMuserverandaretransferredbetweenthe serverandeachclientPCasrequired.Multimediacanalsobestoredonaseparate serverornetworkattachedstoragedevice(SAN).Recorddatatransferredbetween PCsandtheserveronlyplacesasmallloadonthenetwork. MultimediafilestransferredbetweenPCsandtheservermayplacesignificantload onthenetworkdependingonthesizeofthefilesandnetworkthroughput. ThenetworkmustsupportTCP/IP. For direct connection between the EMu client and server a minimum network bandwidthof1GB/sisrequired.Ifthinclienttechnologyisinplace(e.g.Citrixor TerminalServices),thenetworkbetweentheenduserandthinclientgatewaymay be substantially slower than this (with Citrix, even a connection across a dial‐up modemisfeasible). OpenSystems The EMu database engine, Texpress, has a number of standard technologies built intotheDBMSforextractingdata.Thesetechnologiesincludehttp,XML(thetwo mostcommonstandardsusedintheworldtoday)andSQL. TechnologiessuchastheSQLinterfacearedesignedonthestandardsavailable.The SQLinterfaceisdesignedtoSQL92withmanyenhancementsforsubsequentdesign advances.Alloftheinformationrequiredtousethistoolispublished.Manypeople outsideofAxiellhavesignificantexpertiseinthisandotherinterfacestoTexpress. WithEMu'sdataexporttoolitispossibletoexportdatainawiderangeofformats, includingXML,CSV,Blankpadded,andavarietyofothers. 6 EMu Technical Requirements Architecture and Connectivity Licensing LicensingforEMuisbyconcurrentuseraccesstotheEMuserver.ThusEMucanbe madeaccessibletoawideuserbasewiththelicenceonlyaffectingthenumberof userswhorunthesystemsimultaneously. EMuincorporatesafullyintegratedwebinterfacesuitableforInternetandIntranet access.DependingontheexpectedInternet/Intranetloadasitecanchoosetorun oneormoreEMuwebservers(inthenatureoftheweboneEMuwebservercan serviceseveralusersapparentlysimultaneously).Note,however,thateachEMuweb serverisconsideredaconcurrentuserintermsoflicensing. The number of licences that should be allocated to web servers depends on the expectednumberofwebsearches.Ageneralguidelineistodedicateonelicenceout ofeveryfivetotenlicencestoservicingwebsearches.Theallocationoflicencesfor webusemaybechangedatanytime.Themorelicencesdedicatedtowebuse,the morewebsearchescanbeservicedconcurrently. Otherthanoperatingsystemsoftwarenoothersoftwareisnecessaryinordertouse EMu.However,manyothersoftwarepackagescanbeusedinconjunctionwithEMu, inparticularawebbrowser,emailsystem,MicrosoftWordandExcelandCrystal ReportsDeveloper. EMu Technical Requirements 7 Architecture and Connectivity Multimedia EMu can store any multimedia resource. It is able to show / play most common image, video and audio formats directly, and can invoke a separate "helper" applicationforanyotherformats. MultimediaresourcesareaddedinteractivelytotheEMuMultimediamodulefroma local or network drive accessible from a client PC. When a multimedia record is saved,EMuautomaticallycreatesathumbnailandanyotherderivativesasrequired. Themultimediaresource(andderivativesifany)arethenuploadedandstoredon theEMuserver.Oncethemultimediarecordissaved,EMudoesnotkeepareference toamultimediaresourceontheclientPC. MultimediaresourcesaredownloadedtoeachclientPCondemand.Amultimedia cachecanbeusedoneachPCtoreducenetworktraffic.EMuclientoptionscanbe settoconfigurethemultimediadownloadandcachefacilities. ImageSupport EMusupportsover100imagefileformats,including: BMP DCX DNG GIF JP2 JPEG PCD PCX PNG TGA TIFF MicrosoftWindowsBitmapImage ZSoftIBMPCmulti‐pagePaintbrush DigitalNegative CompuServeGraphicsInterchangeFormat JPEG2000 JointPhotographicExpertsGroup PhotoCD ZSoftIBMPCPaintbrush PortableNetworkGraphics TruevisionTargaImage TaggedImageFileFormat EMu’simagecapabilitiesareextensible,automaticallysupportinganyimageformats registeredbyapplicationsinstalledonacomputer. 8 EMu Technical Requirements Architecture and Connectivity ImageMetadataSupport EMusupportsthefollowingmetadatastandards: EXIF ExchangeableImageFileFormat(Version2.3,April2010) IPTC InternationalPressTelecommunicationsCouncil(Version1.1,July2009) XMP ExtensibleMetadataPlatform(RevisionDecember2008) Metadataisautomaticallyextractedfromthefollowingfileformats: Standard Formats supported EXIF JPEG,TIFF IPTC JPEG,TIFF,PICT,PS,PSD XMP JPEG,TIFF AudioSupport AudioformatssupportedbyEMuinclude: AIFF AU MIDI MP3 RMI SND WAV WMA AudioInterchangeFileFormat AudioFileFormat MusicalInstrumentDigitalInterface MPEGAudioStream,LayerIII MIDIinRIFFFileFormat SoundFileFormat WaveformAudio WindowsMediaAudioFile EMu’saudiocapabilitiesareextensible,automaticallysupportinganyMCIcompliant audioCODECsinstalledonacomputer. VideoSupport VideoformatssupportedbyEMuinclude: ASF AVI M1V M2V MP4 M4V MODJVC MPEG WMV AdvancedStreamingFormat AudioVideoInterleaveFile MPEG‐1VideoFile MPEG‐2VideoFile MPEG‐4Part14 EverioGZ‐MG20UDigitalVideoFile MPEG1SystemStream WindowsMediaFile EMu’svideocapabilitiesareextensible,automaticallysupportinganyMCIcompliant videoCODECsinstalledonacomputer. EMu Technical Requirements 9 Example system configurations SECTION2 Examplesystemconfigurations In this section we describe a range of EMu system configurations suitable for an institution with approximately 110 concurrent users. But first we examine what resourcesaretypicallyrequiredtorunEMuinanenvironmentwithapproximately 110concurrentusers. CPU CPU usage varies considerably depending on what activities users undertake: for instance, searches use little CPU resources compared to data manipulation (e.g. sortingandreporting).Ifusersperformlotsofdatamanipulation,itisimportantto havehighspeedCPUsandpreferablyenoughcoressothatoneprocessisnotblocked byanother.Whilemultiplethreadspercoreisusefulasitallowsformoreconcurrent processes,distinctcoreswillgivesuperiorperformanceasloadincreases(inother words, it is better to have lots of cores rather than a single core with multiple threads). ToguaranteethatprocessesarenotCPUbound(processesgoingslowbecausethey cannotgetenoughoftheCPU),thenumberofcoresshouldbesimilartothenumber ofactiveusers.Ingeneral,ofcourse,notalluserswillproduceaheavyloadatthe same time and it is reasonable to have less cores than the number of users. Furthermore,processesneedtostopfromtimetotimetowaitfordisks,soother processescanusethecoreatthistime.Iffewercoresareavailable,thenmultiple threads per core become useful when the load peaks (multiple threads lead to a slowerdegradationofmachineperformance). In a system with 110 concurrent users and a high load, 100 cores would be recommended.Inreality,itishighlyunlikelythatalluserswillbeproducingaheavy loadsimultaneouslyanditisfeasibletohavesignificantlylesscoresandtoallowtwo threadspercoretohandlepeakloads.Inthiscase,maybe50coreswithtwothreads percorewouldbeadequate. Cache Naturally,thelargertheCPUcache,thebetter.EMuisadatabasesystemsothehit rateonthedatacachecanbecomelowwhenmanipulatingalotofdata(asthedata isspreadoutovermemory).However,thelargerthecache,thebetterthehitrate. AlsoalargecacheallowsmoreoftheEMuexecutabletoresideintheinstruction cache, hence improving CPU performance. This is useful when two threads are runningonasinglecoreandmultiplecoresarelocatedontheonechipasallthreads arelikelyrunningthesameexecutable(theEMuserver). EMu Technical Requirements 11 Example system configurations NumberofCPUs ItisbettertohavemorephysicalCPUseachwithlesscoresthanoneCPUwithalarge numberofcores. UnfortunatelythemorephysicalCPUs,thegreaterthecost;however,thebenefits outweigh the cost. In particular, the duplication of memory access hardware required by each CPU allows greater throughput. Also less cores and more CPUs means more distinct caches (one per CPU) which increases the hit rate as less processesareusingthesamecache(allowingeachprocesstousemoreofthecache). Tosummarise:forEMuitisworthlookingformorephysicalCPUswithlesscores anddualthreadspercore.AlargecacheisalsobeneficialandthehighertheCPU speed,thebetter. SPARCvsIntel Until recently SPARC based systems were the architecture of choice for large machinesduetothewellthoughtoutdesignofSPARCsub‐systems(memory,disk, network, etc.). Designed to communicate simultaneously, this provides maximum usage of hardware at any point in time. In contrast, Intel based systems were designedassuperPCs,wheretheneedforsimultaneousoperationswaslow. InrecentyearsIntelandAMDhavemovedintotheservermarketandnowproduce hardware that handles simultaneous operations as effectively as SPARC based systems.Intel/AMDCPUsprovidebetterthroughputattheCPUlevelthanSPARC basedCPUs,anddosoatalowercostanditisourviewthatanIntel(x86_64)based solution will nowadays provide better computing power for a lower cost than a SPARCbasedsolution. Memory EMucanbefairlymemoryintensivewhenaprocessisaccessingalargenumberof tables.InordertoprovidebetterperformanceEMudoesnotfreememoryassociated withtablesafterthetableisnolongerrequired.Astheoverheadofreloadingthe schema,etc.ishigh,itisbettertopaythepricewithmemoryratherthanprocessing poweranddiskaccesses(requiredifthetableneedstobereloaded).Thismeansthat EMuprocessestendtogrowasusersaccessmoretables. The EMu server process starts at about 6 MB before any tables are loaded. The amountofmemoryrequiredforafullyloadedservervariesdependingonthetables installedinaninstitution.Atypicalrangeisfrom200MBto400MB.Ifweassumea worst case of 500 MB and there are 110 users, then about 55 GB RAM would be required by EMu itself. To this we need to add memory for all other system processes,theoperatingsystemandthefilesystemcache:roughly500MBforother EMuprocess(backgroundloads,etc.)and500MBforsystemandoperatingsystem processes. The amount of disk cache depends on the file system used. If using ZFS (recommendedforSolarissystems),thebiggerthecache,thebetter.Asizeofabout 4‐5 GB is useful. This means that about 64 GB would be required to handle the existingload.Thisdoesnotallowformoreusersinthefuture,norfortheaddition ofmorewebservices,etc.Wewouldrecommendincreasingthisnumberto96GBto allowforfutureexpansion. 12 EMu Technical Requirements Example system configurations IdeallythememoryshouldbesplitintobanksbasedonthenumberofphysicalCPUs inthemachine.Thus,iftherearefourCPUs,thenfourbanksof24GBeachshouldbe used.ThisallowseachCPUtoworkonitsownsetof24GBwithminimalneedfor expensivecrossmemorybankreads. Disk Asadatabasesystem,EMucanbediskintensivewhenalargenumberofrecordsis beingmanipulated(sorting,reporting,globaledits,etc.).Sincethedisksmaybecome abottleneck(asallprocessesneedtoaccessthem),itisworthgettingthefastest disksubsystemavailable. If looking at local disks, it is hard to go past SAS (Serial Attached SCSI) drives. If possible,use15,000rpmdrives,otherwise10,000rpmdrives.Inordertoimprove throughput,usingRAID0(striping)isrecommended.Ideallythenumberofdisks stripedshouldbeafactoroftheblocksizeofthefilesystem.Agoodfilesystemblock sizeforEMuis16KB.Thiscouldbeconfiguredasfourdisksinthestripewhereeach diskhasa4096bytesectorsize.Thismeansthateach16KBreadrequiresreading onlyonesectorfromeachdisk.Asthishappensinparallelacrossthefourdisks,the throughputgainislarge. Ifdataredundancyisrequired,RAID1isrecommended.ChecksumbasedRAIDis notrecommendedduetotheslowwritetimesexperienced(e.g.RAID5).If1.2TBof diskisrequiredforinstance,then8x300GBdriveswouldbeideal,assumingRAID 10isused(thatisRAID0and1combined).Ifthisisnotpossible,then4x600GB drivescouldbeusedwithafilesystemblocksizeof8K. Ifnon‐localdisks(SAN)aretobeused,muchthesameappliestotheconfiguration oftheSAN.Ideallythedatashouldbestripedoveralargenumberofdiskstoprovide betterthroughputviasimultaneousread/writes.Alsothelargerthediskcacheon the SAN, the better. Once again checksum based RAID configurations are not recommendedastheymayimpactwriteperformance. Ingeneral,localdisksarepreferabletoanon‐localdiskSANaslocaldisksdonot have to compete with other machines accessing the SAN. Of course, if the SAN is dedicated to the EMu server, then it functions much like a local disk subsystem (providedtheconnectingtechnologyisfast,e.g.fibreoptic,andtheSANoperating systemistunedfordatabaseusage). Hybriddisktechnologiesareappealing.Theyprovidetheappearanceoffasterdisk accessesthroughtheuseoffrontendSSDs.SoprovidedallinformationisintheSSD cache,veryfastreadsarepossible;similarlyfastwritesarepossible(SSDswrite2‐4 fasterthanhighspeedSASdrives).Suchsolutionsaremoreexpensive,butfordisk intensivesystemslikeEMu,significantbenefitscanbeachieved. Finally,SSDonlybaseddrivesmaybeused.Theseprovidesuperiordiskthroughput for both reads and writes. However the throughput comes at a price. SSDs are expensive,butifthefastestsolutionisrequired,theyshouldbeconsidered. SSDs do have a limited number of write cycles, however the limit for most contemporarydrivesisaround10,000writesanditisextremelyunlikelythatthis limitwouldbereachedinafiveyeartimeframe(even,indeed,inunder10years). EMu Technical Requirements 13 Example system configurations OperatingSystem Foramediumtolargeinstitution(>10users)werecommendrunningundersome variantofUNIXratherthanWindows. The two most popular UNIX versions we support are Linux and Solaris. For very large institutions (>=50 users) we would recommend Solaris 10/11 rather than LinuxasSolarissupportsmultiplevirtualCPUs(whereavirtualCPUisathreadona coreonaphysicalCPU)betterthanLinuxwhenthenumberofvirtualCPUsgrows above ten. However EMu runs well on Linux and a choice between the two UNIX versionsreallydependsonthepreferenceofyourSystemAdministrator. Forsmallernumbersofusers(<50)Linuxprovidesaverynicesolution. 14 EMu Technical Requirements Example system configurations SuggestedConfigurations Thefollowingconfigurationsaresuggestionsonlybutshouldprovidesomeideaof configurations suitable for EMu. Which configuration is suitable (and whether tweakingisrequired)willdependonaninstitution'ssize,requirementsandloads. Themachinespecificationsbelowareforasitewithapproximately110concurrent users. Configuration1 SunX4‐4Server 4x15coreE7‐8895v2,2.80Ghzx86_64CPU 37.5MBSharedL3CacheperCPU 128GB(4x2x16GB)DDR3‐ECCmemory 6xSASdrives(300GB) Solaris10/11x86_64 ZFSfilesystem ThisconfigurationusesZFSastheunderlyingfilesystemwithRAID10.Itisamiddle solution(intermsofcostandperformance).ZFSdoesnotsupporttheuseofSSDs fordata,howeveritdoessupportthemforitsZILandARC(caches).Theideahereis topurchaseextramemorytoensurethatmostdatablocksarekeptincorebyZFS (hencethe128GBRAM).InthiscaseSSDdrivesarenotrequired.Youcouldlower thememoryto96GBandincludetwoSSDsfortheZFScaches,butthiswouldaffect thestripingasthenumberofdisksinthestripewouldreducefromthreetotwo.Ifa higherendserver(andmoreexpensiveserver)isrequired,theSunx4‐8shouldbe consideredwiththeaboveconfiguration,exceptthat8xSASdriveswouldprovide betterdiskstriping. Configuration2 SunX4‐4Server 4x15coreE7‐8895v2,2.80Ghzx86_64CPU 37.5MBSharedL3CacheperCPU 128GB(4x2x16GB)DDR3‐ECCmemory 6xSSDdrives Linuxx86_64 XFS/Ext4fsfilesystem Thisconfigurationissimilartoconfigurationone.Theonlydifferenceistheuseof LinuxinsteadofSolaris.ThereasonforthechangeisthatLinux'sXFSandExt4fsfile systems both provide direct support for SSD drives. In this configuration it is possible to use SSD drives exclusively to provide superior disk throughput. The aboveconfigurationwouldprovideafasterdisksubsystem,howeveritisdebatable howmuchofthedisksubsystemisusedgiventhelargeRAMcachepossiblewith 128GBofmemory. EMu Technical Requirements 15 Example system configurations Configuration3 DELLC6220IIServer 4x12CoreE5‐2697v2,2,70Ghzx86_64CPU 30MBSharedL3CacheperCPU 128GB(4x2x16GB)DDR3‐1866memory 8x300GBSASdrives(15K)or8x300GBSSDdrives Linuxx86_64 XFS/Ext4fsfilesystem The above configuration allows for either SSD or SAS drives. Cost will probably decidethebestwaytogo.Giventhelargememorycache,thefilesystemcouldreside onSASdrivesandtheextrabenefitsofSSDmaythereforebeminimal. Configuration4 DELLR920Server 4x15CoreE7‐8890v2,2.80Ghzx86_64CPU 37.5MBSharedL3CacheperCPU 128GB(4x2x16GB)DDR3‐1600memory 8x146GBSASdrives,4x149GBSSDdrives Linuxx86_64 XFS/Ext4fsfilesystem TheaboveconfigurationwouldallowtheSSD/HDDsolution,providingaveryfast disksubsystemthroughtheuseofSSDdrivestofrontendtheSASdrives. 16 EMu Technical Requirements Index S Server • 3 A Apple Macintosh • 5 SPARC vs Intel • 12 Storage • 4 Suggested Configurations • 15 Architecture and Connectivity • 1 T Audio Support • 10 Terminal Services • 5 Authentication • 3 V B Video Support • 10 Backup • 4 C Virtualisation • 4 W Client installation • 5 Configuration 1 • 15 Web server • 7 Configuration 2 • 15 Workstations • 5 Configuration 3 • 16 Configuration 4 • 16 CPU • 11 D Disk • 13 E Example system configurations • 11 I Image Metadata Support • 9 Image Support • 9 Installed software • 3 L Licensing • 7 M Memory • 4, 12 Multimedia • 9 N Network • 7 O Open Systems • 7 Operating system • 3 Operating System • 14
© Copyright 2026 Paperzz