SUPERCOMPUTERS SUBSURFACE AND SOLUTE THEIR USE IN MODELING TRANSPORT D. A. Barry Centre for Water Research University of Western Australia, Nedlands Abstract. Supercomputers offer a wide range of pos- reducedto solutionsof algebraicsystemswhen they are sibilitiesfor advancingknowledgeand solvingdifficult, implementedon computers,algorithmsfor linearalgebraic real-worldproblems. Solutetransport-related theoryand systemsof equationssuitable for vector and parallel applicationshave long been an area of specialization machinesare discussedin some detail. Data analysis, within hydrology. In this review, recentapplications of necessaryboth for evaluatingemergingtheoriesand for supercomputers to solutetransport problemsarepresented. more "routine" applications like determining model is identifiedasanothergeneralareathatis ably Theseapplications arebasedmostlyon theassumption of a parameters, In particular,supercomputers dilute solute, allowing the governing equationsto be handledby supercomputers. decoupled. An applicationof supercomputers is the offer a way to become less dependenton parametric simulation of solute transport in large heterogeneous assumptionsand constraints,substituting,instead, raw spatialdomainsby both direct solutionof the governing computationalpower. Visualization techniques and facilitieshaveflourishedalongsidethe developequations and by particle-tracking methods.An impetus associated for this researchhasbeen the rapid increasein predictive mentof supercomputer centers.Theseoffer thepossibility dispersionmodelsbasedon analyticalstochastictheory. for real-time viewing of solutetransportprocesses,both Up until now, relatively few applicationshave incorpo- real and simulated,althoughlittle hasbeenproducedthus ratedalgorithmsdesignedwith a targetedsupercomputer in far. It is expectedthat many more researchersinvolved mind, with theresultthat the machine'spotentialcannotbe with subsurface solutetransportwill make use of superexploited fully. Since many numericalproblemsare computersin the future. INTRODUCTION ation. There are two basic paths by which computer calculationalratescan be improved. In the first, ratesare Hydrological uses of computers are increasing increasedby speedingup the passageof informationin the monotonically with the passage of time. We are now into machine and the response time of the processors. the third phase of the "computer revolution" where Theoretically, information cannot be transmittedfaster "problemsare beingroutinelydefinedthatcouldnot even thanthe speedof light. Superconductors offer a possible have been conceivedwithout the existenceof computers" way to increasepresentlyavailable transmissionrates. [Wallis, 1987]. Certainly, available supercomputers Reducingthe distances betweenthe machinecomponents continueon this"revolutionary"path. Thereare two rather is another,more obviousmethodof speedingup informaobvioustrendsin theuseof supercomputers. First,thesize tion transfer. The circulararchitecture of the Cray X-MP of problemswhichbecomesfeasiblecontinuesto increase illustratesan approachof this type. This and other by Seitzand Matisoo [1984], who in magnitude. Here the word "size" refers not only to limitationsare discussed computer-related limitations,e.g., speed or number of indicatethatthelimitsof theavailabletechnology maybe processors, amountof memory,input/outputand physical in sight. The secondway to increasecomputationalrates is to storagecapacity,but also to the complexitylevel of the models investigated. Second, smaller, more routine perform more calculationsat the same instant. Vector analysesare carriedout muchmorerapidlyandaccurately, machinesachieve their speed by operatingon vectors sometimes in real time. ratherthanscalars,whereasparallelmachinesincreasethe Supercomputers embodythe notion of speed. Unfor- numberof processors in the computer.With this simple tunately,it canbe difficult to realizehighratesof comput- statementit is apparentthat, on the one hand, a vector Reviewsof Geophysics, 28, 3 / August1990 Copyright1990 by theAmericanGeophysical Union. 8755-1209/90/90RG-00342 pages277- 295 Papernumber90RG00342 $05.00 277 ' 278 ß Barry:SUPERCOMPUTERS AND SOLUTETRANSPORT 28, 3 / REVIEWS OF GEOPHYSICS machineneedsto operateon vectorsmostof the time to speedupof 50 but vectorizesonly 90% of the code will achievespeedscloseto its ratedmaximum. That is, the achievea speedupof only 8.5. Denning [1988] discusses degreeto whichan algorithmwill vectorizedetermines its Amdahl'slaw asit appliesto parallelprocessors. Parallel machinesare affected by factorsapart from suitablityfor runningon a vectorprocessor.On theother hand, parallel machinesachieve the same result by simplythetotalnumberof operations.Boththetime taken betweenprocessors and the synchronizacompletingmanycomputational taskssimultaneously on to communicate differentprocessors.The combinationof an army of tion of the computationaltasks need to be considered. vector processorsproducesa hybrid, vector-parallel Adamsand Crockett[1984] presentmethodsto quantify machine. For a given algorithm, then, the degree of thesetiming costsusinga linear systemexample. More parallelism, or concurrency [Sharp,1987],achievable in its generally,on parallel machinesthere are two ways by implementation is the overridingfactorin determining its which speedupcan be enhanced. First, the problemis computationrate. So far, only "course-grained" paral- broken into independentsubproblems. Monte Carlo lelism has been referred to. The operationsto be per- simulationsprovidean immediateexamplesuitedto this formed, suchas matrix multiplication,can be separated treatment. Other examplesincludematrix multiplication intoindependent component parts,eachof whichis carded or scalar products,both of which can be parallelized. out independently, "in parallel." Often,the instructions to Second,the algorithmcanbe reorderedso thatthe degree a machine central processingunit (CPU) (e.g., loads, of parallelismincreases. stores,adds)can also be cardedout in parallel,yielding This brief introductionindicates that realizing the is often more "fine-grained"parallelism. Very long instructionword potentialperformanceof a supercomputer (VLIW) machines [Fisher, 1984] overlap processor than simply loading and runningan existingcomputer instructionsthat can be carried out in parallel with code. The characteristics of the machinebeing used speedups canbe substantial increases in computation rate. The advantageis dictatethe methodsby which substantial that VLIW machinescan operatein scalarmode,relying obtained. Supercomputers can be usedto providevery on a sophisticatedcompiler to produce fine-grained substantial gainsin solutetransportmodelingcapability of the relativeimportance parallelism in object code. There are, apparently,no and concomitantunderstanding VLIW machines that are supercomputers,although of theprocesses involved. Problemsinvolvinglarge-scale, heterogeneous domainsand complex chemicalreactions minisupercomputers of thistypeareavailable. Simply transferringcodes from serial to vector or becomeopento analysis.However,the time takento mna parallel machinesmay not result in very impressive problem is always a limiting factor to some degree. speedups.Thispointis demonstrated clearlyby Pelka and Thereforethe designof a computercode is worthy of Peters [1986], who comparecomputation ratesfor scalar serious consideration. General methods by which canbe utilizedefficientlyare one focusof and vectorizedfinite elementsolutionsof a groundwater supercomputers flow model. Particularly for parallel machines,the thisreview. An overviewof transporttheoryfor a single flow is presented algorithmsusedmay turnout to be not very efficient. Not solutespeciesin a density-dependent surprisingly,algorithmsfavoredon sequentialmachines first, followed by numericalapproaches to solvingthe tend to be those that minimize the number of arithmetic Effect of Portiol Foctorizotion operations necessary to completethe taskat hand. Vector processorscarry out pipelined operations. Pipelining 102 refersto theprocessby whichthearithmeticoperations are brokendown into a numberof elementarysubcomponents. Eachsubcomponent operationdependson theresultsfrom the previoussubcomponent.By operatingon vectors,the calculationson each pair of operandscan be performed simultaneously in assemblyline fashion,hencethe term pipeline. The natureof thepipeliningis suchthatfor some time no restfitsare availablewhile the necessarysubcom5 ponentoperationsare completedfor the first time. This is known as the start-uptime. It is clear that shortvectors 100 shouldbe avoidedas muchas possible,sincestart-uptime 0.2 0.4 0.6 0.8 1.0 0.0 is incurredfor every new pair of vectorsoperatedupon. The effect of the degreeof vectorizationof an algorithm, Froction Vectorized as givenby Amdahl'slaw [Amdahl,1976;Eramen,1987], Figure 1. Relative performance,or speedup,R/S, plottedon a is presentedin Figure 1, whereit is shownthatthe relative logarithmicscale as a function of the fraction of the code (or actual)performanceis a functionof the fractionof the vectorized.R is the realizedspeedandS is the machine'sscalar algorithmthat vectorizes. The curve labels list possible speed.Threehypotheticalvectorprocessors are considered, with speedup,i.e., the ratio of possiblevector speedto scalar the curve labels corresponding to the value of R/S for fully speedon the computer. A machinethat has a possible vectorized code. 28, 3 /REVIEWSOF GEOPHYSICS governingequations. It is shownthat solutetransport models are certainly candidatesfor numerical solution usingsupercomputers. The foundationof thesenumerical solutionsis solving linear systemsof equationsin an efficientmanner. Modem algorithmsdevelopedfor vector and parallel machines are presented. Data analysis constitutesanotherarea where supercomputers can be of great benefit. A summaryof conceptuallysimple but highly computationallyintensive,nonparametric estimation proceduresis included. The number of studies concerningvariousaspectsof solutetransportthat have usedsupercomputers is relativelylimited. There is little doubt that supercomputer-based researchendeavorswill increase. Even consideringthe existingresearch,it is apparent thatvery littleefforthasbeendevotedto "pushing the envelope"of supercomputer performancein solute transportapplications. Barry:SUPERCOMPUTERS AND SOLUTETRANSPORT ß 279 nonisothermal, multiphase, multispecies, densitydependent solute transport in variably saturated heterogeneousporous media. The parameterizafion requirements of sucha model could well exceedavailable knowledge of the necessary consfitufive or chemical relationships. Even with adequateresolutionof these issues, the numerical solution of such a model is a significanttask requiting,certainly,a supercomputer for results to be obtained in a reasonabletime scale, if at all. Therehavebeenfew attemptsto developgeneralcodesfor theseextreme, but relevant, conditions. The densityof subsurfacewater is very often nearly constant,or at leastit is assumedto be so. The density varieswith changesin temperature, pressure,or presence of dissolvedchemicals. Density changescan become noticeable if thesequantifies undergolargevariations.It is possiblefor dissolvedsolutesto produceextremedensity changes. In some countries, waste disposal in salt formationsis under consideration. In such areas, and in THEORY OF MISCIBLE FLUID FLOW the freshwater/seawater interfacesof coastalregions,the density dependenceof the fluid componentsmust be Solutetransportproblemsmustalwaysbe consideredin considered in any modelingof the system[Huyakornet al., the contextof a particularscale,e.g., a catchmentbasin, 1987; Hassanizadehand Leijnse, 1988; Herbert et al., part of a basin, a geologicalstructure,or a laboratory 1988]. apparatus. Many researchershave developedtheories In groundwaterapplications,densityeffectsof thermal applicableto the macroscale, definedas beingmuchlarger variationsare typicallyignored. Likewise,the problemis thanthe microscaleor pore scale[e.g., Gelhar et al., 1979; simplified further if only nondeformablemedia are Chu and Sposito,1980, 1981; Gelhar and Axness,1983; considered. Both of theseeffects will be importantin Winter et al., 1984; Dagan, 1982, 1984, 1987]. These somecases,and both are the subjectof readily available foundationaltheoriesare not directlyrelevantto supercom- treatments. For example, the problem of heat and mass puterapplications exceptinasmuchastheyarenecessary to transportis discussedby Bear [1972, section 10.7.2]. provide the governingmodelswhich must be solvedby Thermaleffectscannotbe ignored,however,if geologic some means. It is worth noting, however, that the time scalesare considered.Bethke [ 1985] andBethke et al. approachesused in deriving these theoriesare open to [1988] modelreactivechemicaltransportin sedimentary debate. For example,Spositoet al. [1979] identifiedand basinsoverlongtime scalesmakinguseof a machinewith reviewed three approachesused to derive foundational a novelparallel-vectorstructure[Kucket al., 1986]. theoriesleadingto the commonlyusedmacroscopic solute A thoroughderivationof the governingequationsfor a transportequationapplicablein an isothermal,saturated singlesolutespeciesin a rigid, saturated mediumis given porousmedium: by Hassanizadehand Leijnse [1988]. With suitable assumptions theyrecoverthe familiar forms: Oc/Ot= V.(D'.Vc)- v.Vc (1) o•p k (2a) n -• = V.p•.(Vp - pg) wheret is time, D' is the solutedispersiontensor,v is the solute velocity, and c is the solute concentrationin &0 k solution. Equation (1) is just the familiar convectionn-•- - •.(Vp- pg).Vc0 =V.(D.Vc0)(2b) dispersionequation(CDE). The threeapproaches, which are basedexplicitlyon, or are tantamountto, averaging In obtainingtheseequations,Hassanizadeh and Leijnse operationsapplied at the microscale,are all subjectto [1988]haveassumed thepresence of a rigidporousmatrix either mathematicalor physicalassumptions which have and isothermalconditions.No productionor removalof yet to be provedrigorously[Spositoet al, 1979]. soluteis accountedfor. The densityof the fluid is Under appropriatecircumstances, foundationaltheories dependenton both its solute mass fraction c0 and fluid producemacroscopicequationslike (1) that, with some pressure p. A suitablerelationship betweenthesequanassumptions,are applicable in practice. Typically, a tifies is given by the densitystateequation[Bear and numberof coupled,nonlinearequationsare reducedto a Verruijt, 1987, section3.2; Hassanizadehand Leijnse, few, possiblydecoupled linearequations usingsimplifying 1988]: assumptions.It is apparent,however,that complexfield applicationscould require numericalmodelsdescribing p(c0,p)/po: exp [yc0- [•(p - Po)] (3) 280 ß Barry:SUPERCOMPUTERS ANDSOLUTE TRANSPORT 28, 3 / REVIEWS OFGEOPHYSICS Thecoefficients ), and[3in thisformulaarebestconsidered movement(approximately1 m in 120 days)was observed. as fittingparameters whichattainspecificvaluesfor given Identifiedas possiblecausesfor this downwardmovement solutes.The dynamicviscosityof thefluid, •, will change were the presenceof a vertical componentin the local with to as well: groundwatervelocity field due, perhaps, to aquifer g = m(to)go (4) recharge or, second, density differences between the injectedplume and the native groundwater. The relative magnitudeof theseeffectsis uncertainat thistime. where re(to)is a functionto be specifiedfor the specific SoluteTransport solute. If go is definedto be the reference dynamic Stochastic viscositywhento= 0, thenm(0) = 1. The ease with which numerical solutions of the nonlinearsystem(2) are obtaineddepends,amongother things,on the dimensionality and extentof the spatial domainand the degreeof variabilityof the coefficients. These factorsalone, in the contextof a practicalsimulation, make the use of a supercomputer desirable. If the systemis furthercomplicated by, for example,a nonlinear chemicalreactionor the coupledtransportof a numberof solutes,then the need for a supercomputer becomesmore advantageous. To show that (2) reducesto (1) as a specialcase,we consider the definitions of concentration and mass fraction. Solute concentration c isequivalent toMs/Vr, andthemass fraction tois equivalent toMs/Mr Thusc/to= M•Vr-- p, or to = c/p. Figure 1 of Herbertet al. [1988] showsthat, for a solutioncontainingup to 26% of saltby mass, P/Po-- 1 + 0.2to O<to<l (3') For to << 1 (dilutesolutions), to-- c and!x -- go. Under steadyflow conditions,(2b) thenreduces,approximately, to (1). An alternativeexpressionto (3) or (3') for the solutedensitystateequationis availablewhenthe volumes of the componentsin the final solutionare additive Recent, comprehensive reviews of the stochastic approachesto water flow and solute transport in groundwaterhave been publishedby Dagan [1986] and Gelhar [1986]. The startingpoint is the recoupledform of thewaterflow andsolutetransportequations, meaningthat a dilute solute is considered. The logarithm of the hydraulic conductivity,In (K), is treated as a random spatialfunctionwith known constantmean and spatially stationarycovariancefunction. Stationaxityrefers to the assumption that the covariancefunctionis unchangedby spatial translation. The low-order statistics of the piezometrichead (i.e., fluid potential)and flow field can then be calculated. With the assumptionof a constant porositythe statisticsof the groundwatervelocity field follow directly. The stochasticsolutetransportequationis just (1) exceptthatv is now a randomspatialfunctionwith known low-order statistical properties. The solute concentration c is the dependentvariablein (1), and hence it is a randomfunctionas well. Theoreticalanalyseshave focusedon ensembleaveraging,i.e., averagingover all possiblerealizationsof the velocityfield in (1) to derivea governing equation of (c), theensemble meanconcentration [GelharandAxness,1983;Dagan, 1984, 1987, 1988; Neuman et al., 1987]. Theories based on the stochastic CDE essentiallytreatthe solutedrift velocityas a random variable with assumedlow-order statisticalproperties. They are, however,not withoutfundamentalproblems,as (3') discussedby Spositoet al. [1986]. These authorsnoted P/P0= pS/[(1- to)pS+ toP0] that each aquifer representsa single realization of the processunderconsideration, whereasthe results Herbert et al. [1988, Figure 1] show(3') is likely to be a stochastic goodapproximation in practicalcases.However,it should of a stochasticanalysisrepresentthe statisticsfor the be notedthat the dependence of fluid densityon pressure ensemble of realizations. The results of the stochastic CDE approaches,therefore, are not applicable to a hasbeenneglectedin (3') and(3'). are made The governingequations canbe manipulated into forms particularaquiferunlessadditionalassumptions suitablefor numericalsolution[e.g.,Huyakornet al., 1987, [Dagan, 1984]. Numerical simulationsreinforcingthis below. AppendixA; Hassanizadehand Leijnse,1988]. It is very pointarepresented commonto ignorethe couplingbetween(2a) and (2b) for The ensemble mean concentrationstaffsties, aptransportequationsimilarto (1), dilute solutions. The questionof how dilute the solution proximately,a macroscale has to be before this assumptionis valid is not easily exceptthat the dispersion coefficientcomponents are now answered. Even for very dilute solutionsmoving in the functionsof time. For a coordinatesystemalignedwith subsurfaceenvironmentthis assumptionmay need to be themeanflow direction(axis 1), SpositoandBarry [1987] coefficients consideredcarefully, particularlyin aquifers. Sudickyet derivedthe macrodispersion t al. [1983] and Freyberg [1986] analyzed nonreactive solute transportexperimentsat the Borden site. These DMii(t) =I I Sii(v:, t')exp (-•Dii•i) cos (vvq t')d}cdt' 0 i (5) experimentstrackedthe movementof plumesin a sandy, unconfinedaquifer [MacFarlane et al., 1983]. In both experimentsthe solutemassfractionof the injectedfluid whereSii(lc,t) is theFouriertransform of thestationary [Herbert et al., 1988]: wasverysmall(<5 x 10-3),butappreciable vertical plume velocitycovariance function andDii arelaboratory scale 28, 3 /REVIEWSOF GEOPHYSICS Barry:SUPERCOMPUTERS AND SOLUTETRANSPORT ß 281 dispersion coefficients. Oncethefunction Siiis specified, vectors,respectively,to be determined,and the coefficient the macroscaledispersioncoefficientscan be evaluated withoutdifficulty. A major advantageof the theory lies in its practical applicabilitywhensomebasicconditionsare satisfied.For example,the authorscitedaboveassumethe presenceof a large-scalevelocity field with a uniform, constantmean. Becausethe theoryquantifiesthe effect of the local-scale matricesare given by Huyakorn et al. [1987]. The finite element equations(7) representa transformationof the model(6) to a systemof algebraicequations.Note thatall numericalsolutions involvethesolutions of linearsystems. Perhapsthe mostcrucialpart of usinga supercomputer efficientlyis solvingsuchsystems efficiently.Because of the importanceof linearalgebraicsystemsin the solution velocityvariationson solutedispersion, one needs"only" of solutetransportmodels,varioussolutionmethodsare to estimate the low-order statisticsof the hydraulic examined in detail below. conductivityfield in order to predict macroscaledisperHavinga supercomputer availablecaninfluence even sivity, and thus spread, of large-scalesolute plumes. the basicapproachto obtainingnumericalsolutions.The Various researchers have investigated the value of theoreticalpredictionsin the light of field experimental data. The main body of analyticalresultsderived using stochastictheory applies only for ideal tracers under assumptionsthat may not be widely applicable. The degreeto whichthe assumptions canbe relaxedin practice needsto be clearlydefined. Second,theeffectof chemical and biologicalreactionsat the field scale is difficult to determineanalytically. Some of theseissueshave been investigated usingsupercomputer simulations, asdiscussed below. Numerical Solutions finitedifference method applied to theaboveproblem would give a differentlinear systemof equationswhich would also have to be solvednumerically. The trmite element method, although more suited to irregular domains,is usually time consumingto code. Finite differencemethodsare more simpleto set up but not as versatile. A scientist must invest considerable time and effortto setup a problemsothata high-accuracy numerical solutionwith reducedCPU time and storagerequirements will result. The question which then arises is whetherthis investmentis justified in the face of the relativelycheap,highcomputation ratesandlargememory availableon a supercomputer. Fully three-dimensional,transientnumerical solutions of the completegoverningsolutetransportequationshave notbeenthe subjectof manystudiesas the computational MATRIX EQUATIONS: MODERN SOLUTION load is very large. Saltwater intrusionposesa serious METHODS threat to groundwatersupplies in many coastal areas [Newport,1977], and this haspromptedmodelingefforts, In thissection,algorithmsleadingto the solutionof the many of which have beenbasedon the "sharp-interface" linear systemAy = b (whereA is a known, often banded, assumption. A variety of numericalsolutionshave been squarecoefficientmatrixof dimensionN, y is an unknown presentedby Lee and Cheng [1974], Segolet al. [1975], vector, and b is a known vector) will be discussed. Segol and Pinder [1976], Huyakornand Taylor [1977], Elements of these arrays areidentified by aij forthe Taylor and Huyakorn [1978], Frind [1982a, b], and elements of A, etc. The typeof supercomputer available Huyakornet al. [1987]. The Galerkinapproachis used will be the arbiterof the bestalgorithmto use. In partypically. For example,Huyakornet al. [1987] consider ticular, algorithms designed for a singlevectorprocessor the governingsystem cannotbe expected to performwell on a massively parallel machine. The literature devoted to numerical methods for the solutionof linearsystems of equations on supercomputers is very extensive, and only a small overview canbe V-[K-(V• + c)]=Ss -•-+nrl-• Pw attemptedhere. Ortegaand Voigt [1985] reviewgeneral applications of vectorandparallelmachinesto the solution •c of partial differential equations.For the purposes of this V-(D-Vc)-v-Vc=n•-• +q(c- Co) (6b) section,we will refer frequentlyto the degreeof paralP/P0 = 1 + ec/c• (6c) lelismof an algorithm, meaningthenumberof operations that can be carriedout at the sametime [Hockneyand Applicationof the Galerkintechniquefor theseequations Jesshope,1981, section5.1.2]. On a vectormachinethisis yields[Huyakornet al., 1987] givenby themaximumvectorlengththatcanbeprocessed at a singletime, andon a parallelmachineit is thenumber (7a) of processors operatingconcurrently. A•j•j + B•j•- = F• jr = 1,..-, mt •_• a• acpq (6a) ~ do= • Eucj+ B•j•- I = 1,..., mt (7b) Direct Methods The most familiar direct method is Gaussian elimina- form wherethesubscripts referto nodes, mt is thetotalnumber tion,whichsolvesAy = b by reducingA to triangular of nodes,•j andcj are unknownheadandconcentrationand thenproceeding to arriveat the solutionby back 28, 3 / REVIEWSOF GEOPHYSICS 282 ß Barry:SUPERCOMPUTERS AND SOLUTE TRANSPORT substitution. On vector machines it can be shown that Gaussian elimination is not a suitable method of factoriza- tion. written as k+l k • k+l k Yi =Yi+w i - •aiiyi - •,•aii yi a•i• Parallel machines fare somewhat better, since for j=l each step in the reduction,each columnof A can be handledindependently.Voigt [1977] detailsthe efficient implementationof the factorizationphaseof Gaussian eliminationin the Cray-1. Ortegaand Voigt [1985] point out threedrawbacksto usingfactorization algorithmson parallelarrays: (1) the degreeof parallelismis reducedat eachstep,(2) significant communication betweenprocessors is necessary, and (10) j=l i:I...,N k>0 wherew is the relaxationparameterand k indexesiterations. The scheme,which divergesunless0 < w < 2 [Hager, 1988, section7-3], is usuallytermedsuccessive over relaxation(SOR) for 1 < w < 2 or successiveunder (3)it ma:9 be•difficult toschedule theworkefficienfiy. An relaxation(SUR) if 0 < w < 1, while it reducesto the algorithmif w = 1. example of drawback 3 is row pivoting, i.e., swappingGauss-Seidel Althoughthe Jacobimethodis eminentlysuitedto rOwsin the matrix to avoidroundingerrorsinducedwhen on parallelor vectormachines,it converges dividingby smallnumbers.It is likely thatintroducing a computation time,themorerapidly sequential processsuchas a pivotingstrategywouldcause tooslowlytobeuseful.At thesame convergent SOR methoddoesnot presentany easymutes substantial programming difficulties. by parallelmethods.Asynchronous SOR Anotherapproachis to Usethe GiVensor Givens- to computation Householderalgorithms [Young and Gregory, 1972, [Kung,1976;Baudet,1978;BarlowandEvans,1982]is an section4.2] to tridiagonalize A by a seriesof plane alternativefor parallelmachines. Basically,Jacobi-like processors, with rotations. For parallelprocessors the Givensalgorithmis iterationsare carriedout by independent pickingup the mostrecentiteratesfrom a slightly more effective [samehand Kuck, 1978]. The eachprocessor therate resultingtridiagonalsystems canbe solvedby substitution commonmemory.Underfairly generalconditions is not reducedwhen comparedwith the usingthe Thomasalgorithm[Noye,1982]. The Thomas of convergence algorithm,however,is manifestlyserial. For example,a usualserialform of thealgorithm. An interestingiterative scheme,termed the quadrant lowertriangular,uppertriangular(LU) factorization needs interlockingmethod, was developed by Evans and to incorporate operations of the type Hatzopoulos [1979],Evanset al. [1981]andEvans[1982]. ThecoefficientmatrixA is setequalto WZ, where Ui : di - f i ei-1/Ui-1 i: 2,' '', N where the elementsof the upperand main diagonalsand subdiagonals aregivenby d/, ei, andf/, respectively, and theui arethediagonal elements of theuppermatrixof the LU factorization. Thedependency of ui onui-1precludes vectorizationor parallelizationof this statement. VLIW machines,on the other hand, perform very well in this situation.An immediateway to parallelize(8) is to solveit iterativelyin vectorform [Traub, 1973]. The convergence rate of the iterationsincreaseswith increasingdiagonal dominanceof the system. Parallel direct methodshave beendevelopedaswell. Stone[1973]reformulates (8) as 1 (11a) i (9) and whereui = q]qi-1. All theqi canbe foundin log2 N iterations. Bycomparison, serial machines solving (8)take time proportionalto N. Z• Iterative Methods. Iterativemethodstendto be more suitedto paralleland vector machines than direct methods. The Jacobi and Gauss-Seidel methods are well-known examples of iterative schemes. Relaxed Gauss-Seidel iteration is (11b) 28, 3 /REVIEWSOF GEOPHYSICS Barry:SUPERCOMPUTERS AND SOLUTETRANSPORTß 283 The triangularportionsof (11) are filled with elementsto be determined. These unknown elements are calculated beginningwith the outermostelementsin the unknown matrices.The algorithmproceedsby first assigning Zli = au ZNi= aNi i = 1,. --,N (12a) Next, the2 x 2 linear systemsare solved: 1. Incrementk by 1, thencalculate [•: r•r•/p•np• r,+l = r, - •k Apk ^ 1 ----B-1 rk+ 1 rk+ 0•k: rk+1rk+l Pk+l = rk+l + O•kPk i=2,...,N- 1 (12b) •iN•NN3LWiN3 Ld'•J 2. Compute Y,+I= Y,+ •,P, 3. If convergence conditionis notsatisfied,go to step1. If A is not symmetricand positive definite but is Equationssimilar to (12) can be solvedto find the other unknownelementsof W and Z. Eachring of elementscan be calculatedindependently, andthecomputation is suited to parallelprocessing.With theseoperationscompleted, the originallinearsystembecomes invertible, thenpremultiplying theequation Ay = b by Ar givesa systemin therequiredform. The PCG algorithm theninvolvesmorecomputations, however. An alternative is the ORTHOMIN algorithm[Behie and Vinsome,1982]. Qualitatively, theORTHOMIN algorithmis similarto the onegivenabove,althoughthe computational work load is increasedbecauseof the nonsymmetric coefficientmatrix or A. Details of the algorithm,relatedvectorizationissues, Wz = b (13b) andapplications to petroleum reservoirs aregivenby Behie Zy = z (13c) and Forsyth[1983, 1984]. On vectormachines,PCG-typemethodsappearto be Equation (13b) is solved for the intermediateunknown themostefficientavailablefor linearsystems of equations vectorz, whereupon(13c) is solvedfor y. encountered in subsurface flow problems,as shownin a WZy = b (13a) recentstudyby Meyer et al. [1989]. The choiceof The Preconditioned ConjugateGradientMethod preconditioner is related to the type of computerused. The conjugategradientmethod(CG) wasfirst presented Meyeretal. [1989]compared a number of preconditioning by Hestenesand Stiefel [1952]. The methodis basedon schemesimplementedon the Cray X-MP/48, a vector the fact thatthe solutionof Ay = b minimizesthe quadratic machine,and the Alliant FX/8, an eight-processor functional yrAY - 2bryif A is symmetric andpositive vector-parallelmachine. Overall, low-orderpolynomial definite [Hager, 1988, section7-5], and the superscript T preconditioning [e.g.,Saad, 1985] was foundto give the indicatestransposition. It is this functional which is bestresults.The largestproblemsolvedby Meyer et al. minimizedto obtainthe unknownvectory. In the absence [1989] involved 980,000 unknownsand took apof rounding error the method convergesto the exact proximately 1 hourCPUtimeontheAlliant,witha factor solutionfor y in N steps. It has been observedthat for of 15 speedup expectedfor the Cmy. Ababouet al. [1988] some matrix types a reasonableapproximationto the useda Cray-2 to performa large-scalethree-dimensional solutionis obtainedafter relatively few steps,leading to flow simulation but did not use the PCG method. For this the use Of the methodas an accelerationtechnique. The case, Meyeret al. [1989]estimated thatusingpolynominal convergence rate of CG dependson the conditionnumber preconditioning withCG wouldhavereduced thecomputof A. PieconditioningiS therefore used to increasethe ationtimeby an Order of magnitude. Thisexample convergerice rate [Hager, 1988, section7-5]. Concuset al. highlightsthe importance of algorithmselectionfor [1976]notethefollowingadvantages of the method: (1) large-scaleproblems. not all of A needsto be storedin memoryat one time, (2) unliker•laxationschemes, e.g., (10), parameters usedby Other Methods the algorithmare automatically determined, and(3) some Othersolutionmethods,althoughuseful,havenot been restrictions on A requiredby othermethodscanbe relaxed. discussed in thissectionbecauseof spacelimitations.For A vectorizedpreconditionedCG (PCG) algorithm was example, many problems lead to sparse coefficient givenby Kershaw[1982]. matricesthatare solvedefficientlyby usingthe multigrid The first step in the algorithmis to choosea precon- method[McCormick,1987]. Supercomputers have been ditioningmatrix B, relatedto A, suchthat B is symmetric appliedin a varietyof multigridapplications [McCormick, positivedefiniteandthe system Bf = r is easilysolved. 1988]. Spectralsolutionmethodsexistas well [Voigtet For example,B could be the tridiagonalpart of A. One versionof thePCG algorithmis 0. Guess Y0.Solve B•0= b - Ay0= r0. LetP0= •0. al., 1984]. Further information can be obtained from the reviewof Ortegaand Voigt [1985] or the summaryof HockneyandJesshope [1988,chapter5]. 284 ß Barry:SUPERCOMPUTERS ANDSOLUTE TRANSPORT DATA 28, 3 /REVIEWS OFGEOPHYSICS ANALYSIS computationallyintensivetechniquesare reviewedin the following. In doingso,we echothe sentimentexpressed in New or updated solute transport models appear therecentarticleof TichelaarandRuff [1989], who aimed regularlyin thehydrological literature.Like anyscientific to "promote the use of resampling techniquesin theory,modelsmustbe testedand evaluatedby compari- geophysics." Presume that the model under consideration is a scalar sonwith experimentaldatato ascertaintheir usefulness in (known)variablesE practice.Two typesof testsareperformedcommonly.In functionf=f(E; P) of theindependent parameters P, where E is a matrixandP isa the first, more rigorouscase, the model parametersare andunknown determined independently of the experiment,if possible. vector composed of elements pj. Theexperimental data Then, with knowledgeof the experimentalconditions,the modelis appliedto predictsomeinitially unknownaspect of the experimentalsystem.The secondcase,whichis not a modeltest,is simplyto fit the modelto the databy some procedurewhich determinesits parameters.If the model fits well and the parameter values coincide with the analyst's expectations, then the model has been "validated." We refer to thesetwo possibilitiesas testing by predictionandtestingby validation,respectively. An exampleof the experimentaldifficultiesinvolvedin consist of nemeasurements, f/(E/),i = 1, ßß-,ne.Def'me a scalar "distance" function asH(e•,e2),wheree• ande2 are locationsin the spaceof independent variables.Usually, wetakeH(ex,e2)= lex-e 2 12or H(ex,e2)= lex - e21,wheretheformerchoiceis thesquare of Euclidean distance and the latter is the absolute value of the distance. In these and many other cases, H satisfies some "stationarity" assumption, i.e.,H = H•(r),meaning thatH is unchangedby translation. In fitting the model, one minimizes the discrepancies between the model and the applyingtheorypredictively is provided by the Borden databy suitablechoiceof P; i.e., chooseP suchthat aquifersolutetransportexperiment[Robertsand Mackay, 1986]. This extremely valuable and meticulouslyperformed experimentwas carded out in a hydrologically d= •]H[f(E;P)- fi(Ei)1 (14) i=1 uncomplicated aquifer. Known quantifiesof soluteswere injectedinto the aquifer,and their subsequent movement is minimized. Minimizationof d can be achievedby was monitoredby an extensivesamplingarray over a differentiation of (14) with respect to the model 3-year period. By averagingthe experimentaldata over parameters, or the vertical axis, Freyberg [1986] and Sudicky [1986] foundthat the theoryof Dagan [1982, 1984]predictedthe temporal growth of the tracer plume's second central i=1 momentsreasonablywell. Analysesof this experiment havehighlightedthe largeuncertainties introducedby lack wherenp is the length of P. The combination of of knowledgeof the initial configurationof the solutein parametersyielding a minimum d is obtainedby the theaquifer[Barryet al., 1988;Naff et al., 1988]. solutionof thesimultaneous equations: Becausetesting by prediction is not often a viable option, testing by validation becomesthe standardby 3d/3pj= 0 j = 1,..., nt, (16) which modelsarejudged. In a given casethe inclusionof substantial experimentalnoiseensuresthata large classof Equation (16) is, in general, a system of nonlinear models fits the data equally well. This scenariowill equations. In this outline of regressionit has been alwaysfavor moresimple,evenempiricalmodels,as there assumedthat the model outputis scalar. Extensionto a is little point in invokinga complex,albeit more theoreti- vectorof predictions followswithoutdifficulty. cally soundmodel if it offers no advantagesover simpler Traditionallinear regressionresultswhen the system approaches.Thereforeexperimentaluncertaintiescan be (16) is linearin the elementsof P, the functionH in (15) seento havea profound effectonthedegree to whicha measures the square of Euclidean distance, and the new theoryis regardedasbeingan improvement. distributionof predictionerrorssatisfiesthe normality Scientistsand engineersfrequentlywish to characterize assumption.After fitting the model it is necessary to a transportprocessin the light of a selectedmodel. checka posteriori,amongotherthings,thatthe distribution Mechanistically, this case is identical to testing by of residuals, si (wheresi = H(f(E; P) -f/(E/)]), validation in that the goal is to determinethe model's i = 1, ß ß., n,, is approximately normal[CoxandSnell, parameters.Nonlinearregressionis usedto fit the model 1968]. If this conditionis satisfied,it follows that one can and so obtain estimatesof the parameters. Statistical calculate a covariance matrix of the residuals as well. theoryprovidesa very elegantsolutionto this problemfor Lineartheorygivesbothan estimateof P andan indication linear regressionwith normally distributedmeasurement of theaccuracyof thisestimate. errors. Efron [1982, chapter1, 1988] suggeststhat the constraints imposedby parametricstatisticaltheorycanbe The Bootstrap abandonedin an age of readily available, large-scale What happenswhen the residualsdo not satisfythe computersand inexpensivecomputation. Some of these normalityassumption or whenthe modelis nonlinear?A 3d_ 3 •-/[f(E; P)- f•(Ei)]j =1.. n•, (15) 28, 3 /REVIEWSOF GEOPHYSICS Barry:SUPERCOMPUTERS AND SOLUTETRANSPORTß 285 centeredat thoselocations. An interpolationprocedure mustbe performedto estimateCoat unsampledlocations. is to use the data set to define the distribution of residuals. Again,thisinvolvesa nonlinearregression procedure,and in principle, the nonlinear regression procedure described This empiricaldistributionis resampleda largenumberof timesto createsyntheticdatasets. For eachresamplingan above, or any of the methodsdescribedby Yeh [1986] estimateP* of P is obtainedusingsomenonlinearfitting could be used. This includesmethodsfrom spatial methodsuchas that of Marquardt [1963]. The bootstrap statistics[Ripley, 1981], includinggeostafisticalmethods estimate of P is PB,the meanof theP*. The bootstrap [Journeland Huijbregts,1978]. However, as the aim is simplyto estimatethe integralin (18), simplealternatives estimate ofthecovariance matrix Ca ofPais basedon crossvalidationseemreasonable.One procedure 1 • (p,k_PB )(P*k - PB )r (17) couldconsistof definingtheinterpolator: Cz=n*-I useful computer-based method is the bootstrap. Efron [1982, section5.7] outlinesthe method. In brief, the idea k=l wheren* is the numberof resamplings andthe superscript k identifiesthe variousP* for the resampleddata set. Of •g [Hg (x,xi, t,ti);Plc0 (xi, ti) Co (x,t) = course,many variationsof this procedureare possible. (19) Note that the groundwaterliteraturecontainsexamplesof •g [Hg (x,xi,t,ti);el preciselythismethod[e.g.,Yehand Wang,1987],although i=1 the term bootstrappinghas not been usedas such. The researchof Yeh and Wang [1987] was directed at parameterizing solutetransportmodels.Theseauthorscite whereH g is a "distance"functionof both spaceand time, other, similar analyses. The role of supercomputers in e.g., H. m H.(x•, x2, q, t2), and the function parameterestimationis apparentfrom the observation of Yeh and Wang [1987] that the "intensivecomputational parametersin the vectorP. Elementsof P can be usedin effort required by the Monte Carlo method... may the definitionof Ho, e.g., to scalethe distancefunction. prohibit[its] applicationto... largescaleproblems." A Cross validation th%n consists of choosing P soasto morecompletereview of modelparameterization studiesin minimize S•, where nt nt groundwaterhydrology is given by Yeh [1986]. The generalityof thesemethodsis that they can be appliedin Scv:•Hr [Co (Xj, tj) - •co(xi, t/)l (20) j=• i=l conjunctionwith many existing applications. Recent examplesto which this commentappliesare theparameter g(/-/g; P)'generali[es/-/s' toinclude anumber ofadjustable estimation studies of Kool and Parker [1988], who analyzedunsaturated flow, andMishra andParker [1989], who dealtwith both waterflow andsolutetransport. Cross Validation Crossvalidationhas been usedfrequentlyto estimate spatialcovariancefunctions[e.g., Samperand Neurnan, 1989]. The methodhas at its base a very simple idea whichcanbe appliedin manycontexts.For example,data from the solutetransportexperimentat the Bordenaquifer had to go througha multistepanalysisbeforethe desired spatialstatisticswere obtained[Freyberg,1986; Barry et al., 1988]. In the case of soluteplumesthe statisticsof interestare often the plume's zeroth-, first-, and secondorderspatialmovements, from whichonecanestimatethe plumemass,meanpositionand variance. The calculation of thesequantifiesamountsto a numberof triple integrations to be carried out numerically. Central spatial momentsaredefinedby 1(t)I (Xl gij•(t) =M000 - gl)i(x2 - g2)/ ß(x3- !.t3)• Co(x, t) dx If g[ttg(x, x, t, t);P] = o%thenthefunction in (19)will reproducethe measureddata. The cross-validation idea, writtenas an equationin (20), is to try to predicteach measurement usingtherestof the dataset. The sumof the prediction errors, S•, isminimized toyieldP. Themethod givenhereis to try anynumberof differentfunctions, g, in (19), selecting the onethatgivesthe leastS•. Other functionalformscouldbe usedin (19) insteadof thelinear combinationsuggested.Note that g can be definedto includeonly a limitedspatialor temporaldomainin (19). A variationof thisprocedure (calledthe"supersmoother"), discussedby Efron [1988, section6], is to define the functiong as a linearinterpolantand thento selectthe size of thespatialdomainusinga criterionlike thatin (20). Whena suitableg is chosen, Coin (18) is replaced by the interpolantin (20), and the required integrationsare performednumerically.The momentestimatesobtainedin this mannerare point estimatesonly. It is possibleto calculateestimatedvariancesof these point estimates, againby someresamplingscheme. One suchschemeis jackknifing. (18) lackknifing whereMooo(t) is theplumemassandCo(X, t) = c(x,t)0(x,t). Assume,as in (18), thatwe wishto derivea statistic,Q from a dataset. The methodusedto calculateQ (e.g., to be biased. number nt of locations or asaverages oversmallvolumes maximumlikelihood)is knownor suspected The functioncowill be knownor estimatedat only a finite 28, 3 / REVIEWSOF GEOPHYSICS 286 ß Barry:SUPERCOMPUTERS AND SOLUTE TRANSPORT Jackknifing[Quenouille,1956;Gray and Schucany,1972] concentration increments, domainsof higherconcentration is a techniqueto reducebias in the estimateof the statistic are surroundedby more surfacesthan areas of lower When(mathematical) lightraysarepassed Q. Thedataset,consisting of n•measurements, isdivided concentrations. into k groups,eachcontainingrn elements.Eachgroupis throughthe surfaces,they are diffused. Domainscontainremoved,in turn, and the statisticQ(/3, j = 1, ß ß., k, ing relativelyhigher concentrations appearas being corresponding to Q is calculated.The jackknifedestimate relativelymoreopaque.Thereis completefreedomin how thefinal imagesare vieweddueto variations in thelight Q•isthen given by Q: kQ - (k - 1)•j (21) sourceandobservation positions. Somerenditionsof thisprocessappearin Plate 1. The bromideplumeafter462 daysof traveltimein theaquifer is viewedasmoving,approximately, fromtheupperfight k to lowerleft portionsof Platela. Platelb providesa side viewof theplume.Theimageis clearlyveryunsymmetrij=l cal, the front and rear portionscorresponding to the verticallyaveraged bimodaltwo-dimensional plumesnoted appears to behighly Thevariance of Q?Var(Q), is estimated from[Tukey,byFreyberg[1986]. Solutetransport 1958] concentrated alongverystratifiedlayerswithintheaquifer. k The difficultyin achievingcompleteplumesamplingis Var (Qj)= k-k 1•[• _Q(0/)]2 (22)revealedby the laggingedge of the plume apparently j=l extending beyondthe imageintoan unsampled area. Plate Mosteller and Tukey [1968, sectionE2] showthat under lc displaysthe plume movingdirectlytowardthe viewthatsolutetransport is dominated by certainconditions, confidence intervals for Q•,, the point.Thesuggestion populationparameterbeingestimated, canbe derivedfrom the morehighlypermeableregionswithin the aquiferis where reinforced. TO= (QJ- Qt,) [Var(Qj)]-•/2 whereTQhasa t distribution withk- (23) findingof Sudicky[1986] that the ratio of the verticaland horizontalcorrelationlength scalesof the hydraulic at theBordensitesuggests thattheaquiferis 1 degrees of conductivity freedom. A detaileddescriptionof the constructionof confidence intervalsusingthisapproach is givenby Barry and Sposito[1990]. Visualization These observations are consistent with the stronglyanisotropicand, indeed, reflects the lenticular geologicalstructure.Thereis a clearindication,also,that a mobile-immobileregion model might provide an adequaterepresentation of solutebreakthrough in this system,asalreadyfoundby GolzandRoberts[1986]. Improvements in datavisualizationhavekeptpacewith Theimagesin Plate1 arebuta fewof a largegroup.In developmentsin computing hardware. Plotters and termsof computational effort,thetotalCPU timerequired associatedsoftware are standard equipment. Much to produceeachimagerangedfrom 10 rain to 10 hourson specializedhardwareand software is available also. A aneight-processor AlliantX-8 minisupercomputer. powerful combination for real-time simulation is the combinationof supercomputer computational ratesand a dedicatedgraphicsworkstation.An exampleutilizingthis SOLUTE TRANSPORT ON SUPERCOMPUTERS: technology wasundertaken aspartof a moregeneralstudy APPLICATIONS on solute transport. Three-dimensionalvisualizationsof thebromidetracerplumefrom the Bordensiteexperiment Subsurfacehydrology researchersare utilizing the [Roberts and Mackay, 1986] were generatedusing increasinglyavailablesupercomputer time and resources. facilities at the National Center for SupercomputingThe extentof supercomputer use,however,is notextensive Applications (NCSA). Briefly,theraw data,consisting of at the presenttime. An overviewof recentapplications solute concentrationmeasurementsat given spatial follows. locationsfor the 14 plume samplings,were fitted to an interpolationfunctionusing the NCSA's Cray X-MP/48 ParticleTracking computer following the cross-validationprocedure Nomeactivesolute transportcan be modeled quite describedabove. Full detailsare given by Barry and effectively by using particle-trackingtechniques. All Sposito [1990].Dense (4-8x 104points), regular grids of approaches to dateare basedon the dilutesoluteassumpinterpolated concentration datawereproduced, andsurface tion;i.e., thewaterflow modelis solvedseparately to give contours--three-dimensional versions of contour the groundwater velocityfield. This velocityis thenused lines--were computed using the "marching cubes" to provide the deterministic motion of the "solute" algorithm [Lorensen and Cline, 1987]. The contour particlesin the simulation, with an additional random surfaces areallowedto be semitransparent andsoallowthe componentproviding the dispersivemechanism. The passageof light. Sincecontoursare formedat regular theoretical basisof the methodis the Langevinequation. Barry:SUPERCOMPUTERS AND SOLUTE TRANSPORT ß 287 28 3 /REVIEWS OF GEOPHYSICS -6.314 27.179 t = 462 59.849 59.849 x2 -2.155 -7.277 x3 6.467 -2.155 -6.314 27.179 59.849 x2 Plate 1. Three-dimensionalimagesof the bromidetracerin the Bordenaquiferexperiment.In all images,the upperline of data gives the elapsedtime (in days) sincesoluteinjection,and the coordinates(in meters) of the soluteplume boundingbox in a Cartesian system alignedwiththemeangroundwater flow(thex• axis). Thex3 axisis positiveverticallyupwardfromtheground form of this stochastic differential equationis [Gardiner,1983,section4.1] dz/dt- a(z, t) + b(z, t) •(t) 6.467 -6.314 27.179 The one-dimensional -7.277 x3 x2 xl (24) -7.277 x3 6.467 -2.155 surface.(a) The plumeviewedfromabove,with thex• axis passingdiagonallyfrom the upper right to the lower left of the viewpoint. (b) Side view of the plume. The meanflow is from left to right. (c) Frontal view of the plume. In this case the plumeis movingdirectlytowardthe viewpoint. wherea andb areknownfunctions and•(t) is a stochastic forcing function idealizedas white noise. As it stands, (24) hasno physicalmeaningbecausewhite noiseis delta functioncorrelated[van Kampen, 1981, chapterVIII]. If, 288 ß Barry:SUPERCOMPUTERS AND SOLUTETRANSPORT however, we follow the It6 interpretation,then (24) is equivalentto 28, 3 / REVIEWSOF GEOPHYSICS upto107nodes. Simulations ofthis kind arepossible only on supercomputers. Another example of the particle-tracking method, althoughnot supercomputer based,was presented by t+At andDOll[1989]. Theseauthors developed a z(t+At)- z(t)=a[z(t), t]At+b[z(t), t] I •(t')dt' (25) Illangasekare t where At is a finite time interval. Clearly, different realizationsof •(t) will influencethe locationof the particle under consideration. Under suitableconditions, (25) is equivalentto the Fokker-Plankor Kolmogorov forwardequation[CoxandMiller, 1965,chapter5]' •}P 1 •}2(b2p) -3T - • •z2 - •(aP) Oz (26) P(z, t) is the probabilitydensityfunctionof the positionof any particle whose movementis controlledby (25). Conversely,the distributionof a "large" number of particleswill be describedby (26) [Barry and Sposito, 1988]. With appropriatechoicesof the functionsa and b, (26) is equivalentto the equationdescribingthe transport of a nonreactivesoluteunderthe assumption that density effectscanbe ignored. A commonproblemwith particletrackingmethodsis the noisysolutedistributions resulting from the simulations. Distributionsare derived by assigninga solutemassto eachparticleandaveragingover a specifieddomain size. The noise is proportionalto modelof reactivesolutetransportbasedon the discrete kernel[Illangasekare andMorel-Seytoux, 1978]numerical solutionof the verticallyaveragedflow equation. A particle-tracking schemewasimplemented to modelsolute transport.FollowingGarderet al. [1964],soluteparticles were moved deterministically throughthe systemin response to the localdrivinggroundwater velocity. The concentration in eachcomputational cell was determined by arealaveraging assuming theparticlescoveredan area ratherthana pointlocation.Withineachcell an explicit finitedifferencesolutionof the solutetransport equation gave concentration changesto a numberof processes, including dispersion,sorption, and mobile-immobile regionexchange.The arealaveraging scheme wasvery effectivein removingthe problem of noisy solute concentrations. Numerical Solutions Conventional solutions basedon the governing equa- tionsof flow and solutetransportare morecommon. It seemsthat most applications,includingthosediscussed below, use the assumptionof a dilute solute,so that the 1IN'm, where N' is thenumber of particles. Themost governingequations maybetreatedseparately. straightforwardway to reduce noise is to increaseN', Perhaps the mostelaborate solutetransport modelyet although smoothingor filtering of the concentration developedand implemented on a supercomputer is the predictionsmay be more economicalcomputationally. DYNAMIX code[Liu and Narasimhan,1989a]. This is a Similar equationscan be derived for multidimensional multidimensional, multispecies, reactive chemical formsof (24). Thussolutions to (26) canbe obtainedby transport model that includes the transformation using a Monte Carlo approachapplied to (25). Each mechanisms: acid-base aqueous complexation, oxidationparticlein the simulations movesindependently, so (25) is reduction,precipitation-dissolution, and kinetic mineral well suitedto evaluationby a supercomputer. Indeed,the dissolution. The coderepresents theinfluence of superdegree of parallelism can be at least the number of computers in enablingmodelingeffortsthatmoveaway particlesin the simulation. Computationally,particle fromthesimple distribution coefficient (Kd)approach to tracking is algebraicallyuncomplicated and extremely modeling adsorption/desorption reactions [Liu and repetitive,makingfor an easilyvectorizedcode. Narasimhan, 1989a].Thisversionof DYNAMIX updates A recentstudyusingparticletrackingis thatreported by the modelof Narasimhan et al. [1986]. The transport Tompsonet al. [1988]. Theseauthorsreview in detail the modelwill handlelargenumbers of chemical components, theory outlined above, including the multidimensional species,andphases.The numericalsolutionof the model formsof thefunctions a andb in (24) whichareappropri- requiressolvinga set of multidimensional transport ate for simulationsof solutetransportfor both saturated equations, onefor eachchemicalcomponent, anda setof andvariablysaturated flows. The simulations of Tompson thermodynamic equilibriumchemistryequations. An et al. [1988] wereaimedat evaluatingstochastic theories, explicit finite difference scheme is used to solve the as discussedelsewherein this review. To that end,flow in transport equations.The thermodynamic equations forma a heterogeneous domain was generatedby using the nonlinearalgebraicsystemsolvedby usingthe Newtonlarge-scaleflow simulator of Ababou et al. [1988], Raphsonmethod. Within each elementof the numerical developedfor the Cray-2 supercomputer.Ababouet al. grid,completemixingof the aqueous chemicalspecies is [ 1988]usedthe stronglyimplicitprocedure, a lessefficient assumed. forerunner of the PCG method, to solve their linear systems of equations[Meyeret al., 1989]. Tompson et al. [1988] did not discussthe specificalgorithmdesignor computerusedin their simulations.Their code,however, is designed to trackmanythousands of particles ongridsof Exampleproblems solvedby DYNAMIX aregivenby Liu andNarasimhan [1989b].Afterfavorable comparison of DYNAMIX to the one-dimensionalcodes of Walsh et al. [1984] and Carnahan [1986], Liu and Narasimhan [1989b]performed a simulation of solutetransport froma 28, 3 /REVIEWSOF GEOPHYSICS Barry:SUPERCOMPUTERS AND SOLUTETRANSPORT ß 289 domain solutionfrom a single set of Laplace domain and arsenicwere included. Their 30-year simulation solutionvalues. Sudicky[1989b]extendedthe application periodshowed thatbothchemicals werestrongly retarded of the methodto the caseof fracturedporousformations. in the subsurface. The total CPU time needed was 3.4 In the case of the traditionalfinite differenceapproximahourson a Cray X-MP/14. The code does not take tion of the temporalderivative,numericaldifficultiesarise advantageof the Cmy's vector speed capabilities.because of the large differencesbetween the rapid transportin the fracturesand much Rewritingcomputationally intensive portions of thecode advection-dominated slower diffusional flux into the porousmatrix. Sudicky to allow vectorizationwould likely lead to substantial [1989b] combined the Laplace transformmethod with improvements in theCPU timeused. ORTHOMIN .acceleration for the rapid solutionof the A model that explicitly makesuse of supercomputer derivedfromapplication of characteristics in algorithmselectionwas developedby algebraicsystemof equations Chiang[1985](cf. Chianget al. [1989]). The codewas the Galerkinprocedure. The combinedprocedurewas developedto model pumping-induced flow and con- shownto be very efficientand accurate. A 30,651-node taminanttransportin a two-dimensional domain. The simulationof downwardseepageof solutein a medium fracturestook 72 s on a Cmy modelusesa modifiedmethod-of-characteristics procedure containingseveralthousand contaminationsite. The reactive trace elementsselenium to solvethe governing equations.Examplesimulations X-MP/24 to obtain solute distributionsvalid for all times especiallysuitedto vector includedtransport in domains havinga randomhydraulic of interest.By usingalgorithms Sudickyhasdeveloped a codecapableof conductivity field. Realizations of thelatterweredevel- supercomputers, verylarge,field scaletransport problems. opedby usingthe turningbandsmethod[Mantoglou and simulating Wilson, 1982]. The name "turning bands"denotesan Stochastic SoluteTransport Theory algorithmthatgenerates correlated randomvariateson a Evaluating As mentionedalready,the data base from the Borden groupof linespassing through a point. A two-or threeexperiment hasbeenusedto evaluatethe dimensional randomfield, with a predefinedcorrelation aquifertransport structure,is derivedfrom a seriesof orthogonalprojections predictionsof stochasticsolute transporttheory. Both fromthegroupof lines. Tompson et al. [1989]reporton Freyberg[1986] andNaff et al. [1988] comparedtheoretithemethod's performance andefficiency.Chiang[1985] cal predictionsof the vertically averagedtracer plume selectedthe PCG algorithm to provide the necessary dispersivitywith thoseestimatedfrom experimentaldata numericalsolutionto the linear systemof equations. It and foundexcellentagreement.Garabedianet al. [1988] was concludedthat the code developedwas efficient, analyzed reactive solute data from field experiments accurate,and capableof modelingtransportin systems conductedat Chalk River, Ontario, and Cape Cod, Massachusetts, usingtheorydevelopedon the assumption withrapidlyvaryinghydraulic conductivity fields. Frind et al. [1989] reporta three-dimensional simula- that both the flow field and the solute retardation factor are tion of transportof organicsolutesundergoing aerobic random.Again,onlytheplumevariancewasevaluated. Dagan biodegradation.They model a saturatedgroundwater Followingthe successof the two-dimensional systemin whichthe solutetransport process is described [1984] model in predictingthe spreadof the vertically by a nonlinear,coupledset of equations, one for the avemgecltracerplume at the Bordensite, Barry et al. contaminantand one for dissolvedoxygen. The dual- [1988] checkedwhetheror not the actualplumeconcentra- Monodequation[Bordenand Bedient,1986] is usedto tions conformed with ensemble mean concentrations. The model the growth of the microbialpopulation. By theorywas used to predict soluteplumesby using the plumefrom a numberof differentsamplings as formulating the numerical modelto produce a symmetric measured coefficientmatrix, the PCG solutionmethodcouldbe used initial conditionsand then comparingthe predictedand effectively. A 24 x 103 nodesimulation over155time measuredplumesat later times. The data analysisand steps(300days)tookabout2 hoursof CPU timeona Cray generationof the predicted plumes, which involved The authors concluded that comparable integratingthe initial conditionnumerically,were carded two-dimensional simulations were to be avoided, as out usinga vectorizecl codeon a Cmy X-MP/48. It was oxygenavailabilitywas markedlyaffectedby problem foundthat the modelpredictedthe measuredplumequite accuratelyon relatively short time scales, becoming dimensionality. Sudicky[1989a] considered the transportof a dilute progressivelyworse with increasing elapsed time. necessaryin the data analysis solutein a steady,nonuniform, two-dimensional flowfield. However,the assumptions of model-predicted and The spatialderivativeswere discretizecl by using the madeequivocalany comparisons Galerkinfiniteelementprocedure.Insteadof usinga finite measuredplumes. The ensembleapproachhas led to elegantanalytical difference procedure for the temporalderivative, Laplace by Dagan [1987], they cannotbe transforms were used. The Laplace-transformed problem results,but as discussed aremet. A demonstration was shown to not suffer from the numerical dispersion appliedunlesscertainconditions of the possible differences between separate plume problemsassociated with other methods. The Crump realizations is given by Frind et al. [1987]. Theseauthors [1976] algorithmfor numericalinversionof the Laplace- X-MP/24. transformed solution can be used to calculate the time use a two-dimensionalvertical slice model of an aquifer 290 ß Barry:SUPERCOMPUTERS AND SOLUTETRANSPORT 28, 3 / REVIEWSOF GEOPHYSICS with a hydraulic conductivity fieldthatreproduced the the directionof flow (and layering). Generationof the estimatedlow-orderstatisticsof the Bordenaquifer. The random hydraulic conductivityfield was producedby synthetic conductivity fieldinvolveda significant computational Cost, as it involved the numerical solution of a usingtheturning bands method.Thelocalwaterfluxes, nonlinear algebraic system of equations. Theperfectly and hencevelocities,were computedusingthe "principal direction technique" [Frind, 1982c; Frind and Pinder, 1982]. The computedvelocitiesin turn were usedin the solutionof the solutetransportequation.Two simulations stratified aquifer,although certainlyunrealistic physically, is useful becauseit tends to representa "worst case" scenario for ensemble averaging.An interesting resultof the simulations was that the concentration uncertainty wereperformed, oneof which used a spatial gridof 106 tends to increase with decreasingensemblemean elements. It was clearly demonstrated that local variations concentration. in the local hydraulic conductivity dominate plume Black[1988]extended theone-dimensional approach to movement,particularlyat early times. The short-term a verticalslice,two-dimensional model.Theturningbands behavior can have persistent,long-term effects. The methodwas usedto generate realizations of the In (K) simulation of Frind et al. [1987], which used 10 hours of CPUtimeona CrayX-MP/48, revealed theformation of field. Theturningbandsmethodrequires thegeneration of numerous one-dimensional correlated fields. Unlike most two subplumesdue to the initial placementof the solute previousinvestigators who usedthe methodof Shinozuka sourcein a regionof rapidlyvaryingconductivity. The andJan [1972]for thistask,BlackandFreyburg[1990] initialverticaldimension of theplumewasonlya factorof present an alternative dalled the "matrix-factorization 3.5 timesthe verticalcorrelationscaleof the In (K) field. moving average" method. The method, which was As discussed by Dagan [1987], the initial condition implemented on a CrayX-MP/48,wasshownto be both dimensionmust be much greaterthan the characteristic length scale of the conductivityheterogeneityfor the ensembleapproachto be valid. That the factorof 3.5 used by Frind et al. [1987] was insufficient for ensemble averagingto apply indicatesthe importanceof considerationsof scalein the applicationof the analyticaltheory. more efficient and more accurate than that of Shinozuka andJan [1972]. Aftergeneration of thetwo-dimensional correlatedhydraulicconductivityfield, steadyflow conditions wereproduced.Particletrackingwasthenused to simulatethetransport of a soluteplume. It wasshown that as the "verticalmultiplicity,"i.e., the ratio of the Other,similarsimulations reportedby Sudicky et al. [1990] verticallengthdimensionof the systemto the vertical demonstratethat in many cases,lessdramaticresultsare obtained. For example, Sudickyet al. [1990] discuss simulationsin which the solute plume approachesa Gaussian distribution, aspredicted by stochastic theory. Sudickyet al. [1990]go on to considerthetransport of a biodegradable solutein a heterogeneous porousmedium. correlationscaleof the In (K) field, increases,the mean distancetraversedby the plumeincreases becauseof the reducedinfluenceof low-conductivity regions. Also, underthesecircumstances the concentration uncertainty tends to decrease. In thiscasethereare threenonlineargoverningequations, DISCUSSION one each for the solute,the availableoxygen,and the CONCLUDING microbial population [MacQuarrie et al., 1990]. The behaviorof theplumeis quitedifferent.If thebiodegrada- No doubtothersupercomputer applications, apartfrom tion rate is large enough,then the plume will tend to thosediscussed here,areworthyof mention.On theother decrease in size. For thisreason,theplumewill encounter, hand,thetopicscoveredserveto illustrate possibilities for and be influenced by, only a small portion of the supercomputerapplications. Also, the mechanismsfor groundwater velocity field. Under these conditions, realizingsome degreeof the availablecomputational plumes resulting from different realizations will not performance in the solutionof algebraicsystemsof necessarily convergevery rapidlyto thosepredictedby equations havebeendiscussed in somedetail. In general, ensembleaveragingof the governingequations. More choosing algorithms suitedto theclassof computer being comprehensive simulations presented by MacQuarrieand usedcanwellhavea largepayofffor relativelylittleeffort. Sudicky[1990] showin detail that biodegradable solute Researchers withaccess to supercomputers naturallywant plumes evolve differently than plumes composedof to makeuseof existingcodesasrapidlyaspossible.It has nonreactive solutes.In particular,it is suggested thatthe beenpointed outalreadythatalthough codes maymnwith scaledependence of the plumedispersivity, whichis well littleor no modification on differentmachines, including documentedfor field scale transportof tracer solutes supercomputers, the speedup in execution timesexpected [Gelhar, 1986,Figure 12], doesnot applyin the caseof maynotoccurwithoutmodifying thecode.Thereis ample biodegradable, organicsoluteplumes. evidencethatcodesoptimizedfor scalarmachineswill not The appropriateness of ensemble averaging was achievehigh computational rates on currentmachines. investigatedalso by Black and Freyberg [1987], who The vast majority of scientificcodesare written in the simulated solutetransport of an idealtracerin a perfectly FORTRAN language. "Standard"FORTRAN 77 is not stratifiedaquiferconsisting of homogeneous layers. The well suitedto vectoror parallel machines,and manufachydraulicconductivity was assumed to vary transverse to turers have introduced extensions to allow machine 28, 3 /REVIEWSOF GEOPHYSICS Barry:SUPERCOMPUTERS AND SOLUTETRANSPORTß 29! Dagan [1986], in his review of stochasticflow and featuresto be exploited. For example,Pelka and Peters [1986] demonstrate the usefulness of the Cray's solu•tetransporttheory,notedthe vigorousgrowthin the GATHER/SCA•R featurefor vectorizingfinite element numberof researchpapersdevotedto stochastic modeling solutionsof the groundwaterflow equation. The research of groundwaterflow publishedin Water Resources of Sudickyandcoworkerscitedaboveandthatof Meyer et Research. In retrospect,this surge in publicationsis al. [1989] demonstratethe very substantialspeedupsthat entirelyreasonable giventhe applicabilityof thetheoretical can be gainedin a wide classof transportproblems. The approachto modeling fluid transportin heterogeneous developmentof more sophisticated compilerswill reduce permeable formations. Supercomputersoffer another the need for special programmingof codes. In the potentiallyvaluable avenue for the understandingand meantime, if researchersare to realize supercomputer resolutionof solutetransportproblems. Their rapidly computation rates,theywill profitby (1) choosingsuitable increasinguse in future investigationsis therefore a algorithmsand (2) rewritingportionsof their code. On plausibleexpectation. vector machineslike Crays, availabletimings show that linearsystemsof equationsare mostefficientlysolvedby NOTATION preconditioned conjugategradient-type algorithms.If an a(x, t) knownfunctionin theLangevinequation. existingcode is being transferredto a supercomputer, a A squarecoefficientmatrix. very usefulexerciseis to time thevarioustasksperformed. b(x, t) It oftenhappensthat mostof the totalCPU time is devoted knownfunctionin theLangevinequation. b vectorof knownquantifies. to performingonly small sectionsof the code. Large c soluteconcentration, M L-3. speedupswill result by making sure that these small concentration corresponding to maximum sections vectorizeasmuchaspossible. density, M L-3. Besides fast computationrates, supercomputers are often linked with other facilities or have other features that co make possible new ways of going about research. Accessiblememory on supercomputers is generallyvast comparedwith that of usualmainframes. Visualization makespossiblewaysof viewingprocesses or resultsthat cannotbe achievedotherwise. A typicalcaseis that of a graphics workstation connected to a supercomputer. Results of a simulation can then be viewed directly. Alternatively,slidesor moviescanbe generated. In this review an attempthasbeen madeto link aspects of solute transport-relatedresearchand modeling that appear, on the surface, to link some rather disparate Co <c> d sum of residuals. D solute dispersion tensor, L2T4. coefficientalongthe i axis,i = 1, 2, 3, Dii dispersion L2 Tq. DMii macrodispersion coefficient alongthei axis,i = 1, 2, 3, LZT-•. reduced solute dispersion tensor (=D/n),L2T4. location vector,L. army of independent variables. arbitraryscal• function. scientific threads. The common denominator,of course,is supercomputer applications. This situationis just as it shouldbe: solutetransportencompasses both the physics of flow in heterogeneous media and complex chemical reactionsandmixingprocesses over multipletemporaland spatialscales.Diversemathematicaltoolsandanalysesare necessaryto elucidatethe behaviorof specifiedsystems. The role of a supercomputer shouldbe as anotherspecialized instrumentthatcanbe calleduponas necessary.It is emphasized that thereare a wide varietyof circumstances in which supercomputers will aid in yielding valuable insight. One instancewhere this has occurredis the macroscalebehavior of a biodegradable,organic solute plume. MacQuarrieandSudicky[1990]reportsupercomputer simulationsthat illustratethe significantdifferences betweenthistype of plumeandthe well-studiedcaseof an ideal tracer plume. In the latter case,both field experiments and numerical simulations indicate that gravitational acceleration vector(magnitude g) L T-2. H distance function. generalized"distance"function. distance functiondependent on separation only. K scalar version ofk, L 2. permeability tensor, L2. hydraulic conductivity, L T2. K hydraulic conductivity tensor, L Tq. k k function in(4)tobeSpecified fora givensolute. theijkthrawspatial moment, M Li Lj Lk. mass of salt. Mr Ile Il l under known conditionsa reasonablyaccurate macroscopic descriptionof the plume is possible. Biodegradation effectivelylimitsthe spatialdomainsampledby theplume with the result that a macroscaledescriptiondoes not appeartenableon realistictime scales. initialconcentration, M L-3. cO,M L-3. ensemble meanof c,M L-a. totalmass(soluteplussolven0. porosityof theporousmedium. numberof experimentalmeasurements. number of locations. numberof parameters. dimension of A. N' number of particles in a particle-tracking simulation. p P0 thermodynamic pressure of thefluid,M L-• T -2. reference pressure, M L-• T-2. 292 ß Barry:SUPERCOMPUTERS AND SOLUTE TRANSPORT P P p, P. q q probabilitydensityfunction. vectorof parameters. IRIS 3130 workstation.Time on the Cray wasprovidedundera grantfromthe AcademicComputingCenterof theUniversityof estimate of P. California, Riverside. elementof P,j = 1,. ß., n. GarrisonSpositowas the editorin chargeof this paper. He thanksE.A. Sudickyand an anonymous reviewerfor servingas volumetric flow rate offlt•id sources orsinks per technical unitvolumeof porous medium, T-•. fitfidfluxvector,L T4. arbitrarystatistic. populationparameter. relativeperformance. residualvectorin thePeG algorithm. temporaryresidualvector. scalarspeed. sumof squares. Scv cross-validated Sii Fourier transformof the stationarysolutedrift velocitycovariancefunctionalongthei axis,i = 1,2, 3,L5T-2. specific storafivity, L-1. v volumetric water content. wavenumbervectorin Fourierdomain,L-1. elementof •, L-1. fluiddynamic viscosity, M L-1 T-1. meanplumepositionalongtheith axis,L. theijkthcentral moment, Li L• Lt•. reference dynamic viscosity, M L-1 T-1. white noiseprocess. po referees. REFERENCES Ababou, R., L. W. Gelhat, and D. McLaughlin, Threedimensional flow in randomporousmedia,2 vols., 833 pp., Rep.318, RalphM. ParsonsLab.,Civil Eng.Dep.,Mass.Inst. of Technol.,Cambridge,1988. Adams, L. M., and T. W. Crockett, Modeling algorithm executiontimeson processor arrays,Computer,•7(7), 38-43, 1984. Amdahl,G. M., Validity of the singleprocessorapproachto achievinglarge scalecomputingcapabilities,AFIPS Conf. Proc., 30, 483-485, 1967. Barlow,R. H., andD. J. Evans,Parallelalgorithmsfor iterative time, T. solutionto linearsystems, Cornput.J., 25(1), 56-60, 1982. relaxationparameter. Barry, D. A., and G. Sposito,Applicationof the convectiondispersion modelto solutetransportin finite soil columns,Soil distancealong the ith axis in an orthogonal Sci. Soc. Am. J.,52(1), 3-9, 1988. coordinatesystem,L. Barry, D. A., and G. Sposito, Three-dimensionalstatistical velocity(magnitude v),L T q. momentanalysisof the Stanford/Waterloo Bordentracerdata, varianceoperator. WaterResour.Res.,26(8), 1735-1747, 1990. total volume. Barry, D. A., J. Coves,andG. Sposito,On the Daganmodelof vector of unknowns. solutetransportin groundwater:Applicationto the Borden site,Water Resour.Res.,24(10), 1805-1817, 1988. compressibility coefficient ofthesolute, L T2M-1. Baudet,G. M., Asynchronous iterativemethodsfor multiprocescoefficientof solutemassfractionin density sors,J. Assoc.Cornput.Mach., 25(2), 226-244, 1978. formula (3). Bear,J.,Dynamicsof Fluidsin PorousMedia, 764 pp., Elsevier, empiricaldensitydifferencecoefficient. densitycouplingcoefficient. p 28, 3 / REVIEWS OF GEOPHYSICS fluidmass density, M L-•. referencevalueof fluid massdensity(e.g.,fresh water),M L-3. $ massconcentration of solutecomponent, M L-3. massconcentration of water,M L4. freshwaterhydraulichead,L. solutemassfraction, p•/p. referencesolutemassfraction,p9p. spatial deloperator, L-1. New York, 1972. Bear, J., and A. Verruijt, Modeling GroundwaterFlow and Transport,414 pp., D. Reidel,Hingham,Mass.,1987. Behie, A., and P. A. Forsyth, Comparisonof fast iterative methods for symmetric systems,IMA J. Numer. Anal., 3, 41-63, 1983. Behie,A., and P. A. Forsyth,Incompletefactorizationmethods for fully implicitsimulation of enhanced oil recovery,SIAMJ. Stat.Cornput., 5(3), 543-561, 1984. Behie, A., and P. K. W. Vinsome, Block iterative methods for fully implicit reservoirsimulation,Soc. Pet. Eng. J., 22, 659-668, 1982. Bethke, C. M., A numerical model of compaction-driven groundwater flow andheattransferandits application to the paleohydrologyof intra-cratonicsedimentarybasins, J. Geophys.Res.,90(B8), 6817-6828, 1985. Bethke, C. M., W. J. Harrison,C. Upson, and S. P. Altanet, Supercomputeranalysis of sedimentarybasins, Science, 239(4387), 261-267, 1988. Black,T. C., Concentration uncertainty for stochastic analysisof solutetransport in a bounded, heterogeneous aquifer,273 pp., Ph.D. thesis,Civil Eng. Dep., StanfordUniv., Stanford,Calif., 1988. Black, T. C., and D. L. Freyberg,Stochasticmodelingof verticallyaveragedconcentration uncertaintyin a perfectly ACKNOWLEDGMENTS. Supportof theWaterAuthorityof WesternAustraliais gratefullyacknowledged.J.B. Yost of the NationalCenterfor Supercomputing Applications(NCSA) at the Universityof Illinois at Urbana-Champaign wasresponsible for producingthe imagesin Plate 1. Calculationsfor theseimages were performedon the NCSA's Cray X-MP/48 andAliiant FX-8 computers.The hardwareusedfor imagegenerationincludedan stratifiedaquifer,WaterResour.Res.,23(6), 997-1004, 1987. Black,T. C., andD. L. Freyberg,Simulationof one-dimensional correlatedfieldsusinga matrix-factorization movingaverage approach,Math. Geol., 22(1), 39-62, 1990. Borden, R. C., and P. B. Bedient, Transportof dissolved hydrocarbons influencedby oxygen-limited biodegradafion, 1, Theoretical development, Water Resour. Res., 22(13), 1973-1982, 1986. 28, 3 /REVIEWSOF GEOPHYSICS Barry:SUPERCOMPUTERS AND SOLUTETRANSPORTß 293 Camahan,C. L., Simulationof uraniumtransportwith variable Freyberg,L., A naturalgradientexperiment on solutetransport in a sand aquifer,2, Spatial momentsand the advectionand temperature and oxidationpotential: The computerprogram THCC, Rep. 21639, Lawrence Berkeley Lab., Berkeley, dispersion of nonreactivetracers,Water Resour.Res.,22(13), Calif., 1986. 2031-2046, 1986. Chiang, C. Y., Numerical simulationof ground water con- Frind, E. O., Seawaterintrusionin continuous coastalaquifertaminanttransporton a supercomputer with injection-pumping aquitardsystems, Adv.WaterResour.,5, 89-97, 1982a. networksusingthemodifiedMOC andMFE method,148 pp., Frind,E. O., Simulation of long-termtransient density-dependent Ph.D. thesis,Rice Univ., Houston,Tex., 1985. transportin groundwater,Adv. Water Resour., 5, 73-88, 1982b. Chiang,C. Y., M. F. Wheeler,and P. B. Bedient,A modified methodof characteristics techniqueandmixed finite elements Frind,E. O., The principaldirectiontechnique: A new approach methodfor simulationof groundwater solutetransport,Water to groundwatercontaminanttransportmodelling,in ProceedResour.Res.,25(7), 1541-1549, 1989. ingsof the4th InternationalConference on Finite Elementsin Chu, S.-Y., and G. Sposito,A derivationof the macroscopic Water Resources,pp. 13.25-13.42, Springer-Verlag,New York, 1982c. solutetransportequationfor homogeneous, saturated, porous media,WaterResour.Res.,16(3), 542-546, 1980. Frind,E. O., andG. F. Pinder,The principaldirectiontechnique Chu, S.-Y., and G. Sposito,a derivationof the macroscopic for solutionof the advection-dispersion equation,in Proceedsolutetransportequationfor homogeneous, saturated,porous ingsof the loth IMACS Conference,pp. 305-313, Scientific media, 2, Reactive solutes at low concentrations, Water Computing,North-Holland,Amsterdam,1982. Resour.Res.,17(3), 333-336, 1981. Frind,E. O., E. A. Sudicky,andS. L. Schellenberg, Micro-scale Concus,P., G. H. Golub, and D. P. O'Leary, A generalized modellingin the studyof plumeevolutionin heterogeneous media,Stochastic Hydrol.Hydraul.,1(4), 263-279, 1987. conjugategradient method for the numericalsolution of elliptic partial differential equations, in Sparse Matrix Frind, E. O., E. A. Sudicky, and J. W. Molson, ThreeComputations,edited by J. R. Bunch and D. J. Rose, pp. dimensionalsimulationof organic transportwith aerobic 309-332, Academic,SanDiego,Calif., 1976. biodegradation,paper presented at IAHS 3rd Scientific Cox, D. R., and H. D. Miller, The Theory of Stochastic Assembly, Int. Assoc.of Hydrol.Sci., Baltimore,Md., May 10-19, 1989. Processes, 398 pp.,ChapmanandHall, London,1965. Cox, D. R., and E. J. Snell, A generaldefinitionof residuals,J. Garabedian, S. P., L. W. Gelhar,andM. A. Celia, Large-scale R. Star.Soc.,Set. B., 30(2), 248-265, 1968. dispersive transport in aquifers: Field experiments and Crump,K. S., Numericalinversionof Laplacetransforms usinga reactivesolutetransporttheory,Rep.315, 290 pp., RalphM. Fourierseriesapproximation, J. Assoc.Cornput.Mach., 23(1), ParsonsLab., Civil Eng. Dep., Mass. Inst. of Technol., 89-96, 1976. Cambridge,1988. Dagan, G., Stochasticmodeling of groundwaterflow by Garder, A. O., D. W. Peaceman,and A. L. Pozzi, Numerical unconditionaland conditionalprobabilities,2, The solute calculation of multidimensional miscibledisplacement by the transport,WaterResour.Res.,18(4), 835-848, 1982. methodof characteristics, Soc. Pet. Eng. J., 4(1), 26-36, 1964. Dagan,G., Solutetransportin heterogeneous porousformations, J. Fluid Mech, 145, 151-177, 1984. Gardiner,C. W., Handbookof Stochastic Methodsfor Physics, Dagan,G., Statisticaltheoryof groundwater flow andtransport: Chemistry andtheNaturalSciences, 442 pp.,Springer-Verlag, New York, 1983. Poreto laboratory,laboratoryto formation,and formationto regionalscale,WaterResour.Res.,22(9), 120S-134S,1986. Gelhar,L. W., Stochastic subsurface hydrologyfrom theoryto Dagan, G., Theory of solutetransportby groundwater,Annu. applications, WaterResour.Res.,22(9), 135S-145S, 1986. Rev. Fluid Mech., 19, 183-215, 1987. Gelhar, L. W., and C. L. Axness, Three-dimensionalstochastic Dagan,G., Time-dependent macrodispersion for solutetransport analysisof macrodispersion in aquifers,Water Resour.Res., 19(1), 161-180, 1983. in anisotropicheterogeneous aquifers,Water Resour.Res., 24(9), 1491-1500, 1988. Gelhar,L. W., A. L. Gutjahr,andR. L. Naff, Stochastic analysis Denning,P. J., Speedingup parallelprocessing, Am. Sci., 76(4), of macrodispersion in a stratifiedaquifer,Water Resour.Res., 347-349, 1988. 15(6), 1387-1397, 1979. Dongarra,J. J., F. G. Gustavson,and A. Karp, Implementing Goltz, M. N., and P. V. Roberts,Interpretingorganicsolute linear algebra algorithmsfor densematriceson a vector transportdata from a field experimentusing physical pipelinemachine,SIAM Rev.,26(1), 91-112, 1984. nonequilibriummodels,J. Contarn.Hydrol., 1(1), 77-93, 1986. Efron, B., The Jackknife,the Bootstrapand Other Resampling Plans,92 pp., SIAM, Philadelphia, Pa., 1982. Gray, H. L., and W. R. Schucany, The GeneralizedJackknife Efron,B., Computer-intensive methodsin statisticalregression, Statistic,308 pp.,MarcelDekker,New York, 1972. SIAM Rev.,30(3), 421-449, 1988. Emmen,A. H. L., A surveyof vectorandparallelprocessors, in Algorithms and Applications on Vector and Parallel Computers,editedby H. J. J. te Riele,T. J. Dekker,andH. A. vanderVorst,pp. 151-154, Elsevier,New York, 1987. Evans,D. I., Parallelnumericalalgorithmsfor linearsystems,in Parallel ProcessingSystems,edited by D. I. Evans, pp. 357-383, CambridgeUniversityPress,New York, 1982. Evans, D. J., and M. Hatzopoulos,A parallel linear system solver,Int. J. Cornput.Math., B7(3), 227-238, 1979. Evans, D. I., A. Hadjidimos,and D. Noutsos,The parallel solutionof bandedlinear equationsby the new quadrant interlockingfactorization(Q.I.F.) method,lnt. J. Cornput. Math.,B9(2), 151-161, 1981. Fisher, J. A., The VLIW machine: A multiprocessorfor compilingscientificcode,IEEE Cornput.,17(7), 45-53, 1984. Hager,W. W., AppliedNumericalLinear Algebra,424 pp., Prentice-Hall,EnglewoodCliffs, N.J., 1988. Hassanizadeh, S. M., andT. Leijnse,On themodelingof brine trasnportin porous media, Water Resour. Res., 24(3), 321-330, 1988. Herbert,A. W., C. P. Jackson,and D. A. Lever, Coupled groundwater flow and solutetransportwith fluid density strongly dependenton concentration,Water Resour. Res., 24(1), 1781-1795, 1988. Hestenes, M. R., andE. Stiefel,Methodsof conjugate gradients for solvinglinear systems,J. Res.Natl. Bur. Stand.,Sect.B, 49, 409-436, 1952. Hockney,R. W., and C. R. Jesshope, Parallel Computers: Architecture Programming and Algorithms,423 pp., Adam Hilger,Bristol,England,1981. Hockney,R. W., and C. R. Jesshope, Parallel Computers 2: 294 ß Barry:SUPERCOMPUTERS AND SOLUTETRANSPORT ArchitectureProgrammingand Algorithms,625 pp., Adam Hilger, Bristol,England,1988. Huyakorn, P.S., and C. Taylor, Finite element models for coupledgroundwaterflow andconvectivedispersion, in Finite Elementsin Water Resources,editedby W. G. Gray, G. F. Pinder,andC. A. Brebbia,pp. 1.131-1.151, Pentech,London, 1977. Huyakorn,P.S., P. F. Anderson,J. W. Mercer,andH. O. White, Saltwaterintrusionin aquifers:Developmentand testingof a three-dimensional finite element model, Water Resour. Res., 28, 3 / REVIEWS OF GEOPHYSICS Meyer, P. D., A. J. Valocchi,S. F. Ashby,andP. E. Saylor,A numericalinvestigation of the conjugategradientmethodas appliedto three-dimensional groundwater flow problemsin randomlyheterogeneous porousmedia, Water Resour.Res., 25(6), 1440-1446, 1989. Mishra,S., and J. C. Parker,Parameterestimationfor coupled unsaturated flow and transport,Water Resour.Res., 25(3), 385-396, 1989. Mosteller,F., andJ. W. Tukey,Dataanalysis,includingstatistics, in Handbookof Social Psychology,vol. 2, editedby G. Lindzey and E. Aronson,pp. 80-203, Addison-Wesley, 23(2), 293-312, 1987. Illangasekare,T. H., and P. D611,A discretekernel methodof Reading,Mass., 1968. characteristicsmodel of solute transport in water table Naff, R. L., T.-C. J. Yeh, and M. W. Kemblowski, A note on the aquifers,WaterResour.Res.,25(5), 857-867, 1989. recentnatural gradienttracer test at the Borden site, Water Resour.Res.,24(12), 2099-2103, 1988. Illangasekare,T. H., and H. J. Morel-Seytoux,A finite element "discrete kernel generator" for efficient groundwater Narasimhan, T. N., A. F. White, andT. Tokunaga,Groundwater management, in Finite Elementsin Water Resources,edited by C. A. Brebbia, W. G. Gray, and G. F. Pinder, pp. contamination from an inactiveuraniummill tailingspile, 2, Applicationof a dynamicmixingmodel,WaterResour.Res., 1.195-1.212, Pentech, London, 1978. 22(13), 1820-1834, 1986. Journel,A. G., and C. J. Huijbregts,Mining Geostatistics,600 Neuman, S. P., C. L. Winter, and C. M. Newman, Stochastic pp., Academic,SanDiego, Calif., 1978. theoryof field-scaleFickiandispersion in anisotropic porous Kershaw,D., Solutionof single tridiagonallinear systemsand media,WaterResour.Res.,23(3), 453-466, 1987. vectorizationof the ICCG algorithmon the Cray-1, in Parallel Newport,B. D., Salt waterintrusionin the UnitedStates,Rep. Computations, editedby G. Rodrigue,pp. 85-99, Academic, 600/8-77-011, 30 pp., U.S. Environ. Prot. Agency, SanDiego, Calif., 1982. Washington,D.C., 1977. Kool, J. B., and J. C. Parker,Analysisof the inverseproblemfor Noye, J., Finite difference methods for partial differential transient unsaturated flow, Water Resour. Res., 24(6), equations,in Numerical Solutionsof Partial Differential 817-830, 1988. Equations,edited by J. Noye, pp. 1-137, North-Holland, Kuck, D. J., E. S. Davidson, D. H. Lawtie, and A. H. Sameh, Parallel supercomputingtoday and the Cedar approach, Science,231(4741), 967-974, 1986. Kung, H. T., Synchronized andasynchronous parallelalgorithms for multiprocessors, in Algorithmsand Complexity,editedby J. F. Traub,pp. 153-200, Academic,SanDiego,Calif., 1976. Lee, C. H., and R. T. Cheng, On seawaterencroachmentin coastalaquifers,WaterResour.Res.,•0(5), 1039-1043, 1974. Liu, C. W., and T. N. Narasimhan,Redox-controlled multiplespeciesreactive chemical transport,1, Model development, Water Resour.Res.,25(5), 869-882, 1989a. Liu, C. W., and T. N. Narasimhan,Redox-controlled multiplespecies reactive chemical transport,2, Verification and application, WaterResour.Res.,25(5), 883-910, 1989b. Lorensen,W. E., and H. E. Cline, Marching cubes:A high resolution 3D surface constructionalgorithm, Cornput. Graphics,21(4), 163-169, 1987. MacFarlane,D. S., J. A. Cherry, R. W. Gillham, and E. A. Sudicky, Migration of contaminantsin groundwaterat a landfill: A case study, 1, Groundwaterflow and plume delineation,J. Hydrol, 63, 1-29, 1983. MacQuarrie, K. T. B., and E. A. Sudicky, Simulation of biodegradable organiccontaminants in groundwater,2, Plume behavior in uniform and random flow fields, Water Resour. Res., 26, 223-239, 1990. Amsterdam, 1982. Ortega,J. M., and R. G. Voigt, Solutionof partialdifferential equations on vectorandparallelcomputers, SIAM Rev.,27(2), 149-240, 1985. Pelka,W., and A. Peters,Finite elementground-water models implementedon vector computers,Int. J. Numer. Meth. Fluids, 6, 913-925, 1986. Quenouille, M. H., Notes on bias in estimation, Biometrika, 43(3-4), 353-360, 1956. Ripley,B. D., SpatialStatistics, 252 pp.,JohnWiley,New York, 1981. Roberts,P. V., and D. M. Mackay(Eds.),A naturalgradient experiment on solutetransport in a sandaquifer,Tech.Rep. 292, 229 pp., Civil Eng. Dep., Stanford Univ., Stanford, Calif., 1986. Saad,Y., Practicaluse of polynomialpreconditionings for the conjugate gradientmethod,SIAM J. Sci. Stat.Cornput.,6(4), 865-881, 1985. Sameh,A. H., andD. J. Kuck,On stableparallellinearsystem solvers,J. Assoc.Cornput.Mach., 25(1), 81-91, 1978. Samper,F. J., andS. P. Neuman,Estimationof spatialcovariance structures by adjoint state maximum likelihood cross validation,1, Theory, Water Resour.Res., 25(3), 351-362, 1989. Segol, G., and G. P. Pinder, Transientsimulationof saltwater MacQuarrie,K. T. B., E. A. Sudicky,andE. O. Frind,Simulation intrusionin southeastern Florida,Water Resour.Res.,12(1), 65-70, 1976. of biodegradableorganic contaminantsin groundwater,1, Numericalformulationin principaldirections,WaterResour. Segol,G., G. F. Pinder,and W. G. Gray, A Galerkin finite Res., 26, 207-222, 1990. elementtechnique for calculatingthe transientpositionof the Mantoglou,A., and J. Wilson, The tuming bandsmethodfor saltwaterfront,WaterResour.Res.,••(2), 343-347, 1975. simulationof randomfieldsusingline generation by a spectral Seitz,C. L., and J. Matisoo,Engineering limitson computer method,Water Resour.Res.,18(5), 1379-1394, 1982. performance, Phys.Today,37(5), 38-45, 1984. Marquardt,D. W., An algorithmfor least-squares estimationof Sharp, J. A., An Introductionto Distributed and Parallel nonlinear parameters,J. Soc. Ind. Appl. Math., 11(2), Processing,174 pp., BlackwellScientific,Oxford, 1987. 431-441, 1963. Shinozuka,M., and C.-M. Jan, Digital simulationof random McCormick, S. F. (Ed.), Multigrid Methods,282 pp., SIAM, processesand its applications,J. Sound Vibrat., 25(1), 111-128, 1972. Philadelphia,Pa., 1987. McCormick,S. F. (Ed.), Multigrid methods:Theory, applica- Sposito,G., and D. A. Barry, On the Daganmodel of solute tions,andsupercomputing, LectureNotesin Pure andApplied transport in groundwater: Foundational aspects, WaterResour. Res., 23(10), 1867-1875, 1987. Mathematics,vol. 110, 644 pp., Marcel Dekker, New York, 1988. Sposito,G., V. K. Gupta,and R. N. Bhattacharya, Foundational 28, 3 /REVIEWSOF GEOPHYSICS Barry:SUPERCOMPUTERS AND SOLUTETRANSPORTß 295 Tompson,A. F. B., R. Ababou,and L. W. Gelhar, Implementation of the three-dimensionalturning bands random field generator,WaterResour.Res.,25(10), 2227-2243, 1989. Sposito,G., W. A. Jury,andV. K. Gupta,Fundamemal problems in the stochasticconvection-dispersion model of solute Traub,J. F., Iterativesolutionof tridiagonal systems on parallel transportin aquifersandfield soils,WaterResour.Res.,22(1), or vectorcomputers, in Complexityof Sequemialand Parallel 77-88, 1986. NumericalAlgorithms,edited by J. F. Traub, pp. 49-82, Academic,SanDiego, Calif., 1973. Stone,H. S., An efficientparallelalgorithmfor the solutionof a tridiagonallinear system of equations,J. Assoc. Cornput. Tukey, J. W., Bias and confidencein not-quitelarge samples (abstract),Ann. Math. Stat., 29(2), 614, 1958. Mach., 20(1), 27-38, 1973. Sudicky,E. A., A naturalgradientexperimenton solutetransport van Karopen, N. G., StochasticProcessesin Physics and Chemistry,419 pp., North-Holland,Amsterdam,1981. in a sandaquifer:Spatialvariabilityof hydraulicconductivity and its role in the dispersionprocess,Water Resour.Res., Voigt, R. t3., The influenceof vector computerarchitectureon 22(13), 2069-2082, 1986. numericalalgorithms, in High SpeedComputerandAlgorithm Organization,editedby D. J. Kuck, D. H. Lawtie, and A. H. Sudicky,E. A., The Laplace transformGalerkin technique:A time-cominuous finite elementtheoryandapplicationto mass Sameh,pp. 229-244, Academic,SanDiego,Calif., 1977. transport in groundwater, Water Resour. Res., 25(8), Voigt, R. t3., D. Gottleib,and M. Y. Hassaini(Eds.),Spectral 1833-1846, 1989a. Methodsfor Partial DifferentialEquations,276 pp., SIAM, Philadelphia,Pa., 1984. Sudicky,E. A., The LaplacetransformGalerkintechniquefor large-scalesimulationof masstransportin discretelyfractured Wallis, J. R., Hydrology--The computerrevolutioncontinues, Rev. Geophys.,25(2), 101-105, 1987. porousformations(abstrac0,Eos Trans.AGU, 70(43), 1082, 1989b. Walsh, M.P., S. L. Bryant, S. R. Schechter,and L. W. Lake, Sudicky,E. A., J. A. Cherry, and E. O. Frind, Migrationof Precipitation anddissolution of solidsattending flow through porousmedia,AIChE J., 30(2), 317-328, 1984. contaminants in groundwaterat a landfill: A casestudy,4, A natural-gradient dispersiontest,J. Hydrol., 63, 81-108, 1983. Winter,C. L., C. M. Newman,andS. P. Neuman,A perturbation Sudicky,E. A., S. L. Schellenberg,and K. T. B. MacQuarrie, expansionfor diffusionin a randomvelocity field, SIAM J. Assessment of the behaviour of conservative and Appl.Math., 44(2), 411-424, 1984. biodegradablesolutes in heterogeneousporous media, in Yeh, W. W.-G., Reviewof parameter identification procedures in Dynamics of Fluids in Hierarchical Porous Formations, groundwater hydrology:The inverseproblem,WaterResour. Res.,22(1), 95-108, 1986. editedby J. H. Cushman,Academic,SanDiego,Calif., 1990. Taylor, C., and P.S. Huyakom,Three-dimensional groundwater Yeh, W. W.-G., andC. Wang,Identification of aquiferdisperflow with convectivedispersion,in Finite Elementsin Fluids, sivities:Methodsof analysisandparameteruncertainty, Water Resour.Bull., 23(4), 369-580, 1987. vol. 3, editedby R. H. Gallagher,pp. 311-321, Wiley, New York, 1978. Young, D. M., and R. T. Gregory,A Surveyof Numerical Tichelaar,B. W., andL. J. Ruff, How goodare our bestmodels? Mathematics,vol. II, 659 pp., Addison-Wesley, Reading, Mass., 1972. Eos Trans.AGU, 70(20), 593, 605-606, 1989. Tompson, A. F. B., E.G. Vomvoris, and L. W. Gelhar, Numerical solution of solute transport in randomly heterogeneous porousmedia:Motivation,modeldevelopment, and application,Rep. 316, 114 pp., Ralph M. ParsonsLab., D. A. Barry,Centrefor Water Research,Universityof WesternAustralia, Nedlands,Australia6009. Civil Eng. Dep., Mass.Inst.of Technol.,Cambridge,1988. theoriesof solutetransportin porousmedia:A criticalreview, Adv. Water Resour., 2, 59-68, 1979.
© Copyright 2026 Paperzz