1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
In Situ Characterization of Soil Clay Content with Visible Near-Infrared
Diffuse Reflectance Spectroscopy
Travis H. Waiser, Cristine L.S. Morgan*, David J. Brown, and C. Tom Hallmark
T.H. Waiser, C.L.S. Morgan, and C.T. Hallmark Dep. of Soil and Crop Sciences, Texas
A&M University, 2474 TAMU, College Station, TX 77843-2474. D. J. Brown, Dep. of
Land Resources and Environmental Sciences, Montana State University, PO Box
173120, Bozeman, MT 59717-3120. Received 06 June 2006. *Corresponding author
([email protected]).
KEYWORDS
Diffuse reflectance spectroscopy; in situ; VNIR; PLS regression; clay content; water
potential
ACKNOWLEDGEMENTS
The authors would like to thank the Texas Water Resource Institute and Texas
17
Agricultural Experiment Station for their financial support. The extension agents from
18
Erath and Comanche County, Texas were instrumental in putting us in contact with
19
landowners of the study sites. We appreciate the work the Texas Agricultural
20
Experiment Station Soil Characterization Laboratory did in running some of the analyses
21
especially Dr. Richard Drees and Ms. Donna Prochaska.
2
22
In Situ Characterization of Soil Clay Content with Visible Near-Infrared
23
Diffuse Reflectance Spectroscopy
24
ABSTRACT
25
Visible and near-infrared (VNIR, 400-2500 nm) diffuse reflectance spectroscopy
26
(DRS) is a rapid, proximal-sensing method that has proven useful in quantifying
27
constituents of dried and ground soil samples. However, very little is known about how
28
DRS performs in a field setting on soils scanned in-situ. The overall goal of this research
29
was to evaluate the feasibility of VNIR-DRS for in situ quantification of clay content of
30
soil from a variety of parent materials. Seventy-two soil cores were obtained from six
31
fields in Erath and Comanche Counties, Texas. Each soil core was scanned with a visible
32
near-infrared spectrometer, with a spectral range of 350-2500 nm, at four different
33
combinations of moisture content and pre-treatment: field-moist in situ, air-dried in situ,
34
field-moist smeared in situ, and air-dried ground. The VNIR spectra were used to predict
35
total and fine clay content of the soil using partial least squares (PLS) regression. The
36
PLS model was validated with 30% of the original soil cores that were randomly selected
37
and not used in the calibration model. The validation clay predictions had a root mean
38
squared deviation (RMSD) of 61 g kg-1 and 41 g kg-1 dry soil for the field-moist and air-
39
dried in situ cores, respectively. The RMSD of the air-dry ground samples was between
40
the two in situ RMSDs and comparable to values in the literature. Smearing the samples
41
increased the field-moist in situ RMSD to 74 g kg-1. Whole-field holdout validation
42
results showed that soils from all parent materials need to be represented in the
43
calibration samples for maximum predictability. In summary, DRS is an acceptable
3
44
technique for rapidly measuring soil clay content in situ at variable water contents and
45
parent materials.
46
47
Abbreviations: DRS, diffuse reflectance spectroscopy; N, the total number of samples in
48
the validation data; PLS, partial least squares; RMSD, root mean squared deviation; RPD,
49
ratio of standard deviation to RMSD; SD, standard deviation; VNIR, visible and near-
50
infrared; Ypred, predicted values of the validation set using the PLS model; and Ymeas,
51
laboratory measurements of the validation set.
52
53
54
INTRODUCTION
The resolution of a 1:24,000 scale soil survey map is too coarse to capture soil
55
variability within soil mapping units and transitions between soil mapping units.
56
However, soil maps that capture soil variability at a 10-50-m scale are necessary for
57
resource management such as, precision agriculture, non-point source pollution
58
modeling, and resource use planning (Pachepsky et al., 2001; Ellert et al., 2002).
59
Researchers have used external landscape features such as terrain modeling, soil surface
60
reflectance, vegetative cover, and rapid surface sensing techniques like electromagnetic
61
devices to capture the spatial variability of soil properties across landscapes followed by
62
collecting soil cores and laboratory analyses of those cores for model calibration and
63
validation (Moore et al., 1993; Sudduth et al., 1997; Zhu et al., 1997, McBratney et al.,
64
2003, Zhu et al., 2004). Each of these methods has the advantage of creating a high-
65
resolution map of horizontal soil variability, but is limited in the ability to get high-
66
resolution, vertical soil information. Soil profile (vertical) information is limited by the
4
67
capacity to collect and analyze soil cores. Model calibration and validation are currently
68
the weak points in digital soil mapping because collecting and analyzing soil cores to
69
capture vertical variability is time consuming and cost prohibitive. For example, current
70
laboratory soil analyses can take several weeks to months for each soil pedon and cost
71
upwards of $2,000. The lack of soil sensors that can rapidly quantify soil profile
72
information demonstrates the need to find new methods for soil mapping.
73
A proximal soil sensor that might be useful for rapidly quantifying soil profile
74
information in the field is visible and near-infrared diffuse reflectance spectroscopy
75
(VNIR-DRS). Recent research has shown the effectiveness of VNIR-DRS in providing a
76
non-destructive rapid prediction of soil physical, chemical, and biological properties of
77
air-dried ground soil samples in the laboratory (Shepherd and Walsh, 2002). In the VNIR
78
spectrum, absorptions by water bonds associated with clay content and other bonding
79
associated with clay type provide the opportunity to use VNIR for quantifying clay
80
information in soil. Previous research on air-dried, ground soil samples has shown VNIR
81
predictions of soil clay content with r2-values ranging from 0.56 to 0.91 and RMSD’s
82
ranging from 23 to 11 g kg-1 (Ben-Dor and Banin, 1995; Janik et al., 1998, Shepherd and
83
Walsh, 2002; Islam et al., 2003; Sorensen and Dalsgaard, 2005; Brown et al., 2006). Air-
84
drying and grinding the soil sample prior to scanning is believed to improve prediction
85
accuracy of DRS-based models. Air-drying the sample reduces the intensity of bands that
86
are related to water so signals associated with other soil properties are not masked or
87
hidden. Additionally, particle size affects accuracy of the spectral scan. Smaller particles
88
increase reflectance scatter, which reduces the absorption peak height (Workman and
5
89
Shenk, 2004). Soil homogenization, accomplished while grinding, may also be beneficial
90
when scanning with VNIR-DRS.
91
The determination of clay content by VNIR measurements is possible in part due
92
to the distinctive spectral signatures of common clay minerals. The spectral signatures
93
include overtones and combination bands that occur from chemical bonds within soil
94
minerals (Hunt and Salisbury, 1970; Clark, 1999). For example, kaolinite, smectite, and
95
muscovite occur in the clay fraction of soils and have distinct spectral absorption
96
features. Kaolinite [Al2Si2O5(OH)4] is an aluminum silicate with two very strong
97
hydroxyl bands near 1400 nm, OH stretch, and 2200 nm, OH stretch Al-OH bend
98
combination (Hunt and Salisbury, 1970; Clark et al., 1990). Smectite (montmorillonite)
99
[M+0.3Al2.7Si3.3O10(0H)2; M=Ca2+, Mg2+, K+, etc.] has two very strong water bands around
100
1400 and 1900 nm from molecular water and another at 2200 nm (Hunt and Salisbury,
101
1970; Goetz et al., 2001). Muscovite [KAl3Si3O10 (OH)2] displays hydroxyl bands at
102
1400 nm and between 2200 to 2600 nm (Hunt and Salisbury, 1970). Brown et al. (2006)
103
built a VNIR calibration for clay minerals and was able to predict kaolinite and
104
montmorillonite within one ordinal unit from X-ray diffraction data 96% and 88% of the
105
time, respectively. Goetz et al. (2001) investigated using NIR spectroscopy to measure
106
smectite content in selected Colorado soils (r=0.83). Because the literature shows VNIR
107
absorbance associated with clay mineralogy, one may expect that the wavebands
108
significant to predicting soil clay content to be similar to mineralogy wavebands.
109
There have been few reported studies of in situ VNIR-DRS soil characterization.
110
Sudduth and Hummel (1993) tested a portable near-infrared (NIR) spectrometer to
111
measure soil properties in situ on the fly. Their research concluded that VNIR-DRS
6
112
could be used to accurately predict cation exchange capacity, soil organic matter, and
113
moisture content in the laboratory, but the technique was not sufficiently accurate at
114
quantifying soil organic matter in a furrow. The poor prediction with this in situ
115
technique was attributed to movement of the soil past the spectrometer. Hence low
116
prediction accuracy for in situ scanning may be avoided by holding the spectrometer
117
scanning device more stable, and by holding the soil stationary while scanning.
118
In the field, DRS measurements may not work as well as laboratory DRS
119
measurements because of potential problems that have not been addressed by research on
120
air-dried ground soils. The potential problems that may reduce the prediction accuracy of
121
in situ DRS analysis include, varying amounts of soil moisture, varying particle size and
122
aggregation, smearing of soil surfaces, small scale heterogeneity (mottles, accumulations,
123
and redox features), and geographic regionality of calibration models. Sudduth and
124
Hummel (1993) indicated that local buried residue and roughness of the soil surface
125
caused a reduction in their prediction accuracies. Smearing of the soil causes reflectance
126
properties to change, possibly altering prediction accuracies. Geographic regionality,
127
such as changes in soil parent material, may affect prediction accuracies when using
128
VNIR-DRS (Ben-Dor and Banin, 1995; Dalal and Henry, 1986; Sudduth and Hummel,
129
1996).
130
The overall goal of our study is to evaluate the feasibility of VNIR-DRS for in
131
situ quantification of clay content for soil profiles from a variety of parent materials.
132
Specifically, this research addresses the following objectives: 1) evaluate the precision of
133
350 to 2500 nm (VNIR region) soil reflectance measurements in quantifying soil clay
134
content of in situ soils at field-moist and air-dry water content; 2) quantify any change in
7
135
measurement error or prediction accuracy for in situ, field-moist soil with a smeared
136
surface; and 3) quantify any change in prediction accuracies with consideration to field-
137
specific calibrations.
138
This research will help evaluate the feasibility of using a VNIR spectrometer in
139
the field as a proximal sensor to aid soil mapping activities. For VNIR-DRS to be
140
considered useful in the field, it must be an improvement on current field techniques.
141
The VNIR DRS methods should be reliable, meaning that the errors should be
142
comparable to current field techniques with similar data acquisition speeds, and VNIR-
143
DRS should be rapid, meaning that it should quantify clay much faster than any current
144
techniques. Although time and money are required to calibrate the VNIR instrument,
145
once calibrated, VNIR-DRS can scan whole soil pedons more rapidly than hand texturing
146
in the field or laboratory particle-size methods (i.e. pipette and hydrometer), allowing for
147
more soil core information to be quantified in one day. Therefore VNIR-DRS is
148
considered adequate if VNIR-DRS can quantify clay content accurate enough for
149
precision management, watershed modeling needs, or other applications where soil
150
information is need. In general, most precision management strategies and watershed
151
models require a stronger emphasis on the change in soil texture in space (vertical and
152
horizontal) than on lab-quality soil data (Morgan et al., 2003). Quickly collecting higher
153
spatial resolution data is where VNIR-DRS has an advantage over current field
154
techniques.
155
156
157
MATERIALS AND METHODS
Soil Coring
8
158
Seventy-two soil cores were collected from six fields in Erath County (fields 1, 2,
159
3, and 6) and Comanche County (fields 4 and 5), Texas in May 2004 using a Giddings
160
hydraulic soil sampler (Windsor, CO) attached to a truck. Each soil core was collected to
161
a maximum depth of 105 cm or to the depth of a coring-restrictive horizon. The soil
162
cores were contained in a 6.0-cm diameter plastic sleeve that was capped on both ends.
163
The soil cores that were collected were chosen to represent the large variability in soil
164
properties over the two counties. For example, 21 soil series were mapped by the Natural
165
Resources Conservation Service in these fields (Table 1), and the parent material of these
166
soils included alluvium, sandstone, shale, and limestone. The large variability was
167
selected to ensure that the VNIR prediction models would not be mineralogy specific.
168
Additionally, though many cores represented the same soil series, variability within each
169
soil series because of 1) differences of morphology within a series, 2) inclusions of other
170
soils within each mapping unit, and 3) varying landscape positions all lead to
171
representation of high morphological variability within each mapped series.
172
VNIR-DRS Scanning
173
The soil cores were scanned using an ASD “FieldSpec® Pro FR” VNIR
174
spectroradiometer (Analytical Spectral Devices, Boulder, CO) with a spectral range of
175
350-2500 nm, 2-nm sampling resolution and spectral resolution of 3 nm at 700 nm and
176
10 nm at 1400 and 2100 nm. The spectroradiometer was equipped with a contact probe.
177
The contact probe has a viewing area defined by a 2-cm diameter circle and its own light
178
source. A Spectralon® panel with 99% reflectance was used to optimize the
179
spectrometer each day; the same panel was used as a white reference before scanning
180
each core. The 72 soil cores were prepared for the first, moist in situ, scan by slicing the
9
181
plastic sleeve lengthwise with a utility knife, and then halving the soil core lengthwise
182
(surface to subsoil) with a piano wire. One-half of the core was smeared using a stainless
183
steel spatula to simulate possible smearing by a soil probe and the other half was left
184
unsmeared. The smearing test was performed to quantify any reduction in accuracy that
185
might occur if a soil probe, pushed into the ground, smeared the sides of the hole. In this
186
laboratory exercise, smearing moist, high clay content soil required multiple passes with
187
the spatula. Therefore, this test was a worst-case scenario, as the soil was smeared to its
188
maximum capacity. A wire grid was used to identify two 3-cm wide columns and 3-cm
189
wide rows within each core half (Fig. 1). Each row within each column was scanned
190
twice with the FieldSpec® Pro FR with a 90º rotation of the contact probe between scans.
191
Both halves of each core, smeared and unsmeared, were scanned at field-moist water
192
content.
193
Water potential was measured at each soil horizon from each core, with a
194
maximum of six samples per core, using a SC-10 thermocouple psychrometer (Decagon,
195
Pullman, WA) (Rawlins and Campbell, 1986). The water potential samples were
196
collected immediately after scanning each core to minimize drying. Water potential
197
measurements were made to confirm variability of water content and to quantify the
198
range of soil moisture among the soils scanned. The cores were then placed in a drier at
199
44°C for two days for air-dried in situ scans. Before taking air-dried scans, the cores
200
were removed from the oven and left on the countertop in the laboratory to equilibrate to
201
room temperature. The soil core half that was scanned in unsmeared condition was then
202
rescanned, in situ, at air-dry moisture content.
10
203
Each row of the unsmeared soil core was ground and passed through a 2-mm
204
sieve, and re-scanned as air-dried, ground soil. The air-dried, ground soils were scanned
205
from below with an ASD mug lamp connected to the FieldSpec® Pro FR. A Spectralon®
206
99% reflectance panel was used to optimize and white reference the spectrometer.
207
Approximately 28 g of ground soil was placed into a Duraplan® borosilicate optical-
208
glass Petri dish. Each sample was scanned twice with a 90º rotation between scans.
209
210
Pretreatment of Data
The reflectance spectra were spliced where the three detectors overlapped using
211
an empirical algorithm. Reflectance for the 0º and 90º scans were averaged (mean).
212
Scans on the same row, column 1 and 2, were also averaged, creating a mean of four
213
scans for each measured value of clay content. Smoothed reflectance and 1st and 2nd
214
derivatives were extracted on 10-nm intervals (360-2490 nm) from a weighted cubic
215
smoothing spline fit to the mean reflectance data using the R “smooth spline” function (R
216
Development Core Team, 2004), following the methods of Brown et al. (2006).
217
218
Laboratory Analysis
Although every 3 cm of the soil core was scanned, only selected rows were used
219
for laboratory particle size analysis. Only the VNIR scans from those rows were
220
subsequently used in the VNIR models. Whole rows of soil where combined for
221
laboratory analysis; hence, each row of soil had a total of four scans to represent its
222
spectral properties (Fig. 1). Rows were chosen to represent the primary horizons within a
223
soil core. The entire row from each half of the core was used for particle size analysis.
224
Particle size distribution was determined in the laboratory using the pipette method with
225
an error of ±1% clay (Steele and Bradfield, 1934; Kilmer and Alexander, 1949; Gee and
11
226
Or, 2002). Fine clays were measured using centrifugation, USDA NRCS method 3A1b
227
(USDA, 1996). Eighteen samples were selected for determination of clay mineralogy.
228
Clay mineralogy by X-ray diffraction was completed following techniques described in
229
Hallmark et al. (1986); no pretreatment was done. The mineralogy technique used was to
230
determine the absence or presence of clay minerals existing in quantities of greater than 5
231
% by volume of clay fraction.
232
233
234
Model Calibration and Validation
Four types of clay prediction models were built to help determine how in situ
235
scans and variable water content affect prediction accuracies for clay content. Air-dried
236
ground, air-dried in situ, field-moist in situ, and smeared field-moist in situ scans were
237
the four soil preparation types used in the models. The clay content models were
238
produced using 70% of the soil cores randomly chosen as the calibration samples. Scans
239
from an individual soil core were not split between validation and calibration datasets.
240
Entire cores were selected for validation or calibration to maintain independence between
241
the calibration and validation data (Brown et al., 2005). For each of the four pretreatment
242
scans (air-dried ground, air-dried in situ, field-moist in situ, and smeared field-moist in
243
situ), three prediction models for clay content were built using soil reflectance, 1st
244
derivative of reflectance and 2nd derivative of reflectance. The calibration models were
245
built using segmented cross validation PLS method in Unscrambler 9.0 (CAMO Tech,
246
Woodbridge, NJ). The segments for cross validation are randomly chosen and represent
247
four percent of the calibration dataset. The remaining 30% of the cores were used to
248
validate the models.
12
249
Model calibration and validation sets were also completed on whole-field
250
holdouts of moist in-situ scans, as a means to compare to the findings of Brown et al.
251
(2005). Whole-field holdouts were achieved by calibrating a model using PLS with five
252
of the six fields. The sixth field was held out as the validation samples. Six models were
253
created so all six fields were represented as a validation set.
254
The significant wavelengths in each model were plotted to help determine what
255
portions of the spectra were important for clay content predictions. Significant
256
wavelengths were chosen by Unscrambler 9.0 by comparing Student t-values of the PLS
257
regression coefficients to a p-value of 0.05. Negative clay content predictions were
258
changed to zero clay content before comparison of measured to predicted clay.
259
Measured versus predicted values of the validation samples were compared using
260
simple regression. The coefficient of determination (r2), root mean squared deviation
261
(RMSD), ratio of standard deviation (SD) to RMSD (RPD) and bias were calculated to
262
compare the accuracy of different PLS models. Statistical formulas to calculate RMSD
263
and bias follow Gauch et al. (2003), Brown et al. (2005) and RPD follows Chang et al.
264
(2005):
RMSD =
265
∑ (Y
pred
− Ymeas )2 / N ,
[1]
n
266
RPD = SD/RMSD, and
[2]
267
Bias = ∑ (Ypred − Ymeas ) / N;
[3]
n
268
where Ypred are predicted values of the validation set using the PLS model, and Ymeas are
269
laboratory measurements of the validation set, and N is the total number of samples in the
270
validation data.
13
271
RESULTS AND DISCUSSION
272
Sample Descriptions
273
From the 72 soil cores, 270 soil samples were analyzed for clay content and water
274
potential. Of these samples, 188 were used in the calibration model. The other 82
275
samples were used to validate the PLS model. Though selected randomly, the calibration
276
and validation data sets were similar (Table 2). Clay content range of the 270 soil
277
samples was 12 g kg-1 to 578 g kg-1 dry soil. The mean and median for the calibration
278
and validation data were similar, indicating that the samples were evenly split above and
279
below the mean, and that the prediction model produced should not be skewed. The
280
minimum water potential measured, -5.8 MPa, in the calibration samples was from a
281
tilled surface (0-3 cm) of a loamy-textured soil. The range of water potential confirms
282
that in situ moist scans involved a large range of water contents that may be found in
283
most field situations. The maximum water potential, 0 MPa, occurred in deep horizons in
284
several of the irrigated fields. The samples represented a range of clay mineralogy. The
285
minerals found in the clay fractions were: smectite, kaolinite, mica, quartz, calcite, and
286
feldspar (Table 3). The soils in field 1 were notably high in clay content and were dark
287
colored Vertisols; field 3 soils were low in clay content.
288
Field-moist vs. Air-dried Validation
289
The 1st derivative PLS model performed better than the reflectance and 2nd
290
derivative PLS models in the field-moist in situ scans. The r2 value for the 1st derivative
291
model was 0.83 compared to 0.71 and 0.58 for the reflectance and 2nd derivative models,
292
respectively. Because the 1st derivative PLS model performed the best for field-moist in
293
situ, and the 1st derivative worked the best in other VNIR work (Reeves et al., 1999;
14
294
Reeves and McCarty, 2001; Brown et al., 2006), the remaining PLS models reported in
295
this paper all use the 1st derivative of the VNIR spectra between 350 and 2500 nm. The
296
prediction accuracy of three of the four models used to predict clay content were very
297
similar, and the fourth, field-moist in situ smeared, had the largest prediction error and
298
scatter about the validation regression. Table 4 summarizes the prediction accuracies
299
between the four VNIR models. The r2 value of the dried-ground samples were similar to
300
the literature, while the RMSD value of this work was smaller than those cited (Ben-Dor
301
and Banin, 1995; Janik et al., 1998, Shepherd and Walsh, 2002; Islam et al., 2003;
302
Sorensen and Dalsgaard, 2005; Brown et al., 2006). The air-dried ground model was
303
expected to produce the most accurate prediction model because the sample was reduced
304
to a uniform size, moisture was uniform, and grinding homogenized the soil, but the air-
305
dried in situ model did slightly better (Fig. 2). There are two possible explanations for
306
why the air-dried ground model did not perform as well as the air-dried in situ model.
307
One explanation could be random error. The second possible explanation is that in situ
308
soil has a higher bulk density than dried ground soil; and therefore, the in situ soil may
309
have a stronger reflectance signal. The air-dried ground and in situ moist models had a
310
similar number of wavelengths with significant (p-value ≤ 0.05) regression coefficients,
311
83 and 78 significant wavelengths, respectively (Fig. 3).
312
The air-dried ground prediction had the most bias (-16.3), compared to the other
313
predictions. The large bias associated with the air-dried ground prediction was attributed
314
to 60% of the samples being under predicted in clay content and large discrepancies
315
between measured and predicted clay content of soils greater than 450 g kg-1 clay.
316
Because the air-dried in situ model slightly outperformed the air-dried ground model
15
317
using the 1st derivative of VNIR reflectance spectra, we conclude natural soil
318
heterogeneity, as a result of clay films, translocations, redox features etc…, and
319
variability associated with aggregation had little effect on model predictions. Gaffey
320
(1986) found that aggregate size had little effect on prediction accuracy, but Gaffey was
321
looking at carbonate sizes. His results showed a difference in reflectance, but the number
322
of bands, band positions and width, and relative band intensities were unchanged.
323
Another significant finding was that the clay content model using the field-moist
324
in situ model had slightly less prediction accuracy than the air-dried in situ model.
325
However, the field-moist in situ model performed just as well when compared to the air-
326
dried ground model (Fig. 2). These results indicate that the amount of water in the soil
327
sample did not change the prediction accuracy compared to air-dried laboratory
328
situations. These results show that soil water may reduce accuracy somewhat (increased
329
RMSD by 20 g kg-1), but this reduction in accuracy was no more than air-drying,
330
grinding, and scanning the soil through a borosilicate Petri dish (Fig. 2). Chang et al.
331
(2005) showed that r2-values decreased slightly from 0.79 to 0.76 from an air-dried soil to
332
a moist soil, respectively, which agrees with other findings. The field-moist in situ
333
models used 102 significant wavelengths in determining clay content (Fig. 3). Figure 3
334
shows the 32 significant wavelengths that were common in all three models.
335
The same set of VNIR models was created to predict fine clay. Fine clay models
336
have lower RMSD values than total clay models but the RPD values are similar to the
337
total clay models (Table 4). The similar RPD values indicate that the smaller RMSD
338
values for fine clay models were due to smaller standard deviations (smaller range); so
339
the total clay and fine clay models performed similarly.
VNIR-DRS has proven
16
340
effective in identifying smectitic clays (Brown et al., 2006) and studies suggest that
341
Vertisol fine clays are comprised primarily of smectites (Anderson et al., 1972; Yerima
342
et al., 1989). Therefore, our ability to predict fine clay in this study might be due to the
343
mineral signature of this size fraction.
344
345
Field-moist Smeared Validation
Smearing of the field-moist in situ cores reduced the accuracy of the prediction
346
model (Fig. 2). The decrease in the prediction accuracy of the smeared cores was
347
probably due to the change in reflectance properties of the soil. Smearing causes the soil
348
surface to become shiny, which changes the reflectance from diffuse to diffuse specular
349
reflectance. Specular reflectance can mask diffuse absorptions and otherwise confuse the
350
analysis of reflectance spectra (Coates, 1998). When comparing the significant
351
wavelengths between the field-moist in situ and field-moist in situ smeared cores, a
352
number of wavelengths found in the visible portion of the spectra are absent from the
353
smeared core (Fig. 4). The smeared core model had 70 significant wavelengths and the
354
unsmeared core model had 102, with 50 of these wavelengths being common in both
355
models. The absence of these wavelengths may be the source of loss in prediction
356
accuracy. In a field situation, the loss of prediction accuracy due to smearing might be
357
less because it took multiple passes with the knife to smear many of the soil cores.
358
Whole-field Holdout Validation
359
As expected (Brown et al., 2005), whole-field cross-validation yielded degraded
360
predictions relative to random cross-validation. For all six fields, whole field validation
361
statistics (Table 5, RMSD and RPD) were worse than for the randomly selected cross-
362
validation results (Table 4). The RPD statistics are particularly telling, with fields 1 and
17
363
3 having RPD values < 1 (no predictive value), fields 4-6 having RPD values < 1.5 (little
364
predictive value), and only field 2 yielding acceptable results (RPD = 1.92).
365
Both field 1 and 3 contained soil at the extremes for clay content. Field 1
366
consisted of high clay soils (median 46 % clay) that were dark in color through the entire
367
length of the soil core—high clay contents that were not represented in the calibration
368
soils. Not surprisingly, the prediction bias for this field was -128.1 g kg-1 (Table 5)
369
indicating a significant underprediction of these high clay contents. Conversely, field 3
370
contained relatively low clay soils (median 3 % clay) not found in the other five fields.
371
Regional clay calibrations will require a far larger and more diverse soil-spectral library
372
than that presented in this study.
373
DRS as a Proximal Sensor for Clay Content
374
Figure 5 illustrates the performance of VNIR-DRS used as a proximal soil sensor
375
for in-situ soils at field soil moisture. Three very different soil profiles are shown. The
376
first is a Vertisol from field 1. This soil is very high in clay and somewhat uniform in
377
clay content with depth. The VNIR-DRS prediction shows the same uniformity, with 10
378
% clay under prediction (Fig. 5a.). Nonetheless, the prediction does put the profile in the
379
correct clay texture category. The second soil, Fig. 5b., is an Alfisol, with a sandy
380
surface and an E horizon. In the case of the Alfisol, VNIR-DRS predictions show the
381
continuous change of clay content, the A, E, and Bt horizons are clearly seen. The third
382
soil is a Mollisol with secondary carbonate concentrations in the B horizon (Fig. 5c). The
383
VNIR-DRS prediction of this core was encouraging, because the secondary carbonates
384
did not appear add random scatter to the prediction. The concentrations may have biased
385
the clay content prediction toward the bottom of the profile. In summary, though the
18
386
VNIR-DRS predictions do no perfectly agree with the particle size analysis in these three
387
cores, the predictions do provide a rapid, continuous measurement of at least relative soil
388
clay content fluctuations with depth.
389
390
CONCLUSIONS
In this study of 72 soil cores from Central Texas, we found a strong relationship
391
between VNIR reflectance and clay content for air-dried in situ (RMSD = 41 g kg-1),
392
field-moist in situ (RMSD = 61 g kg-1), and air-dried ground (RMSD = 62 g kg-1) scans.
393
Visible near-infrared DRS was capable of predicting soil clay content in-situ at varying
394
water contents. There was a decrease in predictive accuracy for smeared field-moist in
395
situ (RMSD = 74 g kg-1) scans. However, we obtained worse results for ground and
396
sieved samples than for the non-smeared in situ scans—suggesting that soil aggregates
397
and natural heterogeneity did not degrade in situ predictions. Variable water contents did
398
reduce predictive accuracy as evidenced by a comparison of air-dried in situ and field-
399
moist in situ results, but the field-moist in situ were still slightly better than those
400
obtained from scans of air-dried, ground-and-sieved samples. Validation RPD statistics
401
were similar for fine clay and total clay predictions.
402
These results indicate that VNIR-DRS may be useful as a proximal soil sensor in
403
field conditions. In particular, we envision a VNIR-DRS sensor placed in a soil probe for
404
in situ soil profile characterization. Such a probe would give soil scientists the ability to
405
acquire soil profile data (e.g. clay content) at greater spatial and vertical sampling
406
densities than is practical with conventional soil excavation, extraction and
407
characterization techniques. To make this vision a reality, continued research is needed
19
408
on in situ VNIR-DRS applications with greater soil diversity and a wider range of soil
409
properties.
410
411
REFERENCES
412
Anderson, J.U., K.E. Fadul, and G.A. O’Connor. 1972. Factors affecting the coefficient
413
of linear extensibility in vertisols. Soil Sci. Soc. Am. Proc. 37:296-299.
414
Ben-Dor, E. and A. Banin. 1995. Near-infrared analysis as a rapid method to
415
416
simultaneously evaluate several soil properties. Soil Sci. Soc. Am. J. 59:364-372.
Brown D.J., R.S. Bricklemeyer, and P.R. Miller. 2005. Validation requirements for
417
diffuse reflectance soil characterization models with a case study of VNIR soil C
418
prediction in Montana. Geoderma. 129:251-267.
419
Brown, D.J., K.D. Shepherd, M.G. Walsh, M.D. Mays, and T.G. Reinsch. 2006. Global
420
soil characterization with VNIR diffuse reflectance spectroscopy. Geoderma.
421
132:273-290.
422
Chang, C.W., D.A. Laird, and C.R. Hurburgh, Jr. 2005. Influence of soil moisture on
423
near-infrared reflectance spectroscopic measurement of soil properties. Soil Sci.
424
70:244-255.
425
Clark, R.N. 1999. Spectroscopy of rocks and minerals, and principles of spectroscopy.
426
p. 3-52. In A.N. Rencz (ed.) Remote sensing for the earth sciences: Manual of
427
remote sensing. John Wiley & Sons, New York.
428
429
Clark, R.N., T.V.V. King, M. Klejwa, and G.A. Swaze. 1990. High spectral resolution
reflectance spectroscopy of minerals. J. Geophys. Res. 95:12653-12680.
20
430
Coates, J. 1998. A review of sampling methods for infrared spectroscopy. p. 49-91. In
431
J. Workman and A.W. Springsteen (ed.) Applied spectroscopy, a compact
432
reference for practioners. Academic Press, San Diego, CA.
433
Dalal, R.C. and R.J. Henry. 1986. Simultaneous determinations of moisture, organic
434
carbon, and total nitrogen by near infrared reflectance spectrophotometry. Soil Sci.
435
Soc. Am. J. 50:120-123.
436
Ellert, B.H., H.H. Janzen, and T. Entz. 2002. Assessment of a method to measure
437
temporal change in soil carbon storage. Soil Sci. Soc. Am. J. 66:1687-1695.
438
Gaffey, S.J. 1986. Spectral reflectance of calcite minerals in the visible and near
439
infrared (0.35-2.55 microns): Calcite, aragonite, and dolomite. Am. Mineral.
440
71:151-162.
441
Gauch, H.G., Jr., J.T.G. Hwang, and G.W. Fick. 2003. Model evaluation by comparison
442
of model-based predictions and measured values. Agron. J. 95:1442-1446.
443
Gee, G.W. and D. Or. 2002. Particle-size analysis. p. 255-293 In J.H. Dane and G.C.
444
Topp (ed.) Methods of soil analysis. Part 4. ASA, CSSA, and SSSA, Madison WI.
445
Goetz, A.F., S. Chabrillat, and Z. Lu. 2001. Field reflectance spectroscopy for detection
446
of swelling clays at construction sites. Field Anal. Chem. Technol. 5:143-155.
447
Hallmark, C.T., L.T. West, L.P. Wilding, and L.R. Drees. 1986. Characterization data
448
for selected Texas soils. Texas Agricultural Experiment Station. College Station,
449
TX.
450
451
Hunt, G.R. and J.W. Salisbury. 1970. Visible and near-infrared spectra of minerals and
rocks: I. Silicate minerals. Modern Geol. 1:283-300.
21
452
Islam, K., B. Singh, and A. McBratney. 2003. Simultaneous estimation of several soil
453
properties by ultra-violet, visible, and near-infrared reflectance spectroscopy. Aust.
454
J. Soil Res. 41:1101-1114.
455
456
457
458
459
460
461
462
463
Janik, L.J., R.H. Merry, and J.O. Skjemstad. 1998. Can mid infrared diffuse reflectance
analysis replace soil extractions? Aust. J. Exp. Agric. 38:681-696.
Kilmer, V.H., and L.Z. Alexander. 1949. Methods for making mechanical analyses of
soil. Soil Sci. 68:15-24.
McBratney, A.B., M.L.M. Santos, and B. Minasny. 2003. On digital soil mapping.
Geoderma. 117:3-52.
Moore, I.D., P.E. Gessler, G.A. Nielsen, and G.A. Peterson. 1993. Soil attribute
prediction using terrain analysis. Soil Sci. Soc. Am. J. 57:443-452.
Morgan, C.L.S., J.M. Norman, C.C. Molling, K. McSweeney, and B. Lowery. 2003.
464
Evaluating soil data from several sources using a landscape model. p. 243-260. In
465
Y. Pachepsky, D.E. Radcliffe, and H.M. Selim (ed.) Scaling methods in soil
466
physics. CRC Press, Boca Raton, FL.
467
468
469
Pachepsky, Ya.A., D.J. Timlin, and W.J. Rawls. 2001. Soil water retention as related to
topographic variables. Soil Sci. Soc. Am. J. 65:1787-1795.
R Development Core Team, 2004. R: A language and environment for statistical
470
computing. R Foundation for Statistical Computing, Vienna, Austria. (Avaliable on-
471
line with updates at http://www.Rproject.org) (Verified 04 Oct. 2006)
472
Rawlins, S.L. and G.S. Campbell. 1986. Water potential: Thermocouple psychrometry.
473
p. 597-662. In A. Klute (ed.) Methods of soil analysis. Part I. ASA, CSSA, and
474
SSSA Madison, WI.
22
475
Reeves III, J.B. and G.W. McCarty. 2001. Quantitative analysis of agricultural soils using
476
near infrared reflectance spectroscopy and a fibre-optic probe. J. Near Infrared
477
Spectrosc. 9:25-34.
478
Reeves III, J.B., G.W. McCarty, and J.J. Meisinger. 1999. Near infrared reflectance
479
spectroscopy for the analysis of agricultural soils. J. Near Infrared Spectrosc. 7:179-
480
193.
481
482
483
484
485
Shepherd, K.D. and M.G. Walsh. 2002. Development of reflectance spectral libraries for
characterization of soil properties. Soil Sci. Soc. Am. J. 66:988-998.
Sorensen, L.K. and S. Dalsgaard. 2005. Determination of clay and other soil properties
by near infrared spectroscopy. Soil Sci. Soc. Am. J. 69:159-167.
Steele, J.G. and R. Bradfield. 1934. The significance of size distribution in the clay
486
fraction. Report of the 14th annual meeting. Am. Soil Sci. Assn., Bull. 15, 88-93.
487
Sudduth, K.A. and J.W. Hummel. 1993. Soil organic matter, CEC and moisture sensing
488
489
490
491
with a portable NIR spectrophotometer. Trans. ASAE. 36:1571-1582.
Sudduth, K.A. and J.W. Hummel. 1996. Geographic operating range evaluation of a NIR
soil sensor. Trans. ASAE. 39:1599-1604.
Sudduth, K.A., J.W. Hummel, and S.J. Birrell. 1997. Sensors for site specific
492
management for agriculture. p. 183-210 In F. J. Pierce and E. J. Sadler (ed.) The
493
state of site specific management for agriculture. ASA, CSSA, and SSSA, Madison
494
WI.
495
496
497
U.S. Department of Agriculture. 1973. Soil survey of Erath County, Texas. Soil
Conservation Service. U.S. Gov. Printing Office, Washington, DC.
U.S. Department of Agriculture. 1977. Soil survey of Comanche County, Texas. Soil
23
498
Conservation Service. U.S. Gov. Printing Office, Washington, DC.
499
U.S. Department of Agriculture. 1996. Soil survey lab method manual. Soil Survey
500
Investigations report no.42. Version no. 3. USDA-NRCS, Washington D.C.
501
502
U.S. Department of Agriculture. 2005. Official soil series descriptions [Online].
Available at http://soils.usda.gov/technical/classification/osd/index.html (Verified
503
04 Oct. 2006)
504
Workman, J., Jr. and J. Shenk. 2004. Understanding and using the near-infrared spectrum
505
as an analytical method. p. 3-10. In C.A. Roberts, J. Workman, Jr., and J.B. Reeves
506
III (ed.) Near-infrared spectroscopy in agriculture. ASA, CSSA, and SSSA,
507
Madison, WI.
508
Yerima, B.P.K., L.P. Wilding, C.T. Hallmark, and F.G. Calhoun. 1989. Statistical
509
relationships among selected properties on Northern Cameroon vertisols and
510
associated alfisols. Soil Sci. Soc. Am. J. 53:1758-1763.
511
512
513
514
515
Zhu, A.X., L. Band, R. Vertessy, and B. Dutton. 1997. Derivation of soil properties using
a soil land inference model (SoLIM). Soil Sci. Soc. Am. J. 61:523-533.
Zhu, J., C.L.S. Morgan, J.M. Norman, W. Yue, B. Lowery. 2004. Combined mapping of
soil properties using a multi-scale spatial model. Geoderma. 118:321-334.
24
516
Fig. 1. Schematic of a vertically sliced soil core. Columns and rows indicate locations
517
scanned using the contact probe for in situ scans. Dried ground soil samples represent the
518
soil from one row, both columns.
519
520
Fig. 2. Predicted vs. measured clay content of the validation data set for (a) air-dried
521
ground (b) air-dried in situ (c) field-moist in situ (d) field-moist in situ smeared. Clay
522
content predictions were made from models built with partial least square regression
523
using the 1st derivative of visible near-infrared reflectance (VNIR) spectra (350-2500
524
nm). RMSD is root mean squared deviation.
525
526
Fig. 3. The wavelengths that contributed to significant regression coefficients (p-value ≤
527
0.05) in the prediction of clay content are shown for field-moist and air-dry in situ and
528
air-dry ground models. The relative magnitude of each regression coefficient indicates
529
the strength of the correlation. The top plot shows the mean of the regression coefficients
530
common in all three models. All plots are on the same x-axis. Values of the y-axis are
531
not shown, but all y-axes are on the same scale.
532
533
Fig. 4. The wavelengths that contributed to significant regression coefficients (p-value ≤
534
0.05) in the prediction of clay content are shown for field-moist, smeared and unsmeared,
535
in situ models. The relative magnitude of each regression coefficient indicates the
536
strength of the correlation. The top plot shows the mean of the regression coefficients
537
common in both models. All plots are on the same x-axis. Values of the y-axis are not
538
shown, but all y-axes are on the same scale.
25
539
540
Fig. 5. Laboratory measured (+) and visible near-infrared (VNIR) predicted (◊) clay
541
content for three soil cores, a) a dark-colored Vertisol, b) an Alfisol with distinct A, E, Bt
542
horizons, and c) a Mollisol with secondary calcium carbonate concentrations in the B
543
horizons. Laboratory measurements were taken to represent a row within a horizon, and
544
VNIR predictions were made every 3-cm.
545
26
546
Table 1. Series and family classification of the soils mapped by the Natural Resources
547
Conservation Service within the fields which were used for VNIR-DRS predictions
548
(USDA, 1973; USDA, 1977; USDA, 2005).
Soil series
Abilene
Altoga
Blanket
Bolar
Bosque
Brackett
Bunyan
Chaney
Cisco
Denton
Frio
Houston Black
Karnes
Lewisville
Maloterre
Nimrod
Pedernales
Purves
Selden
Venus
Windthorst
549
Family classification
Fine, mixed, superactive, thermic Pachic Argiustolls
Fine-silty, carbonatic, thermic Udic Haplustepts
Fine, mixed, superactive, thermic Pachic Argiustolls
Fine-loamy, carbonatic, thermic Udic Calciustolls
Fine-loamy, mixed, superactive, thermic Cumulic
Haplustolls
Loamy, carbonatic, thermic, shallow Typic Haplustepts
Fine-loamy, mixed, active, nonacid, thermic Typic
Ustifluvents
Fine, mixed, active, thermic Oxyaquic Paleustalfs
Fine-loamy, siliceous, superactive, thermic Typic
Haplustalfs
Fine-silty, carbonatic, thermic Udic Calciustolls
Fine, smectitic, thermic Cumulic Haplustolls
Fine, smectitic, thermic Udic Haplusterts
Coarse-loamy, carbonatic, thermic Typic Calciustepts
Fine-silty, mixed, active, thermic Udic Calciustolls
Loamy, carbonatic, thermic Lithic Ustorthents
Loamy, siliceous, active, thermic Aquic Arenic Paleustalfs
Fine, mixed, superactive, thermic Typic Paleustalfs
Clayey, smectitic, thermic Lithic Calciustolls
Fine-loamy, siliceous, active, thermic Aquic Paleustalfs
Fine-loamy, mixed, superactive, thermic Udic Calciustolls
Fine, mixed, active, thermic Udic Paleustalfs
Field number
4
6
6
1,5,6
5
5
2
4
4
1,6
1,4,5
1
5
5,6
1
3,4
4
1,5,6
3
5
2,3,6
27
550
Table 2. Clay content and water potential summary statistics for soil samples used in
551
calibration and validation datasets.
N
Total clay (g kg-1)
Fine clay (g kg-1)
Water potential (MPa)
552
188
188
188
Total clay (g kg-1)
82
Fine clay (g kg-1)
82
Water potential (MPa)
82
† SD, standard deviation.
Min.
Max.
Calibration samples
12
525
3
380
-5.8
0.0
Validation samples
28
578
18
362
-2.2
0.0
Mean
SD†
Median
255
138
-0.49
139
85
0.60
262
140
-0.35
271
152
-0.48
144
79
0.47
247
147
-0.38
553
554
Table 3. Clay minerals present in soil cores from each field. Mineralogy is presented as
555
a summary by field and not all cores are represented. Cores selected for mineralogy were
556
chosen to represent clay minerals within a field.
557
Clay Minerals
Field no. Vermiculite Smectite Kaolinite Mica Calcite Quartz
Feldspar
1
XXX†
X
XX
X
2
XX
X
X
X
3
t
X
XX
X
X
t
4
XX
X
X
X
5
X
X
X
X
X
6
XX
X
t
X
X
† XXX, greater than 50% of mineral present; XX, 20-50% of mineral present; X, 5-20%
558
of mineral present; t, trace amounts of mineral present.
28
559
Table 4. Prediction accuracies used for total clay and fine clay content models using the
560
1st derivative of visible near-infrared reflectance spectra (350-2500 nm) and partial least
561
squares regression. Prediction accuracies were calculated by using entire soil cores (30 %
562
of the total data set) randomly held out of the calibration model. Also included are the
563
number of principal components (PCs) used in each partial least squares regression
564
model.
r2
RMSD†
g kg-1
Total clay
62
41
61
74
RPD†
Bias
g kg-1
PCs
565
Air-dried ground
0.84
2.32
-16.3
9
Air-dried in situ
0.92
3.51
-2.0
10
Field-moist in situ
0.83
2.36
3.0
9
Field-moist smeared
0.75
1.95
7.4
6
Fine clay
Air-dried ground
0.81
34
2.32
0.65
4
Air-dried in situ
0.85
31
2.55
4.2
4
Field-moist in situ
0.84
32
2.47
4.4
8
Field-moist smeared
0.75
41
1.93
11
4
†
RMSD, root mean squared deviation; RPD, ratio of standard deviation of clay content to
566
RMSD.
567
Table 5. Prediction accuracies of clay content using whole-field holdouts of field-moist
568
in-situ scans. Models were created using partial least squares regression with five fields
569
and then validated using soil cores from the sixth field. Six separate models were
570
calibrated and validated.
Validation field
1
2
3
4
5
6
RPD‡
Bias
RMSD†
-1
---------------------g kg --------------------143
0.72
-128.1
64
1.92
4.6
97
0.98
9.2
89
1.18
44.0
72
1.21
31.7
68
1.09
30.0
29
571
†
572
RMSD.
RMSD, root mean squared deviation; RPD, ratio of standard deviation of clay content to
573
574
575
Ro
Ro
Ro
Ro
Ro
Ro
Ro
Ro
Inserted:
576
F
e
lin
1:
1
lin
300
200
100
400
lin
1
(d)
y= 0.65x + 103
r2= 0.75
e
RMSD= 41 g kg-1
1:
e
(c)
y= 0.76x + 69
r2= 0.83
lin
500
RMSD= 62 g kg-1
1
600
300
200
100
0
RMSD= 61 g kg-1
0 100 200 300 400 500 600
Lab clay (g kg-1)
577
(b)
y= 0.88x + 31
r2= 0.92
400
0
VNIR clay (g kg-1)
1:
1
500
(a)
y= 0.72x + 60
r2= 0.84
1:
VNIR clay (g kg-1)
600
e
30
Fig. 2
RMSD= 74 g kg-1
0 100 200 300 400 500 600
Lab clay (g kg-1)
Significant regression coefficient (b)
31
Similar wavelengths
Air-dried ground
Field-moist in situ
Air-dried in situ
500
578
579
580
Fig. 3
750
1000
1250 1500 1750
Wavelength (nm)
2000
2250
2500
32
Significant regression coefficient (b)
Similar wavelengths
Smeared in situ
Field-moist in situ
500
750
1000
1250 1500 1750
Wavelength (nm)
2000
2500
Fig. 4
581
582
583
584
" Measured
% VNIR prediction
-1
clay content (g kg )
0
0
depth (cm)
20
40
60
80
585
586
2250
Fig. 5
150 300 450 600
a)
0
150
b)
300
450
0
150
c)
300
450