Fun with data in
sports journalism
Jack Gillum
The Associated Press
Analog to digital
Old, paper rosters mTurk Excel for cleaning HTML/PDF files of rosters Nokogiri SQL Master table of rosters HTML: the best option
Solution: Nokogiri to TSV
Using regular expressions for PDFs
if ($number[$k] eq '') {
($number[$k]) = $_ =~ /.*\|\s?([0-9]{1,2})\s?\|.*/;
}
if ($number[$k] eq '') {
($fullname[$k]) = $_ =~ /^.*?\|([A-Za-z,\'\s\.-]{6,})\|/;
}
else {
($fullname[$k]) = $_ =~ /^[\||\s]?$number[$k]\|{1}(.*?)\|{1}.*$/;
}
Solu%on: Perl regular expressions to TSV Those olde tyme paper rosters
– We wanted to go back 10
years, and most records
from the mid-2000s and
earlier were on paper.
– FOI requests -- with
questionable results.
Those olde tyme paper rosters
...and, the results:
19O
97
Aaro#%ii M. Alexander
So.
WR
63f-5 198
7-7% Hahnvi11e, _ _ • /.
Keenan R. wBews#$asley ¦
6-3
222few
9-23-83
C53#alif./ClaieiSont Tv
SCCo. fDE
P0m0ne,.r3
E2
Dwight F. Blac%kledge
So.
RB
6f-0
188
9-3-82
Ventura, Calif./Buena
{USMAPS) "<1 ’':
Solution: Manual input from mTurk to TSV
Standardizing names w/ Mr. People
And, finally:
school year number name_refined posi4on height weight GaTech 2001 11 George Godsey QB 6-­‐2 205 GaTech 2001 16 Nathan Burton DB 6-­‐2 196 GaTech 2001 16 Charlie Saunders WR 5-­‐8 155 GaTech 2001 18 Richard McCabe SS 6-­‐3 199 GaTech 2001 18 Jonathan Warshaw WR 6-­‐1 190 GaTech 2001 20 Chris Morehouse P 6-­‐1 200 The product of Google Refine
(now Open Refine)
The statistics of it all:
PctBMIChg2 PctBMIChg3 PctBMIChg4 ZRE_all 39.6% 40.0% 45.1% 6.02 11.1% 11.1% 44.1% 5.48 17.4% 23.9% 40.5% 5.07 8.8% 23.3% 34.4% 5.05 23.7% 35.7% 38.4% 4.72 24.6% 31.2% 36.9% 4.68 The statistics of it all:
What’s the deal with
these players’ gains?
So, what did the players tell us?
“I just ate. I ate 5-6 times a day,” said a player who
gained 53 pounds in one year.
“I was just a young kid at that time, and I was still
growing into my body,” said an offensive lineman, who
gained 81 pounds since high school.
“Food and good training will only get you so far,” said
player who admitted to injecting stanozolol.
Other reporting threads:
– Database of prescriptions obtained from
sources
– Interviews with former PED dealers
– Ferreting out federal investigations
The result:
WASHINGTON (AP) — With steroids easy to buy, testing weak
and punishments inconsistent, college football players are
packing on significant weight — 30 pounds or more in a single
year, sometimes — without drawing much attention from their
schools or the NCAA in a sport that earns tens of billions of
dollars for teams.
Rules vary so widely that, on any given game day, a team with a
strict no-steroid policy can face a team whose players have
repeatedly tested positive.
An investigation by The Associated Press — based on dozens of
interviews with players, testers, dealers and experts and an
analysis of weight records for more than 61,000 players —
revealed that while those running the multibillion-dollar sport
believe the problem is under control, that is hardly the case.
How to reach me:
Interwebs:
[email protected]
@jackgillum
In real life:
Hotel bar tonight