Measuring Broadband America:
Fixed and Mobile Broadband Performance
Measurement
Open Platforms & Opportunities for
Collaboration
[email protected]
202-418-0807
Chief EMCD/OET/FCC
[email protected]
202-418-7351
Senior Attorney Advisor EMCD/OET/FCC
Disclaimer
The opinions expressed are those of the author
and do not necessarily represent the views of the
Federal Communications Commission or the
United States Government; The Maureen and
Mike Mansfield Foundation; or any Japanese
Ministry or the Government of Japan.
本人の見解によるものであり、アメリカ合衆国
その他の代弁ではないこと をご承知下さい
• Measuring Broadband America
– Recognition of importance of broadband and need to
improve availability of broadband network performance
information.
– Core principles
• Provide open and transparent processes and
information on broadband performance
• Full release of collected data consistent with our
privacy policy
• Collaboration with industry, academia, and other
stakeholders
• Fixed Broadband Tests
– Commission began gathering data in 2011 and has
released three reports on fixed broadband performance.
– Strategic goals to ensure accountability, increase
transparency, and enhance competition in the market.
– Reports have spurred investment, helped consumers make
informed decisions about the marketplace, and helped the
Commission make fact-based decisions.
– Developed infrastructure for nationwide testing and data
collection
• Expansion to Mobile
– Recognition of importance of mobile broadband and need to
expand the program to mobile broadband networks achieved
with releasing of FCC Speed Test App for Android.
– Shares core principles and open methodologies of fixed effort
• Provides information to consumers and interested stakeholders using
Open and Transparent methodologies about mobile broadband.
–Release of collected data consistent with privacy policies
• Encourage improvements in mobile broadband networks and drive
competition and fact-based decision making.
• Utilize collaborative Open approach with support from academics,
industry, government, public interest and other stakeholders
• Consumer privacy top priority.
– Developed policies and procedures working with a diverse team
of privacy experts from the federal government, academia, and
industry.
– Fully anonymous data collection with no personal or identifiable
information collected.
– Client software released as open source for inspection
– Analysis of data’s statistical properties by privacy experts to
ensure anonymity and prevent re-identification prior to release
of data.
– Mobile App Terms and Conditions and Privacy Policy developed
over multi-month “privacy by design” and reviewed by all
stakeholders
• Handset Retains
Personalized
Performance History
– Average and individual
test result history
displays changes in
mobile broadband
network performance
over time.
– Removes need for
personalized data on
backend
• App Functionality
– Available to download from
Google Play.
– Tests run as requested by the
user (manually), and on a
random centrally-scheduled
background basis.
• Measures speed, latency, and
packet loss.
• Structural data including RSSI,
Bearer, Carrier, CellTowerID,
etc.
• Data cap defaults and other
consumer protections
• Shareable Collaborative Datasets
– Openly documented data dictionary
– Structured export capability
– Anonymized data
– Public and unrestricted use of data consistent with privacy policy
• Opportunities for Collaboration
– Use of shared datasets
– FCC App source code available
– Ability to support other data collections
– Leverage collaborative and privacy-centric best practices
• Data Sources and Processing
– JSON Native Files
– Business Rules and SQL Processing Code
– Supporting Analysis and Existing FCC Reports
– Current Aggregations, Thresholds and Mapping Efforts
•
JSON Native Files
– Active Tests
Measurement
Tests
– Wireless Cellular
JHTTPGETMT
– Handset APIs
– Test Conditions &
Error Conditions
JHTTPPOSTMT
JUDPLATENCY
CLOSESTTARGET
Metrics
phone_identity
network_data
gsm_cell_location
•Differentiating Layer 2 and Layer 3 Results
–Definitions of test success and failure
–UDP Latency = 0 Success = 0 && RSSI Bearer etc
•Data Quality and Mobile Privacy
–Sample Counts and mobile’s inherent variance
•Satisfying Confidence Intervals
•Satisfying Diversity of Samples
•National vs. Other Geographies
•Govt. Programs Costs Speed of Deployment
•ACM and IETF meetings in Yokohama
•Limitations for consumer smartphones
ご清聴ありがとうござました!
Questions?
• Consumer privacy Issues
– Direct personally identifiable information
– Unique handset identifiers or data features
– Analysis and data processing policies to allay risks of
reidentification
•
JSON Native Files
Measurement Reference
– Active Tests
– Wireless Cellular
– Handset APIs
– Test Conditions &
Error Conditions
Property
Type
Explanation
Description
_received
Integer
unix_timestamp of
reception
The timestamp recoded at
server side at the moment the result file
is being received.
_sourceip
String
source ip
address
The Internet Protocol (IP)
address of the handset submitting the
results to the collecting infrastructure as
seen by the collecting infrastructure.
enterprise_id
String
FCC_Public
The code for different
panel programs.
sim_operator_code
String
android.telephony.Telepho
•
JSON Native Files
JHTTPGETMT Reference
– Active Tests
Property
•
JHTTPGETMT
•
JHTTPPOSTMT
•
JUDPLATENCY
•
CLOSESTTARGET
– Wireless Cellular
– Handset APIs
– Test Conditions &
Error Conditions
Type
Explanation
Description
type
String
JHTTPGETMT
The active metric type
'JHTTPGETMT' describes
measurement results of the active test
for download performance.
bytes_sec
Integer
154716
The field represents the
throughput experienced during the
transfer period of the test, the value is
obtained dividing the total amount of
bytes transferred during the
“transfer_period” by the time they have
been transferred. This represents hence
the download speed.
datetime
format)
2013
String (Android dtime
Fri Jan 25 15:35:22 GMT
The field represents the
•
JSON Native Files
network_data Reference
– Active Tests
– Wireless Cellular
Property
Type
Explanation
Description
•
network_data
•
gsm_cell_location
•
cdma_cell_location
•
.getTypeName()
The field holds an integer from
cell_neighbour_tower_data
the Android method that identifies the type of
•
[..]
active_network_type
String
android.net.ConnectivityManag
er
.getActiveNetworkInfo()
wireless network that provides Internet
connectivity at the time of the observation.
location
[...]
– Handset APIs
network_operator_code
String
android.telephony.TelephonyMa
nager
– Test Conditions & Error Conditions
•
JSON Native Files
phone_identity Reference
Property
– Active Tests
Type
Explanation
Description
type
– Wireless Cellular
– Handset APIs
•
phone_identity
– Test Conditions &
Error Conditions
String
phone_identityThe passive
metric type 'phone_identity' describes
features of the handset and installed
operating system.
datetime
String
Fri Jan 25
15:35:07 GMT 2013
The unix time
and date of the handset performing the
measurement at the beginning of the
observations.
manufacturer
String
api
android.os.Build.MANUFACTURER
The field holds a string from
the Android method that identifies the
handset manufacturer.
model
String
api
android.os.Build.MODEL
The field
holds a string from the Android method that
identifies the handset model.
•
JSON Native Files
– Active Tests
– Wireless Cellular
– Handset APIs
– Test Conditions &
Error Conditions
•
NETACTIVITY
•
PARAM_EXPIRED
•
CPUACTIVITY
NETACTIVITY Reference
Property
Explanation
Type
Description
type
String
NETACTIVITY
The passive metric type 'NETACTIVITY'
describes the traffic sent and received by the
handset during a test condition period.
bytesin
Integer
0
The field holds an integer value that identifies
the number of bytes received by the handset during
the test condition period.
bytesout
Integer
0
The field holds an integer value that identifies
the number of bytes sent by the handset during the
test condition period.
datetime
String
Fri Jan 25
10:23:21 EST 2013 The unix time and date of the
handset performing the measurement at the
beginning of the observations.
maxbytesin
Integer
10000
The field holds an integer value that identifies
the maximum limit of bytes to be received by the