Radiocommunication Study Groups
INTERNATIONAL TELECOMMUNICATION UNION
Received: Date 2016
Document -E
Date 2016
English only
Technology Aspects
TBD12
EVALUATION METHODOLOGY AND SCENARIOS FOR IMT-2020
1 Introduction
In the last ITU-R WP5D meeting #23 work has stared on evaluation methodology of IMT 2020.
IMT-2020 is expected to push the broadband user experience and network available in contemporary
networks to the new extremes. It will also span the wireless connectivity to new areas such as IoT or
ITS. Although solutions addressing these domains are already available, IMT-2020 will cater for
them under the umbrella of one system. The level of integration between different Air Interface
Variants (AIVs) supporting different IMT-2020 test cases is still under discussion in many
organizations, vendors and research projects. Therefore it is of the uttermost importance to provide a
unified evaluation framework that could be used to appropriate assess proposed technical solution
with respect to the IMT-2020 technical performance requirements.
2 Proposal
[CEPT] propose a framework for IMT 2020 Evaluations included in Annex 1 for consideration in
the development of the new report ITU-R M. [IMT-2020. EVAL].
____________________
1
This contribution has been agreed within the framework of CEPT ECC PT1
2
The content of this contribution originates from the work done in METIS-II but does not
necessarily reflect the viewpoints of all partners in the METIS-II project.
DOCUMENT1 ( )
13.07.17
13.07.17
-2-E
Annex 1
1) Introduction
Considering the IMT-2020 usage scenario as defined in Recommendation ITU-R M.2083, there is a
need of the creation of appropriate performance evaluation framework that will allow for a fair and
comparable evaluation of various technology concepts proposed for the new generation of wireless
communication systems.
2) General considerations for evaluation methodologies
Following the approach adopted in [ITUR-M.2135], characteristics of IMT-2020 technology
solutions could be evaluated using three methods: simulations, analysis and inspection.

Evaluations through simulations contain both system level simulations and link level
simulations.

In case of analytical procedure, the evaluation is to be based on calculations using the technical
information provided by the technology component owner.

In case of evaluation through inspection the evaluation is based on statements.
For system level simulations the following principles should be considered:

System simulations shall be based on proposed deployment scenarios and models captured
later in the document.

Cell assignment to a user is based on the cell selection scheme proposed by the technology
component owner, and must be described. Some examples are:
o Connection to the station received with highest power, considering a handover margin
of 1 dB.
o Connection to the station received with highest power, considering a handover margin
of 1 dB, but with a limit of users per BS.
o Connection to the station received with highest wideband SINR, with or without a
limit of users per BS.
o Connection to the station whose estimation of the Quality of Service (QoS) satisfaction
is more likely. This could be known based on the SINR estimation, the number of
users connected to each station, and their QoS requirements.
It is allowed to have the control plane user plane served by different stations.

In simulations based on the full-buffer traffic model, packets are not blocked when they arrive
into system (i.e. queue depths are assumed to be infinite).

In bursty traffic simulations, packets that are discarded (e.g. as they can’t be transmitted within
given latency requirements) are also included in the overall performance statistics with 0
correctly received bits.
DOCUMENT1 ( )
13.07.17
13.07.17
-3-E

Packets are scheduled with an appropriate packet scheduler(s) proposed by the proponents for
full buffer and bursty traffic models separately. Channel quality feedback delay, feedback
errors, PDU (protocol data unit) errors and real channel estimation effects inclusive of channel
estimation error are modeled and packets are retransmitted as necessary.

The overhead channels (i.e., the overhead due to feedback and control channels) should be
realistically modeled.

For a given drop, the simulation is run and then the process is repeated with the users dropped
at new random locations. A sufficient number of drops are simulated to ensure convergence
in the user and system performance metrics. Performance statistics are collected taking into
account the wrap-around configuration in the network layout, noting that wrap-around may
not be considered in some test cases
3) Test Environments
In the following test environments that would have impact on technical performance requirements of
IMT-2020 are discussed. It should be noted that test environments are focused on how the simulation
will be carried, and the deployment of real networks could differ. From this respect, it is proposed
that for IMT 2020 the following test environments to be adopted:
1. Indoor hotspot: an indoor isolated environment at offices and/or in shopping malls based on
stationary and pedestrian users with very high user density.
2. Dense urban: an urban environment in city centres with high user density and traffic loads
focusing on pedestrian and slow vehicular users, this will consist of HetNet deployment
(macro + outdoor small cells layer)
3. Rural: rural environment with larger and continuous wide area coverage, supporting high
speed vehicles
4. Urban macro: an urban environment with focus on a high number of connected machine type
devices
4) IMT-2020 usage scenarios mapped to test environments and cases
DOCUMENT1 ( )
13.07.17
13.07.17
-4-E
Figure 1 IMT-2020 usage scenarios, mapped to test cases.
As defined in [ITUR-M.2083], IMT-2020 will be based on three usage scenarios. For these three
usage scenarios the following test case are envisaged:
Enhanced Mobile Broadband (eMBB)
 Dense Urban Information Society: the objective of this test case is to provide connectivity for
city dwellers at any place and time. Exemplary applications in this test case are immersive
multi-media applications (including 4K/8K ultra high definition video, virtual reality and real
time mobile gaming). IMT-2020 services should provide in such test case an improved quality
of experience (QoE) for users compared to the IMT-A services.
 Virtual Reality Office: This test case is based on a future indoor setting where improved
wireless technologies will provide extremely high data rates while fulfilling challenging
capacity requirements at a reasonable cost
 Broadband Access Everywhere: this test case emphasizes the need of providing acceptable
broadband user experience practically everywhere, even in areas with sparse network
infrastructure or at very high user speeds.
Massive machine type communications (mMTC)
 Massive Deployment of Sensors and Actuators: this test case is related to to the progressive
trend observed in contemporary networks, where machine type devices use radio access to
transmit data related to the variety of applications. Key challenge for mMTC is a very large
number of low cost devices requiring sporadic access for low payload data exchange. This
challenge is different than for eMBB.
Ultra-reliable and low latency communications (URLLC)
 Connected Cars: this test case focus on the exchange of safety related data between moving
vehicles (or potential vulnerable road users). Such safety related communication is
challenging, as reliability of the transmission can be impacted by the availability of radio
resources (possible concentration of vehicles in a cell) and high velocity (frequent cell change
and challenging transmission conditions caused by the high Doppler effect).
DOCUMENT1 ( )
13.07.17
13.07.17
-5-E
eMBB test cases have similar requirements as of IMT-A (high-speed broadband access), however
these requirements are are pushed to the new extremes (e.g. 1Gbps is IMT-A peak data rate, for IMT2020 this is the expected user throughput in test case of virtual reality office), which reflects the
effects of growing number of broadband devices and high data rates applications. In contrary to IMTA evaluation [ITUR M.2135] which focused to a large extent on spectral efficiency, proposed
approach in this input suggest evaluation of eMBB test cases using absolute data rates to better reflect
impact of some key enhancements for IMT-2020, such as utilization of higher frequency bands and
exploitation of extreme network densification.
On the other hand specific nature of mMTC and URLLC application test cases impose different set
of indicators that should be used for an appropriate evaluation of IMT-2020 performance. mMTC
devices should operate with much longer battery life comparing to eMBB and at much higher
densities. Similarly, for URLLC applications key performance indicators will rely on extremely high
reliability linked with capability to transmit data with very low latency.
In the following an example is given on how test cases and environments could be mapped :
IMT 2020 Usage scenarios Recommendation ITU-R M.2083
eMBB
mMTC
URLLC
Test case
Test
Test case
environments
Test
Test case
environments
Test
environments
Dense Urban
Information
Society
HetNet
(Urban
Macro +
Outdoor
Small Cells)
Urban Macro
HetNet (Urban
Macro +
Outdoor Small
Cells
configured as
RSUs)
Virtual
reality office
Indoor
Hotspot
Broadband
access
everywhere
Rural Macro
Massive
Deployment
of Sensors
and
Actuators
Connected
cars
The following table gives information’s on proposed test environments and parameters to be
applied in simulations evaluations:
Test
environments
Indoor
Hotspot
Dense Urban
Rural
Urban Macro
Layout
Single layer
Single layer
Single layer
Single layer
DOCUMENT1 ( )
13.07.17
13.07.17
-6-E
- Indoor floor:
(12TRPs per
120m x 50m)
Macro layer:
Hex. Grid
- Macro layer:
Hex. Grid
- Macro layer:
Hex. Grid
Two layer
- Macro layer:
Hex. Grid
- Micro layer:
Random drop
(All micro
TRPs are
outdoor)
- 3 micro
TRPs per
macro TRP
Inter-BS
distance
20 m
200 m
1732m for
4GHz and
1732m and 5km
for 700 MHz
500 m
Carrier
frequency for
evaluations
Around 4 GHz,
30 GHz and
70 GHz
Around 4 GHz
and 30 GHz
Around
700MHz and
4GHz
Around 4 GHz
and 30 GHz
Aggregated
system BW
Around 4GHz:
Up to 200MHz
(DL+UL)
Around 4GHz:
Up to 200MHz
(DL+UL)
Around 4GHz:
Up to 200 MHz
(DL+UL)
Around 30GHz
or 70GHz: Up
to 1GHz
(DL+UL)
Around 30GHz:
Up to 1GHz
(DL+UL)
Around
700MHz: Up to
20MHz
(DL+UL)
DL+UL
BS antenna
configurations
Around 4GHz:
Up to 200MHz
(DL+UL)
Around
30GHz: Up to
1GHz
(DL+UL)
700MHz: Up to 64 Tx /Rx antenna elements
4GHz: Up to 256 Tx /Rx antenna elements
30GHz: Up to 256 Tx /Rx antenna elements
70GHz: Up to 256 Tx /Rx antenna elements
BS antenna
height
DOCUMENT1 ( )
3m
25m for macro
cells and 10m
for micro cells
35m
13.07.17
25m
13.07.17
-7-E
UE antenna
elements
700MHz: Up to 4 Tx /Rx antenna elements
4GHz: Up to 8 Tx /Rx antenna elements
30GHz: Up to 32 Tx /Rx antenna elements
70GHz: Up to 32 Tx /Rx antenna elements
BS antenna
model
3GPP model
UE antenna
model
3GPP model
BS noise figure
Below 6 GHz: 5 dB
Above 6 GHz: 8 dB
DOCUMENT1 ( )
13.07.17
13.07.17