EDGE DEVELOPMENT
PHASE 2
Statement of Requirement and
Request for Proposal
Contract Reference: PPRO 04/73/07
Framework Agreement Reference: PPCA 09045
Version 5 final 12/9/12
Page 1 of 16
Contents
1.
Introduction
page 3
2.
Project Aims & Objectives
page 3
3.
Glossary of Terms
page 3
4.
Methodology
page 3, 4
5.
Deliverables
page 4
6.
Programme of Work
page 5
7.
Project Plan
page 5
8.
Project Team
page 5
9.
Management Requirements
page 6
10.
Pricing
page 6
11.
Evaluation Criteria
page 6
12.
Procurement Timetable
page 8
13.
Submission of Proposal
page 8
14.
Points of Contact
page 9
Appendices A-D
pages 10-16
Page 2 of 16
1.
Introduction
1.1
EDGE is the Departments exogenous rail demand forecasting model. It
is currently used internally by the Department as part of the suite of strategic
rail modelling tools. To enable EDGE to be used more widely across the rail
industry, the EDGE user interface, documentation and the set of available
case studies need to be improved.
1.2
In the current economic climate, big capital investment projects are
coming under more public scrutiny. To ensure that we can defend our
prediction of the benefits and costs of these schemes in a robust way, it is
essential the Department understands the limits of our forecasting models.
The second phase of the EDGE project will produce an uncertainty calculation
around the demand forecasts.
2.
Project aims and objectives
2.1
The aims of this project are to produce an update to the existing EDGE
software and user guide which will improve the user interface and create a
clear step by step “how to” guide for use by DfT’s external and internal
stakeholders.
2.2
The software update will be packaged and include multiple case
studies that can be used by DfT stakeholders
2.3
A new module within EDGE will be written that will calculate the
uncertainty around the demand forecasts. The module will; Read in the input
uncertainty data; Produce output files containing uncertainty estimates on the
model outputs ready to be passed into the Network Modelling Framework
(NMF) and MOIRA2.
3.
Glossary of Terms
1) Scenario : defines a set of demand drivers, elasticities and forecast years
over which a demand forecast will be generated
2) Case Study: defines the zone structure over which forecasts are
produced.
3) PDFC: Passenger Demand Forecasting Council
4) UAT: User Acceptance Testing.
5) NMF Network Modelling Framework
Page 3 of 16
4.
Methodology
4.1
Bids should include a full description of the proposed approach.
Specific attention should be given to:
4.1.1 Software familiarisation and approach to dealing with the bug
fixes and functional changes, including testing process. See Appendix
A for examples of many (but not all) of the EDGE bugs/changes.
4.1.2 The EDGE software is programmed in “Microsoft Visual Studio
2010 Professional” and evidence of programming skills in this
environment or similar is required.
4.1.3 Produce a set of case studies (defined in Appendix B) for use by
DfT stakeholders and record the data sources in the EDGE
documentation.
4.1.4 Clear testing process to show that the new EDGE software
accurately converts matrices between zone structures. For example,
base data at NMF zone structure level, case study at RIFF zone level,
and then outputting at NMF zone structure level.
4.1.5 Re-writing the documentation (See Appendix C) The user guide
will be tested by the PDFC community, and some suggestions may
need to be incorporated following UAT.
4.1.6 Package up the software product ready for dissemination to
PDFC. The approach used will allow PDFC/DFT to easily identify each
case study so that each case study can be sent out to separate user
groups
4.1.7 Design and program an uncertainty module add-on, using a
similar method to the approach demonstrated by DFT and described in
Appendix D, and produce associated documentation
4.1.8 Package final software product which includes “uncertainty addon” (g) for DfT.
4.2
Scope of this work covers bug fixing and post user acceptance testing
for a period of 6 months.
4.3
We also request that the bidder provides costing for an independent
review of the uncertainty approach. This item should be priced separately and
will not be part of the evaluation of the bid. This option may not be taken up by
the Department in the event of a successful bid.
5.
Deliverables
5.1
It is envisaged that the project will be completed in two parts.
Page 4 of 16
5.2
The first part will be completed by 30th January 2013 with the following
deliverables:
5.2.1 New version of EDGE which will be released to PDFC members,
which includes all bug fixes and functional changes. This does not
include the “uncertainty module add-on”.
5.2.2 New documentation including “How to guide”. The user guide
and how to guide should be written to publishable standard. All outputs
should be agreed in advance in writing - for example, report outlines
and structure. The Department will need the opportunity to comment
on, and agree the technical user guide. The final drafts of all outputs
should be thoroughly proof-read prior to submission to the DfT. It is
likely that this technical documentation will be published.
5.2.3 Set of case studies for DfT key stakeholders
5.3
The second part should be completed by April 2013. The following
deliverables are required:
5.3.1 New EDGE software that calculates the uncertainty around the
demand forecast according to the Departments desired approach.
5.3.2 Documentation and how to guide relating to this new
functionality.
6.
Programme of Work
6.1
Tenders should include a detailed programme of work setting out how
bidders would meet the objectives of the research. This must demonstrate a
full understanding of the Department's requirements. Tender should also
highlight potential difficulties that might arise and how they would identify
potential risks of delivery and provide a risk-mitigation strategy.
7.
Project Plan
7.1
The tender proposal should include a project plan and time schedule
for the work that identifies the main tasks and key milestones that will be used
to monitor progress towards production of the deliverables, indicating clearly
where DfT is expected to contribute. The plan should also be accompanied by
a breakdown of the resources in person days allocated to each task (a
resource profile).
8.
Project Team
Page 5 of 16
8.1
Tenderers should provide details of the proposed project team, linked
to the project plan, which indicates the grade of staff; number of work days
per staff member, and number of days allocated to specific work areas.
8.2
Tenderers should also provide details of team members' previous
experience of undertaking research of a similar scope and complexity.
9.
Management arrangements
9.1
The contractor is required to appoint a project manager who will
manage the contract and will be regarded by DfT as being fully responsible for
performance of the programme of work.
9.2
The contract with the appointed consultant will be managed by DfT’s
project officer who will expect to be kept in touch with progress and emerging
issues on a regular basis, ideally on a weekly basis.
9.3
A meeting is to take place at the award of contract between the
customer and contractor. The purpose of this inception meeting is to clarify
the direction of the project. Other meetings will be convened as necessary by
agreement between the project officer and the customer project manager.
9.4
Invoices for completed work should be submitted to the project officer
on the completion of the agreed stages of this work.
10.
Pricing
10.1 Fully costed tenders are invited for this work. Tenders should be
submitted on the basis of a firm price to deliver the outputs using the attached
Annex A. In cases where contractors wish to present more than one potential
solution for delivering the outputs, the options should be clearly defined, and a
firm price given for each solution identified. Tenderers should also make
recommendations for a preferred solution. The final contract can be based on
a combination of fixed fee and time and materials but total fee should be
capped.
10.2 Items such as model familiarisation, fixing of known bugs, specified
functional changes, quality assurance, documentation and “uncertainty model
add-on” should be fixed fee.
10.3 Following the completion of the UAT period, the Department will want
to retain a user support component of the work. This should be priced
according to the MAP framework.
11.
Evaluation Criteria
Selection will be based on the evaluation criteria that focus on rewarding
proposals which demonstrates a high degree of overall value for money,
Page 6 of 16
competence, credibility and ability to deliver. The department is under no
obligation to select the lowest costing proposal.
This tender will be evaluated using the following weightings to obtain the
optimal balance of quality and cost.
Quality factors:
Financial factors:
70%
30%
11.1
The quality factors will be assessed against the criteria specified in
the table below:
Primary Criteria
Understanding
of the brief
and project
requirements
Strength of
the approach
proposed
Primary Criteria
Weighting (%)
35
20
Sub Criteria
Understanding the brief
(Sections 4 and 5)
20
Project Requirements
(section 4.1.1 to 4.1.8 and 5)
15
Programme of work
(Section 6)
20
Expertise in scientific
programming (section 8)
Skill /
experience of
named staff,
particularly
35
Demonstrated experience of
upgrading bespoke
transport software systems
(section 8)
Demonstrated experience of
documenting bespoke
software systems (section 8)
Arrangements
for Project
Management
10
Sub -Criteria
Weighting
(%)
including ethics, risk
management and staff
continuity) of the study and
ability to complete the work
within the timetable (section
9.1)
15
10
10
10
Against each of these criteria the quality factors will be assessed using the
methodology in the table below:
Page 7 of 16
Score
5
The proposal demonstrates fully that they can meet the requirement as
detailed in the specification
4
Meets all critical requirements but with minor issues
Meets some requirements but with a few major gaps or issues
3
11.2
Definition of Score
2
Meets some requirements; major concerns
1
Meets few requirements; serious concerns
0
The method of fulfilling the stated requirement is inadequate / not addressed
Price Scoring (30%)
The lowest tendered price will score 10 with subsequent bids base-lined to
this score on a sliding scale (9 for second lowest). The scores will be
multiplied by their weighting.
The vendor with the highest overall score on Quality and price will be awarded
the contract.
12.
Procurement Timetable
Description
Date
Issue Request for ITT
12th September 2012
Closing date for Clarifications of ITT
19th September 2012
Deadline for receipt of bids
11am 1ST October 2012
Evaluation of bids
1-5th October 2012
Clarification of bids (if required)
8-12th October 2012
Contract Award
19th October 2012
This timetable is not binding and may be changed if circumstances dictate.
Suppliers will be notified as soon as practicable of any changes to avoid
adverse impact on their costs. Any likely delay to delivery should be notified
to the Department at the earliest possible opportunity.
13.
Submission of Proposal
13.1 Your proposal should be submitted according to the form of tender
attached to the ITT documents
Page 8 of 16
13.2
14.
Proposals should be submitted by 11am on Monday 1st October 2012
Points of Contact
Procurement
Contact
Contract
Manager
Name
Telephone
Email
xxxxxxxxxxxx
020 7944 xxxx
[email protected]
Address
Ashdown House, Hastings, East Sussex,
TN37 7GA
xxxxxxxxxxxxxx
020 7944 xxxx
xxxxxxxxxxxxx
Name
Tel
Or
alternatively
Tel:
020 7944 xxxx
All queries, questions & clarifications MUST be sent to the procurement
contact shown above
Page 9 of 16
Appendix A Known bugs
1. Manage Data screen: only .exml files to be displayed, not .ebak
2. Main Interface: File Path Report not to display drivers that had box unticked
in the scenario setting screen
3. Scenario Setting screen: add a new column with sequential numbers
(1,2,3,etc.) so that users can see straightaway if they have included all drivers
they meant to
4. change field name in EGE output from “Demand” to “Journeys”
5. fare driver to be treated and inputted as any other driver (EDGE will still need
to know which drivers are the fare drivers in order to calculate revenue), so that it
can be specified at OD level
6. ensure that the functionality of combining employment and population to get
GDP growth rates works with any driver’s name, not only with those containing
the word “employment” and “GDP”
7. ensure that users can select “blank” in drop-downs. Some drop-downs, e.g.,
Base to Case Study Zone Correspondence, do not need populating – i.e., they can
be left blank – but if users change them to a non-blank item, it is then impossible
to change them back to blank
8. Main Interface, Select Assignment Set: it takes ages for the list of assignment
sets to appear, this time needs to be reduced. Also once the list appears, and you
select and assignment set, the chose assignment set takes along time to appear.
So this time needs to be reduced as well.
9. Main interface, takes ages to load list of results files
10. Main Interface, Check Assignment Set: it should check that Case Study
Name and Base Model Name are consistent, if they are not EDGE falls over. And
also check the field labels are consistent across the files used by the assignment
set (for example, flow “Urban Area” to appear in all relevant files with the correct
spelling)
11. EDGE should not open files as read-only as soon as they are saved – only
once they have been used in an EDGE run.
12. users should be able to delete drivers, elasticity files, assignment sets
through the EDGE interface – currently one has to do that through Explorer with
the risk of deleting files that have already been used in previous EDGE runs.
13. does EDGE have a log file reporting any error encountered? If not, it should
be added.
14. Ammend Log files so that they are readable using DFT software e.g. excel
2010
15. At present all indices apart from fares are inputted with reference to base year whilst
fares are inputted with reference to the previous year. It would be useful if the fare input could
be made consistent with the other inputs
16. At present when you are in an assignment set and you select a different demand driver
file on a particular row, it automatically sets the elasticities file to blank. I believe the purpose
of this is to stop you chasing from one type of drive to another (e.g. from fuel prices to car
time) without remembering to update the relevant elasticity file). This is an annoying feature
however as quite often you are simply updating an existing driver and do not want to change
the elasticity file so it would be best if this feature was dropped.
17 When you write over an existing assignment set with the same name with new input files,
EDGE automatically deletes the results files for that assignment set. Instead, EDGE should
not allow the user to create a new assignment set with the same name or filename, nor edit an
assignment set which already has results without changing the name and filename.
Page 10 of 16
18. It is possible in EDGE (and often useful) to create demand driver files that are
disaggregated by OD. Even though this works it creates an error message when you click on
“check assignment set”. Can this error message be removed?
19. We have not been able to get the cross elasticities file to work. Can this feature be fixed
or alternatively can the documentation be improved?
20. In the view results screen, there is a box listing all of the results files, it would be helpful
to be able to order these by a method of the users choosing (years or file type).
21. In order to use the view EGE function you have to click upon each of the EGE files whilst
holding down control. It would be preferable if all of the EGE files within an assignment set
were automatically loaded by clicking “View EGE Results” button.
22. It would be very helpful if the equivalent information for the base year was automatically
loaded when you click the View EGE button.
23. The filename must match the assignment set name
24. Prevent user being able to upload to the Network Modelling framework (NMF) the results
of an assignment set if an assignment set of that name already exists on the NFM database.
(This does not required knowledge of NMF database or the NMF
Page 11 of 16
Appendix B Creating Zone
Correspondence files
A set of EDGE case studies needs to be produced that corresponds to the
PDFC members MOIRA zone structures.
This will mean that for each base demand matrix (listed below) a set of the
following files needs to be created. All the data will be provided, but the files
will need to be put into the EDGE format.
1.
2.
3.
4.
5.
“dummy” demand drivers and “dummy” elasticity files,
market segmentation to case study correspondence
case study to output zone correspondence file
distance matrix
dummy base demand and revenue matrix must be created in the
correct format.
Organisation
Arriva Trains Wales
ATOC
c2c
Chiltern Railways
Cross Country
East Coast
East Midlands Trains
First Capital Connect
First Great Western
First Great Western
First ScotRail
Grand Central
Greater Anglia
Hull Trains
London Midland
Northern Rail
South West Trains
Southeastern
Southern
TransPennine
Express
Virgin Trains
Virgin Trains
Dummy HLOS
assignment set
Code
AW01
OR02
CC01
CH06
XC06
GR04
EM01
FC01
GW05
GW06
SR05
GC02
LE01
HT07
LM01
NT02
SW08
SE07
SN04
TP03
VT02
VT07
Version Name
Arriva Wales
All Operators
c2c
Chiltern Railways
Cross Country
East Coast
East Midlands
Capital Connect
Great Western
First Group
ScotRail
Grand Central
Greater Anglia
Hull Trains
London Midland
Northern Rail
SWT & Island Line
Southeastern
Southern
TransPennine
Express
Virgin Trains
Virgin Trains
XXX
XXXXX
Page 12 of 16
Base demand matrix
Zone size
589
430
231
200
600
229
471
515
480
998
475
253
421
392
431
707
441
387
462
482
271
302
XXX
Appendix C: Documentation
Current EDGE documentation is attached.
This documentation needs to be updated, and a second user guide created
that is a step by step user guide that takes users through from installation to
using the model, explaining each input that is required and the outputs that
are produced.
The existing documentation also needs to clarify any naming conventions that
users have to comply with for EDGE to work correctly. For example, demand
drivers name need to start with "DD_..." otherwise EDGE will not work and the
field names in the different tables need to be in a specific format, or whether
the order of the columns is relevant.
Specific issues identified about the documentation are:


Explaining how the fares cross elasticity function work
The employment population conversion to GDP per capita functionality.
The documentation must be checked for consistency.
Once the documentation has been produced, it will be passed to DFT and the
PDFC community. The bidder should allow for suggestions to be incorporated
following UAT.
Appendix D: Method for calculating
uncertainty estimates.
Calculating the uncertainty on the demand forecast (exogenous)
The Department has investigated the following method of propagating
uncertainty through a single flow of the exogenous demand model and would
like to implement in within EDGE. The method is known as the tangent linear
approach.
The exogenous demand model is made up of a simple relationship (eqn 1).
This makes it ideal to analytically derive the uncertainty.
Eqn 1.
 GDPnew 

I  
GDP
base 

g
p
 Popnew 
 FuelCost new 

 exp( n( NC new  NCbase ))

Pop
FuelCost
base 
base 


Page 13 of 16
f
 Cartimenew 


Cartime
base 

c
 BusCost new 


BusCost
base 

b
 Bustime new 


BusTime
base 

t
 BusHead new 


BusHead
base 

h
 AirCost new 


AirCost
base 

a
 AirHead new 


AirHead
base 

Calculating the uncertainty – General example
The following sets out how to calculate the uncertainty on the output of a
model using a simple example.
e.g. Lets say we have two input variables x1 and x2 which are used to produce
a single output y.
The model converts inputs x to y, We’ll use a simple model
y = 10x1 + 3x2
Both x1 and x2 are subject to errors, which are assumed to be Guassian, and
characterised by standard deviations σx1 and σx2, respectively.
The resultant uncertainty on the output y is
2
 y
  y

  
 x1   
 x 2 
 x1
  x 2

2
2
y
So in the simple example it will be
 y2  10 x1 2  3 x 2 2
Again, the error on y is assumed to be Guassian random variable with a
standard deviation of σy. Note that the uncertainties are added in quadrature.
Demonstrating the analytical approach for demand forecasting
To demonstrate the analytical approach we consider one flow
e.g. Birmingham to London.
From equation 1 lets just take the first term for brevity
 GDPnew 

I  
 GDPbase 
g
Let G = GDPnew/GDPbase
I  G 
g
Page 14 of 16
r
So we can calculate the uncertainty on the output σI

 y
  y
    G     g 
 G
  g

2
2
2
I
y
I
 gG ( g 1) = g
G
G
y
 G g ln G = I ln G
g
As we know σG and σg we can calculate σI
Now lets consider first 2 terms of equation 1;
 GDPnew 

I  
 GDPbase 
g
 Popnew 


 Popbase 
p
Let G = GDPnew/GDPbase and P = Pnew/Pbase
I  G  P 
g
p
So we can calculate the uncertainty on the output σI
2
  y

 y
  y
  y
    G     g     P     p 
 G
  g
  p
  P

2
2
2
2
I
y
I
 P p gG ( g 1) = g
G
G
y
 G g P p ln G = I ln G
g
…etc
So it is easy to calculate the partial derivatives for each of the components of
the equation.
Page 15 of 16
This method should be applied within edge, incorporating methods for reading
in all the input and elasticity uncertainties, storing the outputs and creating
output files that can be read into MOIRA 2 and NMF.
Page 16 of 16