ABC
Department of
Semiotics
-SAMPLEProfessional Services for Semiotic
Monitoring RFP
FAIRNESS MONITOR’S REPORT
OF BID EVALUATION
OUTCOMES
Mmmm dd, yyyy
Prepared by:
RFP SOLUTIONS - Procurement Strategies for Government
Ottawa, Canada
Telephone: (613) 728-1335
-SAMPLEAttestation of Fairness Monitor
I, Paul Probity, having served as the Fairness Monitor during the evaluation of
ABC’s Professional Services RFP, and having participated in the proposal
evaluation process associated with this RFP, do hereby attest that:
1.
The evaluation of the proposals presented to the Evaluation Committee was
undertaken in accordance with the pre-established evaluation criteria
described in the RFP, and that the pre-established evaluation criteria were
developed in advance of the RFP being placed on GETS;
2.
Any and all proposals considered by the Evaluation Committee which were
deemed to be non-compliant for failing to meet any one of the established
mandatory requirements as set out in the RFP, were duly analyzed,
documented and verified prior to rendering this decision;
3.
Where clarifications/questions were submitted to bidders by ABC during the
evaluation process, that these requests for clarification/questions were
submitted by ABC in writing for dissemination to the appropriate bidders;
4.
The proposal evaluation methodology used by the Evaluation Committee
was based on the “rules of evidence approach”, with determinations being
made on the basis of whether or not evidence was supplied by bidders in
their proposals against each of the previously established proposal
evaluation criteria described in the RFP; and
5.
The proposal evaluation process was conducted in a consistent, fair, evenhanded, objective and transparent manner, in accordance with the criteria set
out in the RFP, and that the specific criteria and tests of evidence were
Page 1
consistently and fairly applied to all proposals considered by the Evaluation
Committee.
Page 2
-SAMPLEIn consideration of the above, and in consideration of my efforts as Fairness
Monitor to verify the process followed in evaluating responses to this RFP and to
seek clarification, as required, for the rationale behind any Evaluation Committee
decision, I am satisfied that the evaluation process associated with this RFP was
conducted in a fair, open, consistent and transparent manner.
Signed at LOCATION, this 25th day of June, 2003:
Paul Probity, M.B.A.,
Assistant Director
Fairness Ltd.
Tel:
Page 3
-SAMPLE-
FAIRNESS MONITOR REPORT
PART A: OVERVIEW
Purpose
The purpose of this document is to describe the process and outcomes associated
with the solicitation and evaluation of proposals in relation to ABC’s RFP # 1234-5678, as well as the work undertaken by the Fairness Monitor to verify the
fairness and probity of the process and outcomes.
Mandate of the Fairness Monitor
The services of a Fairness Monitor (Fairness Ltd.) were retained by ABC to act as
an independent and impartial third-party observer of the procurement process
associated with this RFP.
The Fairness Monitor (Paul Probity, M.B.A.) was mandated to oversee and
observe the proposal evaluation process and to provide the Department with a
report outlining his impartial observations and findings with respect to the
compliance of the process with both the principles and practices of fairness.
In all respects, the Fairness Monitor served as a neutral and objective third-party
during the process, with no interest (financial or otherwise) in the outcome of the
proposal evaluation exercise other than ensuring that an open, fair and transparent
process was followed.
The Fairness Monitor attended and participated in all of the deliberations and
Page 4
-SAMPLEdiscussions held by the Evaluation Committee to determine the compliance of
bidders, and participated in discussions to obtain clarification on any outstanding
points or issues. All of the documentation submitted by the bidders was made
available to the Fairness Monitor.
The Fairness Monitor is the Assistant Director with Fairness Ltd., an Ottawabased firm specializing in RFP development and proposal evaluation services.
Paul Probity is the former Head of Inquiry at Examination Inc. Mr. Probity holds a
valid Secret-level government security clearance.
Fairness Ltd. also maintains a strict corporate ethics policy which, among other
things, requires that Mr. Probity divulge nothing regarding the RFP process to
outside parties, before, during or after the RFP process, without the explicit written
authorization of ABC.
Profile of RFP Issued by ABC
The RFP’s (see below) were posted on the Government Electronic Tendering
System on mmmm dd/yy, with a closing date/time of mmmm dd/yy, 00:00 hrs.,
EST (including extensions).
RFP Number
12-34-5678
RFP Name
Professional Services for Semiotic Monitoring
Page 5
-SAMPLEMembers of the Evaluation Committee
The members of the ABC Evaluation Committee evaluating proposals received in
response to the RFP were:
1.
2.
3.
4.
Adam Administration, ABC
Patricia Proposal, ABC
Paul Probity, Fairness Ltd. (Fairness Monitor)
Carla Contract, ABC (Recording Secretary - Non-voting)
Summary Outcomes of the Bid
Evaluation Process
For all bids deemed to be in compliance with the mandatory requirements, the Bid
Evaluation Committee completed a detailed evaluation in order to assess the
evidence presented in their proposals against the rated criteria stipulated within the
RFP. The summary-level results of this analysis are presented in the tables below:
1a. Professional Services:
Bidder
Mandatory
Criteria
Met (Y/N)
Total
PointRated
Score
Ranking
# of
Qualified
Resources
Connie Contractor
Yes
85/100
1
1
Peter Procurement
Yes
70/100
2
1
Adele Acquisition
No
n/a
n/a
n/a
Page 6
-SAMPLEMandatory Criteria Within the RFP
Bidders’ proposals needed to meet ALL of the following mandatory criteria for
their submissions to be considered for further evaluation. Failure on the part of
bidders to meet these requirements resulted in their proposals being deemed noncompliant, with the proposal being given no further consideration.
MANDATORY CRITERIA
1.
Bidders must include within their bid a detailed curriculum vitae (c.v.) for each
proposed Professional Services resource named in their bid; c.v.’s must include
chronological work experience (indicated in years/months) as well as a detailed listing
of the educational and professional designation attainments, as well as all other
academic credentials for each proposed Professional Services resource.
2.
It is mandatory that all proposed Professional Services resources must be able to
demonstrate within their c.v.’s, a minimum of twelve (12) months previous work
experience during the past ten (10) years (on a cumulative basis) in the provision of
Professional Services in the specific subject area of Semiotic Monitoring. Proposed
Professional Services resources lacking these respective experience requirements will
not be considered by the Department.
3. Bidders must include within their bid the names of at least three (3) impartial references
who have knowledge of the bidder’s experience in the provision of professional services in the
specific subject area of Semiotic Monitoring. At a minimum, the name, address, and telephone
number for each named reference must be included. Note: It is not necessary to provide
references for each proposed resource.
Page 7
-SAMPLEPoint-Rated Evaluation Methodology
Bidders’ proposals were evaluated and point-rated against the following criteria,
using the evaluation factors and weighting indicators specified for each criteria.
Only those bidders whose aggregate point-rating score on the whole exceeded the
passmark threshold (65/100) were considered as responsive.
Certain of the following point-rated criteria apply to information submitted by
bidders at the firm/bidder level (#’s 3, 4, 5 & 6), while other criteria relate
specifically to information provided by bidders concerning each of their proposed
resources (#’s 1 & 2). Each proposed Professional Services resource submitted by
the bidder will be evaluated on a category-by-category basis against criteria # 1 &
2, and an average point-rating score derived for each category. The average pointrating score for criteria # 1 & 2 (by category) will be added to the bidder’s score
on criteria # 3, 4, 5 & 6 and totalled, leading to an aggregate point-rating score for
the proposal on the whole.
Page 8
-SAMPLEPOINT-RATED CRITERIA
POINT-RATED
CRITERIA
1.
For each proposed Professional
Services resource, bidders
should indicate their knowledge
of the historical, legal, political
and cross-cultural elements
affecting Semiotic Monitoring
in Canada, as well as their
knowledge of the documentary
and evidentiary requirements of
the Semiotic Monitoring
process, as evidenced through
previous work experience
and/or formal education in this
general subject area.
WEIGHT
/20
EVALUATION
FACTORS/METHODOLOGY
2 points (or part thereof) for each year of previous
work experience and/or formal education in this
general subject area, as listed in the c.v. of each
proposed Professional Services resource, to a
maximum of 20 points (calculated as an average per
category, based on the rating score of all proposed
resources by category). Evaluations will be
undertaken for each proposed Professional Services
Resource on a category-by-category basis.
Based on evidence contained within proposals, the
Evaluation Com m ittee determ ined the total
relevant experience and education. Bachelor
degree (any field) - 3 years; Bachelor degree with
Honours (any field) - 4 years; Relevant Masters
degree - 2 years; Relevant PhD degree - 1-3 years
(based on evidence supplied; Experience as per
evidence supplied of relevant experience.
(X months education and experience/12),
m ultiplied by 2 = points awarded up to a
m axim um of 20.
Page 9
2. For each proposed
Professional Services
resource, bidders
should describe the
proposed resource’s experience in the
general area of Professional Services as
evidenced through previous work
experience in this general area.
-SAMPLE-
a) Professional Services:
responsible for the planning,
managing and direction of
research projects and teams
working to develop complete
and factual briefs for semiotic
codification (e.g. Research
Plans, Research Reports,
Statements of Claim/Reference
Analysis, Background
Documents, Briefing Materials);
b) responsible for identifying
primary documents and other
materials of relevance to
Semiotic Monitoring; extracting
relevant data from primary
documents, transcripts and
secondary sources; identifying
distinct issues of relevance to
monitoring in such sources;
compiling annotated
bibliographies; writing document
summaries and file summaries;
producing annotated document
indices; reading and transcribing
hand-written documents;
working with computerized
database information systems
used for monitoring; and
entering data derived from
research into such
/20
2 points (or part thereof)
for each year of previous
work experience in this
area, as listed in the c.v.
of each proposed resource, to a maximum of 20
points (calculated as an average per category, based
on the rating score of all proposed resources by
category). Evaluations will be undertaken for each
proposed Professional Services resource, on a
category-by-category basis.
Based on evidence contained within proposals, the
Evaluation Com m ittee determ ined the total
relevant experience per resource, per category.
Relevant Professional Services experience (a) was
considered to include only that which was directly
related to work experience Professional Services
activities (i.e. planning, directing, managing) in
the specific area of Semiotic Monitoring (defined
to also include Semiotic M onitoring equivalents
such as codification, transm ission and education).
Relevant Professional Services experience (b) was
considered to include only that which was directly
related to work experience Professional Services
activities (i.e. identifying prim ary docum ents,
compiling annotated bibliographies, etc.) in the
specific area of Sem iotic M onitoring (defined to
also include Sem iotic Monitoring equivalents such
as codification, transm ission and education).
(X months experience/12) m ultiplied by 2 = points
awarded up to a maxim um of 20.
Page 10
3.
Bidders should
indicate,
through a
written
description of no more than
1,000 words, their
understanding of ABC’s
requirement as stated in this
RFP, together with their
proposed approach and
methodology which would be
used to manage any
tasking/call-up assignments
awarded to them.
-SAMPLE-
Up to 2 points for
demonstrating evidence
of having understood
ABC’s requirement
/10
Up to 4 points for describing an approach and
methodology that is consistent with the principles
and practices of sound Semiotic Monitoring
Professional Services, including investigation and
analysis techniques, research planning, the
preparation of clear, concise and objective historical
reports, document/file summaries, report forms,
annotations and database entries; accurate source
attribution; effective document management; and
sound resource budgeting/scheduling
Up to 4 points for describing an approach and
methodology that is consistent with sound project
management, quality control and reporting practices
4.
Assessments from named
references.
/5
1/3 point per affirmative response, to a maximum of
five (5), based on five (5) standardized questions
asked to each of the three (3) supplied references.
5.
Proposal Quality
/5
5 points for presenting proposals in a clear and
logical fashion, and in a manner which facilitates a
clear and straightforward evaluation, based on the
information requested in the RFP, as evidenced by
the following:
2 points for ordering/structuring the proposal to
match the order and sequence of the mandatory and
point-rated factors in the RFP;
1 point for including tabs/separations between
sections of the proposal;
2 points for the overall clarity and ease-of-use as it
relates to the structure, presentation, layout and
design of the proposal.
Page 11
6.
-SAMPLE-
Bidders should
indicate the
applicable allinclusive per
diem rate ($CAN) for an 8.0
hour day, for each proposed
resource during the term of the
Standing Offer. Quoted per
diem rates which vary over the
period of the SOA must be
specified and will be evaluated
on a weighted average basis.
/40
The average of the
Professional Services per
diem rates quoted will be
used for evaluation
purposes on a category-by-category basis, with full
points going to the bidder with the lowest weighted
average rate per category, and lesser points being
awarded to all other bidders on a pro rated basis
based on the percentage differential of their
weighted average rate per category from that of the
bidder with the lowest weighted average rate per
category. Per diem rate evaluations will not be
undertaken for the Document Management
category.
Average per diem rates per category were
calculated by the Evaluation Com m ittee on a
category-by-category basis, using the quoted
average per diem rates for only those proposed
resources who were deemed to have m et
Mandatory Requirem ent #2 for their category.
Only the average rates of these qualified resources
were used to establish the weighted average per
diem rate per bidder.
TOTAL:
/100
Page 12
-SAMPLEPART B:
Profile of Proposals Received
The table below provides a listing of the proposals received by ABC in response to
the RFP:
Name of Bidder
Mandatories
Met?
Professional
Services
Document Support
Connie Contractor
Yes
T
T
Peter Procurement
Yes
T
T
Adele Acquisition
No
T
Non-Compliant Bids
The bid received from Adele Acquisition was deemed as non-compliant for failing
to meet the requirements of Mandatory Criteria #3. The Adele Acquisition bid did
not contain any references.
Page 13
-SAMPLEOutcomes of Evaluation Process
The Evaluation Committee completed a detailed evaluation of each proposal in
order to assess the evidence presented within the proposals against the mandatory
and point-rated criteria stipulated in the RFP. All proposals which failed to meet
one or more of the stated mandatory requirements were set aside by the Evaluation
Committee and given no further consideration (see above).
Page 14
-SAMPLESummary of Point Rated Evaluation
Bidder: Connie Contractor
Point-Rated Criteria
Professional
Services
Rationale
#1 Knowledge (Education &
Experience)
20/20
40 months relevant experience and 80 months
relevant education
#2 Specific W ork Experience
8.5/20
40 months relevant experience applicable to both
categories
#3 Understanding of
Requirement
9/10
Understanding too narrowly defined to one
specific type of monitoring activity
#4 References
5/5
n/a
#5 Proposal Quality
2.5/5
Tabs missing; ease-of-use and layout was lacking
#6 Cost
40/40
Professional Services: $400.00
TOTAL:
85/100
Bidder: Peter Procurement
Point-Rated Criteria
Professional
Services
Rationale
#1 Knowledge (Education &
Experience)
20/20
60 months relevant experience and 65 months
relevant education
#2 Specific W ork Experience
8/20
50 months relevant experience
#3 Understanding of
Requirement
8/10
Transmission activities understanding not fully
documented
#4 References
5/5
n/a
#5 Proposal Quality
4/5
Tabs missing
#6 Cost
25/40
TOTAL:
70/100
Professional Services: $800
Page 15
-SAMPLE- Appendix 1 Requests for Clarification
During the evaluation process, the Evaluation Committee sought clarification from
each of the following bidders with respect to information which could not be
located within the bidders’ proposals. Questions were presented in writing to
bidders by ABC, with bidders being given two (2) working days to respond in
writing.
1.
Peter Procurement.
Mandatory Criteria #3 within the RFP required bidders to include within their bid the
names of at least three (3) impartial references who have knowledge of the bidder’s
experience in the provision of professional services in the specific subject area of
Semiotic Monitoring. At a minimum, the name, address, and telephone number for each
named reference must have been included. The RFP also noted that it was not necessary
to provide references for each proposed resource.
Among the references provided by the bidder with respect to their proposed resources, the
Evaluation Committee is unable to identify which of these references are in compliance
with the requirements of Mandatory Criteria #3, and therefore which of these references
should be contacted by the ABC Evaluation Committee to obtain an assessment of the
bidder organization’s experience and capabilities.
From the references provided by the bidder with respect to their proposed resources, can
the bidder specifically identify the three (3) references to be contacted by the ABC
Evaluation Committee?
Page 16