This deliverable defines the competitive evaluation strategy which will be used for the assessment of the results of the suppliers at the end of the design phase 1. The competitive evaluation strategy serves to choose those suppliers which will pass the design phase 1 and will continue with the prototyping and testing phases.

Evaluating and comparing suppliers requires us to identify two distinct processes:

• evaluation process: during this process each supplier is individually examined and it is scored according to its characteristics. The evaluation process is formalized through the evaluation matrix, as described in Section 2.1. The outcome of the evaluation process is the supplier score, that is a number representing the scoring achieved by the supplier;
• comparison process: once the suppliers have been scored, they are compared with each
  other on the basis of their supplier scores. The comparison process is formalized through the comparison matrix, as described in Section 2.2. The outcome of the comparison process is a ranking of the suppliers, based on their scorings.

In particular, as detailed in Section 3, the PREFORMA evaluation matrix consists of four categories: Impact on the Challenge, Technical Approach, Quality of the Tender, and Price/Cost. Each category contains several items which are scored using a likert scale ranging from 1 (bad) to 5 (excellent), according to well-established standards, like ITU-T P800. The score for each item is obtained with a weighted average of the scores assigned by different reviewers which belong to the three following reviewer types: technical expert, domain expert, and external expert. The score of a category is the weighted average of the scores of its items and the total score for a supplier is the weighted average of its category scores.