The following scorecard is for sousa_2-5-x-1 released on July 16, 2008. All evaluations were done by Mark Norton, the project leader of the Sousa Project.
The following evaluation methods are proposed for the scorecard items selected in the Scorecard BOF at the Ninth Sakai Conference in Paris, France. Please feel free to edit in place, add comments in other colors, or append comments below.
Note that point scales for individual items vary considerably. This is not a problem if we develop a weighting schemes when scores are rolled up into the four evaluation dimensions.
See also Scorecard Item Definitions
| No. | Scorecard Item | Max Points | Score | Comments |
|---|---|---|---|---|
| 1.0 | User Experience | 60 | 16 | 26% |
| 1.1 |  Consistency / Best practices |
10 | 5 | Some conventions are used. Sakai CSS. |
| 1.2 | Usability |
10 | 5 | Improvements are needed. |
| 1.3 | Accessibility |
10 | 2 | No formal assessment. Basic support included. |
| 1.4 | Internationalization |
20 | 2 | Only one langauge (English) |
| 1.5 | Ease of use |
0 | ||
| 1.6 | User testing |
10 | 2 | Very limited user testing, informal feedback. |
| 2.0 | Technical | 148 | 23 | 15% |
| 2.1 | Browser support |
6 | 3 | FF2, FF3, IE6 |
| 2.2 | Code review |
10 | 0 | No formal code review yet. |
| 2.3 | Unit testing |
10 | 0 | No test, no results. |
| 2.4 | Functional regression testing |
10 | 0 | No test, no results. |
| 2.5 | Integration testing |
10 | 0 | No tests, no results. |
| 2.6 | Performance testing |
10 | 0 | No test, no results. |
| 2.7 | Internationalization |
10 | 5 | Strings externalized. UTF-8 support. |
| 2.8 | Licensing |
10 | 5 | Currently on a personal license, not Sakai. |
| 2.9 | Outstanding JIRA bugs |
10 | 0 | Not part of Jira yet. |
| 2.10 | Packaging (code structure) |
10 | 8 | Conforms to naming conventions. Maven 1/2 support. |
| 2.11 | Static code review |
10 | 0 | No tests, no results. |
| 2.12 | Validation/spec conformance |
10 | 0 | No test, no results. |
| 2.13 | DB support |
10 | n/a | Based on CHS, no databases. |
| 2.14 | DB best practices |
10 | n/a | Bsed on CHS, no databases. |
| 2.15 | Security review |
10 | 2 | Check for basic permissions. |
| 2.16 | Technical |
0 | ||
| 2.17 | Event tracking |
2 | 0 | No support for event tracking. |
| 3.0 | Descriptive | 48 | 29 | 60% |
| 3.1 | Bundled Help |
10 | 0 | No bundled help. |
| 3.2 | Test plan |
10 | 6 | Test plan exists, needs work. |
| 3.3 | Javadocs |
10 | 9 | Very good coverage. |
| 3.4 | Design Documentation |
3 | 9 | Well documented. |
| 3.5 | Wiki/website |
0 | ||
| 3.6 | Deployment doc |
3 | 2 | Exists, needs work. |
| 3.7 | End user external docs. |
3 | 3 | Good documentation. |
| 3.8 | Issue Tracking (Jira) |
3 | 0 | Not included in Jira at this time. |
| 3.9 | Events documented |
3 | 0 | No support for events. |
| 3.10 | Licensing Documented |
0 | ||
| 3.11 | Configuration |
3 | 1 | No configuration properties at this time. |
| 4.0 | Community Support | 47 | 16 | 34% |
| 4.1 | Team size |
20 | 1 | |
| 4.2 | Team diversity |
4 | 1 | |
| 4.3 | Responsiveness |
10 | 5 | I do what I can. |
| 4.4 | Production experience |
3 | 1 | Still new after one year of development. |
| 4.5 | Communications and openness. |
10 | 8 | YMMV. |