Multiple Evaluator Workflow - Functional design

Added by Mark Breuker, last edited by Mark Breuker on Apr 22, 2008  (view change)
This is the first iteration of a functional design for multiple evaluation workflow

See also evaluation workflow related requirements

Scenarios

Looking at the user stories we identified three basic scenario's for evaluation:

  1. Single evaluation - Student receives one evaluation per submission by one evaluator (current situation).
  2. Multiple independent evaluations - Student receives N evaluations per submission, created by N different evaluators that work independently.
  3. Team evaluation - Student receives one evaluation per submission created by N different evaluators that work together. Usually one of the evaluators on the team is the lead evaluator that communicates the evaluation to the student. The lead evaluator uses the evaluations of the other evaluators as an advice.
Scenario 1 workflow Scenario 2 workflow Scenario 3 workflow

Terminology

We use the term evaluatable item as an abstract term for things that can have an evaluation workflow. An evaluatable item can be a matrix cell or portfolio presentation. Since the assignments team is also working on evaluation workflow an evaluatable item could perhaps also be an assignment.

UI Mockups

These mockups give an impression of the screens a site administrator uses for defining the workflow.

Screen 1 Screen 2a Screen 2b
 
Screen 3a Screen 3b  

These mockups give an impression of the screens an evaluator uses creating an evaluation.

Screen 5 Screen 6a Screen 6b

Use cases



Use case name Evaluate evaluatable item
Version 1.0
Goal The evaluator wants to provide formative or summative feedback to the student and assess the formal progression of the student through the curriculum.
Actors Evaluator
Preconditions
  • The evaluator is logged in to the system.
  • A student has submitted an item for evaluation.
  • An evaluation workflow has been created for the item under evaluation.
  • The evaluatable item has been routed to the evaluator or group of evaluators the evaluator is a member of.
Triggers The evaluator has received a notification that an item needs evaluation.
Basic course of events
  1. The evaluator decides to evaluate an evaluable item. The system shows the evaluable items that are routed to the evaluator or to the group the evaluator is member of.
  2. The evaluator selects an evaluatable item from the overview. The system shows the contents of the evaluatable item with the submitted result of the student.
  3. The evaluator reads the contents of the evaluatable item and other available evaluations he has access to (if any).
  4. The evaluator fills out the evaluation form, provides a rating and then saves the form when complete. The system stores the evaluation with the evaluatable item.
  5. The evaluator completes the evaluation workflow step. The system initiates the next step in the workflow.
    1. If the evaluator is the final evaluator then the system marks the evaluatable item as EVALUATED and sends an optional notification to the student 
    2. If the evaluator is not the final evaluator then the system routes the evaluatable item to the next evaluator or group of evaluators. The system sends an optional notification to the next evaluator or to the evaluators in the group that have not yet contributed an evalutation to this evaluatable item. The status of the evaluatable item remains UNDER EVALUATION.
  6. The system removes the evaluatable item from the overview of items he can/should evaluate.
Alternative paths Step 4. If the evaluator decides to complete the evaluation on a later time he saves the evaluation without completing the workflow step. The system shows the evaluatable item.
Goto step 1.
Postconditions
  • The evaluatable item has been evaluated
  • The evaluatable item can no longer be evaluated by the evaluator
  • The student has received an evaluation OR the evaluatable item is routed to the next evaluator if the evalation criteria are not yet met.
Business rules  
Notes See also screens 5, 6a & 6b
Author and date Mark Breuker, Hugo Jacobs
April 4th, 2008


Use case name
Design evaluation workflow
Version
1.0
Goal
The administrator wants to allow an matrix cell or portfolio to be evaluated by an evaluator.
Actors
Administrator
Preconditions
  • The administrator is logged in to the system
  • An evaluatable item has been created
  • No students have previously submitted the evaluatable item
Triggers
A portfolio or matrix cell has been created that needs to be evaluated.
Basic course of events
1.      The administrator selects the evaluatable item for which s/he wants to define evaluation (workflow) settings. The system renders a link to the evaluation settings for the evaluatable item.
2.      The administrator follows the link to the evaluation settings.
3.      The administrator selects a workflow type: paralllel or sequential.
4.      For each step in the evaluation the administrator defines an evaluation step.
5.      If the administrator selected a sequential workflow s/he arranges the steps in the appropriate sequence.
6.      The administrator sets the global evaluation options.
7.      The administrator saves the settings. The system shows the evaluatable item.
Alternative paths
Step 4: The administrator removes a previously defined evaluation step. Goto step 5
Step 4. The administrator edits a previously defined evaluation step.
Goto step 5
Postconditions
An evaluation workflow has been defined for the evaluatable item.
Business rules
 
Notes
See also screen mockups 1, 2 & 3
Author and date
Mark Breuker
April 7th 2008


Use case name
Obtain evaluation for evaluatable item
Version
1.0
Goal
The student wants to obtain a formative or summative rating of an evaluatable item.
Actors
Student
Preconditions
  • The student is logged in to the system.
  • The student is permitted to submit the evaluatable item
  • The evaluatable item has not yet been submitted (status = READY)
Triggers
The student has completed work on the evaluatable item.
Basic course of events
1.      The student submits the evaluatable item;
2.      The system determines the first evaluator(s) in the evaluation workflow and changes the status of the evaluatable item to PENDING;
3.      The system sends a notification to the evaluator(s) if the evaluator indicated he wishes to be notified of new items in his list of items for evaluation;
4.      The students waits until the evaluatable item changes its status to EVALUATED. Optionally the system sends a notification to the student when the status changes;
5.      The student opens the evaluatable item. The system shows a list of all available evaluation(s);
6.      The student opens each evaluation to read the contents. Once the student has read the evaluation the system marks the evaluation as read.
Alternative paths
 
Postconditions
  • The student has obtained one or more evaluations for his submitted evaluatable item.
  • The status of the evaluatable item is EVALUATED
Business rules
 
Notes
If the evaluation in unsatisfactory to the student and the student is allowed to resubmit the item then the basic course of events is repeated.
In step 4 the status changes to EVALUATED when the evaluation workflow is completed.
Author and date
Mark Breuker
April 7th 2008
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.