Approaches to Criteria for Application and Evaluation of Exemplary courses (Maggie Lynch)
From the literature, and a review of several different rubrics and course evaluation instruments used in evaluating online courses, I've divided the elements into 6 primary categories and an "other important" category.
- I. Online Learning Community/Collaborative Presence
- II. Cognitive Presence (knowledge/content)
- III. Teaching Presence (teacher/assessment)
- IV. Social Presence (learner-centered)
- V. Media Design Presence
- VI. Innovative Presence
- VII. Other Important Measures
- Questions to Consider in Establishing Criteria
- Minutes from 2-1-2008 about this topic
I. Online Learning Community/Collaborative Presence
- Communication variety
- Opportunities for sharing and helping each other
- Options for going beyond the set assignment/incorporating real life experience
- Interaction (among learners, learner-instructor, learner-materials)
- Electronic collaboration mechanisms
II. Cognitive Presence (knowledge/content)
- Clear learning outcomes/objectives/competencies/benchmarks
- Module progression (logical flow, reasonable size)
- Prompt feedback
- Time on task
- Active learning techniques
- Communication of high expectations
- Respect for diverse talents and ways of learning
- Course goals/objectives/competencies/outcomes
- Instructional Activities integrated with content
III. Teaching Presence (teacher/assessment)
- Clear contact information
- Course specific resources
- Audience analysis
- Teaching strategies
- Feedback systems
- Pacing control
- Communication variations (synchronous/asynchronous)
- Connections between offline and online work are clear
- Progress tracking/student evaluation is clear
- Assignment feedback/grading enhances learning/improves performance
- Expectation communication
- Overall course management
IV. Social Presence (learner-centered)
- Opportunities to build social rapport
- Instructional activities engage students based on their background & experience
- Opportunities for students to showcase application of theory to their real-world
- Students impact course design/assignments
V. Media Design Presence
- Information presentation
- Interface design (navigation, consistency, easy to read, links work, fast loads)
- Multimedia elements
- Aesthetically pleasing
- Appropriate use of /integration with third-party software tools
VI. Innovative Presence
- A variety of technology tools used to facilitate communication and learning
- New teaching methods applied to enhance student learning
- New means for interactive engagement developed
- A variety of multimedia elements and/or learning objects relevant to learning are used
VII. Other Important Measures
- Accessibility
- Course Evaluation
Questions to Consider in Establishing Criteria
- The selection of criteria will define a pedagogical model, are we okay with that? (i.e., lean toward constructivism vs instructivism)
- Do we like the category designations?
- Are there other elements to include within the categories?
- Shall we use a rubric format both for self-evaluation in the application and for judging?
- If we use a rubric, how many options do we want?
- a. WebCT had 6 options: Exemplary, Accomplished, Promising, Incomplete, Not evident, and Not Appropriate
- b. Chico uses Baseline, Effective, and Exemplary
- c. Others use a typical Likert scale of qualities such as few, minimum, moderate, above average, and high level
- Is there a certain percentage of Exemplary ratings that is required to be a finalist?
- Does one category (i.e., innovation or collaboration) outweigh other categories?
- Do all categories have to be present for a course to be exemplary?
Minutes from 2-1-2008 about this topic
- Janet and Kate are concerned about the "course as a whole approach" versus "blended" or "activity" approach.
- Let's not require people to submit a whole course. Josh
- John: classes are hybrid. How does a specific practice enhances the whole course?
- These are the things we are looking for and you should fit in one category.
- Are we looking for innovation or examples?
- Demonstrate a shift in teaching and learning? (Josh)
- Janet: We can collect practices in other ways.
- Robin: we don't need examplars, nothing shiny. Good practices are the most significant.
- Maggie: must be integrated with the whole course.
- Josh: Small usages belong to the repository.
- Have multiple Honor roll mentions (runner-up).
- Eddy: Get Apple involved to find more goodies.
- Include travel for runner-ups.
- Some work tweaking could do the trick.
- Use simple language: http://www.campustechnology.com/innovators/
- Add: Syllabus, Instructional Goals, Assessment of outcomes.
- Assessment is very large in itself... Find a way to present it in an easy way.
- Breaking Teaching Presence in two. Much tweak that term to make it more broad.
- Maggie would like to write down the rubrics and send them to the group (Sue, Kate, Matt).
- Be aware of international terms.
- Show our work to faculty members.
- Have people be able to submit in their own language. Suggested in English (form only).
- Susan Roig and Melissa Zhuo, Learning Styles and the Online Environment - Is This Possible in Sakai? (From the Amsterdam Sakai Conference)