Best Practices for Administrative Evaluation of Online Faculty

10 Apr 17

jsackhoff

OnlineEvaluation

visiting an online course in order to observe
instructor-student interaction can be set up, not according to time (as in the traditional classroom observation) but according to course unit

17 Mar 17

Enoch Hale

evaluation faculty teaching_evaluation online

28 Feb 17

dwstuart

ETC645 Lesson 7 Assessment

07 Oct 16

acadtechnology

teaching online

19 Feb 16

lmsscrementi

24 Jun 15

Pip Ferguson

Evaluating online teaching, 2004, Tobin, Southern Illinois Uni

onlineteaching evaluation

15 Sep 14

Geoff Cain

evaluation elearning standards

19 Mar 14

April Adams

08 Feb 14

Lisa Kidder

dissertation article evaluating

02 Nov 13

Mary Maine

15 Jul 13

David Migliorese

administration teaching evaluation elearning standards faculty

12 Apr 13

Janetta Garton

elearning

03 Mar 13

Arlynne Awayan

evaluation faculty elearning

11 Dec 12

Anne Klingen

evaluation elearning administration online

03 Dec 12

Elizabeth Graves

evaluation administration elearning faculty standards

There has not, however, been a concomitant increase in research about how administrators can best evaluate the performance of the instructors whose courses they are tasked to assess. Fortunately, several of the instruments that have been created for purposes other than evaluating faculty performance contain criteria and questions germane to the traditional administrative tasks of rating instructors and suggesting improvements.
"if a course offered online is equivalent to a traditional classroom based course?"
Sal Monella, in his hypothetical food safety course, displays all of the outward signs of good classroom teaching: he has a syllabus that is tied to the academic calendar, he uses the "glitter factor" of multimedia to hold his students' interest, he is quick to respond to students questions, and he provides a wealth of resources to help students to achieve. Mel Ted Butter, on the other hand, seems not to evince these traits, going so far as to let students questions sit for days at a time. It appears as though the students in Butter's class are having to teach themselves.
Dean Bineerawile had something a quandary on her hands. Her evaluative instrument told her that Monella was the better instructor, but the students seemed to favor Butter. What has Monella done that allowed his performance to rank so high on the evaluative instrument? It seemed that quantity was the driving factor. Monella

· had a lot of discussion,
· posted a lot of documents,
· used a lot of flashy multimedia, and
· generated a lot of web links to outside resources.

However, his students seemed to want not just the information, but wanted to know how to assess, use, and create more of it themselves-something that Butter seemed to able to provide. The dean makes a few assumptions about online learning that echo those warned against in Susan Colaric and David Jonassen's The Web in Higher Education, such as "when a teacher asks students to use the Web to find an answer, the teacher most likely (a) assumes that the student is using problem-solving skills to define the problem, and (b) expects that the student will extrapolate the information from the various sites that are found" (163). Likewise, Colaric and Jonassen relate that they "have witnessed numerous presentations of instructional Web sites that proudly point to a number of other informative Web sites. What is disturbing is that these [course] authors too often convey credibility in the links that they list without critically evaluating the information contained in those Web sites" (165).
Judith Eaton, in a 2002 report about the future of the accreditation of distance learning, established several principles which good DL programs share. Although these principles are aimed at entire programs, it is instructive to see how well many of them help to answer some of the questions with which we began:

· Learning is dynamic and interactive, regardless of the setting in which it occurs; . . .
· Institutions undertake the assessment and improvement of their quality, giving particular emphasis to student learning. (Eaton 26)
Principle 1: Good Practice Encourages Student-Faculty Contact

Lesson for online instruction: Instructors should provide clear guidelines for interaction with student
Principle 2: Good Practice Encourages Cooperation Among Students

Lesson for online instruction: Well-designed discussion assignments facilitate meaningful cooperation among students.

In our research, we found that instructors often required only "participation" in the weekly class discussion forum. As a result, discussion often had no clear focus. We have developed guidelines for creating effective asynchronous discussions, based on substantial experience with faculty members teaching online. In the study, we applied these guidelines as recommendations to encourage meaningful participation in asynchronous online discussions:

· Learners should be required to participate (and their grade should depend on participation).
· Discussion groups should remain small.
· Discussions should be focused on a task.
· Tasks should always result in a product.
· Learners should receive feedback on their discussions.
· Evaluation should be based on the quality of postings (and not the length or number).
· Instructors should post expectations for discussions.
Principle 3: Good Practice Encourages Active Learning

Lesson for online instruction: Students should present course projects.
Principle 4: Good Practice Gives Prompt Feedback

Lesson for online instruction: Instructors need to provide two types of feedback: information feedback and acknowledgment feedback
Principle 5: Good Practice Emphasizes Time on Task

Lesson for online instruction: Online courses need deadlines.
Principle 6: Good Practice Communicates High Expectations

Lesson for online instruction: Challenging tasks, sample cases, and praise for quality work communicate high expectations.
Principle 7: Good Practice Respects Diverse Talents and Ways of Learning

Lesson for online instruction: Allowing students to choose project topics incorporates diverse views into online courses.
At California State University at Chico, Jon Knolle has created a self-assessment instrument for online instructors that shows promise. Knolle's instrument defines online instruction principles such as "emphasis on time-on-task" and "asks open-ended questions." For the time-on-task principle, Knolle inquires "How effectively were you able to use the technology available to emphasize time on task in your class?" (3) and then asks the instructor to identify the specific tools provided by the course management system (in this case, WebCT) the instructor used to put the principle in action. By listing the specific tools available to instructors, Knolle essentially asks them to provide a principles-based road map to their courses, one which administrators can then use to "observe" a class in action. Knolle provides a sample assessment already filled in with uses for specific WebCT tools:
· Goals: Post course goals and learning objectives, then ask students which two are meaningful to them and why.
· Calendar: Set time-achievement expectations that are laid out at the beginning of the course.
· Tracking: Monitor student activity on content pages and provide feedback to students based on their activities; permit students to track their own progress through the content.
· Discussions: Set limits for the number and type of postings by each student; limit the length of each message.
· E-mail: Keep messages succinct; be sure that the subject lines are specific and meaningful; avoid over-reliance on attachments; set guidelines for file format of e-mailed attachments, require virus-checking of all attached files; require progress reports from students periodically.
· Bookmarks: Maintain accurate and up-to-date links to external sites. (3)
Perhaps more useful for administrators is the "Checklist for Online Interactive Learning (COIL)," authored by Dennis W. Sunal, Cynthia S. Sunal, Michael R. Odell, and Cheryl A. Sundberg (Appendix A). The COIL is centered on measurable outcomes of online instruction, with focus on four major areas: student behavior, faculty-student interaction, technology support, and the completeness of the learning environment (40). The best feature of the COIL is its high degree of objectivity: most of the statements in the instrument are verifiable in terms of course output or content, and thus predict a high correlation with actual instructor quality.
M. D. Roblyer and Leticia Ekhaml have compiled an excellent rubric for determining interactivity

14 more annotations...

14 Feb 12

wayne anderson

eLearning education evaluation peerReview quality eLearningDevelopment

23 Aug 11

Laura Knudson

evaluating online teachers!

evaluation elearning administration faculty onlineteaching standards assessment

23 Jul 11

Alec Barron

Interesting take on evaluation with a framework.

DSEP7230_Barron_wk7

22 Dec 10

jennmarco

elearning course_evaluation

19 Aug 10

Luis Montesinos

Associate Dean Felicidad Muerte Technology

10 Aug 10

eLearning Consultant

elearning course_evaluation

16 Jul 10

Sonja Janousek

evaluation elearning administration online_teaching faculty practices for best

30 May 10

Joh Fra03

6705 evaluation faculty administration

17 Apr 10

Stella Porto

evaluation elearning practices administration online teaching faculty assessment

04 Mar 10

ecollinsbrown

Course Evaluation online

16 Feb 10

Leinda Peterman

online teachers evaluation elearning

16 Jan 10

Maria Filomena Teodoro

18 Nov 09

AJ Williams

paper presented at DLA 2004 proceedings, Jekyll Island, Georgia, May 23-26, 2004 by Thomas Tobin from Souther Illinois University

faculty standards onlineteaching evaluation dlab psdlab

25 Mar 09

Lisa M Lane

article, includes application to Seven Principles, and checklist at end

evaluation

29 Jan 09

Greg Walker

best practices online_teaching

12 Jan 09

JP Bosman

Online training Evaluation

Best Practices for Administrative Evaluation of Online Faculty
how to evaluate the materials and teaching in online courses
However, his students seemed to want not just the information, but wanted to know how to assess, use, and create more of it themselves-something that Butter seemed to able to provide. The dean makes a few assumptions about online learning that echo those warned against in Susan Colaric and David Jonassen's The Web in Higher Education, such as "when a teacher asks students to use the Web to find an answer, the teacher most likely (a) assumes that the student is using problem-solving skills to define the problem, and (b) expects that the student will extrapolate the information from the various sites that are found" (163). Likewise, Colaric and Jonassen relate that they "have witnessed numerous presentations of instructional Web sites that proudly point to a number of other informative Web sites. What is disturbing is that these [course] authors too often convey credibility in the links that they list without critically evaluating the information contained in those Web sites" (165).
"overall, attitudes of faculty at all schools (as perceived by academic leaders at those institutions) remain more conservative with regard to the quality of online education and its ability to equal face-to-face learning" (23). Their report also indicates that administrators at all levels of higher education predict that, going forward, online learning will become an increasingly important part of the offerings of their institutions, and that they will be responsible for guaranteeing the quality of their online offerings to regional and national accreditation agencies
Seven Principles for Good Practice in Undergraduate Education
Figure 1. "Seven Principles of Effective Teaching: A Practical Lens for Evaluating Online Courses"
Principle 1: Good Practice Encourages Student-Faculty Contact
Principle 2: Good Practice Encourages Cooperation Among Students
required to participate
small.
focused on a task
result in a product
receive feedback
quality of postings
post expectations
Principle 3: Good Practice Encourages Active Learning
Principle 4: Good Practice Gives Prompt Feedback
Principle 5: Good Practice Emphasizes Time on Task
Principle 6: Good Practice Communicates High Expectations
Principle 7: Good Practice Respects Diverse Talents and Ways of Learning
At California State University at Chico, Jon Knolle has created a self-assessment instrument for online instructors that shows promise. Knolle's instrument defines online instruction principles such as "emphasis on time-on-task" and "asks open-ended questions." For the time-on-task principle
Goals: Post course goals and learning objectives, then ask students which two are meaningful to them and why.
· Calendar: Set time-achievement expectations that are laid out at the beginning of the course.
· Tracking: Monitor student activity on content pages and provide feedback to students based on their activities; permit students to track their own progress through the content.
· Discussions: Set limits for the number and type of postings by each student; limit the length of each message.
· E-mail: Keep messages succinct; be sure that the subject lines are specific and meaningful; avoid over-reliance on attachments; set guidelines for file format of e-mailed attachments, require virus-checking of all attached files; require progress reports from students periodically.
· Bookmarks: Maintain accurate and up-to-date links to external sites. (3)
level of interaction among the students, the instructor, and the outside worl
Minimum interactive qualities
Moderate interactive qualities:
Above average interactive qualities:
High level of interactive qualities:
An overwhelming majority of students perceived the evaluation used for traditional courses to be appropriate for distance education courses (over 90%). Of those few students (less than 8%) who did not feel that the Web form was appropriate for distance education courses, none provided suggestions when given the opportunity. Using the online course evaluation, students gave both the courses and instructors very high ratings. Ratings from the first testing of the online version evaluation exceeded a rating of 9.0 on a 10-point scale. This is supportive of the belief that the evaluation form used for traditionally taught courses is appropriate for distance education courses, as judged by the students completing the form. The Web-based course evaluation form has now been accepted by the university as appropriate for distance education courses. (Holcomb, King, and Brown 10)
Critical commenting: create narrative summaries of the strengths and weaknesses of the instructor from administrators and peers.
· Developmental testing: during the preparation of online materials, encourage "dry runs" to test the effectiveness of techniques.
· Revision of materials in light of formative evaluation: build in a feedback mechanism for faculty to make changes based on the sum of student, peer, and administrative evaluation.
· Feedback from students: require that student evaluation of instructor performance carry significant weight in the overall assessment.
· Definition of the extent of utilization: ask instructors to list the resources absolutely necessary for the completion of essential course tasks, and review their validity and currency on a regular basis. (291-98)
Appendix A: Checklist for Online Interactive Learning (COIL)

28 more annotations...

11 Jan 09

Tony Searl

This introductory-level
presentation demonstrates how to evaluate the materials and teaching in online
courses. Topics covered include similarities with evaluation of on-ground
teaching, factors unique to online courses, technological considerations,
helping administrators unfamiliar with online courses, and national standards,
rubrics, and benchmarks.

evaluation faculty online_teaching

24 Nov 08