Guidelines for Observation of Online Teaching

Gallaudet University values excellence in teaching.  Teaching quality and effectiveness include a collaborative effort by the individual faculty, the program, the department and the University.  Accordingly, the evaluative process should be a shared responsibility and should be reflected within the evaluation process.  

The University Faculty Handbook discusses faculty engagement with evaluation and observation in different areas of their professional career.  Evaluation should stimulate professional growth, academic freedom and yet provide the basis for personnel action. While there is no single best way to deliver instruction, the evaluation and observation of online teaching should reflect best practices. An evidence-based evaluation criterion assists in providing guidelines for best practices and what the faculty member needs to do in order to be considered for reappointment, tenure, promotion and/or merit increase.  

Since the University Faculty Handbook specifies that departments develop their own evaluation guidelines for course evaluations, classroom observations, and reflective self-evaluation (p. 17), this guideline includes recommended areas to evaluate when observing online teaching.  Departments and programs are welcome to adapt the following guidelines to fit their specific field of study, and utilize certain areas for hybrid and online courses.  For online courses, we also include guidelines for evaluating synchronized and asynchronized formats.

The purpose of this guideline is to provide a systematic basis for online observation and evaluation that is as equitable as possible for all disciplines and styles of teaching, and a way to document our observations.  The guideline comes in a combination of a checklist, a rating sheet and written analysis.  This form can be used to mentor new adjuncts and instructors, assist colleagues in continued improvement and promote teaching excellence.

The resources that supported the development of the following guidelines include but are not limited to Quality Matters rubric and courses, Online Learning Consortium workshops, different University peer evaluation guidelines, and Evaluating Online Teaching:  Implementing Best Practices by Tobin, Mandernach and Taylor (2015). 

Best Practices for Online Teaching Observation

Bilingualism

Gallaudet University is committed to providing bilingual education in ASL and English in all courses, both onsite and online, integrating the four principles of bilingual education at Gallaudet:  

1) direct, visually accessible communication,

2) inclusion and guidance of academically qualified students with diverse language and educational background and abilities,

3) intentional and integrated use of academic discourse in both ASL and English, and

4) enhancing students’ social and cultural experiences.

Our university mission, vision and bilingual education principles were taken into account when developing the following evaluation guidelines.   

Framework

In 1987, Seven Principles for Good Practice in Undergraduate Education was published by Chickering and Gamson.  They summarized 50 years of higher education research that addressed good teaching and learning practices.  Their findings have been widely used to guide and improve college teaching.  Their principles provide a useful framework to evaluate the effectiveness of teaching and learning in onsite and online settings.

The Seven Principles

Good practice:

1. Encourages contact between students and faculty;
2. Develops reciprocity and cooperation among students;
3. Encourages active learning;
4. Gives prompt feedback;
5. Emphasizes time on task;
6. Communicates high expectations; and
7. Respects diverse talents and ways of learning.

Those principles are integrated into the guidelines below.

Objective and Subjective Approaches

In developing the following guidelines based on best practices in teaching observation, there are two approaches one can take towards observation, that are not necessarily mutually exclusive:  an objective and a subjective approach.  

1)  Objective observation provide a more measurable format, often using a rating scale involving numbers.  These types of observations tend to be a part of overall faculty evaluation process.  

Some objective observations include statistics downloaded from the course learning management system (LMS, currently Blackboard) and other course elements that may be outside of the LMS (e.g. amount of time spent in Blackboard, length of time in responding to student discussion posts or grading assignments).  This is not recommended, for multiple reasons beyond the scope of this guideline.  See Tobin, Mandernach and Taylor (2015) for an in-depth explanation as to why.

2)  Subjective observation can be useful in exhibiting quality of online teaching that may not be able to be meaningfully measured by an objective observation.  This often includes a narrative-based question, response and/or summary.    

Integrating a hybrid of both styles are highly recommended.    

Summative and Formative Approaches

Formative approaches towards online observations can be especially helpful to faculty. Thus, a faculty member may request multiple observations throughout the semester. Observers can login multiple times throughout the semester, instead of logging in once.  A formative approach also gives the faculty member opportunities to improve their online course and their teaching before their next peer evaluation. The observations are coordinated and agreed on by the instructor and peer observers.

Not only that, faculty observers are recommended to meet with the faculty being evaluated, in order to better understand the faculty’s decisions in implementing a certain approach, assignment or group activity prior to completing the peer evaluation, which adds a sense of a formative nature towards evaluation.  Evaluators are responsible for providing specific feedback and suggestions.  The focus of the discussion should be the theories, best practices and ideas about what constitutes excellence in teaching.  

Eskey and Roehrich (2013) research reports that distance learning peer observations should focus on quality education that meets student educational and access needs, serve as a mentoring process for adjunct and face-to-face instructors, and in conjunction with current university policies.  Mentoring is an important to improve instruction, technology and collaboration. It is suggested that an online program should be “built based on the face-to-face evaluation model, [university] online policies, online best practices, a team of evaluators, a team of faculty mentors, instructional designers, course developers, a distance learning advisory committee (DLAC), departmental buy-in, and support from the University administration” (Eskey & Roehrich (2013, pp. ).

Online peer observations researchers, Eskey and Roehrich, suggest four categories of course facilitation and evaluation:

1. Building Community in the Classroom
2. Discussion Facilitation and Instruction
3. Assessment and Grading
4. Online Course Learning Environment

These considerations would be crucial in considering cultural and language diversity that would be encountered at Gallaudet University’s Online Program.

Delivery Systems for Courses

Gallaudet courses may be taken in a number of different formats. While most of our classes are offered as face-to-face classes on campus, an increasing number of courses are involving some degree of technological transmission. Many of our teachers combine face-to-face teaching with online materials. In addition, we offer a variety of online courses, both for college credit and for professional studies (PST) credit. Through our extension programs, we can also bring credit courses or contract classes to requested sites given an adequate group of participants. Gallaudet University also has up-to-date facilities to provide video conferencing on the main campus and at the Gallaudet University Regional Centers in various parts of the United States.

The following categories and descriptions of course delivery formats available at Gallaudet University are:

Face-to-Face  Courses:

Classroom Course: Course activity is organized around scheduled face-to-face (in-person) class meetings. Classroom courses require students to be present on campus.

Distance Education/Distance Learning Courses include:

Online Courses: Online courses can be asynchronous or synchronous.

Online Asynchronous: Students are NOT required to login at a specific time. Online Asynchronous courses achieve the required credit hours primarily through time spent online at the time of the student's choosing.

Online Synchronous: Students are required to login at a specific time. A synchronous course will indicate this information either within the course description or list a specific online meeting time.  Time online could vary between 1% to 100% of course time and activities.

Hybrid Courses: Courses that include a face-to-face component and an online component. A course is designated hybrid if time online is substituted for 51% or more of required in-class time. Hybrid courses can be asynchronous or synchronous. Students are required to attend class meetings or exams on campus, as well as, to participate in an online element.

Hybrid Asynchronous:  The online component of hybrid asynchronous courses achieve the required credit hours primarily through time spent online at the time of the student's choosing. Students are NOT required to login at a specific time.

Hybrid Synchronous: The online component of a hybrid synchronous course requires students to login at a specific time. A synchronous course will indicate this information either within the course description or list a specific online meeting time.  Time online could vary between 1% to 100% of course time and activities.

Observer Criteria

According to the University Faculty Handbook, observation should be completed by a regular faculty (tenured, tenure-track, non-tenure-track, and pre-tenure-track) who are members of a Department/Program of Instruction.    

Online:  Faculty members completing the online teaching observation form need to complete Gallaudet’s required online teaching training and have previous experience teaching online courses.

Observer Procedure

This is the recommended observation procedure for teaching online observation.  The process can be cyclical, for instance, an observer can go through the below process several times before submitting a final observation.  The instructor and peer evaluator should agree upon the observation procedure.

Pre-observation

It is essential that the observer and the faculty being observed are absolutely clear on what areas are being observed and the criteria/point system for each prior to the observation.  The faculty being evaluated will need to supply the observer with access to the course syllabus and materials covered during the observation period.  Both the observer and the faculty being observed will need to discuss the goals of the observation, which categories will be covered in the observation (see below) and some background about the course (e.g. content already learned, student background and dynamics, and content that will be covered during the observation).     

Observation

Online Asynchronous (OA):  Exploring the online course may be better near the end of the course, as the evaluator will be able to observe documented student participation and teacher responses across a period of time.  

Online Synchronous (OS):  Consider having the live video discussion recorded and observed at a later time in order to allow the best bandwidth possible for the faculty and students.  Be sure to safeguard the recording in a secure place (e.g. Google Drive where a login is required), allow only certain employees to see the video, for instance, the faculty evaluator and the faculty member being evaluated.  Delete immediately after use in order to adhere to FERPA regulations.    

Post-observation

A post-observation meeting should be arranged between the faculty evaluator and faculty being evaluated shortly after the observation.  Discussion regarding the faculty evaluator’s observation and notes is essential to ensure that the faculty being evaluated can clarify questions or concerns prior to the submission of the faculty evaluation.  

 

Scope of Observation

Teaching involves numerous different areas that can be observed.  The following categories and questions are meant to serve as a guide for possible areas a peer faculty may be able to observe.  Those categories are not intended to be used as a definitive list of required behaviors or products for teaching online observation.  

1. Learning Management System (LMS) Course Structure  

1. Are the required materials for the course clearly listed?
2. Is the course cleanly designed with a minimal number of icons and navigational choices for users?
3. Are course materials viewable within three clicks when starting from the course homepage?
4. Is faculty availability (e.g. office hours) listed on the course LMS?
5. Are alternate contact methods, such as a college e-mail address or office telephone number, provided in case students need to contact the instructor outside of your course LMS? (e.g. if the LMS is down).
6. Is the course layout, navigation and design consistent throughout? (e.g., ordering content in similar learning modules, maintaining type size, font, heading styles, and other page elements).
7. Are the course syllabus and getting starting documents in the course LMS easy to identify?
8. Does the course contain information about being an academic learner and include links to campus resources for students?
9. Is content chunked into manageable portions? Are large blocks of text divided into smaller sections with hyperlinks and a minimum of scrolling? Are lengthy videos split up in sequential videos or are hyperlinks provided for certain points throughout the video to assist with searching?  
10. Does the faculty member integrate current subject matter into the course LMS?
11. Does the course include web resources such as links to relevant websites, and online videos?
12. Does the course include multi-modal and multi-media resources?
13. Is redundancy used to minimize searching for commonly sought information?
14. Do all of the course links and documents work?  (e.g. broken links, missing photos, removed videos)
15. Are the amount of English text and ASL videos balanced?  
16. Are the technology tools utilized in the course appropriate and effective in facilitating learning?
17. Are copyright and fair use laws adhered to?  
18. Does the faculty member give attention to accessibility issues?  Are the videos captioned and/or transcripts provided?  Are there descriptions of images and videos?

1. Faculty and Student Interaction
   1. Does the course promote student interaction via discussions and collaborative assignments? Do students actively participate?
   2. Does the instructor encourage students to introduce themselves to one another informally and to post short bios and photos? Do students engage with one another?
   3. Does the course provide an informal discussion area for students? Do they form a community of learners?
   4. Are the assignments in the course learner-centered? Do students engage with the material?
   5. Does the instructor encourage student engagement via open-ended questions and modeling of expected behavior?
   6. Does the instructor provide guidance on how to receive help and ask questions? Are there opportunities during class for students to ask questions? Is a questions forum provided on the course LMS?
   7. Does the instructor encourage all students to participate and ask open-ended questions to prompt engagement and draw out shyer students?
   8. Does the instructor seek feedback from students and provide additional resources or explanations when necessary?
   9. Do the students and instructor show respect for one another?  Does the faculty member discuss and model collegial discussions?  Does the instructor provide “netiquette” guidelines for online interactions?
   10. Does the faculty member participate effectively in threaded discussions and respond to appropriately to student questions within the discussion area?
   11. Is the instructor regularly “present” in the course via announcements, discussion posts and individual feedback?
   12. Is the faculty member’s signing and writing style and tone clear and welcoming?
   13. Does the faculty member personalize his/her exchanges with students by using names and salutations?
   14. Does the faculty member seem to express enthusiasm in engaging with students and the content?
   15. Is the classroom dynamics respectful?  Do the students interact well with each other?  Is the faculty member open and responsive to questions?  

1. Subject Area Mastery (Department/Program Level)
   1. Does the faculty member show evidence of content area expertise?
   2. Are they familiar with current trends, research and publications in their field?
   3. Are the required readings/viewings appropriate for the course?  Is the degree of difficulty suitable to the course?  Are the required topics, readings and viewings redundant or repeated in multiple courses with similar assignments/goals?
   4. Are they able to answer questions, guide and support students in mastering the content?
   5. Are the required materials relevant to the purpose of the course?

1. Instruction
   1. Did the instructor present clear objectives or plans for the lesson?
   2. How clearly did the instructor communicate the objectives or plans to the students?
   3. Did the instructor carry out the objectives?
   4. Did the instructor present content at a pace appropriate to the length of the lesson, or the course?
   5. Did the instructor use various presentation formats (i.e., text, graphics, digital slides, video), to appeal to different student learning styles and abilities?
   6. Does the instructor approach difficult concepts in a variety of ways: e.g., written English explanations, ASL explanations, use of photos, animation, graphs or charts?
   7. Would a student new to Gallaudet’s academic environment know what to do during class?  Would a student with no prior online experience know what to do when he/she logs into the course?
   8. Are the objectives cohesive?  Does each objective build off the previous objective?  Is there a good flow among topics covered?  Are the topics and activities sequenced logically?
   9. Is there a clear connection/relationship between the objectives, content and the material?  
   10. Is the teaching of critical thinking emphasized?  Is the faculty member challenging students to understand complex ideas, analyze, compare/contrast, evaluate arguments considering different perspectives and draw conclusions?
   11. Is academic integrity emphasized?  Are there examples of violations of academic integrity given?  Are students given examples of how to ensure academic integrity in their work?
   12. Are copyright and fair use laws adhered to?  
   13. Does the faculty member give attention to accessibility issues?  Are the videos captioned and/or transcripts provided?  Are there descriptions of images and videos?  

1. Active and Personalized Learning
   1. Are students actively signing, writing, typing or utilizing other forms of self-expression?
   2. Do students have opportunities to gather information, synthesize, analyze and solve problems?
   3. Are students engaged in collaborative learning activities?  Are there paired and small group activities?
   4. Do students have opportunities to customize their learning by having assignments tailored to their professional interests and needs?
   5. Are students participating in, designing, developing, performing, reflecting, and more?
   6. Are there some flexibility in completing assignments in different formats or independence in picking a topic?  

 

1. Assessment
   1. Does the syllabus/course LMS clearly communicate course and assignment grading criteria?  
   2. Are rubrics clear and provide meaningful feedback for students?
   3. Are there opportunities for improvement throughout the course?  Are there drafts accepted and/or large projects split up in smaller assignments that build up to the final project?
   4. Is the gradebook accessible to students?  Is the gradebook consistently updated, feedback and grades posted within a reasonable amount of time?
   5. Are there examples of the assignments for students?  Are students clear on what is expected from them?
   6. Are there opportunities for informal feedback that does not involve grading? (e.g. class discussion, paired feedback, small group draft presentation)
   7. Are there opportunities for students to provide feedback for the instructor and course throughout the semester?  Does the faculty member survey students for feedback?
   8. Are the feedback for students meaningful?  
   9. Are assignments adequately challenging and required within a reasonable time frame?
   10. Does the faculty member use formal and informal student feedback in an ongoing basis to help plan instruction and assessments of student learning throughout the semester?
   11. Does the faculty member discuss feedback on assignments in class?  Or does the faculty member alert students about when and where feedback on assessments will be provided so that students can easily access instructor comments and grades?

1. Inclusion
   1. Were student characteristics such as race, gender, class, ability/disability, religion, language, geographic region, sexual orientation, first generation in college and so on taken into consideration?
   2. Are different perspectives and viewpoints included in the course content?
   3. Are the pedagogical choices available and diverse?  (e.g. lecture, team-based learning, Socratic method, role-play, debate, service learning).
   4. To what extent do the course outcomes appeal to a range or variety of student interests?  Do the outcomes appeal to and reflect the interests and learning needs of different types/groups of students?
   5. To what extent do the learning outcomes appeal to different domains?  (e.g. cognitive, affective and psycho-motor domains)
   6. Does the course material represent a variety of voices?  Are the required readings/viewings reflective of different perspectives?  Are the authors of those required materials diverse themselves? 
   7. Is divergent, creative thinking rewarded?
   8. Do the teaching activities meet the needs of diverse learners, diverse learning styles, and diverse ways of processing information, diverse performative styles?
   9. Are there implicit messages/hidden biases in the curriculum?  (e.g. only one type of assessment to determine grades)
      
      

Observation of Online Teaching Form

 

Faculty Member:  _____________________   Evaluator:__________________________

 

Course(s):_______________________________________________________________

*Indicate whether a course is hybrid or online by adding HY or OL respectively.  Onsite courses do not have a designated acronym.

*Be sure to include course section if multiple sections are offered.

 

Pre-Observation

 

Meeting Date(s):_________________________  Time Duration(s):_________________

 

Scope of Observation: _____________________________________________________

 

Goals:__________________________________________________________________

 

Observation

 

Date(s):__________________________________  Time Duration(s):________________

 

Content Observed:________________________________________________________

 

Post-Observation

 

Meeting Date(s):__________________________  Time Duration(s):________________

 

Rating:  0 - No evidence, 1- Needs extensive mentoring, 2 - Needs improvement, 3- Satisfactory, 4- Done Well and 5 - Truly Exemplary, and N/A - does not apply.

 

1.  Course LMS:  Is the course layout user-friendly, clear, and navigable?  Is the design consistent throughout?  Is the content chunked in manageable portions?
Notes:
2.  Faculty & Student Interaction: Does the course delivery and design promote genuine academic interaction between faculty and students?  Between students?  Do all students have opportunities to contribute?
Notes:
3.  Subject Area Mastery: Does the faculty member show evidence of content area expertise?  Are they familiar with current trends, research and publications in their field?  Are they able to answer questions, guide and support students in mastering the content?
Notes:
4.  Instruction: Does the faculty member present clear objectives or plans for the lesson?  Was the pacing appropriate?  Were various presentation formats used?  Were difficult concepts approaches in a variety of ways?
Notes:
5.  Active and Personalized Learning:  Are students actively signing, writing, typing or utilizing other forms of self-expression?  Do students have opportunities to gather, synthesize, analyze information?  Are there collaborative learning activities?  Are there alternatives or flexibility in format submissions or topic selection for students?
Notes:
6.  Assessment:  Are the course and assignment grading criteria communicated clearly?  Are the rubrics clear and provide meaningful feedback for students?  Are large projects split up in smaller assignments?  Are there opportunities for students to receive informal feedback?  Are feedback and grades posted within a reasonable amount of time?  Is the grading center updated frequently and accessible?
Notes:
7.  Inclusion:  Are student characteristics such as race, gender, class, ability/disability, religion, language, geographic region, sexual orientation taken into consideration in the course syllabus, design and delivery?  Are different perspectives and viewpoints included?  Does the course material represent a variety of voices?
Notes:

 

Total:__________

 

Additional  Notes:_________________________________________________________

 

________________________________________________________________________

 

________________________________________________________________________

 

 

Recommended Areas for Improvement:________________________________________

 

________________________________________________________________________

 

 

Signature of Observer:__________________________ Date:______________________

 

Faculty Member Signature*:______________________________ Date:______________ 

*The faculty member’s signature means that they have read this report fully, and does not necessarily mean that they agree with the observation. 

Resources Used in Developing Online Teaching Observation Guidelines

 Many components of this guideline were derived, revised and adapted from other resources including the following:

Miami-Dade College

Pennsylvania State University

University of Central Florida

University of Texas Pan-American

References 

Chickering, A. W., & Gamson, Z. F. (1987).  Seven Principles for Good Practice in Undergraduate Education. Journal of American Association for Higher Education Bulletin, pp. 3-7.

Edkey, M. T. & Roehrich, H. (2013). A faculty observation model for online instructors: Observing faculty members in the online classroom.  Online Journal of Distance Learning Administration, 16 (2). Retrieved from: http://www.westga.edu/~distance/ojdla/summer162/eskey_roehrich162.html

Gallaudet University Faculty (May 2015). Gallaudet University Faculty Handbook.

Tobin, T.J., Mandernach, B.J., & Taylor, A.H. (2015). Evaluating Online Teaching: Implementing best practices. Hoboken, New Jersey: Wiley & Sons Publishing.