September 2003

In this issue:

Evaluation of your teaching: Don't leave it entirely in the hands of students

For most faculty members, when it comes to decisions about promotion, tenure, and merit increases (which go into our base salaries), research and teaching are supposed to have an equal weighting (of 40%), and service usually counts for the remaining 20%. So it behoves us, as early in our careers as possible, to pay attention to how our teaching is evaluated, and by whom.

I recently came across what I consider quite a useful article, which elaborates on some ideas to avoid the bias that can result from putting all your eggs into the basket of student evaluations, or student evaluation of instruction (SEI) (Baldwin & Blattner 2003). Here, I will mention the main points these authors make, but feel free to contact our office (trace@watserv1.uwaterloo.ca) if you would like a copy of the article in its entirety.

SEI's are often the only means the department chair (or the Promotion and Tenure Committee of your department) has to decide how good a teacher you are, and there is a lot of research showing that, no matter how well designed they are, there are potential problem areas with these evaluations. Probably the best they can do is indicate very crudely whether you're dreadful, OK, or brilliant at teaching. What can you do to mitigate the effects of less-than-brilliant evaluations?

  1. Use several evaluation methods.
  2. Consider adding specific questions to the standard form - questions that will highlight your particular teaching strategies and objectives for the course. Also, don't wait until the end of the course to administer an evaluation. Students are more motivated (quite understandably) to help you improve the present offering of the course than to fill out a form that will only benefit future students.  A good idea is to give them 2 or 3 open-ended questions about 3 weeks into the term. Take their responses seriously, discuss them with the class, and make whatever changes seem reasonable. They appreciate your efforts. I hand out a sheet (and leave space after the numbered questions) that says the following: I am interested in how you think this course is going so far. Can you think of how I could do things differently that would make it much easier for you?  Are there things you particularly like or dislike about the way the lectures are presented? I would honestly welcome constructive suggestions at this stage of the course, because there is time to improve things for this year's class (unlike end-of-term evaluations).  Do let me know if there are things you like, so that I can keep doing things that way!
  3. What are the things you particularly like so far?
  4. What do you think I could be doing differently that might help you?
  5. Do you have comments on teaching assistant (TA) support and/or tutorials, videos, etc.?

More examples of formative (as opposed to summative) evaluation questions are available from our office.

  • Create a teaching dossier.

    A teaching dossier (or portfolio, as the Americans call it) is a detailed 'picture' of who you are as a teacher: your philosophy of teaching, how you put that into practise, and evidence from a variety of sources substantiating what you've written. It has been forcefully argued that the very act of putting together a dossier makes one a better teacher (Seldin, 1991). To this end, collect formal and informal feedback from students and colleagues about your teaching (unsolicited email messages, thank-you cards from honours students, evidence of students' work, course syllabi, etc.) to add to your promotion-and-tenure package and your annual review.

  • Have colleagues attend your classes.

    Some departments do peer evaluations as a matter of course, and some don't. Regardless of your department's stance on this, it can be very useful to have colleagues attend your classes to give different perspectives from that of a student. It can be very informal, for example someone at your own level who would give no official feedback to you; or, it could take the form of a more formal evaluation by a more senior person in your field, known for his or her good teaching. Finally, don't forget that there are trained observers (both faculty members and graduate students) in the TRACE office who will provide this confidential service.

  • Remember the context.

    Keep in mind that the type of course you teach, the composition of the class, and even the time the course is offered can influence evaluations. For example, required quantitative courses in the social sciences are not ones that traditionally elicit wonderful evaluations, no matter who teaches them! Also, most students would rather start a lecture at 9:30 in the morning instead of 8:30. Be aware of this if you teach a similar type of course; make it your business to find out how people in the past have fared in being evaluated on 'difficult' courses in your department, and point out where you stand with respect to these norms.

References

Baldwin, T., & Blattner, N. (2003). Guarding against potential bias in student evaluations. 
     College Teaching, 51, 27-32.
Seldin, P. (1991).  The teaching portfolio: A practical guide to improved performance and 
     promotion/tenure decisions. Bolton, Mass.: Anker Publishing, Co.

Barbara Bulman-Fleming

Tomorrow's professor today!

You can subscribe to a listserv called Tomorrow's Professor that sends messages bi-weekly from Stanford University. The messages, which are often excerpts from articles or books, would appeal mostly to graduate students, post doctoral students, and newly hired faculty members, but there are often quick tips of advice that any faculty member would find useful. The list now has more than 15,000 subscribers worldwide. Here are some titles of recent postings: "Establishing rules for groups," "The constructivist view of learning," and "Guided notes - improving the effectiveness of your lectures." The listserv recently surpassed message #500!  If you're interested in subscribing, email Majordomo@lists.stanford.edu. Do NOT put anything in the SUBJECT line, but in the body of the message type: subscribe tomorrows-professor.

On being a new faculty member at Waterloo: An interview with Sarah Michaels

For Sarah Michaels, assistant professor in Waterloo's School of Planning, securing a faculty position here was a case of coming full circle. Sarah completed her BIS at Waterloo before heading off to Simon Fraser University and then the University of Colorado to complete her education.  Now she is back at Waterloo, a new faculty member as of October 2001.  Since coming back she has received a 3-year SSHRC grant, successfully completed numerous refereed publications and conference presentations, and taught three different courses. I asked Sarah for her most positive teaching experiences and her advice to other new faculty members.  Here's what she shared.

Her positive teaching experiences all revolved around seeing her students rise to a challenge and put course material into practice. In one course on policy analysis, which included both undergraduate and graduate students, she had invited the chief and deputy librarians for the City of Waterloo to class. They were interested in ideas to help raise the public profile of the library. After a presentation by the librarians and some general discussion, one undergraduate student suggested that the class use the basic methods of policy-analysis that Sarah had taught to generate alternative ideas about how to raise awareness about the needs of the public library.  The class banded together and generated a number of feasible alternatives, then helped the librarians identify the best ones. The students stayed past the end of class time that night (it was a 3-hour class!) and applied their skills and knowledge in a new situation.  Sarah's delighted response was, "I couldn't have asked for anything more." In another course on resource management, she took students on a canoeing field trip. Approximately 75% of the class participated, even though a number had never canoed before. The students saw for themselves how the conservation principles discussed in class were reflected, or not, in management practices on one stretch of the Grand River.

Sarah has worked on her teaching over the past few terms and has advice that all faculty may benefit from, but particularly new faculty. 

  • Get as much help as you can as soon as you can. For Sarah, this has meant contacting TRACE and master teachers in her area. Have a colleague come to a class or two to give you feedback. Teaching is a public event, so acknowledge it as such and be open to getting input from others.
  • Before you begin teaching, find out what the teaching expectations are in your unit and Faculty. Review the standardized performance evaluation form to see what is emphasized. This may help you make decisions about your course design and delivery.
  • Be organized in advance. The time that you spend on creating a detailed course outline is well spent because the outline is a  contract between you and your students. It's great to have it to refer to throughout the term to keep your course on track and to help you think through and plan for the logistics of running your course. 
  • Know your teaching assistants. When you know their strengths, you may be able to adapt your course to capitalize on them. Find out about their career aspirations and interests and see if you can help further their development in your course.
  • Have students complete written midterm course evaluations. Getting feedback isn't always as easy as you may think. Even if you feel that you're approachable, students may not speak up unless asked.
  • Even the best laid plans go awry. Sarah has found that having a sense of humour and thinking on your feet are two critical skills in teaching. The bottom line is to do what's fairest for the majority of the students.
  • Have fun!

Sarah attended last year's programming for new faculty members as well, which increased her knowledge of Waterloo as an institution and the varied expectations across campus.  She enjoyed the sense of camaraderie and having the chance to meet others who were "in the same boat."  She also appreciated having access to materials tailored to new faculty and their issues - available both at the lunch-and-learn sessions for new faculty and on the website for new faculty (https://uwaterloo.ca/watport/).

Donna Ellis

Rethinking writing assignments

Educators agree that writing is a powerful learning tool: it encourages critical thinking, dependent learning, the exploration and explanation of ideas, focused thoughts, personalisation of ideas, and the retention of concepts. 

However, as university class sizes increase, professors may be tempted to cut back on the writing they assign to their students to limit the amount of marking that they and their graduate teaching assistants (TAs) need to do. Participants at the TRACE workshop on July 17, 2003, sought to address this problem: how do we help students glean the benefits of writing without burying ourselves in paper?

One potential solution is to replace some high-stakes writing assignments with low-stakes writing tasks. High-stakes writing is formal, structured writing that is graded and is usually worth a large portion of a student's overall mark, whereas low-stakes writing assignments or activities involve informal writing and grading. One of the benefits of low-stakes writing is that low-stakes assignments can achieve the same teaching goals as high-stakes assignments while at the same time decreasing the stress level for students and being less time-consuming for professors and teaching assistants to grade.

For example, if the goal of a formal essay (high-stakes writing) is to have students identify and synthesise the key research questions and ideas within a particular field, having students do an annotated bibliography (low-stakes writing) would still allow them to learn about the key areas of research, and would encourage the students to summarise concepts in their own words. Or, at the end of a class have students write a one-sentence summary (low-stakes writing) of a particular concept discussed in class. Alternatively, do not give them a particular concept; simply ask them to summarise the lecture in one sentence, picking the most salient points. Not only do students have to think about the ideas learned in that particular class, but they will retain those ideas better, enabling them to potentially perform better on exams. Other low-stakes writing ideas were presented at the workshop and can be found on the TRACE tips sheet, "Low-stakes writing assignments."

Another way to make writing assignments more manageable is to adjust how you respond to students' writing. For example, write comments only if the students will read and benefit from them. If the comment will do more harm than good to the student, don't write it. Or, rather than writing a long comment on each paper, try giving general feedback to the class, either in writing by email or orally at the next class. You could also choose not to read every student's writing every time, instead only collecting some writing each week.

To simplify marking tasks, especially when responding to high-stakes assignments, you may find heuristics and rubrics helpful. A heuristic is a set of questions that guides one's attention to certain aspects of a piece of writing (e.g., is the writing clear and precise?). So, when you read a paper, you ask yourself these questions. Your written response to the paper is then focused on only those particular aspects. A rubric is a scoring grid or scale that features a description of the primary traits or essential components of a completed piece of writing, along with associated grades/evaluation levels. Essentially, a rubric consists of a set of criteria that you've identified as important for a piece of writing and a graded scale based on how well the student has achieved each of the criteria. Heuristics and rubrics can help to streamline the marking process as well as make marking more consistent. A TRACE tips sheet entitled "Responding to writing assignments: managing the paper load" gives additional details on how to respond to high- and low-stakes writing effectively and efficiently, including how to design and use heuristics and rubrics.

Daniel Olsen & Cara DeHaan

Programming for new faculty

Under the direction of Tom Carey and Bruce Mitchell, TRACE and Learning and Teaching Through Technology (LT3) staff members recently coordinated the 2nd annual half-day welcoming events for faculty members new to Waterloo. On September 3rd, more than 50 new faculty members attended a luncheon with senior Waterloo administrators, deans, and department chairs.

After lunch, the new faculty heard a panel presentation on "Success through balance," then participated in break-out groups to discuss their research and teaching challenges. The session concluded with an invitation to apply for a faculty learning community, which is a small group that meets throughout the year to discuss an overarching theme or issue. 

The day concluded with a barbecue for new faculty and their partners at President Johnston's farm.  It was a great day to start building a Waterloo community!

Ongoing programming is also provided to new faculty members - three lunch-and-learn events on the following topics:  research grants, tenure and promotion, and teaching evaluations. New faculty will be invited before each event.

Please direct any questions about the program to Donna Ellis at ext. 5713.

Learning and Teaching Through Technology Centre interim director

On July 1, Liwana Bringelson assumed the role of interim director of the Centre for Learning and Teaching Through Technology (LT3) for a one-year term. LT3 was started in 1999 by founding director, Tom Carey, as an innovation catalyst for learning technology at Waterloo. Until moving into its current physical space in 2000, Dana Porter Library (LIB 328), LT3 shared space with TRACE in MC. This symbolizes the emphasis that both TRACE and LT3 place on the importance of teaching and learning. Both of these organizations are part of the learning resources and innovation portfolio, which is headed by Tom Carey, as an associate vice-president.

Bringelson, previously a research project manager in LT3 and a research associate professor in systems design engineering, will direct the innovation and support projects of the LT3 team. Through both internal and external funding, LT3 is able to engage with faculty members who are interested in exploring how technology impacts the teaching and learning environment. These projects range from using a learning object to explain a difficult concept to restructuring a course through the use of online discussions. For more information on LT3 projects and activities visit http://LT3.uwaterloo.ca.

Certificate in University Teaching guidelines revised

As of September 1, 2003, the Certificate in University Teaching (CUT) is running under revised guidelines. All active CUT participants should go to the TRACE web site at http://www.trace/tacert.html and read through all of the guidelines.

The most significant change involves the teaching dossier workshops. In the past, the CUT required that participants attend two workshops on the teaching dossier (Parts 1 and 2) and write a response paper for each one as part of Graduate Studies 901. They chose their own remaining six workshops. Under the new guidelines, only the revised teaching dossier workshop (a combined version of the original two) will be offered and required for GS 901. Participants starting the CUT as of September 1, 2003, or those who have not completed either of the original teaching dossier workshops, will need to take the new workshop and write a response paper, plus complete seven more workshops and corresponding papers. Participants who started before this date and have completed either teaching dossiers part 1 or part 2 (now discontinued workshops) will be required to take the revised dossier workshop. Anyone who has already completed the original teaching dossiers part 1 and 2 (complete with accepted response papers) does not need to take the revised workshop.

We have also added heuristics for all of the written work required for the program so participants can self-assess their work before submitting it.  Heuristics are questions that focus a reader's attention on certain aspects of a document. TRACE evaluators will also be using these heuristics when assessing participants' work.

Other small revisions have been made to increase the clarity of the guidelines. If you have any questions or comments about the revised guidelines, please contact Donna Ellis at ext. 5713 or by email at donnae@uwaterloo.ca.

Several small grants available for course internationlizaiton

Up to 7 grants of $1500 each will be made available to faculty members who are willing to modify their courses in such a way as to incorporate an international component. This money will be made available by Gail Cuthbert Brandt, the associate vice-president, academic.

The University has committed to carrying on the work that began in the spring of 2002 when Bruce Mitchell and I were awarded one of the three CBIE (Canadian Bureau for International Education) matching grants for universities in Canada. We hired a part-time research assistant, Elise Ho, to work on the project; she made great progress thanks to the help of those who participated in her focus groups. The aim was to develop some approaches through which both international students and domestic students who have international experience could be engaged by instructors to share their expertise in order to help internationalize the curriculum. We submitted our final report to CBIE in August, and will use the funds recently made available by the University to hire Elise's successor, Candace Newman (see below). Candace will help instructors put into practice Elise's ideas, as well as look into other possible internationalization initiatives.

If you are interested in curricular internationalization, or would like more information on these small grants, please contact Candace at cmnewman@fes.uwaterloo.ca. Because a limited number of these grants are available, priority will be given to those professors who have already expressed an interest in course internationalization. 

Barbara Bulman-Fleming 

Internationalizationresearch assistant - Candace Newman

Candace Newman is a PhD candidate in the Department of Geography. For her dissertation research, Candace has travelled to Indonesia where, through experiential observation, she assessed potential socio-cultural inputs that could be used to improve the operational use of satellite imagery of coral reef ecosystems for managers and management plans. She is focusing on the challenges and solutions to intercultural communication, with an accent on how effectively remote sensing technology can be used as a communication tool. Candace is interested in issues surrounding intercultural communication and is looking forward to being a part of the TRACE team that will assist professors to enrich university curricula.

New Certificate in University Teaching coordinator

Geneviève Desmarais, a PhD candidate in the behavioural neuroscience division of the psychology department, will rejoin TRACE as senior teaching assistant (TA) developer and CUT coordinator, replacing Cara DeHaan.  After being a TA developer from January to December 2001, as well as in fall 2002, she has been a lecturer for 'Physiological Psychology,' a second-year psychology course, as well as a coordinator for the same course in its distance education version. Geneviève enjoys discussing teaching with her fellow graduate students and offering her expertise. She is thrilled at the prospect of returning to TRACE, and looks forward to the opportunity of gaining further experience through this appointment. To book observations, contact Geneviève at: gdesmara@watarts.uwaterloo.ca

"Stressed…?"  response

Last issue's article about stress (http://www.adm.uwaterloo.ca/infotrac/tmMay03.html) struck a nerve. I received over a dozen responses from faculty, staff and graduate students. With one exception, people agreed that their jobs were more stressful now than was the case a few years ago. One person pointed out how lucky we are as faculty: free to ask interesting research questions, and relatively free to come and go as we please as long as we get our work done. There are no magic cures for stress, but communication (with friends, family and/or professionals) is critical, so keep that in mind as the term progresses. Also, I have come across a little story that, I think, puts things in perspective nicely. It's called "Fill Life" http://www.mcgeesmusings.net/2003/06/06.html.  Personally, writing the article was a very good thing - I've made wonderful new friendships with two of the responders. 

Barbara Bulman-Fleming

Announcing September 2003 TRACE events

TRACE will be offering many interesting and informative workshops in the fall term which are open to all Waterloo faculty members, staff instructors, and graduate students.

Tutorials and teaching September 18 12 - 1:30 p.m.
Teaching critical thinking October 15 12 - 1:30 p.m.
Creating CVs and cover letters October 30 12 - 1:30 p.m.
Teaching sossiers (revised) November 4 9:30 - 11:30 a.m.
The academic job interview November 19  12 - 1:30 p.m.
Understanding the learner December 3 OR
December 11		 9:30 a.m. - 12:30 p.m.
1:30 - 4:30 p.m.

For more specific details, watch for notices in your department and via the certificate listserv. To join the listserv, email trace@watserv1. 

CUT participants, please note that all of these workshops partially fulfill CUT requirements for GS 901 and 902. The teaching dossier workshop is required for the CUT and will be offered every term.

You may register via our web site at: http://www.trace/workhp.html