What is Crowdmark?
Crowdmark is an online, collaborative marking system available to instructors and students at Waterloo without charge. It is commonly used in STEM courses, but is available for both STEM and non-STEM exams.
What are Assigned Assessments?
Assigned assignments are distinct from Administered Assessments, which many of us have used for conventional exams, often in conjunction with Odyssey. Assigned Assessments are ideal for remote exams. You set these up directly in Crowdmark, not through Odyssey.
Who might this work for?
- Students need to give full written answers or solutions
- Students need to draw graphs or other figures
- Instructors want to see a student’s work
Who will this not work for?
Assigned assessments are not appropriate for multiple-choice exams that are typically auto-graded. Mobius or LEARN quizzes are a better option in those situations. However, this may be a good compromise if you have a mixture of written and multiple choice. You would need to be creative about how the multiple choice answers are provided by students.
How do you use Assigned Assessments in Crowdmark?
The main goal of this document is to help you decide if this platform will work for you and to provide some initial guidance on how this tool works. For those who have decided to use it, there is excellent on-campus support (firstname.lastname@example.org), online support through Crowdmark via their website and email@example.com, and a detailed handout on setting up these assessments on the UW Crowdmark help page.
How do you set up the exam?
- Instructors create a Course, then an Assigned Assessment in Crowdmark, including the Due Date (end of exam).
- It is recommended a PDF version of the exam is also posted in Crowdmark so that students can visualize spacing, etc. They may even choose to print it and write their answers on it.
- Instructors tell Crowdmark to send email to the students in their Crowdmark assessment. This can be scheduled in advance.
- We recommend you set up a mock exam to help students to become familiar with the steps.
- Crowdmark integrates with LEARN so that rosters for emailing students can be pulled into Crowdmark from LEARN if the entire class is writing the same exam under the same timelines. You can also create separate exams (different versions, different start times, extra time for AAS students), and manually enroll students by uploading their email addresses to these separate exams.
- Students can either type their answers in a text file or write/draw their answers to each question on paper. Whether typing or writing, students should start each questions on a new page.
- students must upload their answers before the end of the allotted time (Due Date),
How do students submit their answers?
- If typing answers into a document: save as PDFs and drag/drop them into the spaces for each question.
- With handwritten answers on blank paper or an exam hardcopy: take separate photos of each page transfer to computer (if necessary, and drag/drop into spaces for each question.
- Files must be PDF, JPG or PNG. There is no limit to the number of pages uploaded per question. If students are uploading as a single file (with all their answers) into each question space, they will need to delete the pages not relevant to a particular question. Students can view/reload pages before submitting. Even after submitting, they can edit/reload pages as long as it is before the time limit. There are a number of free apps to convert cell phone photos to PDF, such as Notebloc and GeniusScan).
How are late submissions handled?
- Optional penalties can be applied to students who submit late. This time is determined by the instructor. E.g., if the exam is submitted between 21:00 and 22:00 apply a 5% penalty, submissions after 22:00 automatically receive a 0
- The penalty can be set at 0% and not advertised to students to allow for those students who don’t leave themselves enough time to submit all their work.
- If necessary, instructors can manually upload on behalf of a student if they email their files to the instructor.
- Instructors can also see submission in real-time, so they can notify students immediately if they notice a student has submitted a duplicate question or something of that nature (only practical for small class or a long submission window).
- Crowdmark provides detailed submission logs, similar to the Quiz Log in Learn.
Even if you and your students are veterans of Crowdmark’s Administered Assessment, practice with all these steps well before the exam is recommended, to make sure students (and instructors) are comfortable with the following:
- how to log into Crowdmark (pre-emptively fixing the usual issue with multiple UW email versions leading to courses being in multiple Crowdmark accounts)
- how to answer questions – typing in separate Word docs, modified approaches to multiple choice, filling in tables (include in your mock exam questions of the formats you will use)
- how to submit their answers – Word, cell phone, etc
How successfully students navigate a mock exam will give us feedback on what amount of time should be allowed. We can even create a mock exam, and enroll ourselves as a student (using an alternate email address), then experience it as instructor and student.
How do we grade exams?
- After the deadline, instructors can grade the submissions online themselves, or add Graders (TAs) to their Team. Crowdmark has several visual and statistical tools to allow instructors to see that questions are marked consistently between different markers.
- Instructr0s can also set up default grading comments that a TA can easily apply, e.g., If a student forgets this step, automatically take away one mark. This further avoids some grade inconsistency between markers.
- Crowdmark is a blind marking tool in that graders can’t see the student’s name while they are marking.
- Instructors can send final exam grades back to the LEARN gradebook (make sure to hide the LEARN gradebook item) or export the grades to CSV, then upload as usual (it is recommended to export to CSV for backup).
WARNING: If there is not already a grade item with the *identical* name as the Crowdmark assessment (case sensitive), Crowdmark will create a new grade item (that will not be hidden). If hiding the final exam grade is important to you, uploading manually from a csv file is safer.
- Excellent support, both on campus and through Crowdmark; lots of users on campus
- Instructors can see physical handwriting. If Crowdmark was used in the past, an instructor can look at prior assessments to see if one student's handwriting is similar to another's (which might point to possible academic dishonesty).
- Uses technology most students already have. Students are only required to have Internet access to download the exam PDF and then to upload the images, plus a cell phone.
- A Keep Learning sanctioned resource. CEL has a lot of experience with this tool
- Connected to LEARN gradebook; direct grade export or upload as CSV file.
- Does not alleviate academic integrity concerns. Students can still collaborate or use resources to answer the questions.
- Additional time needs to be allotted to allow for uploading, technical issues, and students accessing the exam in different time zones.
- Some instructors may not be familiar with the Crowdmark software. This option requires initiation by the instructor with minimal involvement by CEL/CTE.
- No multiple-choice auto grading, unlike Mobius and LEARN quizzes