UC Davis Canvas Fall Quarter 2016 Final Grading Period Report

Share on FacebookShare on Google+Tweet about this on TwitterShare on LinkedIn

During the Move to Canvas project (to transition courses from SmartSite to UC Davis Canvas, our new learning management system), we continue to learn and adapt as instructors and students gain experience with the system. In this report, we want to share with our community some lessons learned during the fall quarter 2016 grading period, which provided our first experience with wide-scale use of the grade-publishing feature in UC Davis Canvas.

We recognize that final grading is essential, and that having confidence in the new learning management system is critical for our instructors.

The grade publishing process (see Fig. A) transfers grades from UC Davis Canvas to the UC Davis final grading system (i.e. the bubble sheet), and then to the Banner student information system (SIS). We tested the process prior to the grading period, and addressed several potential concerns. We created a knowledge base article that explained how to submit final grades using either the integrated “Publish grades to SIS” feature or manual options, and placed announcements in the Canvas interface outlining the grade submission process, with links to the appropriate support articles.

Even with these efforts, we expected to see additional issues based on the varied uses of UC Davis Canvas, so we closely monitored support requests to identify any new concerns. To help us improve our processes, systems, documentation, and communications, we are documenting and sharing the lessons we learned.

Fall Quarter 2016 Final Grading Issues

  • No Grading Scheme
    • Experience:
      • Instructors encountered errors when reaching the UC Davis final grading system.
    • Impact:
      • Instructors could not easily submit grades, and either had to contact support for help, or manually select every grade in the final grading system.
      • Most support requests fielded by Canvas Support and by our local support team were related to this issue.
    • Reason:
      • A grading scheme was not enabled for the course, and therefore no letter grade was available to be passed to the UC Davis final grading system.
    • Actions Taken:
      • New documentation guided instructors through the process of enabling and selecting a grading scheme for the course.
      • The need to enable a grading scheme had been documented and communicated in advance, but the Canvas grading announcement and UC Davis knowledge base article were subsequently updated to clarify the message.
      • The grade publishing page was augmented with more guidance in order to reiterate the need to complete this step before proceeding (see Fig B)
      • We worked on a process to set a default grading scheme in every course.
    • Planned Actions:
      • Apply a default letter grade +/- grading scheme to all course shells that do not already have one enabled, and automate the process to apply this during the course shell creation process. In conjunction, provide instructions on how to change the course grading scheme and how instructors can create their own.
      • Update the UC Davis Final Grading system to inform the instructor when there is not a grading scheme set for the course, and advise the user to enable a grading scheme in the UC Davis Canvas course.
  • Non-Zero Blank Grades
    • Experience:
      • Final grade for individual students displays lower than expected in the UC Davis final grading system.
    • Impact:
      • Instructors spent time overriding deflated grades manually in the UC Davis final grading system bubble sheet before submitting.
      • A small number of students who received slightly deflated grades emailed instructors for a grade review.
      • We received only a small number of support requests, but we believe the impact was broader than reported.
    • Reason:
      • Blanks in the gradebook are not counted as zeroes in the grade displayed by default in the Total column of the gradebook, but are considered zeroes as part of the final grade. In most cases this is what the instructor wants, but in some cases instructors prefer blanks to be excused.
      • For weighted assignment groups with the “drop lowest grade” feature, the lowest scored assignments, rather than the blank entries, were dropped.
    • Actions Taken:
      • We updated Canvas grading announcement and UC Davis knowledge base articles to emphasize the importance of filling in blanks in the gradebook with “zeroes” or “EX” (excused) to ensure that assignments are graded as desired.
      • We added more guidance on the grade publishing page in UC Davis Canvas on the topic of finalizing the gradebook before submitting (see Fig B).
      • We reviewed submitted grades against Canvas current grades to confirm that only a small set of grades were lower than displayed in Canvas.
      • We communicated to the Canvas Customer Success Manager that improving this process is critical to our campus, and requested better identification and messaging within the system. Specifically, functions should be added to the Canvas gradebook to provide
        • warnings to instructors when there are blanks in the gradebook
        • the ability to update blanks to zeroes, or to excuse blanks en masse for all assignments.
      • We designed updates to the UC Davis final grading system to provide this identification and messaging in case Instructure / Canvas cannot add this functionality (see Roadmap).
    • Planned Actions:
      • We will update the UC Davis Final Grading system to inform the instructor when there is a discrepancy between the final grade passed from Canvas and the grade that is displayed in the Canvas gradebook, display information about each discrepancy, and advise the user to update the gradebook to ensure the proper grade is being submitted.
      • We will investigate custom developed LTI grading tools that other universities have created, to see if any can address this issue within Canvas.
      • We will create a knowledge base article with grading best practices.
  • Muted Grades
    • Experience:
      • Students (from many to all) in a course displayed lower grades than the instructor intended when the grades reached the UC Davis final grading system.
    • Impact:
      • The instructors spent time overriding deflated grades manually in the UC Davis final grading system bubble sheet before submitting.
      • To submit grades, some instructors had to use the manual export/import method, or manual entry method, rather than the Canvas integrated feature. This was necessary because the final grade passed from UC Davis Canvas to the final grading system does not equal the final grade displayed in the gradebook export in cases when the gradebook contains a muted assignment.
      • Grade change forms were submitted by department personnel for two courses that submitted deflated grades due to muted assignments. This impacted approximately 600 grades.
      • We only received a few support requests, though we have learned of a small number of additional incidents.
    • Reason:
      • The scores for assignments that are set as Muted are not accounted for in the final grade calculation when the “publish grades to SIS” feature is utilized.
    • Actions Taken:
      • We worked with Canvas Support to identify the source of the course-wide deflated grades.
      • We posted a warning announcement in Canvas specific to the muted assignment issue and behavior.
      • We reviewed and identified courses with muted assignments that still hadn’t submitted grades, and sent direct communications to instructors of record for those courses.
      • We updated Canvas grading announcement and knowledge base articles to highlight the importance of unmuting grades in the gradebook.
      • We added more information on the grade publishing page in UC Davis Canvas, to provide guidance on finalizing the gradebook before submitting (see Fig B).
      • We communicated to the Canvas Customer Success Manager that this behavior is a serious flaw in their system, that the behavior does not match any of their existing documentation, and that the final grade being passed does not match the final grade displayed in the gradebook export.
      • We communicated to the Canvas Customer Success Manager that improving this process is critical to our campus, and requested a site visit with the Product Manager for the Canvas gradebook tool.
      • We designed updates to the UC Davis final grading system, to identify these cases and provide messaging, in case Canvas cannot update the system in this way (see Roadmap).
      • We queried other universities using Canvas to see if they have developed their own LTI tools to address this grading discrepancy (see Roadmap).
    • Planned Actions:
      • We will update the UC Davis Final Grading system to inform an instructor of a muted assignment in the Canvas gradebook, and advise the instructor to unmute the assignment in the gradebook to ensure the proper grades are being submitted.
      • We will investigate custom developed LTI grading tools that other universities have created, to see if any can address this issue within Canvas.
      • We will create a knowledge base article with grading best practices.
  • Grade Overrides
    • Experience:
      • Final grade override changes made in the UC Davis final grading system do not update the Canvas grade that displays to students.
    • Impact:
      • Students receive final grades different than what they see reported in UC Davis Canvas.
      • We saw a moderate amount of support requests for this item.
    • Reason:
      • The Canvas gradebook does not allow grade overrides at this time.
    • Actions Taken:
      • Canvas Support provided a workaround to manually provide the updated grades.
      • We communicated to the Canvas Customer Success Manager that improving this process is critical to our campus, and that we need a timeframe for the roll-out of this item. It is already on their roadmap.
    • Planned Actions:
      • We will continue to follow up with Instructure on their planned development of the gradebook feature to allow instructors to override grades directly within the gradebook.
      • We will investigate custom developed LTI grading tools that other universities have created, to see if any can address this issue within Canvas.
  • Non-English language stalls grade publishing
    • Experience:
      • The process to publish grades to SIS stalled and did not reach the UC Davis final grading system
    • Impact:
      • The instructor could not use the “publish grades to SIS feature,” and had to use the manual export or manual entry method to submit grades.
      • We identified a single faculty request.
    • Reason:
      • Courses which had the language set to something other than English did not display the dialog box with the button required to initiate the passing of the grade file to the final grading system.
    • Actions Taken:
      • The script that initiates the grade publishing process was updated to address foreign language differences.
    • Planned Actions:
      • No additional actions are planned. This issue is resolved.

Roadmap of planned actions

  • Apply a typical letter grade +/- grading scheme to all course shells that do not already have one enabled, and automate the process to apply this during the course shell creation process.
    • Apply to Winter quarter course shells – COMPLETED 2/8/2017
    • Apply to Spring quarter/semester course shells – COMPLETED 3/2/2017
    • Apply to Fall quarter/semester 2017 course shells – COMPLETED 3/3/2017
  • Update the UC Davis Final Grading system to warn instructors about potential issues in their grade submissions from UC Davis Canvas, advising them to
    • Check for muted assignments in the Canvas gradebook
    • Check for grading scheme enabled in the course
    • Check for grades that arrive lower than the current displayed grade in Canvas

This update was deployed to the Test instance on 3/1/2017, and deployed to the production instance on 3/14/2017 – COMPLETED.

  • Instructure plans to update the gradebook to allow instructors to override grades directly within the gradebook.
    • Instructure has not committed to a hard deadline to achieve this, but they have communicated their expectation to release this functionality by December 31, 2017.
  • Investigate custom developed LTI grading tools that other universities have created, to see if any can solve some of our gradebook issues.
    • Contact selected universities for demonstrations – COMPLETED 2/6/2017
    • Review tools by and propose a recommendation by 4/3/2017
    • If a tool is selected, implement in Summer Session
  • Create a knowledge base article with best practices for grading – by 3/17/2017

 

Fig A) Grade submission progression:

Fig B) Updated messaging added on the grade publishing page.