For Consultation: Final Oral Examination Review Working Group Report

Overview

At the University of Toronto, the thesis is the culminating research project for a doctoral degree. The final oral examination (FOE) is the evaluation of that thesis and its oral defense. More than 1,000 final oral examinations take place at the university every year. Therefore, as the graduate education landscape continues to evolve – globally and at the University of Toronto – it is important to review the current FOE regulations and guidelines to ensure alignment with innovative practices, foster quality scholarship and academic rigour, and facilitate a quality experience for all individuals involved. To this end, SGS initiated a review of the final oral examination in January 2024 and the Final Oral Examination Review Working Group commenced in September 2024.  

Two documents were produced by the Final Oral Examination Review Working Group for consultation:

Reports

1

Final Report

2

Proposed Recommendations for Consultation

Final Report

Background and Rationale

At the University of Toronto, the thesis is the culminating research project for a doctoral degree. The final oral examination (FOE) is the evaluation of that thesis and its oral defense. More than 1,000 final oral examinations take place at the university every year.

The final oral examination committee represents the School of Graduate Studies (SGS) Graduate Education Council and through it, the university. The committee is therefore responsible for upholding the standard of the doctoral degree at the University of Toronto and possesses the full authority of SGS with respect to the examination.

Members of the graduate community have shared challenges with the current SGS Calendar examination regulations and guidelines, including a lack of clarity on key components of the FOE and complicated or resource-intensive processes and procedures. Given the presence of two similar documents, the SGS Calendar and the SGS FOE Guidelines, there is confusion between what exactly are the regulations versus what are the suggested best practices that units can choose to implement depending on their context. Currently, the SGS Calendar only presents the minimum core regulations, while the SGS FOE Guidelines cover both regulations and best practices in greater detail. As a result of these challenges, some inconsistencies in application of these regulations and guidelines across divisions are observed. It is critical that the foundational regulations and guidelines are clear, differentiated, simple and flexible to enable individual graduate units to adhere to the regulations while incorporating discipline-specific practices that align with scholarly norms in their fields.

Therefore, as the graduate education landscape continues to evolve – globally and at the University of Toronto – it is important to review the current FOE regulations and guidelines to ensure alignment with innovative practices, foster quality scholarship and academic rigour, and facilitate a quality experience for all individuals involved. To this end, SGS initiated a review of the final oral examination in January 2024 and the Final Oral Examination Review Working Group commenced in September 2024.

Purpose

The goals of the working group in undertaking a review of the FOE regulations and guidelines are: 

  1. To increase the sense of occasion of the FOE.  
  2. To ensure the FOE guidelines allow for innovation, as well as disciplinary and programmatic flexibility.  
  3. To increase clarity of the FOE guidelines for all stakeholders. 
  4. To ensure the FOE guidelines align with both the SGS thesis guidelines and degree level expectations. 
  5. To ensure alignment between the FOE guidelines and other SGS regulations.  
  6. To increase the operational impact of current processes and any recommended changes. 
Membership

The following individuals served as members of the Final Oral Examination Review Working Group:

  • Vina Goghari, Professor, Graduate Department of Psychological Clinical Science, University of Toronto Scarborough, and Vice-Dean, Research and Program Innovation, School of Graduate Studies​ (Chair)
  • Greg A. Jamieson, Professor and Associate Chair, Professional Programs, Department of Mechanical & Industrial Engineering, Faculty of Applied Science & Engineering
  • Josée Johnston, Professor and Acting Chair, Department of Sociology, University of Toronto Mississauga
  • Joseph Flessa, Professor and Associate Dean, Programs, Ontario Institute for Studies in Education​
  • Kajri Jain, Professor and Graduate Chair, Department of Art History​, Faculty of Arts & Science and University of Toronto Mississauga
  • Katherine Tamminen, Professor and Associate Dean, Graduate Education, Faculty of Kinesiology & Physical Education​
  • Pamela Klassen, Professor, Chair and Graduate Chair, Department of Religion, Faculty of Arts & Science ​
  • Paul Kushner, Professor, Chair and Graduate Chair, Department of Physics, Faculty of Arts & Science​
  • Sarah Peake, Director, Student Academic Services, School of Graduate Studies
  • Stephanie Waterman, Associate Professor, Department of Leadership, Higher & Adult Education, Ontario Institute for Studies in Education​ and former Provostial Academic Advisor on Indigenous Curriculum and Education
  • Tania Watts, Professor, Department of Immunology, Temerty Faculty of Medicine
Pre-Working Group Consultations and Environmental Scans

To help inform the primary foci of the working group, SGS conducted pre-working group consultations with various stakeholders in the graduate community from February to September 2024. Consultations included focus groups with graduate students and attendance at an SGS Graduate Student Advisory Committee meeting, survey of faculty (n=418 responses) and facilitation of a Conversations with SGS session, survey of administrative staff (n=48 responses) and attendance at the Graduate Administrators Community of Practice, and a survey of external appraisers who have participated in an FOE at U of T (n=68 responses).   

In general, stakeholders consulted were satisfied with the current FOE regulations, guidelines and resources. However, there were a few core areas where stakeholders suggested improvements, including:

  • Revising pre-exam procedures, such as obtaining an FOE chair and the criteria required for an external appraiser
  • Reviewing existing examination formats and their associated value to the student experience (i.e., in-person vs. online or hybrid exams, closed vs. public exams)
  • Increasing flexibility in the regulations to allow for department-specific FOE practices
  • Enhancing clarity across FOE voting categories and more streamlined voting procedures
  • Enhancing the sense of occasion of the post-examination experience and examples of unit-specific practices that have worked well

In conjunction with stakeholder consultations, several environmental scans were also conducted to review the common practices at peer institutions for the core components of the FOE (i.e., FOE chair, external appraiser, types of examination, closed vs. public defense). Each of the environmental scans reviewed FOE practices at U15 institutions in Canada and select peers in the United States, including University of California, Berkeley, Cornell University, University of Minnesota, Ohio State University and the University of Southern California.

Working Group Process

The FOE Working Group met seven times to discuss the survey results and environmental scans, review FOE regulations and guidelines, and develop recommendations for consideration.

The first task of the working group was to identify the core principles necessary for a successful, academically rigorous, and fulfilling FOE experience. Once identified, these principles, listed below, grounded discussions and helped the working group evaluate different domains of the FOE. Consistently referenced by working group members during meetings, the principles were employed to help guide decisions and build informed, thoughtful recommendations.  

Each meeting of the working group was structured around key domains of the FOE. The FOE domains reviewed by the working group included: FOE chair, external appraiser, composition of the FOE committee and quorum, type of examination, voting, adjournments and reconvened exams, closed exam vs. public defense, and the FOE time length. Equity, diversity and inclusion, as well as Indigenous practices were identified as cross-cutting factors guiding all discussions of the FOE domains.

Data and consultation insights were presented to the working group at the beginning of each domain review to help foster evidence-informed dialogue and help working group members become familiar with the experiences of all the stakeholders and the practices of peer institutions. Specifically, the working group examined and considered the benefits and disadvantages of different approaches and models used at peer institutions for key domains of the FOE, which assisted in evaluating practices at U of T and guiding the development of recommendations. Finally, the working group dissected the FOE regulations in the SGS Calendar, and as relevant, the FOE Guidelines on the SGS website. This practice helped pinpoint discrepancies, issues of clarity, and challenges with implementation, and thus, helped identify necessary improvements and develop recommendations.

Finally, the working group dissected the FOE regulations in the SGS Calendar, and as relevant, the FOE Guidelines on the SGS website. This practice helped pinpoint discrepancies, issues of clarity, and challenges with implementation, and thus, helped identify necessary improvements and develop recommendations.

Core Principles

The core principles identified as necessary to a successful, academically rigorous, and fulfilling FOE by the Final Oral Examination Review Working Group are as follows:

  • Academic rigour: Ensuring that FOE process and outcomes are underpinned by scholarly excellence, integrity, innovation, and an enriching learning experience.
  • Disciplinary and programmatic flexibility: Ensuring that the appropriate regulations are in place to meet OCAV’s (Ontario Council of Academic Vice-Presidents) degree level expectations and SGS Calendar requirements, while allowing for unique discipline- or program-specific practices.
  • Wellness and exceptional experience for all stakeholders: Ensuring that the FOE process and procedures prioritize stakeholder well-being and strive for a fulfilling experience for all involved.
  • Equity, diversity and inclusion: Ensuring that the FOE processes promote equity and inclusion and are responsive to our diverse stakeholder groups.
  • Professional development: Ensuring that the FOE process serves as an opportunity for students to develop and advance their academic and professional skills and professional network.
  • Sense of occasion: Ensuring that the FOE process and environment demonstrates a sense of significance and achievement given the importance of this culminating milestone.
  • Public accountability: Ensuring that, as a public university, there are appropriate mechanisms for publicly disseminating research and engaging with the community.  
  • Transparency: Ensuring that all FOE regulations, processes and procedures are clear, accessible, and understandable by all stakeholders.
  • Streamlined and operationally efficient: Ensuring that FOE regulations, processes and procedures can be implemented in a well-organized, timely and effective manner.  

These core principles should underlie and be reinforced by the FOE process and procedures.

Recommendations

The working group recommends that students, faculty and staff would benefit if SGS were to:

  • Streamline processes and increase efficiencies with FOE procedures.
  • Increase flexibility for graduate units by allowing for, where advisable, unit autonomy in implementation of regulations.
  • Simplify final oral examination guidelines on the SGS website.
  • Clearly differentiate between SGS regulations that must be implemented for all FOEs by all units and best practices that graduate units are encouraged to implement (i.e., FOE guidelines), if relevant to their context.
  • Maintain core elements of the FOE that are valued by the stakeholders (e.g., inclusion of the external appraiser; allowing the student to receive the external appraisal letter prior to the FOE; allowing for an adjournment rather than a fail after the first examination).
  • Revise core elements of the FOE that could be enhanced for the stakeholders (e.g., clarify and simplify voting categories and processes; create more flexibility in who can chair the FOE; be more flexible in who can attend the examination; encouraging in-person FOE participation).

Based on these recommendations, and the core principles listed above, the working group has developed more detailed recommendations for the Dean of the School of Graduate Studies to guide internal consultations going forward (document submitted to the Dean of the School of Graduate Studies).

Next Steps

The working group provides this report and a list of sample practices to help enhance a sense of occasion for the FOE that could be implemented by graduate units and/or supervisors to the general U of T community. In addition, a detailed set of recommendations are provided to the Dean of SGS for further consultation, delineation and implementation. The ultimate output of the working group will be the revised SGS Calendar for the FOE section and the reworked SGS FOE Guidelines webpage and materials. All revisions to the SGS Calendar for the FOE section will need to be approved by the Graduate Education Council.

Proposed Recommendations for Consultation

This detailed set of recommendations is provided to the Dean of the School of Graduate Studies (SGS) for further consultation. Once the recommendations have been accepted or revised, further delineation of some of the models or options will be necessary, in addition to implementation.

Appointment process
  • 3 models considered: Outside of the graduate unit, outside of the FOE committee, and within the FOE committee.
    • Please note: All three models were found to retain academic rigour as the Chair of the FOE committee must hold full graduate faculty membership (GFM), as per SGS’s existing GFM guidelines. 
  • Recommendation: SGS should extend the parameters of the current “outside of the graduate unit” model for the FOE Chair to “outside of the FOE committee.” This would allow faculty members to serve as the FOE Chair for students within their own graduate unit, if they are not active members of the student’s FOE committee. (SGS Calendar: 8.3.5)
  • Principles: Allows for increased disciplinary and programmatic flexibility and a more streamlined and operationally efficient model in securing a chair. Maintains key principles such as academic rigour, sense of occasion, and an exceptional experience for all stakeholders.

Mandatory component of the FOE
  • 2 models considered: Retain external appraiser or remove the external appraiser involvement.
  • Recommendation: SGS should retain the external appraiser. (SGS Calendar: 8.3.3)
  • Principles: Promotes academic rigour, professional development, a sense of occasion, exceptional experience for all stakeholders, equity, diversity and inclusion, and public accountability.
Criteria
  • 2 models considered: Retain the current external appraiser criteria (i.e., arm’s-length, two-year restriction on participation) or modify the current external appraiser criteria.
  • Recommendation: SGS should add the following statement from the SGS Guidelines to the SGS Calendar criteria for an external appraiser: “An external appraiser should not normally be used more frequently than once every two years by a single supervisor.” It was noted by the working group that the application of this criteria will be more nuanced in emerging or smaller fields.
  • Recommendation: Remove from the SGS Guidelines “an external appraiser should not be used more frequently than once every two years for committees with substantive overlap of members with whom the external has served within the past two years i.e. three or more voting members.”
  • Recommendation: SGS should create an arm’s-length attestation process that is completed by the external appraiser, supervisor(s) and student, and reviewed by the graduate chair as a component of the external approval process. Where there is conflict or ambiguity regarding the application of the criteria for the external appraiser, the graduate chair will consult with the Vice-Dean, Students.
  • Principles: Retains academic rigour, while providing disciplinary and programmatic flexibility, and a streamlined and operationally efficient process.
Appointment process
  • 2 models considered: Retain the current process for approving external appraisers (i.e., SGS approves all external appraisers) or create a more streamlined process at the graduate unit level (i.e., graduate chair approves external appraisers).
  • Recommendation: To streamline the external appraiser approval process, the graduate chair will be responsible for granting approval of external appraisers. Where there is conflict or ambiguity regarding the application of the criteria for the external appraiser, the graduate chair will consult with the SGS Vice-Dean, Students. SGS should replace its current process of reviewing every external appraiser with a periodic review of external examiners within each graduate unit. (SGS Calendar: 8.3.3)
  • Principles: Retains academic rigour, while providing disciplinary and programmatic flexibility and ensuring processes are streamlined and operationally efficient.
Defining arm’s length
  • 2 models considered: Retain the current definition for arm’s length or revise definition of arm’s length to extend parameters and allow for a wider pool of possible external appraisers.
  • Recommendation: SGS should revise the following sentence of the arm’s length definition: “in the past six years, been a departmental colleague of the candidate or the supervisor” to “in the past seven years, been a departmental colleague of the candidate or the supervisor.” This change would more accurately reflect the normative length of the PhD program. (SGS Calendar/Guidelines)
  • Recommendation: SGS should remove “scholarly work” from the following sentence in the definition of arm’s length in the SGS Guidelines as it was found to be overly broad, as well as define a time frame for the collaboration as seven years: “has collaborated on a research project, scholarly work or publication, with either of them.” Once revised, the sentence would be: “in the past seven years, has collaborated on a research project or publication, with either of them.” (SGS Calendar/Guidelines)
  • Recommendation: SGS should add to the arm’s length exclusion criteria: Close family/friend relationship with the supervisor(s) or student. (SGS Calendar/Guidelines)
  • Recommendation: SGS should add the following sentence from UTQAP’s definition of arm’s length into the SGS Calendar/Guidelines: external appraisers “… should not be chosen who are likely, or perceived to be likely, to be predisposed, positively or negatively, about the FOE and its outcomes.” (SGS Calendar/Guidelines)
  • Recommendation: SGS should review UTQAP’s examples of what may or may not violate arm’s length and determine if any of the examples listed should be included on the SGS website to further clarify what arm’s length encompasses. (SGS Calendar/Guidelines)
  • Principles: Retains academic rigour, while providing disciplinary and programmatic flexibility, ensuring equity, diversity and inclusion, and promoting processes that are streamlined and operationally efficient.

Composition of committee
  • 2 models considered: Keep the current examination committee requirements or change the composition requirements for the examination committee.
  • Recommendation: SGS should consider explicitly naming the supervisor(s) in the list of voting members for the final oral examination, in addition to the one to two members of the supervisory committee selected as voting members. The regulations should also state that the supervisor(s) normally attend the final oral examination. (SGS Calendar: 8.3.4)
  • Recommendation: SGS should consider explicitly stating in the SGS Calendar that the external examiner normally attends the final oral examination. (SGS Calendar: 8.3.4)
  • Principles: Promotes academic rigour, an exceptional experience for all stakeholders, a sense of occasion, professional development, and public accountability.
Examination Committee and Quorum
  • 2 models considered: Retain the current process for approving examination committee membership (i.e., SGS approves all members) or create a more streamlined process at the graduate unit level (i.e., graduate chair approves examiners).
  • Recommendation: To streamline the examination committee approval process, the graduate chair will be responsible for granting approval of the examination committee. Where there is conflict or ambiguity regarding the make-up of the examination committee, the graduate chair will consult with the SGS Vice-Dean, Students. (SGS Calendar: 8.3.3) SGS should replace its current process of reviewing every examination committee with a periodic review within each graduate unit.
  • Recommendation: Retain quorum regulations. (SGS Calendar: 8.3.4)
  • Principles: Upholds academic rigour, while providing disciplinary and programmatic flexibility and ensuring processes are streamlined and operationally efficient.

  • 2 models considered: Retain flexibility in type of examination by keeping all three options (i.e., fully in-person, hybrid and fully online), or shift to in-person and hybrid (where student should attend in-person) only.
  • Recommendation: Retain flexibility in type of examination by keeping all three options in line with universal design for learning principles. However, SGS should explicitly state the benefits of in-person examinations for all members involved, but particularly for the student, and indicate a preference for in-person examinations in the SGS Calendar. (SGS Calendar: 8.3.6)
  • Principles: Ensures disciplinary and programmatic flexibility, wellness and exceptional experience for all stakeholders, a sense of occasion for students, and equity, diversity and inclusion (e.g., universal design for learning principles).

Nomenclature
  • 2 models considered: Retain the current nomenclature for the voting categories or revise the voting categories to enhance clarity, applicability and ease of implementation.
  • Recommendation: SGS should clarify the current FOE voting categories by updating the nomenclature and descriptions to align with the normative length of time required to complete the revisions. (SGS Calendar: 8.3.12)
  • Principles: Ensures academic rigour, while also increasing transparency, and promoting streamlined and operationally efficient processes.
Current language vs. proposed revisions
  1. Current: Acceptable as it stands – the candidate must make arrangements to submit the thesis to the digital research repository within one week of the date of the examination.​ (SGS Calendar: 8.3.12)
    • Proposed revision: Change to “Accepted as it stands”
  2. Current: Editorial corrections – the candidate must complete the corrections within one month of the date of the examination, and the supervisor will inform the candidate of the necessary corrections. ​(SGS Calendar: 8.3.12)
    • Proposed revision: “Revisions that must occur within one month” – The supervisor will supervise the proposed revisions. ​Revisions must be completed within one month of the date of the final oral examination. ​The examination committee must decide the nature of revisions, but it is intended that these revisions could be more substantial typographical errors, corrections in style, expanding on material, such as adding a limitation, and less than major changes in the thesis. (SGS Calendar: 8.3.12)
  3. Current: Minor revisions – ​a subcommittee will supervise the proposed revisions. One member of the subcommittee is designated by the chair, with the approval of the examination committee, as the convenor. The convenor will be responsible for the preparation of a statement detailing the revisions required. ​Revisions must be completed within three months of the date of the oral examination. ​The examination committee must decide the nature of minor revisions, but it is intended that minor revisions should be more than corrections in style and less than major changes in the thesis. A typical example of minor revisions might be clarification of textual material or qualification of research findings and conclusions. (SGS Calendar: 8.3.12)
    • Proposed revision: “Revisions that must occur within three months” – ​The supervisor or a subcommittee of the examination committee will supervise the proposed revisions. If there is a subcommittee, one member is designated by the chair, with the approval of the examination committee, as the convenor. During the FOE, the examination committee may determine whether a subcommittee is necessary when the revisions require more substantial input and/or the expertise of multiple committee members to complete the revisions. The supervisor or convenor (depending on the option chosen) will be responsible for the preparation of a statement detailing the revisions required. ​Revisions must be completedwithin three months of the date of the final oral examination. ​The examination committee must decide the nature of revisions, but it is intended that the revisions should be less than major changes in the thesis. A typical example of revisions might be more substantial clarification of textual material or qualification of research findings and conclusions. (SGS Calendar: 8.3.12)
Striking a subcommittee for minor revisions
  • 2 models considered: Mandatory creation of a subcommittee to oversee minor revisions process or increased flexibility in oversight of minor revisions process.
  • Recommendation: SGS should allow the examination committee to determine whether additional oversight by a subcommittee is necessary in instances where a student must complete minor revisions (term to be revised) to the thesis following the final oral examination. In circumstances where additional oversight by a subcommittee is not needed, the examination committee may opt for review by the supervisor(s) only. (SGS Calendar: 8.3.12)
  • Principles: Enhances disciplinary and programmatic flexibility, while ensuring academic rigour, as well as streamlined and operationally efficient processes.
Adjournment and reconvening of the FOE
  • 2 models considered: Adjourn and reconvene if voting consensus is not reached on first attempt or revise the process to fail on first FOE attempt and have a second exam.
  • Recommendation: Keep current process as is; however, SGS should review its process and procedures for adjournment, reconvening, and outcomes for the second FOE for clarity. For example, SGS could consider in its revision of the SGS FOE Guidelines web page providing information on different options that can be discussed with students by supervisors and graduate faculty administrators in the event of an adjournment (e.g., preparing for a second FOE, process to appeal, withdrawal) and unsuccessful second exam (e.g., process to appeal, termination). (SGS Calendar: 8.3.12/Guidelines)
  • Principles: Promotes wellness for students.

Closed defense vs. public component
  • 2 models considered: Retain the current model (i.e., largely closed, with an option for GFM members able to attend) and revise the current model to formalize the option for a public component (i.e., open to colleagues, community members).
  • Recommendation: SGS should add language to the SGS Calendar to ensure graduate units are aware of the choice to include a public component in their final oral examinations, in addition to the required closed component. (SGS Calendar: 8.3.9)
  • Principles: Academic rigour, disciplinary and programmatic flexibility, wellness and exceptional experience for all stakeholders, sense of occasion, professional development, and public accountability and transparency.
Time length
  • 2 models considered: Keep the guidelines broad in relation to time length or state a normative time length for the exam.
  • Recommendation: SGS should explicitly state a normative length of time for the final oral examination. If a graduate unit chooses to have a public seminar as a component of the final oral examination, the public component will normally last up to a maximum of one hour. The closed examination component of the final oral examination will normally be two to three hours. (SGS Calendar)
  • Principles: Ensures some disciplinary and programmatic flexibility and equity, diversity and inclusion, while maintaining academic rigour, transparency, sense of occasion, wellness and an exceptional experience for all stakeholders.

Submit Your Feedback

FOE feedback