Procedures for ensuring that key assessments of candidate performance and evaluations of unit operations are fair, accurate, and free of bias.
Candidate Assessment
There are procedures in the Professional Education Unit (PEU) to ensure that assessment of candidate performance is fair, accurate, and free of bias. One mechanism for this assurance is that all programs work with the Director of Assessment Support to ensure that current best practices in assessment are followed. Rubrics used for all programs have been developed by faculty and assessment personnel, vetted by stakeholder groups, and refined as needed. The Director of Assessment support holds calibration exercises with faculty for high stakes exit rubrics.
Initial Programs
Midpoint assessments across the PEU are conducted using rubrics that specify criteria that must be met to achieve specific ratings. For teacher credential candidates, rubrics are used to assess embedded signature assignments, special education assignments, school counseling midpoint student artifact submissions, and course-embedded centerpiece assignments in the administrative services programs. Practicum students are evaluated on criteria appropriate for each discipline by trained supervisors. The criteria for evaluation are tied to California Commission on Teacher Credentialing standards. In addition, school counseling candidates’ evaluations are linked with standards set by the Council for Accreditation of Counseling and Related Educational Programs (CACREP). Rubrics used to assess candidates for administrative services credentials practicum respond to the standards of the Interstate Leaders Licensure Consortium (ISLLC) and the California Professional Standards for Educational Leaders (CPSEL).
There are several standardized exit assessments for initial teacher education programs at the elementary and secondary levels. Candidates must take and pass a standardized test, the California Subject Examinations for Teachers (CSET). Each candidate takes the test that is appropriate for his/her credential area. They must also successfully complete the Performance Assessment for California Teachers (PACT). The PACT teaching events are uploaded into TaskStream and evaluators conduct a blind review of candidate work. In addition, assignments are made to ensure that PACT evaluators do not know the candidate being evaluated. In addition, elementary and special education initial credential candidates must pass the Reading Instruction Competence Assessment (RICA), designed to ensure that California-trained candidates for Multiple Subject Teaching Credentials and Education Specialist Instruction Credentials (special education) possess the knowledge and skills important for the provision of effective reading instruction to students.
Advanced Programs
Masters level teacher education candidates (MAT and MEd) are evaluated at midpoint using rubrics to assess readiness to move to the thesis level or action research project. This formal assessment is embedded in a required course, EDUC 500.
At midpoint, school counseling credential and master of arts candidates are assessed using the Clinical Instruction Benchmark Assessment (CIBA). The CIBA includes faculty and candidate examination of multiple measures, including coursework and prepracticum assessments, to determine readiness for internship. They are assessed by experienced clinical supervisors and faculty members using standardized observation rubrics in their fieldwork that assess multiple criteria across these dimensions: relationship/attending skills, assessment skills, and professional skills.
School counseling candidates are also assessed at the exit point by multiple measures. Their fieldwork (internship) is assessed using the same criteria applied during prepracticum and practicum. In addition, school counseling candidates must pass a comprehensive examination that assesses key areas in their development within a detailed case study analysis and reflective essay regarding personal identity as school counselors. Candidates who require remediation in any of the facets of their evaluation are assigned a faculty mentor with whom they work until they meet the requirement. Beginning in 2011-2012, the action research project will be used as an exit requirement for master’s level students.
The cohort based preliminary and professional administrative services programs are able to provide ongoing assessment because the number of candidates in each cohort is small. As a culminating evaluation for the credential, administrative services candidates are assessed on their performance within a detailed portfolio project and their oral defense of the portfolio. The portfolio rubric is designed to assess performance on specified artifacts for the seven CPSEL standards.
Program/Unit Assessment
All courses in the School of Leadership and Education Sciences are evaluated using a standardized course evaluation. This process has always had provisions to ensure that they are fair, accurate, and free of bias. From Fall 2007 through Fall 2009, SOLES used the standardized course evaluation form produced by the University of Washington. On the last day of class, the instructor would leave the room and each student would complete a paper evaluation form that was then collected by one student and delivered to the Dean’s office. Paper course evaluations were then sent to the University of Washington for tabulating and reports were sent back that summarized the data by course and instructor, by department, and by the whole school.
Starting in Spring 2010, a new anonymous on-line course evaluation form was implemented for use across the PEU. The new form has questions that are specifically designed for the School of Leadership and Education Science. The course evaluation is delivered electronically using SurveyDig, a survey tool that links to course and student record information in USD’s centralized data system, Banner. Students access their course evaluations via their USD e-mail accounts. No student level data are available to anyone in the School Leadership and Education Sciences. Student comments to the previous course evaluations were hand written. With the new anonymous delivery system, students type in comments, which are presented in a summary table. This new format dramatically reduces the time lag between students taking the evaluation and instructors receiving their evaluation results, compared with the previous method. Student feedback has been positive about the ease of use and the ability to provide more considered responses. The course evaluation process is facilitated by the Office of Assessment Support.
All programs use exit surveys at the time of program completion to obtain information from recent graduates regarding how programs could be improved. (sample exit survey.) Exit surveys and follow up surveys are all managed by the Office of Assessment Support to ensure anonymity of respondents. Program completers are sent exit program assessment surveys electronically using the Qualtrics Survey tool and all responses are anonymous. The Director of Assessment support develops reports for faculty that present the results aggregated across all responders. Comments are either grouped into themes and presented collectively, or, if they are presented as written, any identifying characteristics are removed.
Likewise, follow-up surveys are conducted in an anonymous manner. Elementary, secondary and special education graduates from two cohorts were followed into the field through the PEU’s participation in the Comprehensive Evaluation of Teacher Preparation, conducted by the Center for Teacher Quality at Stanford. Recently, because of he cost of participation in the CTE survey and because of relatively low response rates, the decision was made to use an in-house follow-up surveys. These surveys are conducted by the Office of Assessment Support and results are brought to faculty meetings for review.