Program Review

Why use professional standards of practice?
A standard to guide practice is an essential characteristic of any established profession. The Council for the Advancement of Standards in Higher Education (CAS) was founded in 1979 as a profession-wide entity to establish standards to guide practice by student affairs, student development, and student support service providers employed by institutions of higher learning. The standards, intended to describe quality practice, are developed through collaboration and consensus and are designed to apply broadly across institution types and sizes.

Program review using the CAS standards
Because CAS believes in the importance of self-assessment, the standards and guidelines are offered as criteria that can be used in multiple ways toward the goal of assuring and enhancing quality practice. CAS does not prescribe or proscribe ways of using the standards; they are intended to be tools for practitioners to use to improve practice. The CAS Self-Assessment Guides (SAGs) and CAS Program Review module in the Campus Labs Platform are examples of possible program review tools.

The most thorough and, perhaps, productive use of the standards involves a self-study process for program evaluation. This process involves others at the institution (and sometimes those external to it) in examining evidence to determine collectively whether the program is in compliance with the standards. Involvement of others serves several purposes; it ensures a broader and more objective perspective, increases knowledge and awareness of the program across the institution, and develops support for implementation of identified improvements. CAS recommends the several basic steps for implementing a program review based on a self-study model. The chart below provides an overview of the full model, and the following text details additional nuances of several steps:
 
1. Plan the Process
Map out steps for process, develop timeline, build buy-in with all stakeholders, and explicitly identify desired outcomes of the self-study
4. Conduct and Interpret Ratings using Evaluative Evidence
Clarify team’s rating criteria; employ a process for rating [small group, individual, staff]; negotiate rating differences; and manage group ratings
2. Assemble and Educate Team 
3-5 (program) to 8-10 (division) comprised of stakeholders including students; train team on self-assessment concepts and principles
5. Develop an Action Plan
Identify discrepancies, corrective action, and recommended steps (e.g., identify strengths, weaknesses, benchmarks, resources, timeframe)
3. Identify, Collect, and Review Evidence
Define what constitutes evidence; then gather, collect, manage, and review evidence
6. Prepare a Report
Identify audience for report(s); describe self-study, evidence gathering, rating process, evaluations, strengths, weaknesses, and action plan; draft executive summary
7. Close the Loop
Put action plans into practice; navigate politics and secure resources; identify barriers; and build buy-in to the program review results
 

Assemble and educate the team
Division and program leaders first need to define the reason for the project and the desired outcomes (e.g., accreditation processes, institutional program review cycle, internal goals and needs). The full staff of the program to be reviewed should be involved in the initial planning stage of the self-study process, including support staff members and knowledgeable students and faculty members when feasible. This approach provides opportunity for shared ownership in the process. This group can assemble available documentation, identify areas in particular need of examination, and provide insight to the review team.

A smaller team should be established to conduct the review. The chair of the team should not be from the area to be studied, to ensure greater objectivity and credibility. Generally the team includes a member from the area to be studied, along with other members with useful perspectives. Initially, the team should familiarize itself with the relevant CAS functional area standard by examining it carefully before making individual or group judgments. It is important that all members come to understand and interpret the standard in similar fashion. Team training should be conducted to ensure that members’ interpretive differences are resolved before initiating the study. Likewise, ground rules for the study should be established and agreed upon. Team members should realize and accept that disagreement is natural, healthy, and probably inevitable, but the resulting debates will usually strengthen the team’s understanding and ultimate consensus on the matter.

It is helpful to have relevant documents and evidence collected prior to initiating the review. While the team may identify additional information that is needed to complete the review, having basic documents from the functional area available at the outset will assist them in making progress.

Conduct and Interpret Ratings using Evaluative Evidence
Using the available documentation, team members review the standards criteria statements and assign a rating to each one, using the scale provided to reflect degree of compliance (from Does Not Meet to Exceed). Generally, team members do this individually and then meet to compare ratings, discuss and resolve discrepancies, and finalize their collective evaluation.

Another approach that can be used is to have the entire team work together to review criteria, examine documentation, and reach agreement about the ratings. There are advantages (e.g., different perspectives are shared from the beginning, building team understanding) and disadvantages (e.g., more full-group time required, minority viewpoints may not be articulated) to this approach.

Documentation must be identified to support each area evaluated. Essential documentation includes relevant publications (e.g., student and staff handbooks), program descriptions (e.g., workshop or training outlines), program assessment results (e.g., evaluation summaries), institutional data (e.g., student profiles), and research initiated for the self-study (e.g., student survey or focus group results). No review can be considered complete without relevant data and related documentation to support and validate the team’s judgments. It is particularly important that data includes assessment of student learning and development outcomes.

Following the rating procedure, it is helpful for the study team to invite the full staff to review and discuss the team’s initial assessment of program compliance with the standards. Staff members may be able to provide additional insight or data to assist in finalizing the evaluation. This approach also increases ownership of the results by the staff that will be affected by them.

Develop an action plan
In summarizing the results, the team should identify those areas that were determined not to be in compliance with the standards and describe in detail the adjustments that need to be made. Additionally, they should suggest priorities for the actions to be taken, reflecting the areas most in need of attention.

The results should also reflect the areas that are judged to be exemplary or exceeding the standards. In addition to identifying areas for improvement, this process should identify areas of strength that can be highlighted and further enhanced.

Prepare a report
In most cases, the program review results are then shared with the staff in the program for follow up, as well as being submitted to the appropriate unit relevant to the purpose of the study (e.g., unit supervisor, division leader, institutional effectiveness office). While the program review team can identify the areas in need of attention, and can suggest the changes needed, the program staff need to consider the results in light of program mission, goals, and resources. Program staff can then develop their own comprehensive action plan, based on the results, to establish priorities, set a timetable for addressing deficiencies, assign responsibilities, and identify measures to assess outcomes.


Further information about CAS, the standards, and the self-assessment approach can be found at www.cas.edu and in our primary publication, CAS Professional Standards For Higher Education (9th ed.).
 

Setting a Ne
w Standard in Partnership
In an exciting, evolving partnership, CAS and Campus Labs are working together to promote the use of data, assessment and self-study for the continued development of quality programs and services that promote learning, development, and achievement across all elements of higher education.

"The mission of CAS is the promotion of professional standards and the assessment of those standards to ensure quality in education endeavors. Through our Campus Labs relationship, CAS can evolve our mission by offering more technically enhanced solutions to support campuses as they answer critical questions around quality and effectiveness," said Marybeth Drechsler Sharp, Executive Director for CAS.

As part of this relationship, CAS materials such as self-assessment guides are available through the Campus Labs platform, in addition to being available in their present form through CAS. Campus Labs and CAS will be collaborating on new products and enhancements to each respective platform.

"We are excited to begin offering the self-assessment guides through our program review tools to help campus administrators build quality programs that support student learning and success,” said Eric Reich, co-founder and Senior Vice President of Campus Labs. “While this initial offering will be a significant advancement for the profession, CAS and Campus Labs are most looking forward to how we can work together to create new, even more powerful tools that can continue to advance assessment practice.” 

If you are looking for additional resources or support for a CAS-Campus Labs Program Review, check out the CAS Resource Center and select the folder for "CAS-Campus Labs User Resources."


img