Program Review

Why use professional standards of practice?
A standard to guide practice is an essential characteristic of any established profession. The Council for the Advancement of Standards in Higher Education (CAS) was founded in 1979 as a profession-wide entity to establish standards to guide practice by student affairs, student development, and student support service providers employed by institutions of higher learning. The standards, intended to describe quality practice, are developed through collaboration and consensus and are designed to apply broadly across institution types and sizes.

Program review using the CAS standards
Because CAS believes in the importance of self-assessment, the standards and guidelines are offered as criteria that can be used in multiple ways toward the goal of assuring and enhancing quality practice. CAS does not prescribe or proscribe ways of using the standards; they are intended to be tools for practitioners to use to improve practice. The CAS Self-Assessment Guides (SAGs) and CAS Program Review module in the Campus Labs Platform are examples of possible program review tools.

The most thorough and, perhaps, productive use of the standards involves a self-study process for program evaluation. This process involves others at the institution (and sometimes those external to it) in examining evidence to determine collectively whether the program is in compliance with the standards. Involvement of others serves several purposes; it ensures a broader and more objective perspective, increases knowledge and awareness of the program across the institution, and develops support for implementation of identified improvements. CAS recommends the several basic steps for implementing a program review based on a self-study model. The chart below provides an overview of the full model, and the following text details additional nuances of several steps:
 

1. Plan the Process
Map out steps for process, develop timeline, build buy-in with all stakeholders, and explicitly identify desired outcomes of the self-study
4. Conduct and Interpret Ratings using Evaluative Evidence
Clarify team’s rating criteria; employ a process for rating [small group, individual, staff]; negotiate rating differences; and manage group ratings
2. Assemble and Educate Team 
3-5 (program) to 8-10 (division) comprised of stakeholders including students; train team on self-assessment concepts and principles
5. Develop an Action Plan
Identify discrepancies, corrective action, and recommended steps (e.g., identify strengths, weaknesses, benchmarks, resources, timeframe)
3. Identify, Collect, and Review Evidence
Define what constitutes evidence; then gather, collect, manage, and review evidence
6. Prepare a Report
Identify audience for report(s); describe self-study, evidence gathering, rating process, evaluations, strengths, weaknesses, and action plan; draft executive summary
7. Close the Loop
Put action plans into practice; navigate politics and secure resources; identify barriers; and build buy-in to the program review results
 

Assemble and educate the team
Division and program leaders first need to define the reason for the project and the desired outcomes (e.g., accreditation processes, institutional program review cycle, internal goals and needs). The full staff of the program to be reviewed should be involved in the initial planning stage of the self-study process, including support staff members and knowledgeable students and faculty members when feasible. This approach provides opportunity for shared ownership in the process. This group can assemble available documentation, identify areas in particular need of examination, and provide insight to the review team.

A smaller team should be established to conduct the review. The chair of the team should not be from the area to be studied, to ensure greater objectivity and credibility. Generally the team includes a member from the area to be studied, along with other members with useful perspectives. Initially, the team should familiarize itself with the relevant CAS functional area standard by examining it carefully before making individual or group judgments. It is important that all members come to understand and interpret the standard in similar fashion. Team training should be conducted to ensure that members’ interpretive differences are resolved before initiating the study. Likewise, ground rules for the study should be established and agreed upon. Team members should realize and accept that disagreement is natural, healthy, and probably inevitable, but the resulting debates will usually strengthen the team’s understanding and ultimate consensus on the matter.

It is helpful to have relevant documents and evidence collected prior to initiating the review. While the team may identify additional information that is needed to complete the review, having basic documents from the functional area available at the outset will assist them in making progress.

Conduct and Interpret Ratings using Evaluative Evidence
Using the available documentation, team members review the standards criteria statements and assign a rating to each one, using the scale provided to reflect degree of compliance (from Does Not Meet to Exceed). Generally, team members do this individually and then meet to compare ratings, discuss and resolve discrepancies, and finalize their collective evaluation.

Another approach that can be used is to have the entire team work together to review criteria, examine documentation, and reach agreement about the ratings. There are advantages (e.g., different perspectives are shared from the beginning, building team understanding) and disadvantages (e.g., more full-group time required, minority viewpoints may not be articulated) to this approach.

Documentation must be identified to support each area evaluated. Essential documentation includes relevant publications (e.g., student and staff handbooks), program descriptions (e.g., workshop or training outlines), program assessment results (e.g., evaluation summaries), institutional data (e.g., student profiles), and research initiated for the self-study (e.g., student survey or focus group results). No review can be considered complete without relevant data and related documentation to support and validate the team’s judgments. It is particularly important that data includes assessment of student learning and development outcomes.

Following the rating procedure, it is helpful for the study team to invite the full staff to review and discuss the team’s initial assessment of program compliance with the standards. Staff members may be able to provide additional insight or data to assist in finalizing the evaluation. This approach also increases ownership of the results by the staff that will be affected by them.

Develop an action plan
In summarizing the results, the team should identify those areas that were determined not to be in compliance with the standards and describe in detail the adjustments that need to be made. Additionally, they should suggest priorities for the actions to be taken, reflecting the areas most in need of attention.

The results should also reflect the areas that are judged to be exemplary or exceeding the standards. In addition to identifying areas for improvement, this process should identify areas of strength that can be highlighted and further enhanced.

Prepare a report
In most cases, the program review results are then shared with the staff in the program for follow up, as well as being submitted to the appropriate unit relevant to the purpose of the study (e.g., unit supervisor, division leader, institutional effectiveness office). While the program review team can identify the areas in need of attention, and can suggest the changes needed, the program staff need to consider the results in light of program mission, goals, and resources. Program staff can then develop their own comprehensive action plan, based on the results, to establish priorities, set a timetable for addressing deficiencies, assign responsibilities, and identify measures to assess outcomes.


Further information about CAS, the standards, and the self-assessment approach can be found at www.cas.edu and in our primary publication, CAS Professional Standards For Higher Education (9th ed.).
 



The Campus Labs partnership with the Council for the Advancement of Standards in Higher Education (CAS) provides the resources for a more holistic program review process.
 
With the addition of the Campus Labs Program Review tools to the already invaluable CAS materials, you’ll be able to coordinate a meaningful and informative process that is dynamic and moves your assessment practice forward.

-  The entire library of CAS standards and self-assessment guides is built right into one system, where an unlimited number of users can access them. New and updated standards are made available the moment they are released by CAS.
-  Choose the applicable set of standards and assign them to the appropriate department or office. Incorporate additional fields for institution-specific assessment processes.
-  Organize, track, and review everything through a central online location. You’ll have the reassurance that comes with a consistent process, but still benefit from the flexibility of immediate access.
-  Manage workflows with ease and clarity by customizing templates, leveraging annual assessment and planning efforts, and identifying areas for improvement.
-  Build upon your existing processes by making the most of the data you already have. In addition to our user-friendly management tools, you’ll be able to link your assessment data and connect it back to your program review.
-  Access the CAS Do-It-Yourself Guide and work directly with a Campus Labs Assessment Consultant for expert assistance launching a program review process and assessment culture.


Discover more about becoming a Campus Labs + CAS Program Review Member and visit the Campus Labs website to schedule a demo.