State Policy Analysis Framework

 
Project Overview
State Self-Study Tools
State and Regional Policies
Assessment Policy Types and Models
Policy Development
Inventory of Instruments and Measurements
Data Collection and Analysis
Publications and Presentations

 


A review of the substantial amount of information received from the states regarding their various assessment policies and practices led to the development of the conceptual framework to facilitate an analysis for each state. The framework’s broadest function is to provide a picture of the inputs, processes, outcomes, and impacts of each state or system’s assessment policy. Within several of these overarching categories, however, the framework also requires answers to several specific questions.


This framework and its categories are presented here to offer one means by which state policymakers could examine their own state assessment policy, or that of others. The framework also permits a comparative analysis of policies across states so commonalties, differences, trends, and patterns can be discerned.

 



Original Initiative and Authority

For the purposes of this analysis, any assessment activity initiated by a state legislature, state (higher) education executive agency, or state college/university system governing or coordinating board is considered public policy. No policy is, however, in fact, a policy; it is a policy decision not to act.

We also noted how each state’s assessment policy was initiated. Often, a state’s assessment policy can have combinations of these means of initiation, or will have a history in which these forms of authority/initiation are related.

Legislative Authority

Statutory action requiring compliance, typically the collection and reporting of data

Intervention designed to place pressure on institutions, such as establishing a Task Force or Commission to study higher education

SHEEO Authority

State agency adopts policy encompassing one or more of the six policy types described below

Statewide planning and the type of state higher education agency

Gubernatorial Authority

Requiring State Higher Education Executive Agency to take action, such as collect data, improve quality, study institutions, etc.

Placing public pressure on higher education to adopt reforms

State System Authority

Policy is self-initiated by state public higher education system to respond to governmental pressure or to avoid further intervention.

 

Top

 


 

Policy Context

The policy context consists of four elements: historical, political, economic, and social inputs.

Historical Inputs

The perceived need(s) for assessment, if any, and prior policies, if any, which address that need

Political Inputs

The political circumstances in the state that led to the adoption of the policy

The entities that influenced the policy content

Quality of relationships among prominent policy actors

Public opinion regarding higher education

Economic Inputs

Financial or other budgetary concerns that led to the development of a policy

Economic needs/concerns of the state

Expenditures/Appropriations for higher education

Social Inputs

Demographic concerns, such as the need for college educated citizens, or the shift or loss in population across the state

 

 

Top

 


Policy Type

Regulatory

The policy is designed to encourage/ensure compliance with regulations; resources may be distributed, in part, based on successful compliance.

Reforming

The policy is designed to encourage/ensure reform of some type.

Quality Assurance

A policy focus that typically emphasizes assessment practices which seek to improve quality or at least provide the public some assurance that quality is a priority for state policymakers.

Accountability

Designed to make institutions accountable to some higher authority, typically the governor and state legislature. each college and university is held responsible by an external authority for reporting achievement of particular standards of performance, such as the governor or the legislature. Typically, the authority defines minimal levels of performance, or the institution and the authority agree upon targets/goals. The targets are typically defined at the level of the individual institution.

Institutional Improvement

Refers to improving educational programs, institutional management, and teaching and learning. Assessment becomes a means of achieving the goal of improvement.

Student Learning

In this policy schema colleges and universities are required to demonstrate student-learning gains. The institution needs to have students exhibit levels of performance on either measures of general skills and competencies or on tests of specific knowledge related to general education and/or major field curricula.

 

Top

 


 

Policy Stage

Our policy process model for higher education identifies five stages in the development process for state assessment policies:

Problem Formation

The period when the need for a state-level assessment policy was first recognized

Policy Formulation

Development of pertinent and acceptable proposed courses of action for dealing with public problems

Policy Adoption

Development of support for a specific proposal such that the policy is legitimized or authorized

Policy Implementation

Application of the policy to the problem

Policy Evaluation

Attempt by the state to determine whether the policy has been effective

 

Top

 


State Guidelines

What actions, assessment activities, reporting tasks, etc. are required/mandated by a state entity?

 


 

Programs/Positions

What programs will be/have been created to carry out the policy?

Who is responsible for data collection, analysis, and reporting?

What entities continually monitor assessment activity?

What entities periodically review and evaluate the policy and its elements?

 

Top

 


Indicators and Outcomes

What common indicators for assessment and/or outcomes are mandated across all institutions?

To what extent do indicators/outcomes vary across institutional types and sectors?

How were these indicators established?

What criteria are used for evaluating achievement of success?

 


 

Instruments/Methods

What common instruments to assess learning outcomes are mandated across all institutions?

To what extent do instruments/assessment methods vary across institutional types and sectors?

What instruments are nationally-normed and which are locally developed?

What instruments and assessment methods are currently in use?

 

Top

 


 


Teaching-Learning Elements

What specific elements of the assessment policy are focused on faculty instruction and/or student learning?



Public Reporting

How are results/outcomes from assessment activity shared amongst the prominent policy actors in the state?

What is the state’s pattern for sharing assessment information with the public?

How effective has the policy been at enabling policymakers to gather, organize, and analyze information from institutions?


Top


Database

What form of state-level computerized database exists?

1. Comprehensive statewide database at the SHEEO level
2. Non-comprehensive statewide database at the SHEEO level
3. Multi-institutional database, not at SHEEO level
4. Limited multi-institutional database
5. No multi-institutional database

How is the database structured?

What elements are collected and used in the database?



Budget

What financial inducements are incorporated in the policy design, or accompanied by budgetary provisions, to foster institutional compliance with the policy?

How much, if any, state funds are provided to institutions to offset the costs of adopting assessment practices?

Our research has revealed a trend toward four approaches that states use to finance and otherwise provide incentives to colleges and universities to implement assessment activities. They are:

1. Performance-based funding
2. Budget lines for assessment activities
3. Student fees for assessment purposes
4. Reimbursements to colleges and universities for assessment activities


Top

 


Regional Accreditation Association Relationship

What are the criteria and approaches set forth for institutional accreditation regarding assessment and the requirements?

What are the methods and processes for assessment that the association prescribes for institutions, including reporting, testing, and data collection?

What types of institutional support are offered to campuses as they implement these requirements, including training, resources, and services?

What is the association’s relationship with state agencies, including the influence of the association on institutions, and the coordination, cooperation, communication between the association and state higher education agencies?

Has there been a formal evaluation of the association's criteria and standards, including a review and reflection upon its effectiveness in fostering assessment activity at the institutional level?


Top

 


Disciplinary Accreditation

To what extent is disciplinary accreditation used as a means of assessment?

To what extent can the questions for the regional associations be used to examine relationships at the disciplinary/program level?

 

 

 


Technology Focus

In what ways are institutions evaluated on the extent to which they incorporate new methods for delivery of instruction into their assessment programs?

 

Top

On this page

State Policy Framework

Initiative and Authority


Policy Context


Policy Type


Policy Stage


State Guidelines


Programs/ Positions


Indicators and Outcomes


Instruments/ Methods


Teaching-Learning Elements


Public Reporting


DataBase


Budget


Regional Accreditation

Association Relationship


Disciplinary Accreditation


Technology Focus

 

 
   
© 2003, National Center for Postsecondary Improvement, headquartered at the
Stanford Institute for Higher Education Research