|
Diablo
Introductory
Guide for Assessment
of Student Learning Outcomes Fall 2005 Guide for Assessment
of Student Learning Outcomes The purpose of this document is to enhance understanding
and provide guidance to the faculty in assessing student learning outcomes at
·
Introduction
·
Philosophy
Statement
·
Guiding
Principles
·
Effective
Assessment
·
Assessment
Model
·
Accreditation
Standards
·
Assessment
Template Introduction However, there is more to be done in order to enhance
student learning and therefore improve students’ chances for success in either
the transfer or the vocational programs. Enhancing the quality of learning will
depend largely on the existence of an effective, systematic, and ongoing
process for assessing student learning outcomes. For the assessment process to be effective, it must be comprehensive (encompassing all academic
and educational support services of the college), participatory (inclusive of faculty, staff, administration and
students), realistic, flexible, and relevant to the educational and teaching environment of the
college. Furthermore, assessment of student learning outcomes must be dynamic, subject to continuous updating
and revision; and must adapt as the college itself evolves and changes. In 2002, the Leadership Council at DVC took the initiative
to begin a discussion of student learning outcomes. Two committees were established to develop
the guidelines and oversee the development and implementation of assessment of
student learning on the campus. Theses two committees include a college-wide
umbrella committee and a Faculty Senate committee. The Senate committee, which
began its work in 2003, has the responsibility of coordinating the
implementation of assessment of student learning outcomes at the instructional
level. On the other hand, the college-wide committee will oversee the
assessment of student learning for the institution as a whole including student
services and other educational support units. The following discussion is based
on the work of the Faculty Senate Committee. Philosophy
Statement Assessment at [Assessment]
involves making our expectations explicit and public; setting appropriate
criteria and high standards for learning quality; systematically gathering,
analyzing, and interpreting evidence to determine how well performance matches
those expectations and standards; and using the resulting information to
document, explain, and improve performance. When it is embedded effectively
within larger institutional systems, assessment can help us to focus our
collective attention, examine our assumptions, and create a shared academic
culture dedicated to assuring and improving the quality of higher education.
(Angelo, T., 1995, p.7) Assessment is important because it
·
helps
to demonstrate and improve student learning and student
success;
·
includes
systematic collection of data and information from courses, program, and the
institution;
·
is
an integral part of teaching and learning and is used to help both;
·
supports
·
focuses
on outcomes, but also includes processes, especially in seeking ongoing improvement;
and
·
is
needed for accreditation, accountability, and institutional effectiveness. The focus of the assessment effort is enhanced student
learning and a process by which program accomplishments may be identified and
validated. The faculty will drive this effort and use gathered data for
academic program improvement. Guiding
Principles
1.
Assessment
of student learning outcome processes at
2.
Assessment
efforts will be respectful of academic freedom, faculty responsibility for the
curriculum, and shared governance.
3.
Assessment
will be decentralized process that will build upon departmental assessment
efforts already in place, efforts that are directed by the faculty.
4.
Assessment
will include systematic collection of reliable data that will be used to
improve student performance.
5.
Assessment
results will be used to guide practices affecting student learning outcomes and
student success. It is a means to improve student learning; it is not an end in
itself.
6.
Assessment
results will be used for identifying strengths and shortcomings in academic
programs based on analysis and reflection by the faculty.
7.
Assessment
will be a process of ongoing dialogue and analysis of data that will result in
improved student success.
8.
Assessment
should be ongoing and institutionalized and become part of the accepted culture
of the college.
9.
Assessment
will not be used for any faculty evaluation.
10. Assessment will not be used for
any personnel or student evaluation. Effective
Assessment An effective assessment process consists of the following
five basic components:
The order in which these five components are listed may
create the impression of a linear process wherein one neatly-packaged set of
activities follows another in an established, orderly sequence. The reality is
that institutions are likely to be involved simultaneously in assessing
yesterday’s results, addressing today’s problems, and planning tomorrow’s
possibilities. The assessment process is therefore understood to be dynamic,
interactive and ever-changing. Assessment Model The assessment model adopted at DVC consists of five
elements that embody the above. Characteristics:
1.
Statement of institutional purpose
2.
Goals or expected learning
outcomes
3.
Assessment methods including the
criteria for judging success
4.
Assessment results
5.
Use of results for improvement 1. Statement of
Purpose: The statement of institutional
purpose is frequently identified as the mission statement and it should
describe in general terms the desired results of the educational programs, and should
also define the fundamental basis for evaluating institutional effectiveness.
In effect the statement should establish a general “blueprint” for the
development of processes for assessing and improving institutional
effectiveness and should also provide a framework for the formulation of more
specific goals, including expected student learning outcomes. Individual
divisions, departments, and programs may articulate their own statements of
purpose that express the institutional purpose at their own respective levels. 2. Goals or Expected Student Learning Outcomes While goal setting may occur at various levels of the institution, the
congruence of those goals is important, and the connecting theme is their
relationship to institutional purpose. Again, that relationship is critical
because it is through the evaluation of goal achievement that the institution’s
effectiveness in fulfilling its purpose will be judged. As a starting point, goal statements may be formulated at three
levels:
·
Institutional
level
·
Program
level
·
Course
level Institutional Level:
At the institutional level, goals may include the outcomes of general
education, the transfer program, and developmental education. With respect to
general education, the following outcomes may be may be considered. General education has comprehensive learning
outcomes for the students who
complete it, including the following: a. An understanding of the basic
content and methodology of the major areas of knowledge, including humanities
and fine arts, the natural sciences, and the social sciences. b. A capability to be a productive
individual and life long learner: skills include oral and written
communication, information competency, computer literacy, scientific and
quantitative reasoning, critical analysis/logical thinking, and the ability to
acquire knowledge through a variety of means. c. A recognition of what it means
to be an ethical human being and effective citizen: qualities include an
appreciation of ethical principles; civility and interpersonal skills; respect
for cultural diversity; historical and aesthetic sensitivity; and willingness
to assume civic, political, and social responsibilities locally, nationally,
and globally. Program Level: All degree programs include focused study in at least one area of inquiry or in an established
interdisciplinary core. Student learning outcomes at the program level should
focus on student competencies related to degrees and certificates approved by
the State of Course Level: Student learning outcomes should
also be set for each individual course. Multiple-section courses should share
the same learning outcomes regardless of the instructor (full-time vs.
part-time) or the length of time for offering the course (12 weeks vs. 18
weeks). Course Objectives and Student
Learning Outcomes (SLOs) It is important to distinguish between
course objectives and student learning outcomes. SLOs describe the broadest
goals for the class, one that requires higher level of thinking. Course Objectives
·
Objectives describe skills, tools or content that
students will master by the end of one course.
·
Objectives require the use of basic thinking skills
such as knowledge, comprehension and application.
·
Objectives do not necessarily result in a
product. Objectives are combined to
produce something that measures outcomes.
Student Learning Outcomes
·
Outcomes describe over-arching goals that students
will be able to demonstrate by the end of a course or program.
·
Outcomes require the use of higher level thinking
skills such as analysis, synthesis and evaluation. Students are asked to
synthesize many discrete skills or areas of content.
·
Outcomes result in a product that can be measured or
assessed. SLOs ask students to produce something--papers, projects, portfolios,
performances, or exams--that applies what they have learned and require the
faculty to evaluate or assess the product to measure a student’s mastery of
outcomes. Evaluation of SLOs It is important for the intended
educational outcomes to be consistent with the institutional mission or the
institutional purpose. Furthermore, these outcomes should be reasonable and
attainable given the ability of the students enrolled at the college. The following factors should be considered in evaluating statements of
student learning outcomes:
·
The statement of student learning outcome should be consistent with
the expanded statement of purpose or mission. Examples Statement of purpose /college
mission for To become one of the outstanding community colleges in the Bay Area Outcome:
Ø
Add
baccalaureate level programs- Not acceptable
Ø
Conduct
research regarding the environmental issues of concern to the community- Highly questionable
Ø
Graduates
will be able to identify current social problems – Yes
·
The statement should be reasonable given the ability of students. The
criteria for success and student achievement should not be set at levels far
beyond the ability of most students. Examples Students completing the general education program will score on the
average at the________ percentile on the ACT collegiate Assessment of Academic
Proficiency.
Ø
25th
percentile – No challenge
Ø
50th
percentile – a reasonable
stretch
Ø
80th
percentile – Sure
disappointment
·
Program outcome statement should target key concepts or change points
in each program. Guidelines
Ø
Program
faculty members may begin by brainstorming ideas related to student learning
outcomes.
Ø
Prepare
a long list of intended student learning outcomes.
Ø
Extract
from the long list those learning outcomes that are considered as priorities
for the current year.
Ø
Target
three to five concepts per program. The limitation to a smaller number reduces
the paper load, assessment efforts, and cost.
·
The statement should be clear and
accomplishment is ascertainable.
·
The statement should be singular.
·
The statement should be rotated
when validated. 3. Assessment
Methods The college should identify the
specific means of evaluating each of the goals set forth by various components
of the institution. In describing both expected student learning outcomes and
assessment procedures, the following conditions should be met:
·
Use
clear and explicit language with action verbs
·
Specify
appropriate time frames
·
Address
both minimum standards and targets for excellence
·
Designate
responsibility for implementing assessment and for assuring appropriate use of
assessment results. Assessment methods can be
indirect, using criteria that assume learning has taken place, or they
can be direct, that is using criteria that measure student learning directly.
Examples of indirect measures include grades, success rates, retention rates,
enrollment patterns, degrees and certificates awarded, placement, outcomes in
special programs, and student equity data. Direct student learning outcomes are
measures of competencies or attainment levels reached by students, i.e. skills,
abilities, knowledge. In evaluating student learning
outcomes in given program, one should examine the designed curriculum, the taught
curriculum, the learned curriculum,
bearing in mind that grades are not the best evidence of student learning. The
designed curriculum is what is in the college catalog and in official courses
outlines, the taught curriculum can be found in syllabi, and the learned
curriculum is what assessment is all about---what have students learned? What can
they do? What do they care about? Included among the methods commonly used to
assess student learning are:
·
capstone
experiences (research papers, ways that demonstrate attitudes
·
and
values, case studies)
·
classroom
assessment techniques
·
dance
productions, music productions
·
evaluation
by advisory councils
·
evaluation
by employers
·
exit
exams
·
exit
interviews
·
faculty-developed
rubrics for scoring student work
·
licensure
information
·
locally
developed tests
·
paintings,
drawings, newspaper articles, computer programs
·
portfolios
(student work is collected and reviewed for evidence of
·
learning
and development)
·
practicum
and internship evaluation
·
standardized
tests Regarding general education,
the starting point should be the examination of the goals and philosophy of
general education in the college’s catalog. The goals and objectives of course
outlines should guide the faculty in teaching the courses. The utility of goals
and objectives may serve as the basis for assessment. General education courses should
introduce students to the humanities and fine arts, the natural sciences, and
the social sciences and that the institution can provide evidence that students
know how to: investigate, analyze, use critical thinking skills, relate to and
negotiate with others, be sensitive to the values of others, develop a sense of
responsibility for actions, and develop and enthusiasm for learning on a
continuous basis. Student services should be part of the program
review process. These services should seriously examine their role as
contributors to student development and student learning. Some of the methods
used by student services might include:
The typical output measure of
student services include course completion rates, basic skills completion,
retention rates, persistence rates, graduation rates, transfer rates, success
after transfer, and job placement. 4. Assessment
Results Assessment results should be collected and analyzed. Discussion of the results should include all
the faculty and staff involved in the assessment process. 5. Use of Results An important component of this model is the clear communication of
pertinent assessment results to the institution’s academic constituents. Data
or information provided to or by each department should focus primarily upon
that unit’s operations and/or the accomplishments of its students. Furthermore,
there should be a commitment on the part of the institution to use the results
with a keen awareness of the extensive interdependency that may exist among the
college components. Accreditation
Standards The Accrediting Commission for Community and Junior
Colleges (ACCJC) revised the accreditation standards in 2002 and placed
emphasis on improving student learning outcomes. The approved standards of June 2002 state
that the primary purpose of an ACCJC-accredited institution is to foster
learning in its students. An effective institution ensures that its resources
and processes support student learning, continuously assess that learning, and
pursue institutional excellence and improvement. An effective institution
maintains an ongoing, self-reflective dialogue about its quality and
improvement. The new accreditation standards consist of four parts as follows: Standard I: Institutional |