Evaluation
of Classroom Learning, BER 550
Spring
2003
3
Credit Hours
INSTRUCTOR:
Dr. Ann Godfrey
OFFICE: 313B Carmichael
OFFICE HOURS: Monday 4:00 - 5:45, Tuesday 2:00 - 3:45
PHONE: 348-7582 (office);
348-0683 (fax); 348-7575 (secretary)
E-mail: agodfrey@bama.ua.edu;
CATALOGUE DESCRIPTION
A comprehensive approach to the
development and application of objectives, teacher-made tests, published tests,
and other measures used in the evaluation of teaching and learning in the
classroom.
CONCEPTUAL FRAMEWORK
Experiences in academic programs are devoted to developing
individuals' understanding of knowledge construction, learning, pedagogy, and
responsible professional practice in the contexts of education. The
University of Alabama's College of Education seeks to prepare professionals who
value and demonstrate reflective practice and ethical decision making
through respecting diversity, honoring difference, and promoting social
justice.
COURSE OVERVIEW
This course is designed as an introduction
to the basic concepts of tests and measurements for classroom teachers. Prerequisites for this course are
satisfactory completion of MA 109 and BEP 205.
Objectives for this course are based upon the 1990 National Council in
Measurement, National Education Association, and the American Federation of
Teachers "Standards for Teacher Competence in Educational Assessment of
Students." Prospective teachers
will be skilled in the understanding of how to choose, develop, administer,
score and interpret external and teacher-produced assessment methods, utilize
grading procedures, communicate assessment results, and recognize unethical and
illegal assessment procedures and use of information. Emphasis will be on the understanding and
basic application of these standards.
KNOWLEDGE BASE
BER 550 will address reflective practice
and ethical decision making. In
constructing teacher made assessments the students of this course will respect
diversity, honor differences, and promote social justice. The students will learn how to score and
communicate assessment results in an ethical manner.
COURSE METHODS
Class instruction will involve lecture
and the use of an overhead and PowerPoint presentations. Computer grading will be demonstrated in the
computer lab allowing hands-on application.
Students will also do supervised class work involving statistical
analyses of tests.
COURSE OBJECTIVES
NOTE:
All of the following objectives must be met to receive course credit for
BER 550. All Objectives are tied to the
teachers/prospective teacher=s field of study. It is assumed that for the development,
implementation, and outcome of any assessment that the contextual elements of
the classroom are taken into consideration.
This assumes that the prospective teacher understands the cultural,
gender, exceptional populations, and ethnic diversity of her/his students. All objectives are to be interpreted for use
in the teachers/prospective teacher=s content area and/or interdisciplinary
studies.
1. (2)(e)1.(ii) Educators will have the knowledge of how
to select, construct, and use a variety of appropriate assessment strategies
(Projects 1, 2, and mid-term exam).
2. (2)(c)1.(iv) Educators will have the knowledge of when
and how to adjust plans based on student responses and other contingencies.
3. Educators
will be able to develop multiple choice, essay, matching, true/false items to
measure instructional objectives (Project 2 and mid-term exam).
4. (2)(e)1.(i) Educators will have the knowledge of
characteristics, uses, advantages, and limitations of different types of
assessments. (Projects 1, 2, and
mid-term exam).
5. Educators
will know how to develop a variety of instruments that will monitor student
learning in five target areas (knowledge, reasoning [thinking], skills
[behaviors], products,
and affective) for implementing informal
assessment, paper-pencil measures, and performance assessment.
6. Educators
will be able to calculate and interpret basic statistics used to establish
properties of test scores, e.g., measures of central tendency and measures of
variability (Project 4 and Clinical).
7. (2)(e)1.(iii) Educators will have the knowledge of
measurement-related issues such as validity, reliability, norms, bias, scoring
concerns, and ethical uses of tests and test results.
8. Educators
will be able to interpret and calculate a correlation coefficient as it is used
to calculate reliability, validity, and inter-rater agreement indices (
Mid-term and final exams).
9. Educators
will be able to calculate and interpret the standard error of measurement
(Final exam).
10. Educators
will be able to calculate and interpret item analysis procedures for paper
pencil tests with dichotomous data.
Basic procedures include item difficulty and item discrimination (Final
exam).
11. Educators
will be able to interpret means, standard deviations and item to total
correlations for use in item analysis for continuous data (Final exam).
12. Educators
will utilize various cognitive, affective, and psychomotor methods to assess
Higher Order Thinking Skills (HOTS) (Projects 1, 2, 3, and mid-term exam).
13. Educators
will be able to develop performance assessment instruments for classroom
use. Instrument development will include
weights or rating scales for observations, performance products, writing and
portfolio development (Project 3).
14. Educators
will be able to score paper pencil and performance assessment instruments
(Projects 2, 3, mid-term). Prospective
teachers will implement scoring procedures that are teacher evaluated and student
evaluated.
15. Educators
will understand and interpret how norms are constructed for individual and
class use (Final exam).
16. Educators
will interpret the basic psychometric properties of reliability, validity, and
utility of teacher made and standardized tests for use in the classroom.
17. Educators
will be able to interpret standardized test scores to students, parents, and
other educational personnel (Final exam).
18. Educators
will be able to utilize standardized test results to design curriculum at the
class and individual level.
19. Educators
will develop a personal grading philosophy (Project 4).
20. Educators
will know how to implement a grading model consistent with their grading
philosophy (Project 4). They will be
able to interpret grading information to students, parents, and administrators.
21. Educators
will be able to utilize teacher made assessments of classroom learning in order
to adapt curriculum for maximum student learning (Projects 1, 2, 3, and
mid-term exam).
22. Educators
will recognize ethical and legal assessment procedures and uses of assessment
information (Projects 1, 2, 3, 4, and mid-term exams).
23. Educators
will integrate assessment with curriculum (Project 1 and mid-term exam) in
planning, presenting, and assessment.
24. Educators
will apply measurement/assessment principles and strategies in a clinical setting
for the purpose of evaluating student learning.
Assessment activities must be totally integrated with the content and
purpose of the teaching/learning environment. (Projects 2, 3, and mid-term
exam).
25. Educators
will apply the research in assessment, motivation, and learning to measuring
student learning in their respective disciplines (Midterm and final exams).
26. Educators
will incorporate the use of computer for assessment purposes. Items and performance measures can be stored
and quickly edited with the use of word processing. Calculations of central tendency,
variability, and standard scores are to be accomplished using a calculator. Grading is accomplished using a computer
program. Students will be able to do
these things by hand and on the computer (Project IV, clinical, and final
examination)
27. Educators
will be able to apply and reflect on their assessment practices integrated with
curriculum, and understanding of students= learning to adjust teaching and improve
the teaching-learning process.
28. (2)(e)1.(iv) Educators will have the knowledge of how
to evaluate one=s performance as a teacher.
REQUIRED TEXT
Kubiszyn, T., & Borich, G.
(2000). Educational testing
and measurement: Classroom application and practice (6th ed.). New York: John Wiley & Sons, Inc.
COURSE REQUIREMENTS FOR PRACTICUM/CLINICAL COMPONENT
All students registered for this course must complete the clinical
experience. They must also attend a
meeting organized by The University of Alabama Clinical Experiences office
before starting the clinical practicum.
PLEASE BE AWARE THAT THE PRACTICUM IS TO BE DONE IN A K-12 SETTING.
The clinical experience is designed to provide students an opportunity
to apply measurement principles and techniques in a real classroom testing
situation. It enables students to
better understand the reflective model of planning, delivery, and
assessment. The 40-point clinical
project is an extension of either project II or project III depending on your
agreement with your instructor and the classroom teacher. The clinical project consists of three
parts: (1) a log of your observation of a unit taught in the class, (2)
paper-pencil test items or a performance instrument you have developed to
measure pupils' mastery of the unit, and (3) follow up statistical analyses,
and (4) reflective analysis. This
project involves knowledge construction, learning, responsible professional
practice, reflective practice, ethical decision making, and diversity from the
Conceptual Framework of the College of Education, The University of Alabama.
A ten-hour practicum with a minimum five-hour classroom observation of
the unit being taught by the classroom teacher is required. This should be scheduled as early in the
semester as possible. However, the
actual observation cannot occur till after the instruction on paper-pencil
tests or performance assessment has occurred.
Students should seek agreement and suggestions from the classroom
teacher as to when, what to observe, and what project best suits the classroom
needs. A log of instructional events,
materials, and activities is required for observation. The Agreement Form for Teacher and Student
should be completed and turned in with the projects. You and
your classroom teacher should decide which project to develop. Projects should be constructed according to
the requirements specified in the BER 550 class.
Using the classroom assessment data, students are required to conduct
a statistical analysis of the results of their clinical assessment
experience. Whether you choose to do
paper-pencil items or a performance assessment, you must include measures of
central tendency and variability, standard scores, item analysis, and
interpretation of the results. A
reflective analysis includes a brief summary of your clinical experience, a
paragraph reflecting on this experience that includes lessons learned, and
recommendations for future use of your instrument (items). This analysis is required as the last
component.
Flexibility is allowed for the due date of the clinical project in
order to accommodate the various classroom assignments. However, all project components are due
before the last day for tests as listed on the current university calendar.
PROJECT 1:
Prospective teachers will write a unit table of specifications (UTS)
using at least two of the following achievement target areas; knowledge,
reasoning, skills, products, and
affective. Methods of assessment and cognitive levels
will be identified. A matching
curriculum will be designed using behavioral objectives. (Knowledge
construction, learning)
PROJECT 2:
Prospective teachers will construct a small unit paper-pencil test
consisting of multiple choice, true/false, matching, completion, and essay test
items. (Knowledge construction,
learning, reflective practice, diversity)
PROJECT 3:
Prospective teachers will construct a performance measure. (Knowledge
construction, learning, reflective practice, diversity)
PROJECT 4:
Prospective teachers will construct a grading model based on individual
philosophy of grading and apply it to a set of class grades provided by the
instructor. (Knowledge construction, learning, reflective practice, diversity,
ethical decision making)
GRADE BOOK
You will be expected to keep a current
grade book (spreadsheet). Grades and
corrections to the grade book will be provided throughout the semester. You must keep the spreadsheet current as I
will make periodic checks throughout the semester. (Knowledge construction)
QUALITY STANDARDS
No handwritten work will be accepted
except the design of the unit table of specifications or test blueprint. All work must be typewritten or completed
with a word processing program.
Projects
are to reflect a professional level. I will deduct points for grammatical
mistakes. Projects are to be turned in
on the assigned date. LATE PROJECTS
WILL NOT BE ACCEPTED.
The Graduate College grading scale will
be used. Students are responsible for maintaining
records of their course standing. The
instructor will assume the responsibility to communicate course standing for
only the final grade.
90 - 100% A
80 - 89%
B
70 - 79%
C
60 B 69%
D
59 and below F.
PROJECT POINTS ALLOCATION GRADING
MODEL
Project 1
40 points Class
Projects 160 points
Project 2
40 points Clinical
Project
55 points
Project 3
40 points Grade Book 25 points
Project 4
40 points Mid-term 100 points
Clinical Project 55 points Final
Exam 100 points
Projects Total: 215 points Worksheet 1
30 points
Worksheet 2 50 points
Presentation 20
points
TOTAL 540
points
POINT ACCUMULATION FOR GRADE ASSIGNMENT
486 - 540 points = A
432 - 485 points = B
378 - 431 points = C
324 - 377 points = D
323 points and below = F
ATTENDANCE
Students will be expected to attend all
scheduled sessions of the course.
Missing one session is equivalent to missing almost a complete
topic. The course is very cumulative,
and losing an entire unit would almost certainly result in a serious loss of continuity
for the student. Documented illness,
family emergency, official conflicts, and the like will be treated as excused
absences, but it is ultimately the students' responsibility to maintain their
standing in the course. WARNING:
ASSIGNMENTS WILL NOT BE TAKEN LATE. If
you miss a class, please get the notes from another student. Make‑up exams will be given only if
accompanied by a valid documented excuse before the exam. The student is responsible for contacting
the instructor in the event of a missed exam.
ACADEMIC MISCONDUCT
Any act of dishonesty will constitute
academic misconduct. The Academic
Misconduct Disciplinary Policy will be followed in the event of academic
misconduct. The use of ready made
assessment materials or duplication of past or present projects for
assignments and tests will be considered a breach of academic misconduct
policy.
REASONABLE ACCOMMODATIONS
It is the policy of the College of
Education to make reasonable accommodations for qualified individuals with
disabilities. A student requesting
these accommodations should be registered with the Office of Disability
Services, 348-4205. If you are a person
with a disability and desire accommodations to complete course requirements, in
compliance with the Office of Disability Services, please notify the course
instructor in writing as soon as possible to discuss your request.
Students in need of reasonable
accommodations relative to class attendance or arrival, course requirements, or
related aspects of performance are to initiate such requests with the
instructor prior to their anticipated need.
Such requests will be accommodated within constraints of fairness and
timeliness with regard for all other students enrolled for the course.
EQUAL TREATMENT
The instructor and students in this
course will act with integrity and strive to engage in equitable verbal and
non-verbal behavior with respect to differences arising from age, gender, race,
physical ability, and religious preferences.
CLASS
SCHEDULE
The following represents a tentative
schedule, specific dates and assignments may be changed at the instructor=s
discretion.
DATE TOPIC
ASSIGNMENT
1/14 Introduction
to Assessment Chapters
1, 2, & 3
Professional Responsibilities and Ethics
1/21 Behavioral
Objectives/Cognitive Levels Chapters
4 & 5
Integrating Content and Assessment
Unit Table of Specifications (UTS) and
Test Blueprint
1/28 Paper
Pencil Assessments (PP) Chapter
6
Item Writing: Objective Items
2/4 Project
1 due Article
II
Item Writing: Subjective Items Chapter 7
Restricted and Extended Response
Validity, Reliability, and Correlation Chapters 14, 15,
& 16
2/11 Review
Project 2 due
2/18 MIDTERM
EXAMINATION
2/25 Performance
Assessments (PA) Chapters
9 & 10
Constructing Scoring Rubrics Articles
III and IV
Higher Order Thinking Skills (HOTS)
3/4 Statistics:
Introduction Chapters
12 & 13
Measures of Central Tendency and Variability
Percentiles
3/11 Administering,
Analyzing, and Improving the Test Chapter
8
The Normal Distribution Chapter
13
Statistics: Standard Scores Article I
Worksheet 1 due
Grading and Reporting Student Progress Chapter 11
Computer Grading Articles V
and VI
Project 3 due
3/18 Computer
lab
Grade Book will be
completed in class and turned in
Worksheet 2 due
3/25 Project
4 due
Types of Standardized Tests Chapter
19
Finding Published Tests Article VII
Interpreting Standardized Achievement
Tests Chapters 17 & 18
Accuracy and Error
4/1 Special
Education Chapters
20 & 21
4/15 Catch up and review for test
4/22 Test
4/29 Clinical
Project due
Presentation*
*Present an overview of your clinical
experience. Without giving the name of
the teacher and the school location of the clinical assignment, critique the
teacher according to topics discussed in class.
The following articles may be of interest
to you. The articles follow certain
topics that we discuss in class.
ARTICLES
I ericae.net/pare/getvn.asp?v=4&n=10
II ericae.net/pare/getvn.asp?v=4&n=6
III ericae.net/pare/getvn.asp?v=4&n=3
IV ericae.net/pare/getvn.asp?v=6&n=2
V ericae.net/pare/getvn.asp?v=4&n=8
VI ericae.net/pare/getvn.asp?v=6&n=5
VII ericae.net/pare/getvn.asp?v=4&n=2
Project
I (UTS)
SSN:
Topic /2
Subject /2
Grade Level /2
Target Areas (at least two) /5
Methods of Assessment /4
Cognitive Levels (Bloom) /5
Objectives (at least 5) /5
Test Blueprint /15
Total /40
The first seven items must be word
processed. The table of specifications
(test blueprint) may be done by
hand. The objectives must be complete
sentences. Each objective must contain
only one action. Please turn this rubric
in with you project. Please do not
write your name on the project. If your name is anywhere on the project, I will
deduct 2 points. Please do not
place pages in plastic sleeves, I will deduct 2 points if you do so. This is a professional class. Grading will reflect the level of
professionalism that is expected.
BER
550 Project #2
Kindergarten
- 2nd Grade
SSN:
Test /8
Layout (Student Copy)
Layout (Teacher Copy)
Lettering
Vocabulary Level
Appropriate Use of Item Types /22
True/False (2)
Matching (1 set)
Multiple Choice (2)
Completion (1)
Cognitive Level for Each Item* /6
Scoring Key /4
Marked Teacher Copy
Breakdown of Point Values
Total /40
*This should be on a separate sheet of
paper from the test.
BER
550 Project #2
Grade
3 and Up
SSN:
Test /5
Layout
Vocabulary Level
General Test Directions
Appropriate Use of Item Types (adherence
to guidelines) /20
True/False (2)
Matching (1 set)
Multiple Choice (3)
Completion (2)
Essay (1)
Appropriate Use of Directions for Types
(adherence to guidelines) /5
Cognitive Level Identification
(Bloom) for Each Type Item* /5
Scoring Key /5
Marked Teacher Copy
Model Answer
Breakdown of point values
Total /40
*This should be on a separate sheet of
paper from the test.
BER
550 Project 3
Performance
Assessment
SSN:
Title of Unit /1
Grade Level /1
Objective(s)
Specific project objective(s) /4
Cognitive Level (Bloom) /4
Context
Performance to be evaluated /4
(Subject, type of performance
assessment, and target areas)
Method
Performance to be rated by /3
(Type of rubric, portfolio, audio, or
video tape)
Performance to be evaluated by /3
(Teacher, peer, other)
Criteria: Standards to judge proficiency /10
(Teacher rubric, point value)
Student Instructions /5
(Instructions given to the student)
Student Grading/Score Sheet /5
(On same page as instructions to the
student, points)
Total /40
BER
550
Project
#4
SSN:
Personal Grading Philosophy
A. Grading Model /6
1. Criterion or Norm Referenced
2. Advantages/Disadvantages
B. Elements of Performance Incorporated /2
(e.g., tests, homework, performance assessment, etc.)
C. Components incorporated in grade specified /2
(e.g., achievement, participation, behavior, etc.)
D.
How elements will be combined or weighted for /3
six weeks grade
E.
How
elements will be combined or weighted for /3
for semester grade
F.
Meaning
of letter grade /3
G.
How grades will be distributed /2
H. Provision for special circumstances /2
(e.g., sickness, borderline cases, etc.)
Assignment of Grades
A. Term 3 grade /7
B. Semester grade /7
C. Copies of spreadsheets /3
Total /40
COURSE
REQUIREMENTS FOR PRACTICUM/CLINICAL COMPONENT
BER
550 EVALUATION OF CLASSROOM LEARNING
OBJECTIVE
Prospective teachers will apply measurement/assessment principles and strategies in a clinical setting for the purpose of evaluating student learning. Assessment activities must be totally integrated with the content and purpose of the teaching/learning environment.
BER 550
The clinical experience is designed to provide students an opportunity to apply measurement/assessment principles and strategies in a clinical setting for the purpose of evaluating student learning. It enables students to better understand the reflective model of planning, delivery, and assessment. The 40-point clinical project is an extension of either project 2 or project 3 depending on your agreement with the classroom teacher. The clinical project consists of three parts: (1) a log of your observation of a unit taught in the class, (2) paper-pencil test items or a performance instrument you have developed to measure pupils' mastery of the unit, and (3) follow up item analyses.
A ten-hour practicum with a minimum five-hour observation of the unit being taught by the classroom teacher is required. This should be scheduled as early in the semester as possible. However, the actual observation cannot occur till after the instruction on paper-pencil tests or performance assessment has occurred. Students should seek agreement and suggestions from the classroom teacher as to when, what to observe, and what project best suits the classroom needs. A log of instructional events, materials, and activities is required for observation. The Agreement Form for Teacher and Student should be completed and turned in with the projects. The first component of this clinical project is really your project 2 (PP) or project 3 (PA). You and your classroom teacher should decide which project to develop. Projects should be constructed according to the requirements specified in the BER 550 project grading sheet.
Using the classroom assessment data, students are required to conduct an item analysis. Whether you choose to do paper-pencil items or a performance assessment, you must do an item analysis. This analysis is required as the last component.
Flexibility is allowed for the due date of the clinical project in order to accommodate the various classroom assignments. However, all project components are due before you start working on project 4.
If you are already teaching in a higher-ed classroom, you may use your class for the clinical. However, you must observe the required five hours in a K-12 classroom.
BER 550 CLINICAL COMPONENT - Grading Criteria
SSN:
Log of Clinical Experience
Annotated Log /10
Instructional Materials
Objectives
Assessment Instrument
Teacher Copy /10
Student Copy
Scoring Key or Rubric
Adherence to sound principles of construction
Statistical Analysis
Measures of Central Tendency /25
Measures of Variability
Standard Scores (z scores and percentiles of each score)
Item Analysis*
Interpretation of Results
Reflective Analysis
Brief Summary of Clinical Experience /10
Reflection on Clinical Experience
Appropriateness/Usefulness of the assessment instrument
Lessons learned
Recommendations
TOTAL
CLINICAL PROJECT
/55
* For the item analysis do a difficulty index and a discrimination index for each objective question. Include in the interpretation what you could do to make any of your items better based on the item analysis.
Agreement
Form For Teacher and Prospective Student
Student is permitted to come to my
(subject matter) class in grade at
(school). She/he will be observing an instructional unit on
(unit/module topic) for a minimum of five hours beginning
(date) and ending (date).
The student will be responsible for developing (write in number) paper-pencil test
items to be administered separately or integrated into one of your tests, or a performance
instrument based on the unit she/he will observe. I will provide the student with instructional
materials and objectives for the unit of study. I agree to facilitate this clinical activity under the
condition that the student's observation, assessment, item-analysis fit into my schedule. I
understand that I may determine whether or not to incorporate the assessment activity into my
students' grades.
(Signature of the teacher) Date
(Signature of the prospective teacher) Date
This practicum experience (check one) is or is not part of another clinical experience.
January, 2003
Dear Cooperating Teacher:
Thank you for assisting us in the practicum for BER 550. Students taking BER 550, Evaluation of Classroom Learning at The University of Alabama are required to incorporate a clinical component into their regular course curriculum. They are to spend approximately 5 hours observing or assisting you in teaching a unit/module in your classroom. Students are to construct an assessment instrument, either a few paper-pencil test items or a performance assessment based upon instruction. The data from this assessment activity will enable students to conduct a follow up item analysis and an assessment evaluation.
For BER 550 you and your practicum student have a choice of content and one of the following two assessment projects. The two projects are: 1) paper‑pencil items (3 - 5 for primary grades; 5 - 8 for intermediate grades; and approximately 10 for secondary grades) and 2) a performance assessment. An analysis of the items and/or the performance assessment is required. You do not have to count these items or performance measure for students' grades. Paper-pencil items may be incorporated into one of your tests. You may drop the items from your test and/or not count the performance measure for students' grades. If you so choose, you may count the assessment(s) for students' grades.
The clinical experience will give the students a valuable opportunity to apply measurement techniques to a real classroom testing situation. The observation and assessment will be done at a time convenient for you with your permission. Please complete the Agreement Form for teacher and student. Your cooperation and support in this process are highly appreciated. If you have any questions, please feel free to contact me at (205) 348-7582.
Sincerely,
Ann Godfrey, Ph.D.
Assistant Professor
References
American Federation of Teachers, National Council on Measurement in Education, & National Education Association. (1990). Standards for teacher competence in educational assessment of students. Educational Measurement: Issues and Practice, 9(4), 30-32.
Crooks, T. J. (1988). The Impact of classroom evaluation on students. Review of Educational Research, 58, 438-481.
Stiggins, R., & Conklin, N. (1992). In teacher=s hands: Investigating the practice of classroom assessment. Albany, NY: SUNY Press.
Wise, A. E. (ed.). (1996). Quality teaching for the 21st century. A special issue of Phi Delta Kappan, 78, 190-224.