Problems with Institutional Assessment

Assessment dominates education from K-12 through college.   There are different types of assessment, formative (helps students improve) and summative (grading of students).  However, institutional assessment  involves the bigger picture of how an institution or a department is doing academically.

In institutional assessment, teachers enter data into a  mega-database. For example, teachers may enter their students’ grades  on each section of the final. Then someone, often a department head,  analyzes the overall results using the online data, to assess the student learning across specific courses and across the department.

Institutional assessment has some basic flows
1) Most institutions have not identified a specific  enough curriculum that can be assessed.  Many contain very general statements of learning.  For example, English might state that  students will write a well-written essay. Has the English department specified what constitutes a well-written essay?  Likewise, a Modern language department may have the curriculum statement  “The student should speak in sentences that have relatively simple structures and concrete vocabulary”.  What does “speak” mean?  Does it mean to be able to talk about one’s life, to hold a conversation. to repeat from memory?  When there are only general  learning statements, there cannot be any  meaningful assessment.

2) If departments have identified specific learning goals, what is the priority of those learning goals? For example, in English the purpose of writing is to communicate ideas or feelings.  Shouldn’t the organization of ideas be more important than the spelling?  Or does spelling/grammar have the same assessment weight as organization?  Likewise,  in  Modern Languages, are all skills (listening, speaking, reading and writing) treated equally in assessment weighting even though both in class and in the real world, people listen and speak almost double the amount that they read and write?  Have the specific learning goals and their priority been communicated to the teachers/students through a department website/wiki?

3) The departments do not have exemplars that show the quality that they expect of students.  Does the English department share  electronically with all English teachers essays that show what constitutes a high level paper,  an acceptable paper, and a non-acceptable paper?  Again, are these exemplars on the department website for each course?  Does the Modern Language department share audio files of  a good ten sentence conversation through their website or an their department app?

4) They have vague assessment tools.  The English department has a generic rubric (has good organization,  conveys ideas, etc.) that can be interpreted differently by different people.   What type of essay will be the written? An autobiographical essay requires a very different approach than a contrast essay.  In Modern Languages, how will writing be assessed – holistically or analytically?  If different educators can come up with different scores for the same student, then the assessment tool does not accurately measure learning.  Teachers can receive a digital image of the rubric and work assessed using that rubric.   How well does the assessment tool match up with how the information was taught in class?  Is the assessment tool such as the final developed  at the  competency level or at the highly competent level?  Students may be competent but not highly competent

5) The departments do not do a thorough analysis to get at the root problem once they have discovered a gap.   If the students do not achieve well, was it due to the  students’ lack of effort, a misunderstanding of  how to answer the  assessment question, a specific word in the  assessment question,   the thinking level of the test question,    the structure of the assessment item,  the textbook, the textbook’s powerpoints,  the teacher’s explanation, the homework, or  the online work?  Usually much additional exploration is needed to determine the real reason for the gap. Once the  department identifies the gap, what  specific strategy will help the students over come this gap?  Will the department suggest  technology-based strategies that appeal to students such as Youtube videos, interactive websites,  interactive apps  and that help the students directly overcome the gap?

6) Most important of all, how does the institutional assessment help  students improve in the course right now?  Most institutions assess once a semester.  After the analysis, the department  focuses on  what changes will happen in the future year.  Unless regular assessment is done in small intervals  throughout the year and changes made almost instantly, then the assessment does not benefit  the present students.  Next year’s students may be very different than the students who took this assessment.  Classroom teachers need access to the online data and analysis so they can take class time to provide  the students new learning strategies.  Then, students can be successful learners!

How does your institution assess  student learning?

My Spanish spontaneous speaking activities (20+) includes Modified Speed Dating (Students ask  a question from a card-whole class), Structured Speaking (Students substitute in or select words to communicate in pairs),  Role Playing (Students talk as people in pictures or drawing from 2-4 people) and Speaking Mats (Can talk using a wide variety of nouns, verbs and adjectives to express their ideas- pairs or small group),  Spontaneous Speaking (based on visuals or topics in pairs),  and Grammar speaking games (pairs or small group). Available for a nominal fee at Teacherspayteachers:  http://bit.ly/tpthtuttle

My three formative assessment books:   http://is.gd/tbook

Advertisement

No Real Destination No Real Learning

I enjoy “talking” with other teachers.  However, I am often amazed at how  they  describe the final learning destination.

A writing teacher may say, “We are doing contrast essays”, a modern language teacher says, “I want my students to speak the language”, a social studies teacher says “My students will be citizens of the world.”  What does each of these mean in a practical sense? How will these teachers know if their  students have reached the final destination  and how well they have done it?  Would another educator be able to identify the  same success?

I have been working in developing some foreign language assessments so that I can quantify the various types of speaking and how well students do on each.  I do not want to say “My students speak the language”  which is a vague concept but to specify what type speaking they do  and how well they do each. Once I can identify what the end learning looks like/sounds like, I can work backward to provide scaffolded  steps to help the students get there at a high level. In addition, I can build in formative assessment to monitor their upward process.

In order for athletes to win, they visualize exactly what they will do; they visualize their success.  Our students need to visualize exactly what they are to do.

How do you identify the end learning for the students so they can become successful at it?  How do they climb the ladder of success in that learning goal?

My  book,  Successful Student Writing Through Formative Assessment, is available through Eye on Education.

My book,  Formative Assessment: Responding to Your Students, is available through Eye on Education.

Aligning activities and end goal

When a  colleague was teaching a remedial English course, she was told to focus on the creative aspects of writing.   Those beginning college students needed to able to write academic  papers for college, not to write poetry and personal narratives.  They would  become good academic writers by focusing on the standard forms of academic writing  of writing such as contrast and argument papers.

Although this is an extreme example, I wonder how often teachers forget the real purpose of the students’  learning and have their students spend time doing something that does not directly lead to learning that goal.    A question to ask  is “How does this directly and efficiently lead students to being proficient in this skill?”  I remember watching a social studies class where  ninth grade students were coloring in the countries of South America, each country in different color.  The real purpose was to learn where each country was on the map but they spent about a third of the class on coloring.    Locating  countries on a map is a memorization activity so do memorization activities such as study a map with the names shown for  half the countries  and then , see a map without the names and put the names in.  Likewise, in a modern language class, students do not improve in their speaking ability by doing grammar drills. A student can be proficient in grammar but not  be a proficient speaker.  Students become more skillful in speaking as they speak more in class through structured activities.  Likewise, English students do not become better writers by just reading and answering questions about long literary essays.  When educators take students through the writing process with a focus on the specific type of writing, then these students become better writers.

So how does what your students do directly and efficiently lead them  to being proficient in a specific skill?

My  book,  Successful Student Writing Through Formative Assessment, is available through Eye on Education.

My book,  Formative Assessment: Responding to Your Students, is available through Eye on Education.

Looking Ahead For Better Learning

I attended my every three year Defensive Driving Course to get a reduction in my insurance. The AAA instructor and the DVD said that we should always be looking 20-30 seconds ahead on the road or about a third of a mile forward so we can be prepared for what is ahead.

I wonder how often we take our eyes off of our current learning to remind ourselves and our students of what is ahead, the standard. It is too easy to get focused on the moment so that we forget where we are really headed. By being focused only on the present activity, we may not connect our present activity into the bigger picture. The present activity may not seem to serve any purpose except when seen in the bigger picture. When students know where they are headed, they are more likely to get there and to be able to assess their progress. As we check what is ahead, we can help modify our instruction to make sure our students get there.

How do you help your students to see the standard or the big concepts of the year?

For any one who is interested in implementing formative assessment in the classroom, my book, Formative Assessment: Responding to Students is available through Eye-on-Education.

Learning goals curriculum or textbook illogic

I’ve been examining a writing/grammar textbook and I’ve noticed that there seems to be no logical learning flow in the writing patterns that students do.   When we show students that one type of writing is similar to another type, they can more easily make the transition.  For example, if students have done a narrative writing, they can easily transition to process writing.  Both types require a time line or sequence of events. Both types usually have the events in a chronological order.

How do you arrange your learning goals so that students transition from one type of learning to a similar type of learning?

Quickly Find Power Points for a Learning Topic

I do like to visually guide my students through a learning goal by creating Power Points but it takes me a long time to create them.  I’ve been using another method, finding an existing Power Point on that learning goal and then adding my own Power Point  for any missing points or things I want to emphasize.  An easy way to find Power Points is to put the category such as narrative writing in quotations “narrative writing” and add .ppt (the ending for Power Point files) so the search would look like “narrative writing” +.ptt.  A search for a Civil War Power Point would look like “Civil War” +.ppt while a search for a Power Point on the Three Little Pigs would appear as “The Three Little Pigs” +.ppt.

I found that within a few minutes of searching I can usually find a Power Point that captures much of what I want the students to learn. Then I create a mini-Power Point to add any additional information and I call it the topic plus “more” such as “NarrativeMore”.  I have cut my creation down drastically and often have a learning tool that is much better than I had thought of.

Changing the student grade paradigm

I taught an English honors class in high school where the constant question was “What is my grade?” The students were much more focused on their grades than on their learning.

How do we help students to focus on their learning and not their grades?  Even if we give them their grades for each homework, test, and project as soon as possible though an online program such as Blackboard, they still are focused on the grade.  To change the paradigm, we can use standards or goals checklists with the “grades” of AP -advanced,  P-proficient, B-basic, and S-starting. We  word process the checklist and give them a copy either physically or digitally.  Students can check off their goals as they achieve them. They can see their progression. We can conference with them about their progress and what they need to close the learning gap. These students know their “grades”.

How do help students to see their progress in learning goals?

Assessing How School’s Academic Priorities are Achieved Through Technology Use

By Harry Grover Tuttle, Ed. D.
Consultant
harry.g.tuttle at gmail.com
Blog: https://eduwithtechn.wordpress.com

NYSCATE Conference

Purpose: To improve students’ achievement of the school’s academic priorities through technology use by carefully analyzing how teachers and students use technology and improving those conditions to promote greater student learning.

School Technology and Learning Profile -Analysis:
Meta-Impressions from many schools visits based on collecting factual information, not stories.

Questions:
How does your local school support its academic priorities through technology use?

How do state assessment and state standards become translated into classroom technology-infused activities? How do you know?

What are some learning and technology issues that may not contribute to the school’s academic priorities?

How can you do make-overs to dramatically improve student standards-based technology-infused learning?

How can you support alternative standards (i.e. Visual literacy) through technology? How can you assess their success?

Sources:
Tuttle, H. G. (2007). Standards-Based Learning: Helping Students Achieve. Classroom Connect Connected Newsletter 14(4), 4-6.
Tuttle, H. G. (2007). Digital-Age Assessment. TechLearning. 27 (7), 22-24.
Tuttle, H. G. (2006). Creating a Tech-Infused Culture. TechLearning. 26 (11), 12-16.
Tuttle, H. G. (2005). Assessing How Schools’ Academic Priorities Are Achieved through Technology Use. Society for Information Technology & Teacher Education Proceedings, 5, 2860-2864.
Tuttle, H. G. (2004). Learning and Technology Assessments for Administrators. Ithaca, NY: Epilog Visions. (book of 54 ways of assessing learning and technology in school)

How do you support your school’s academic priorities through technology?

Does Your Classroom Technolgy Use Support School’s Academic Priorities-Part 2

Usually when I do a workshop or presentation on this topic, someone in the audience says something like, “We should not base everything we do in the classroom on state assessments.” I agree. However, whatever we define as our school’s academic priorities must be something we can assess in order to see student growth. The problem with education is that we have been fuzzy in our goals and therefore we have never known if we have reached the goals or not. If your school wants “global citizens,” how do you know how the students are progressing in this wonderful goal? If your school is a 21st Century School, how does it measure the students’ growth in these skills? If your school is a character school, how do you assess and improve the students growth? If you cannot define it, you do not have it as a priority! If you cannot assess it, you do not have it as a priority!

If you cannot define exactly what the “skill” is and you cannot assess it, then you never will be able to use technology to help students reach it!

When I taught Spanish, I wanted my students to be fluid speakers. I defined this as saying at least 15 sentences about a given topic in a minute or as having a conversation in which at least five different questions were asked about a topic with the other person answering those questions. Once I defined it, I could plan how to help students achieve it. I used technology of cameras, television, images, recordings to help them be successful. I measured them frequently to determine how I could help them to be better.

How do you define the priorities within your subject area? How do you use technology to help students achieve those specific priorities?

© Harry Grover Tuttle, 2007

—–