Archive for the 'Test' Category

Problems with Institutional Assessment

Assessment dominates education from K-12 through college.   There are different types of assessment, formative (helps students improve) and summative (grading of students).  However, institutional assessment  involves the bigger picture of how an institution or a department is doing academically.

In institutional assessment, teachers enter data into a  mega-database. For example, teachers may enter their students’ grades  on each section of the final. Then someone, often a department head,  analyzes the overall results using the online data, to assess the student learning across specific courses and across the department.

Institutional assessment has some basic flows
1) Most institutions have not identified a specific  enough curriculum that can be assessed.  Many contain very general statements of learning.  For example, English might state that  students will write a well-written essay. Has the English department specified what constitutes a well-written essay?  Likewise, a Modern language department may have the curriculum statement  “The student should speak in sentences that have relatively simple structures and concrete vocabulary”.  What does “speak” mean?  Does it mean to be able to talk about one’s life, to hold a conversation. to repeat from memory?  When there are only general  learning statements, there cannot be any  meaningful assessment.

2) If departments have identified specific learning goals, what is the priority of those learning goals? For example, in English the purpose of writing is to communicate ideas or feelings.  Shouldn’t the organization of ideas be more important than the spelling?  Or does spelling/grammar have the same assessment weight as organization?  Likewise,  in  Modern Languages, are all skills (listening, speaking, reading and writing) treated equally in assessment weighting even though both in class and in the real world, people listen and speak almost double the amount that they read and write?  Have the specific learning goals and their priority been communicated to the teachers/students through a department website/wiki?

3) The departments do not have exemplars that show the quality that they expect of students.  Does the English department share  electronically with all English teachers essays that show what constitutes a high level paper,  an acceptable paper, and a non-acceptable paper?  Again, are these exemplars on the department website for each course?  Does the Modern Language department share audio files of  a good ten sentence conversation through their website or an their department app?

4) They have vague assessment tools.  The English department has a generic rubric (has good organization,  conveys ideas, etc.) that can be interpreted differently by different people.   What type of essay will be the written? An autobiographical essay requires a very different approach than a contrast essay.  In Modern Languages, how will writing be assessed – holistically or analytically?  If different educators can come up with different scores for the same student, then the assessment tool does not accurately measure learning.  Teachers can receive a digital image of the rubric and work assessed using that rubric.   How well does the assessment tool match up with how the information was taught in class?  Is the assessment tool such as the final developed  at the  competency level or at the highly competent level?  Students may be competent but not highly competent

5) The departments do not do a thorough analysis to get at the root problem once they have discovered a gap.   If the students do not achieve well, was it due to the  students’ lack of effort, a misunderstanding of  how to answer the  assessment question, a specific word in the  assessment question,   the thinking level of the test question,    the structure of the assessment item,  the textbook, the textbook’s powerpoints,  the teacher’s explanation, the homework, or  the online work?  Usually much additional exploration is needed to determine the real reason for the gap. Once the  department identifies the gap, what  specific strategy will help the students over come this gap?  Will the department suggest  technology-based strategies that appeal to students such as Youtube videos, interactive websites,  interactive apps  and that help the students directly overcome the gap?

6) Most important of all, how does the institutional assessment help  students improve in the course right now?  Most institutions assess once a semester.  After the analysis, the department  focuses on  what changes will happen in the future year.  Unless regular assessment is done in small intervals  throughout the year and changes made almost instantly, then the assessment does not benefit  the present students.  Next year’s students may be very different than the students who took this assessment.  Classroom teachers need access to the online data and analysis so they can take class time to provide  the students new learning strategies.  Then, students can be successful learners!

How does your institution assess  student learning?

My Spanish spontaneous speaking activities (20+) includes Modified Speed Dating (Students ask  a question from a card-whole class), Structured Speaking (Students substitute in or select words to communicate in pairs),  Role Playing (Students talk as people in pictures or drawing from 2-4 people) and Speaking Mats (Can talk using a wide variety of nouns, verbs and adjectives to express their ideas- pairs or small group),  Spontaneous Speaking (based on visuals or topics in pairs),  and Grammar speaking games (pairs or small group). Available for a nominal fee at Teacherspayteachers:  http://bit.ly/tpthtuttle

My three formative assessment books:   http://is.gd/tbook

Advertisements

Students Paired Oral Testing Better Than With Examiner Modern Language

Based on Brooks, L. (2009). “Interacting in pairs in a test of oral proficiency; co-constructing a better performance”. Language Testing 26(3): 341-366.

Brooks’  research shows that students who are tested in pairs outperform students who are tested one-on-one with the examiner.  In addition, the students’ interactions were more complex and revealed that students co-constructed a more linguistically demanding performance. In addition, when students worked in pairs, they more closely resembled the oral interactions typical of a real conversation.  In paired testing students demonstrated a wider range of interactions (17) to the individual format (10).   The paired students mostly commonly had these interactions: seeking confirmation, asking a question, asking for agreement, clarification requests, and prompting elaboration, finishing sentences, and referring to partner’s ideas.  Over half of all interactions in the one-on-one with the examiner was asking a question.

As Modern Language teachers, we will want to encourage oral communication in the classroom.  We can have our students do more oral work in pairs.  We can structure students speaking  from very basic conversations up to free-flowing spontaneous conversations about common topics. Our scaffolding will allow our Second Language students to have more complex and personally meaningful conversations.

Most of the  Spanish activities I have developed are for pairs. A few of them are

Spanish Tell Me About Yourself Substitution Sentences    (Partners substitute in their own answers to tell about themselves

Spanish Conversation Questions Spontaneous Speaking Partners  (Partners ask basic questions and then variations on those questions)
Spanish Friend /Family Member Detailed Description – Partner Talk   (Each partner talks about a family member using possible words)

My Spanish spontaneous speaking activities (20+) includes Modified Speed Dating (Students ask  a question from a card-whole class), Structured Speaking (Students substitute in or select words to communicate in pairs),  Role Playing (Students talk as people in pictures or drawing from 2-4 people) and Speaking Mats (Can talk using a wide variety of nouns, verbs and adjectives to express their ideas- pairs or small group),  Spontaneous Speaking (based on visuals or topics in pairs),  and Grammar speaking games (pairs or small group). Available for a nominal fee at Teacherspayteachers:  http://bit.ly/tpthtuttle

My three formative assessment books:   http://is.gd/tbook

 

Final In the Course What is it really?

We are within a few weeks of  finals.  Some good questions to ask are   What is a final?  What learning do we want the students to  show on the final?

A history teacher  tells his students that the final is on  Chapters 1-15 and all they have to do is know that information. Obviously, the students become overwhelmed because they do not know what is really important in the chapters.  They do not know the format of the final – multiple guess or essay writing? They have no idea of how to study for the final.

Teachers give paper and pencil finals, scantron finals or online computer scored.

Let’s look at some possible types of  written final (not project based):

Wikipedia  defines a final as a big unit test.  The final covers the same material that has been previously covered and in the same way but covers more of it in one exam.  An American History final is just  parts of previous tests;  instead of 50 questions, the students have 200 questions.

– Some teachers give a put-it-all-together test in which students have to integrate what they have learned during the course.  For example, an 8th grade  Science final involves students reading and critiquing an experiment on the health of a local stream.  They have covered everything previously in individual sections such as  the biological or physical aspects but they have not had to go to the big picture of the whole stream.

– Some teachers create a final that consists of  the final improvement on previous work.  For example, students have revised a Contrast essay previously in English class and they do a final revision as  their final.

– Some teaches create  a final that goes  far beyond what the students  have learned in class. The final  includes brand-new material such as many vocabulary words  the students have never seen and it may ask  them to do tasks that they have never done before in class. For example, in class students have only answered literal questions on  reading passages  but the final has mostly inference reading questions.   The final generally does not test the regular forms but focuses on all the irregular forms or exceptions.  Only the A+++ students might pass this final.

Some questions about a final:

Do the  students know what precise learning goals will be on the final? Do they know which learning goals are the most important for success in the final?
Do they know how these learning  goals  will be tested such as multiple guess or try to fill in the blank?
Does the final reflect the same level of learning as  done during the course?
Do the students have a sample final that mimics the final  both in content and format?  Do they have an online practice that explains the wrong answers?
Is the final an opportunity for the students to show how much they have learned?

What type final do you give?

I have 20 Spanish spontaneous speaking activities at Teacherspayteachers:  http://bit.ly/tpthtuttle

My formative assessment books:   http://is.gd/tbook

Having Students Go from Proficient to Above Proficient Through Improvements

In my Oral presentations (speech) class, I’m grading their final speeches on how much they have improved from when they originally gave the speech. They have to show me their original speech, the rubric in which I indicated their strengths and gaps, and a sheet which explains how they are overcoming their gaps. Their final (two speeches that they select from those they have done) are graded on improvement.  If they show the three  improvements, they get an A. For each learning gap that is not changed into a strength, they loose ten points.  So far students have shown drastic improvements, their speeches have gone from being below proficient or being proficient to being above proficient. They have learned to support their speeches with image-based PowerPoints that drive home their messages. When we raise the bar and prove ways for students to improve, they go over the bar!

How do you have your students improve and become above proficient?

My book, Formative Assessment: Responding to Students, is available through Eye-on-Education.

Reponding to Your Students

Improving Students’ Self-Assessments Skills for Increased Learning

Part of the handout for the conference session:

Reasons for student self- assessment

Closed- ended assessment

Some examples:

Check answers against a paper or digital “answer” key.

Take online quiz.

Transitional assessment

Some examples:

Take online tests until ready for “real” test

Learn the quality in an assignment

Open -ended assessment

Some examples:

Self-assess and change strategy if necessary

Digital portfolio updates

Students' Self Assessment Growth Chart

Students

For any one who is interested in implementing formative assessment in the classroom, my book,
Formative Assessment: Responding to Students is available through Eye-on-Education.

Finals Do Not Reflect Standards

How well does your final match up with your standards and particularly those standards which you identified as being critical for the course?

A colleague shared with me  a  course final which focuses on a topic not mentioned in the course proficiencies (standards).   Since it was her first time in teaching the course, she had not studied the final.  When she did, she wondered where it came from.  She went back through the course proficiencies again and still could not find the topic.  Then she went to the textbook and searched it for the topic; it was not there. Someone had decided that the final had to be a certain topic which was neither in the standards nor the textbook.

How well does your final match up with the specified standards and the high level of thinking in those standards?  Does your final measure all the standards and all of their parts?  Does it measure some of the standards and even just some of those learning goals?

Embedded Testing or Random Testing

A colleague was sharing a humorous story about assessment. The students in his college have to take a post-writing test to show their improvement from their first writing at the college. They have to write an essay based on the same five topics.

However, the irony is that the students who are in his business writing course which is the final English course for the students are the ones being asked to write the post-writing. In the business course, they write business letters according to very strict formats.

Having them write an essay in the business class is like assessing a pizza maker on how well he writes checks. The pizza maker certainly writes checks but those checks are a very small percent of his/her time. The pizza maker spends most of the time in making pizza. Maybe the pizza maker can be observed as he/she makes pizzas just as our students can be assessed doing regular classroom writing in their essay class, not in their business class.

How and when do you assess students? Are they doing things that they would normally do as part of your class or is the assessment a random event outside of their normal tasks?


RSS Education with Technology

  • Tech Integration Teacher, What time is it? August 23, 2016
    When someone asks what time it is, that person wants to know the time, not the history of the clock, not how a clock works, and not what other types of clocks there are. Classroom teachers want to help their students improve their academic learning through technology. Sometimes they need help with technology so they go […]
    hgtuttle
  • Curriculum Focus, Not Technology Focus July 28, 2016
    In my public school career I have been a classroom teacher, a technology integration specialist and a technology administrator. In my technology role, I served under the Assistant Superintendent for Instruction. She had a simple mission: Improve students’ academic learning. My mission was equally simple: Improve students’ academic learning through technology […]
    hgtuttle
  • Students React to Digital Badges: Pros, Cons and Interesting June 22, 2016
      ISTE 2016 By Harry Grover Tuttle, Ed. D. College World Language Students’ Preferences Digital Badges – 52%        Paper Certificates – 48% World Language: Can-Do Digital Badges Digital Badges Pro- – Breaks down proficiency more – Shows all badges at once – Is more attractive – Is more appropriate since we use […]
    hgtuttle
  • Digital Badges: Naming the Badge October 29, 2015
    Once teachers have selected what learning and what digital badges (individual or category badges; see previous blog), the teachers encounter another decision. What will they name each badge? Will they use the full name of the Common Core Standard or the national proficiency? For English, under “Speaking and Listening,”will they write out SL.2 “Integrate and […]
    hgtuttle
  • Digital Badges: Better Than Grades? October 19, 2015
    Teachers understand that the grade in a course consists of many different factors such as homework, participation , projects, tests, etc. Blodget observes that sometimes grades reflect attitude, effort, ability and behavior (http://www.academia.edu/9074119/Grading_and_Whether_or_not_Grades_Accurately_Reflect_Student_Achievement). Equally important, a letter […]
    hgtuttle
  • World Language Students Use of Mobile Devices in the Classroom October 5, 2015
    Do world language students use technology n the classroom? Do their  teachers go beyond having their students use technology simply for the drill and practice in vocabulary and grammar? Students can use laptops and mobile devices to hear authentic language, read authentic texts, read tweets about famous performers, see up-to-the-moment culture,  watch video […]
    hgtuttle
  • Digital Badges: Individual or Categorized Learning Badges? September 12, 2015
    The idea of digital badges sounds appealing for the digital children in classes. As teachers start thinking about digital badges, they have to figure out what badges will be awarded. The teachers can award social or academic badges. If teachers decide to use academic badges, then the teachers may base their badges on the Common […]
    hgtuttle
  • English +Common Core +Mobile = Success (ISTE2014 Poster -details) June 30, 2014
    Here are the ten examples I showed at my English + Common Core  + Mobile ISTE 2014 Poster Session: Based on CCSS Anchor Statements: L.2 Take a Conventions Mobile Online Quiz  to pick the  incorrect sentence from four choices (capitalization) SL.2  Evaluate audio recording of a  book chapter on mobile and predict for next chapter. […]
    hgtuttle
  • Global Cultural Learning Using Mobile Devices (ISTE Mobile MegaShare Presentation) June 28, 2014
    Based on my presentation at ISTE 2014 Mobile Megashare Why teach about other countries? Location: Large view to small on maps. Culture or culture. Find six similarities in a  mobile picture from another culture (“Wars are caused by differences, not similarities.”-Tuttle.) Tell one piece of information from each different Internet visual from a place in that […]
    hgtuttle
  • English + Common Core + Mobile = Success in Learning Poster Session at ISTE 2014 June 25, 2014
    In my ISTE Sunday 8-10 am poster session, I demonstrate many diverse mobile activities to help students achieve the English Language Arts Common Core Anchor Statements through mobile devices. The mobile activities focus on free common tool apps that are available on both the Android and the iPad. The students use the apps as a seamless […]
    hgtuttle

Blog Stats

  • 779,653 hits
Advertisements