School Party
Screen 7 – Drinks—Lemonade or Water
By the beginning of Screen 7 with four items left in this PSI (2 items on this screen and 2 on Screen 8), the percent of not-reached students increased to 11 percent. Of the students that entered Screen 7 with assessment time remaining, 15 percent still omitted 7A.
Screen 7’s introductory information included: 1) the cases of drinks contain 24 bottles, and 2) 400 people will need 17 cases of drinks. 7A involved determining the smallest number of cases that would provide enough lemonade for 100 people. To provide their answers, the fourth grade students were asked to use a slider tool to drag a yellow arrow along a number line with 17 unit marks to indicate the 17 cases of drinks (but without any numbers). The slider tool was intended to be engaging and reduce the number of calculations necessary in 7B, where students were asked to determine the total cost of the drinks. However, the purpose of the slider tool may not have been clear to the students.
Item 7A
To solve 7A, students were expected to determine (one way or another) that 4 cases of 24 bottles of lemonade would not be enough lemonade for 100 people (because it is 4 bottles short). Then, if they had been using the typical eTIMSS response mode for a numerical response, the students would have used their number pad to enter 5 (which would have been recorded in the answer box). However, in this particular item students were asked to “drag the yellow arrow along the number line to show how many cases to buy.” The intention was for them to move the arrow 5 units, which automatically would make 5 appear in the box labelled “Cases of Lemonade” and 12 appear in the box labelled “Cases of Water.”
Click video to play
Maximum Score Points: 1
Content Domain: Number
Topic Area: Whole Numbers
Cognitive Domain: Reasoning
Results 7A
However, only 15 percent of the students on average across countries moved the arrow 5 units (Exhibit 12). The highest achievement, 25 to 29 percent correct responses, was in Hong Kong SAR, Norway, Singapore, Korea, and Chinese Taipei. Further analysis of response data indicated that some students (8% on average) moved the arrow four units (causing 4 to appear by the cases of lemonade and 13 to appear by the cases of water). It may be that the students moving the arrow four units just did not round up to 5 cases for some reason. On average across countries, boys had higher percentages of correct responses than girls.
Based on a cursory look at the process data, it appears that the fourth grade students did not actually understand that the slider tool was the way to submit their answers to the item. Most moved the slider tool, so maybe they found it interesting. Yet 71 percent of students on average; that is, most of the students—except the 15 percent responding correctly and the 15 percent that omitted the item—left the arrow at some spot other than 5. Consequently, their Part A did not show the correct data about the cases of lemonade and cases of water, but every possible combination of incorrect data. As an important consideration in developing innovative response options, no matter how engaging, if the students do not understand the purpose of the tools or how to us them, then the tools may be more of a distraction than anything else.
Item 7B
Click video to play
Maximum Score Points: 1
Content Domain: Number
Topic Area: Whole Numbers
Cognitive Domain: Reasoning
Do not use items that depend on correct answers to previous items. 7B, isolated from dependency on where the slider tool was expected to be in 7A, could have been straightforward. Students should have been given new conditions for cost with new values that would have made the calculations in 7B relatively easy like was originally intended. Based on the current 7A and 7B, only the students with a correct answer to Part A have the correct multipliers for the cases of lemonade and cases of water necessary to calculate a correct answer to 7B.
Results 7B
Exhibit 13 containing the results for 7B shows only 8 percent of the students on average managed the correct calculations for the total cost of the drinks. Even with correct values, finding the solution involved three steps. That is: 5 cases × 20 zeds = 100 zeds for the lemonade and 12 cases × 10 zeds = 120 zeds for the water. Then, 100 zeds + 120 zeds = 220 zeds. The scoring guide also included tracking for the students whose calculations for the total cost matched up with an incorrect number of cases of lemonade (any number other than 5). This elevated the 8 percent correct on average by another 15 percent, with nearly reaching one-fourth of the students providing correct multiplication and addition in 7B.