Data Interpretation

Becoming a Clinical Practitioner - Semester 3, March 2021

GUTTMAN CHART

ASSESSMENT TASK REVIEW

a.   Suitability of items

By sorting the assessment data into a Guttman chart, we can determine that this task and items / criterions were suitable for the given students. High performing students have been positioned at the top and highly achieved tasks towards the left of the chart. This creates a clear diagonal slant in the data demonstrating what each student has achieved (1) and what they are yet to achieve (0), indicating the gradual progression and difficulty of the items and task. A regular student’s or indicator’s data would as such be presented as mainly 1’s at the beginning, a pattern of 1’s and 0’s towards the middle, and mainly 0’s at the end. A good example of this is presented in item 10.3. Generally, all items within this assessment follow that pattern, with an increase in 1s due to the high achievement of the cohort. There are no items from this task that could be seen as concerning due to lack of achievement, as low achieving items such as 5.3 and 9.4 are presented to challenge students’ higher order thinking skills. Therefore, the items within this task present a suitable range of difficulties to accurately determine almost every students’ zone of proximal development (ZPD).

b.   Suitability of task

Over 80% of the chart contains 1s, suggesting that this task was at an appropriate level for the participating students, while still providing enough information to determine their future learning goals and ZPD. This is also evidenced by the groupings of skills within the assessment and the students’ ability to meet their requirements. Every student was able to meet the requirements of the orange group and sort the data correctly (bar one - but given that this student has met harder requirements of the task, we can assume that a simple mistake was made). Most students were then able to understand and evaluate the data with the requirements in the yellow group, however the later groupings and requirements of analysing and employing the data were found more challenging. This outcome is expected, as most students should be able to understand the task at hand while still being challenged by the higher order thinking skills that are interrogated in the later requirements. Student 1 was able to meet all requirements of the task, and along with students 10, 13, 14 and 27 who only missed one requirement, may benefit from a more challenging task in order to accurately assess their ZPD.

c.   Impact of noise

Upon first glance, there does seem to be a fair bit of noise around the diagonal slant of the chart, however it is not a pattern that would warrant a review of the assessment as a whole. Many of the rogue 0s in this Guttman chart, especially towards the left of the chart, can assume were because of simple mistakes. However, noise closer to the diagonal slant can be examined further. 

Item 12.1 could be seen as an issue of concern in terms of noise, as higher achieving students were unable to meet it’s requirements whereas lower achieving students could. Other inconsistencies occur in items 7.2 and 4.2, as they have been achieved beyond certain students' ZPDs. Student 19 displays a lot of noise within their irregular pattern. They were unable to meet the requirements of the yellow grouping of criterions (understand and evaluate data) however have successfully achieved the most difficult requirements of the task that some of the highest achieving students were unable to do. Due to this irregularity, another assessment should be conducted to get a better understanding of students ZAD and ZPD.

There are a multitude of reasons for this noise to appear in the data. As this is a hypothetical dataset, I cannot determine the impact of the noise entirely without observing the participating students and the context. Certain criteria or instructions may not have been presented clearly, however there is not enough inconsistency within a single item to confirm this. Scaffolding may have been lacking in certain aspects of the task, or students may simply have been absent during lessons. Context is the main source of noise. Students’ own individual contexts play a role in the academic performance, including their energy levels, ability to focus, mental wellbeing and health. Overall, however, this amount of noise has only had a slight impact on the assessment and has not influenced the placement of students’ ZPDS.

PAIRWISE REVIEW

a.   Relative difficulty review

Reflecting upon the Guttman chart and the original matrix for the assessment, the pairwise comparison seems to be fairly accurate, however there are some differences between the groupings and difficulty progression established. Certain items have been positioned lower (7.1 & 4.2) or higher (10.2 & 11.2) on the matrix than they appear in the difficulty progression in the Guttman chart. Criteria 11 as a whole was ultimately easier than initially expected, with two of the items belonging in the same ZPD grouping. Items 9.1 and 9.2 appear next to each other in the difficulty progression, and although they are of different levels in the matrix, perhaps these criterions should be adjusted and combined into one for a more clearer understanding.

The level statements in the matrix and the big ideas behind the ZPD groupings are ultimately quite similar. Both statement progressions discuss the process of sorting the data to ultimately using the data to inform future teaching strategies. 

STUDENT PERFORMANCE REVIEW

a.   Ability of the Guttman chart to determine ZPD for individual students

The Guttman chart has become a useful tool in enabling the identification of students’ ZPD groupings. Using a Guttman chart across multiple assessments would provide a better understanding of students’ ZAP and ZPD and make aspects of noise more evident. By reviewing this Guttman chart alone, clear ZPD groupings can be made along the diagonal slant indicating the difference in students’ abilities. Five groupings were recognised, each with a corresponding big idea that students must understand before moving on to the next. 

The orange group, sort data, was achieved by all students and therefore no one has fallen into this ZPD grouping. The yellow group, understand and evaluate data, is the ZPD for an individual student. Student 4 presents areas of concern, as they met all the requirements for the first grouping of skills but none after that. As this student has presented results inconsistent with a Guttman chart (no pattern of ones and zeros, only a block of ones followed by a block of zeros), another assessment and subsequent Guttman chart would be required in order to properly identify and develop this student’s ZPD. 

The green group, analyse data, is the expected level for students to be sitting. These students have been able to sort and evaluate data, and have demonstrated early skills to be able to analyse the data, but are yet to do so completely. For this group to progress further, they must be able to justify the decisions they have made. 

The differences between the green and blue group came down to criterions 10 and 11. These criterions are ultimately the same but apply to two different situations where feedback needed to be given. These items, despite being separate, present the same skills and as such each corresponding item in criterions 10 and 11 (10.1 & 11.1, 10.2 & 11.2, and 10.3 & 11.3) would have to be in the same ZPD grouping. 

The blue group and purple group have the same big idea, employ data, however they have been divided into two groups. The blue group sit well above the expected level, but can further develop their learning by developing their scaffolding techniques. The purple group could also fit into the blue group, however they have demonstrated a firm understanding of this concept already and after fixing a few mistakes, they would need a new assessment to better understand their ZPD.

All but two students have fit into these ZPD groupings. Student 19 displays an irregular pattern, as they are able to meet the requirements in items that the top students were unable to meet, but have struggled with items that are easier on the progression. Their data is too inconsistent to be able to accurately determine his ZPD. Contrarily, student 1 has met all the requirements for each item assessed, and has demonstrated that their skills go beyond that of this particular task. Further assessment would be required here to determine both their ZPDs. 

LEARNING GOAL

The green grouping (analyse data) has demonstrated that they are able to sort and understand the data, but have difficulty analysing and interpreting the data in order to further use it to develop their future teaching and learning goals. As such, the goal for these students is to develop their language consistent with the development model of learning in order to explain and justify their decisions for their ZPD groupings by discussing patterns presented in the chart.