Context more important than ability?

Nov 5, 2017 by

James Thompson –

A UK charity, The Sutton Trust, has urged universities to take in students with grades which are two levels below the usual entry requirements, arguing that some students are capable of doing well at university, but have low scholastic attainments because of environmental circumstances: being poor, being at a bad school, and having to look after sick relatives.

At this point you may imagine that I am lining you up to tell you, once again, that intelligence tests were expressly designed to identify true ability, and that their use has led to the better detection of such bright, but adventitiously disadvantaged youngsters. Not so. No testing of ability is involved.

It seems that many UK universities have such “contextual” programs, but the University of Bristol features prominently in this debate so I will use it as an example. It describes its Contextual Offers requirements thus: being in the bottom 40% of schools in terms of attainments or progression to higher education; or living in a postcode area with low progression to higher education; or having completed a University of Bristol outreach program; or having spent more than 3 months in local authority care. All of these are misfortunes, but none of them include any measures of cognitive ability.

As is my usual habit, at this point I give you due warning that you may wish to stop reading here. However, a very striking claim is made: that such students do just as well as students admitted in the usual way, based on scholastic attainments. Worth a look? I thought so.

The Sutton Trust research measured scholastic attainments using the best 3 A Level results. That in fact underplays the attainments of the best candidates, who do 4 or 5 A levels, and sometimes another half A level. This will have an impact on their calculations about the differences between normal and contextual admissions at the most prestigious universities, which are most able to demand the highest academic attainments. The authors then calculate what the entrance requirements were likely to have been, using an approximation, since they did not have actual data. Further, I cannot find any evidence that they know which degree courses were followed by candidates. Some disciplines are more demanding than others, and command correspondingly higher premiums in the occupational market place.
Anyway, on page 28 of their publication they lay out their case for these offers not being associated with poor outcomes. They say:

But is there any evidence that universities who appear to be more likely to contextualise are also more likely to see higher dropout rates, lower degree completion rates and lower percentages of students getting firsts or 2:1s? We find little evidence of this, at least using the two potential measures of contextualisation described above.

Figure 4.5 presents correlation coefficients showing the relationship between the degree of contextualisation evident from Figures 4.3 and 4.4 with average degree outcomes for students on these courses. If contextualisation were adversely affecting degree outcomes, then we would expect to see a negative correlation between the percentage of courses on which universities appear to offer lower entry grades to students from low participation neighbourhoods and dropout rates, and a positive correlation with degree completion and degree class. By contrast, we would expect to see a positive correlation between the difference in A-level grades of students from low participation neighbourhoods and the standard offer and dropout rates, and a negative correlation between this gap and degree completion and degree class results.
Sutton Trust contextualized offers

It is not easy to make sense of these correlations, other than to observe that they are rather small, particularly with only 25 data points. None of these correlations are remotely significant in the statistical sense. The bigger problem is to understand what they mean.

Administrative data from 25 universities who admit some student on this “contextual” basis has been looked at, even though we do not know what percentage of students were so admitted. It seems likely to be a small percentage, say about 3% of the whole student body. These students would be worth studying as a group, if one could identify them. I imagined, hearing the bold statements during radio interviews, that such a comparison between normal admissions students and “contextual” admissions students had been carried out, and I could see the results of a t test comparison. Instead, assumptions have been drawn from a chain of prior assumptions, as shown in Table 4.1 which shows very little impact from all these “contextual” factors, except perhaps the “free school meals” low income measure. There is little difference between the implied groups, which leaves me bewildered about why they are considered important factors for contextual admission in the first place. Their whole argument hinges on the assumption that children from poor families will be, in intellectual terms, no different from children from wealthier families. The possibility that families who do not require free meals are brighter does not figure in these discussions.

In my view the results provide no grounds for saying that one should discard scholastic attainment as a way of regulating entry to university. On the contrary, this publication asserts but does not prove that being poor is an indicator of undetected ability. If so, why not use the tests designed to identify such talents?


Source: Context more important than ability? – The Unz Review

Print Friendly, PDF & Email

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.