Selecting University Students: Perspectives from Selection and Recruitment

I was reading the article in The Age reporting on a discussion paper at The University of Melbourne reviewing undergraduate entry requirements.

The discussion paper mentions other means for selecting students in addition to their performance in the final year of high school (ENTER).

I see the issue as involving two elements:

1. What criterion of an effective selection system does the university want to use?
2. What measurement system can be put in place to maximise this criterion?

The first issue is a matter of values and policy. The second issue is empirical.

From The Age article I gather that there is concern for: representation from disadvantaged student groups and academic potential.

Many other values could be mentioned, including transparency, procedural justice, cost of administration, and so on. No doubt any decision would be a synthesis of such concerns. However, if the choice of criterion could be resolved, the question of a measurement system is largely an empirical question.

Many lessons can be taken from the selection and recruitment literature on. For a nice review, see: Sackett, P. R., & Lievens, F. (2007). Personnel selection. Annual Review of Psychology, 59, 419-450.

Without going into the details I just wanted to comment on a few points mentioned in The Age.

“ 'ENTER scores can also give a false sense of precision in assessing academic prowess. Just how large is the difference in academic potential with a student of 86 and one with 87, even though a single point might separate entry to and exclusion from a particular course?'' asks the discussion paper, Refining Our Strategy.” - The Age

I agree that any predictor may create a false sense of precision. However, if ENTER is a valid predictor in that it correlates with the criterion (e.g., university performance), then this issue is not really relevant. The predicted performance of an applicant with 87 should be selected because they will on average do better than someone with an 86. As the difference between scores on the predictors becomes smaller, so does the difference that we can expect on the criterion. If ENTER is the wrong predictor, then change the predictor. Change it to some composite of ENTER, low socio-economic status, and ability tests. Change it to whatever empirically maximises your criterion. However, the end result will still be a set of applicants rank ordered on some predictor variable. And a line will need to be drawn on who will and will not be selected. We know that the difference between two applicants on either side of the line is small, yet to the best of our knowledge the person on the selected side of the line is predicted to do fractionally better than the person on the side of the line not selected.

“The paper said students in schools with average lower ENTER scores tend to perform better once they get to university than their counterparts from high-performing schools, suggesting that ''preparation in some schools is focused on achieving ENTER, rather than skills for university'” - The Age

This seems to confuse the issue of defining a criterion and developing a valid measurement system. It implies that ENTER is less than optimal both in terms of predicting university performance and in maximising representation of disadvantaged students. This may be true. And it would strengthen the argument for searching for alternative selection devices. However, it is important to keep the values and the empirical questions separate.

The article summarises a few suggestions for improving the selection system:

Ability tests: I’ve seen research in the past showing that ENTER correlates highly with ability test measures. Based on a quick search, some research shows a correlation of 0.64 between ENTER and the GAT (a test completed by all VCE students that is similar to an ability test). Given this fairly high correlation this suggests that an ability test would not add much prediction over and above ENTER scores. However, having two tests might improve overall predictive validity. Equally, it may be that ability tests are less influenced by school practices, which may make ability tests more equitable. However, this is an empirical question and it would be necessary to be aware of coaching strategies that might bias results.

Civic test or personal essay: My intuition from the selection and recruitment literature is that this would be a poor predictor of actual academic performance. Rating of a personal essay would also probably be highly subjective. However, I imagine it would make for some good stories for the university news letter.

An expansion of special-entry access schemes: I assume that such schemes are based on the aim of improving representation of students from lower socio-demographic groups. This sounds like a noble aim. The key to this is to actually design a system which validly measures whoever is special.

In summary, such policies need to clearly articulate what are issues of values and what are empirical matters.

DISCLOSURE: I am a lecturer at the University of Melbourne, but I have no involvement in the above policy document. My views are my own and do not necessarily reflect those of the university.