Our Sector: Getting behind the headlines of NSS 2017

The dominant press narrative surrounding this year’s NSS results centred on NUS’ boycott of the survey- which depending on your viewpoint was either a resounding success in disrupting a tool of neoliberalism, or as dangerous exercise that both revealed the lack of influence NUS has on its own membership, whilst robbing students of much evidence to bolster arguments about the need for improvement.

But behind the headlines other narratives have played out. One of the key stories from the results wasn’t about response rates or boycotts in the Russell Group- that is noise. The signal is that more than 1 in 4 students aren’t happy with the way they’re being assessed or the feedback they get. The sector’s continued failure to get a grip on this basic and crucial component of provision is shocking. Almost 3 in 10 students can’t agree that marking criteria was clear in advance, over a quarter don’t agree that marking and assessment has been fair, and one in four students don’t agree that comments were helpful.

There are policy implications that go beyond the inclusion of Assessment and Feedback as a TEF metric. The Competition and Markets Authority- whose initial guidance on Higher Education assumed that the service/product on offer was a course’s teaching- must surely broaden the focus of its guidance to the complexity of the services wrapped up in an HEI’s offer. Given assessment and feedback are so crucial to students for both learning and sorting, intervening to enable students to enforce rights to have it done properly is crucial.

It is also not clear why we go to the trouble of organising a national survey without requiring HEIs to publish a response and action plan, given we do so successfully on access. It’s clear that merely publishing satisfaction data is distortive and only partially successful- whereas a comprehensive plan (with input from student representatives) would drive more balanced improvement and retain institutional autonomy. OfS should consider it.

At an institutional level, the heart of the assessment and feedback problem is tied up in the management of performance. There has been plenty of HEA work on the pedagogy of effective practice in this area, but the real issues are about understanding the complexity of administrative systems and the cultural difficulties involved in causing academics to perform to a standard and deadline. These require real work, of the sort only possible if the promise of an integrated Higher Education Development Agency (rather than just a glued together HEA and Leadership Foundation with an Equality Challenge Unit conscience) is realised.

The other key story emerging from the results was one that concerned students’ unions. For the majority of the sector, Question 26 on the survey- which invited students to evaluate the effectiveness of their union in representing students’ ‘academic interests’- became its new lowest scoring item.  In the bulk of HEIs this new low was challenged only by the relative failure of institutions to communicate the impact of feedback gathered from students when compared to a University’s willingness to gather it, represented by a similarly poorly scoring Question 25.

Given the result it would be tempting to argue that the Q26 score indicates systemic failure and the need for serious reform if students’ unions are to continue to command funding and access to University decision making bodies. But the question and its results demand some scrutiny- some of which is assisted by a study commissioned by 18 students’ unions earlier in 2017 (“Union Futures”) to inform and enable a proactive response to the new question. Yielding a core dataset of over 17,000 students, the study provides a source of insight and reflection for the sector, and a platform for the development of both tactical and strategic actions.

A driving agenda behind changes to NSS this year was HEFCE’s desire to focus the survey more closely on the student academic experience, dropping questions on personal development in the process. This was a strange move given the wider focus of the TEF, and leaves large parts of the HE sector’s provision (both services and co-curricular) oddly unevaluated. In his pamphlet on HE and the Public Interest for HEPI, Bill Rammell argued that we need a framework that enables universities (in partnership with students’ unions) to “articulate and evidence their development of co-curricular and extra-curricular learning environments”, given the contribution of this activity to personal and civic development of students.

The shift in focus also left a clear dilemma for the question on students’ unions. In the previous NSS Q23 asked students to indicate the extent to which they agreed with the statement “Overall I am satisfied with my students’ union (association or guild)”.

But on the assumption that all SUs do have a representative role to play in students’ education, cognitive testing of a question to test it proved difficult. HEFCE’s own research suggested that previous iterations of Q26 confused students, some of whom appeared not to associate the SU with having a role in the student academic experience at all. And NSS 2017 bears out the testing, with a higher “neither agree nor disagree” score than for any other question. A question on SU impact on the academic should survive- but not if it is this easily misinterpreted.

The language of “interests” was problematic too. Although the “student interest” is a term familiar at sector level, in the Union Futures study a substantial number of students appeared to interpret the concept literally- evaluating their SU on the extent to which it reflected subjects and activities they were ‘interested in’.

And even when students did understand the concept, the evidence suggests that a significant proportion of students evaluated the effectiveness of their Union at delivering co-curricular educational opportunities rather than influencing institutional academic provision- ironic given NSS’ direction of travel. It is critical that OfS looks again at the provision of student evaluation of and public information on the co-curricular, and arguably should go wider still in inviting students to assess the breadth of services and functions that form the wider student experience sold to students in prospectuses.

There is a strong correlation in NSS 2017 between Question 25 and 26, both of which vie for last place in most HEI results. These are interesting, because taken together they question satisfaction not just with the effectiveness of student input, but also success in communicating its effectiveness to students. There will always be a drag on being able to implement change that results from student intervention and views, but given the relative level of expertise and investment in marketing to applicants (HE choosers) when compared to marketing to current students (HE users) there is space for professional development in this area for HEI and SU communications professionals alike. At the very least HE leaders need to be more open to giving credit to SU Officers that have long argued for a change or investment when it is announced.

There are clear lessons for Students’ Unions themselves. In the Union Futures work, students prioritised four areas where the SU should focus to deliver on their academic interests:

  • Helping students be more employable when they graduate
  • Quality of teaching
  • Placements/work-based experience
  • Mental health support

NUS’ Quality Students’ Unions scheme suggests that the majority of Students’ Unions have Trustee Boards adept at strategic planning and effective paid staff and management, but tend to focus these resources on the management of delivery of services within the SU rather than longer term efforts to influence and improve the student experience at an HEI- often leaving these aspects to elected student officers. The Union Futures work suggests that improving the effectiveness of students’ unions will involve being less rigid about the respective role of SU Officers and Staff, with HEIs recognising that whilst the tip of the iceberg is the student President in the committee, it is only investment in the iceberg that enables that President to make valuable and evidenced contributions.

The Union Futures work tells us other things. Student reps are the most visible and accessible source of representation for students within the academic sphere − with 6 in 10 survey respondents in the study declaring that they would be likely or very likely to contact a student rep to support them in their academic interests. Where reps belong to the students’ union they can therefore make or break perceptions of the performance of the union on Q26. The huge disparity in investment in student representation between institutions deserves investigation, but more broadly the overall lack of investment is an important story. Almost all SUs have budgets that deliver the academic representation function on little more than a couple of junior staff and an elected sabbatical officer- so for it to be judged by over 50% of students as effective is miraculous rather than shameful- no wonder students with little other SU contact respond with “don’t know”.

Students’ Unions have tended to intervene at institutional level with the bulk of NSS inquiring about course level impact. My own experience at UEA- where dedicated investment in a school based representation and activities partnership with the SU has resulted in traditionally miserable Nursing and Midwifery students rating questions 25 and 26 in the top three of UEA’s 26 schools, should be an example researched and replicated where possible. Across the piece, Union Futures found that student reps must have sufficient power to make the student voice audible, be seen as approachable by both staff and students, be properly trained, understand the role of the students’ union and their function within it, and be true ambassadors. This requires real resource.

There are things we don’t know. The theory of student representation is that data fused with user representatives wielding it for accountability purposes drives improvements that data alone cannot generate- but in an era set to be dominated by metrics, this does need testing. And given the significant power imbalance between students and HEIs, students’ willingness to offer negative feedback to a tutor or department or submit a complaint about provision or behaviour requires research- with the role of SUs in giving students confidence to do so needing recognition.

This is especially true in private providers and FEIs, where there are not the same traditions of Students’ Union development and activity, and regulation on what “counts” as a students union often does not apply. The answer is not to allow these aspects to become competitive options for providers of HE but instead for OfS to recognise their centrality, requiring all providers to reflect on the way they enable both individual advocacy for and collective representation of students within their culture.

We also don’t know about the enormous contribution made by many students’ unions to the rest of their HEI’s scores in NSS 2017. Sector level actors often assume that the mere publication of data causes behavioural changes at a granular level, but it is often the usage of the data by student representatives that levers the results. Of course students’ unions need to think about the nature of the relationship they and their reps have with their HEI- with neither aggressive opposition nor passivity embodying the kind of assertive partnership needed. But more broadly, if the publication of data to applicants helps drive change for choosers, then OfS also needs to understand what it is that can drive change in the interests of users- testing the widely held hypothesis that it is students unions and reps within an HEI that convert research into action.

What we do know is that UK HE- at least in public institutions- has a Students Union sector unique when compared to counterparts in both Europe and North America. Its fusing of individual and collective advocacy, along with collectively owned services and an extraordinary breadth of co-curricular activities- all run democratically- is a low-cost jewel in UK HE’s crown. Perhaps the real real story is not how poorly SUs score compared to libraries or academics, it’s how well they score when doing all this for less than £50 per student per year- and the opportunity there is for investment, development and research into the SU sector to drive improvement in all of NSS’ questions in the student interest.





Posted in Uncategorized.

Leave a Reply

Your email address will not be published. Required fields are marked *