UCAS publishes report analysing its verification service

The service is run on behalf of students, universities, and colleges to help prevent anyone gaining an unfair advantage, or securing a place in higher education, by deception.
Posted Thu 31 May 2018 - 00:00

The report (198.64 KB) analyses all UK domiciled applicants applying to UCAS Undergraduate courses over the period 2013 to 2017, and looks at where applications have been flagged for investigation for potential fraudulent activity, and subsequently cleared, cancelled, or withdrawn, by ethnic group.

Applications can be flagged as a result of automated screening for fraud (using industry standard software), similarity of personal statements, duplicate detection, or if universities, colleges, or others have concerns (known as Alerts). The report examines each of these in detail.

Our analysis shows the following:

  • The total number of applications flagged and cancelled every year is small. Of the 2,913,525 applicants over the last five years, 5,160 (0.18%) were flagged, and 2,085 (0.07%) had their applications cancelled.
  • Typically, around 40% of flagged applications are cancelled. The percentage of applications cancelled is broadly proportionate to the percentages flagged in each ethnic group. This suggests that the verification activities undertaken by UCAS are generally robust and fair. In addition, all cancelled applications from 2017 and 2018 have been reviewed, confirming all cancellations were made for genuine reasons. 
  • While ethnicity is not used in any of the processes that can lead to an application being flagged, there are differences by ethnicity in applicants flagged. Overall, a greater proportion of flagged applicants are Black (52%), compared to the proportion of Black applicants in the applicant population (9%). This equates to 2,675 Black applicants being flagged out of an applicant population of 260,550.    
  • Industry standard fraud detection software is used to screen applications against a set of reference data, using combinations of different criteria. The data does not include information about applicants’ ethnicity or nationality. Over the five years analysed, a greater proportion of flagged applicants (65%) were from the Black ethnic group. While there are likely to be several reasons for this, UCAS believes that the accumulation of historical data, and the inclusion of all applications that have not been cleared in the reference dataset, could be contributing to the results observed. UCAS has already enhanced this service and cleansed the reference data, so applications are matched against six years of historical data, which is an industry standard. All reference data will be reviewed and cleansed annually.
  • A greater proportion of Black applicants are flagged for similarity of their personal statement (53%), compared to the proportion of Black applicants in the applicant population (9%). As this software only looks at the text of an applicant’s personal statement (and doesn’t include information about an applicant’s name, address, ethnicity, or nationality), UCAS believes the flagging results are not influenced by an applicant’s ethnicity.
  • The Alerts process results in proportionally more applicants from the Black ethnic group being flagged (37%), compared to the proportion of Black applicants in the applicant population (9%). Analysis shows that alerts were spread widely across 67 universities and colleges in 2017, with the majority flagging only one or two applicants. Universities and colleges do not have knowledge of an applicant’s ethnicity when these alerts are raised.

Actions we are taking:

  • Annual figures on the verification service will be published as part of our end of cycle reporting, which currently produces two million application and acceptance statistics each year. These data will inform application guidance for all students.
  • UCAS will be reaching out to representative organisations to strengthen the voices of Black, Asian, and minority ethnic students in our decision-making and processes.
  • UCAS will introduce an annual survey for applicants whose applications are flagged. This will help improve our communications about verification activities, and information and advice for students who may need additional help to make good quality applications.
  • We will share our findings with the higher education sector, to encourage the development of good practice on raising Alerts, and verification activities more generally.

Clare Marchant, UCAS’ Chief Executive, said: ‘UCAS uses industry standard software to screen applications for fraud on behalf of universities and genuine applicants. This analysis gives us confidence we are only cancelling applications where there’s clear evidence of fraud or missing information.

‘However, there is more work for us to do to ensure that flagging is as robust as it can be across all areas of the verification service. We’ve already made enhancements to our fraud detection service, introduced an additional review of applications prior to cancellation, and ensured all staff involved in verification activities have had up-to-date unconscious bias training.  

‘We have set out a programme of work to further review verification processes in collaboration with universities and colleges, and we will be inviting organisations that represent Black, Asian, and minority ethnic groups to work with us to help us take a thorough look at key areas of our business, to improve the experience for all applicants.’

ENDS


UCAS Press Office contact details

Email: communications@ucas.ac.uk

Phone: 01242 545 469


Notes to editors

Download the full report (198.64 KB).

View additional data tables on the verification service (114.79 KB).

The industry standard fraud detection software, Hunter, (excluding reference data) is provided by Experian Ltd.

Copycatch is the similarity detection software that analyses personal statements, and is provided by CFL Software Ltd.

UCAS, the Universities and Colleges Admissions Service, is a charity, and the UK's shared admissions service for higher education. We manage almost three million applications from around 700,000 people each year for full-time undergraduate courses, at over 380 universities and colleges across the UK.

Related news