Abstract: Objective: To investigate trainee doctors’ and trainers’ perceptions of the validity of the Annual Review of Competence Progression (ARCP) using Messick’s conceptualisation of construct validity. Design: Qualitative semi-structured focus groups and interviews with trainees and trainers. Setting: Postgraduate medical training in London, Kent Surrey and Sussex, Yorkshire and Humber, and Wales in November/December 2015. Part of a larger study about the fairness of postgraduate medical training. Participants: Ninety-six trainees and 41 trainers, comprising UK and international medical graduates from Foundation, General Practice, Medicine, Obstetrics and Gynaecology, Psychiatry, Radiology, and Surgery, at all levels of training. Main outcome measures: Trainee and trainer perceptions of the validity of the ARCP as an assessment tool. Results: Participants recognised the need for assessment, but were generally dissatisfied with ARCPs, especially UK graduate trainees. Participants criticised the perceived tick-box nature of ARCPs as measuring clerical rather than clinical ability, and which they found detrimental to learning. Trainees described being able to populate their e-portfolios with just positive feedback; they also experienced difficulty getting assessments signed off by supervisors. ARCPs were perceived as poor at identifying struggling trainees and/or as discouraging excellence by focussing on minimal competency. Positive experiences of ARCPs arose when trainees could discuss their progress with interested supervisors. Conclusions: Trainee and trainer criticisms of ARCPs can be conceptualised as evidence that ARCPs lack validity as an assessment tool. Ongoing reforms to workplace-based assessments could address negative perceptions of the ‘tick-box’ elements, encourage constructive input from seniors and allow trainees to demonstrate excellence as well as minimal competency, while keeping patients safe.