Our blog series ‘Examining the Experts’ asks leaders and
experts in assessment – practitioners and consultants,
authors and solution providers – to share their
experience and knowledge on examination design,
certification management and the changes brought by
online assessment software and examination technology.
What piece of advice would you give to anyone planning
a career in assessment?
Start with a large international company, learn good
practices, soak up wisdom from cross-cultural
team-working and be exposed to well-known exam owners
with big challenges. Get a maths and/ or statistics
qualification if you’re interested in test development
or analysis. Assessment and Psychometrics degrees are
rare in the UK, but do exist in the USA.
If the delivery of tests stimulates you, get a
qualification in facilities management, geography or
logistics. If you want to manage an assessment
company, get an engineering degree and an accountancy
qualification. Every engineer I’ve met who works in
assessment totally gets it: the process, procedure and
system inter-linkages, as well as the overarching big
Question assessment leaders who have never taken a
high-stakes exam after compulsory education. Do they
think exams are for ‘other people’ or do they not
believe in assessment as a career?
Thinking back over your career, what would you rank as
the most important innovation in management of
Social media changed everything. Finding study groups,
tips on exam preparation, malpractice and actually
locating those exam owners – social media ripped up
the marketing book. The black box of advertising in
magazines and 96-sheet billboards still exists, but we
have the new black box of algorithms and a fluid ‘Wild
West’ of global compliance and legality to work out.
In many countries, the tri-net
(Google-Facebook-Amazon) has direct influence over 70%
of internet traffic. In Portugal, ISPs offer cheaper
internet access based on ring-fenced tri-net access,
not the full internet. So for some key countries, exam
owners are compelled to align with tri-net providers
as the only realistic way to be visible to learners,
candidates and members.
What would you say was the biggest challenge facing
awarding bodies today?
I’ve codified 12 common challenges, including
malpractice and integration. Exam owners can meet
these challenges by understanding which to address
first, what’s ‘Business As Usual’ and what grows their
organisation. Most only handle three challenges at a
time, for example, internationalisation.
One exam owner I work with believed their home market
was saturated. Setting up new routes to new markets,
tailoring products, setting up new risk management
processes can be incredibly disruptive. Ironically,
admitting that the skills and experience to make that
happen is not “in the room” is often the biggest
Exam owners and suppliers are also often held back by
legacy ways of working and business culture that have
not moved with contemporary social norms and
Do you think that the use of paper in examinations
will be phased out or will it always have a role?
Paper has a role in three approaches:
1. To build a new qualification and use
on-screen assessment as a sandbox to get stakeholder
2. Have parallel delivery of on-and-off
screen tests to maintain equivalence.
3. Phase in a completion conversion over a
number of years.
Also, many countries will always lack traditional
internet infrastructure, so it’s mobile-first: PCs and
laptops are not ubiquitous. However, exam owners often
frown upon summative exams performed on small screens,
so that’s where paper will also persist.
It's bizarre that some experts claim paper is still
needed to accommodate candidates with vision or
motor-skills challenges, or have IT unfamiliarity.
Paper delivery actually locks out special-needs
candidates. Candidates with special needs are better
catered to with screen-based delivery, which can be
adapted to their requirements.
Familiarisation and opportunities to practice are
omnipresent: test prep is a massive industry,
especially in countries such as China and India.
What do you think have been the most useful technical
advancements in assessment software?
Technical advances such as plagiarism detection
software, identity/credential checking and on-screen
marking of digitised scripts have gained traction. I
believe the most useful technical advancements to
getting the job done are outside of the assessment
Collaboration tools have the biggest business and
useful personal impact because they enable colleagues
and exam owners to quickly corral knowledge,
understanding and experience.
Online messaging tools (Slack, Hangouts), workflow
software (Trello, Asana) conferencing (Zoom, Skype)
and sharing features within item authoring software
empower exam owners and suppliers to get things done
and offer continuously improving service. Why wait
weeks for an answer when it can be crowd-sourced, or
you can reach out easily to an expert?
What are the most dramatic changes you have seen in
the running of exams over the course of your career?
Competition and Markets Authority investigations
notwithstanding, this sector is not known for drama.
As an old boss said to me, “If you want drama, go and
work on your Hamlet!”, but some causes célèbres in the
last two decades come to mind:
2018: Indian Railways: 9.2 million high-stakes exams
delivered in three days
2008: Microsoft introducing simulation into its
1998: VicRoads in Australia introducing a hazard
perception video into its driving theory test, three
years before the Brits got there.
The speed of introduction, often using agile
techniques, has also catalysed change. Some of the
changes involve facets we take for granted in Western
society. For example, exam owners are trying
country-specific and culturally-acceptable payment
methods such as M-Pesa in Nigeria, AliPay in China,
and Paytm in India where candidates do not have a bank
This is quite a change from the early ‘00s when
cheques, postal orders and cash-at-the-centre were
common payment methods for UK-based exams. The
currency plans being formulated by the social media
giants will also create drama in this context.
Should we be running assessments differently? What
would you change if you could?
Did you know that most exams are still created by a
single person? I read a paper on England’s GCSE
Physics exams and how those exams are still created by
a single person, rather than compiled from an item
bank. The paper claimed that there is no strong
evidence to say one method produces ‘better’ papers
than the other. I would make item banking the default
There are multiple upsides to item banking, so I’m
puzzled why it’s not the default. Multiple exam
series, ‘test-when-ready’, adaptive test delivery,
LOFT, as well as giving protection and confidence to the
qualification. It seems strange not to change.
Changing awareness is also important. For example, in
areas such as the professionalisation of the
has been the big driver to start that change – because
it is far better to have an invigilator who performs
the task everyday, than a warm body handing out papers
for minimum wage and their petrol money for six weeks
of the year.
What’s the most unusual assessment story you have
encountered or experienced?
It’s difficult to stop becoming blasé to cheating
stories, but tales about beating the cheater still
make me chuckle. One exam owner told me they caught a
candidate who had an implanted a subcutaneous radio
behind their ear and was reading the exam questions to
a remote scribe. Apparently, the implant is a
technique used for duckling and seal tracking,
although I’ve read they can also induce the receiver
to empty their bladder!
Have you had a professional mentor who was especially
influential in your career? If yes, what lessons or
advice have proven to be most impactful for you?
Professor Cliff Beevers at Heriot-Watt University was
always accessible, easy to relate to and personable to
me. I’ve found that keeping his maxims to hand such as
‘good practice, not best practice’ and ‘nobody knows
everything, and if they did, nobody likes a smart
alec!’ helps to prevent hubris and conceit.
What social media platforms do you use regularly, and,
of these, which one do you find to be the most useful
in your professional life?
I tweet as
@geoffchapman_ most days to share good practice, and LinkedIn
rarely: nothing else.
The level of aggression and bullying on Twitter is
frightening. I’m amazed that teachers stick with it
for CPD. Maybe I should follow new people! LinkedIn
seems back-slappy, insubstantial and incredibly clunky
for retrieving useful information. If assessment
leaders are on social media in a business capacity,
they must show leadership – not just retweeting their
boss, company press release, or some social media
It’s a daily occupational hazard to avoid social media
noise, ads, kudos-giving and FOMO. Filtering and
analysis tools are wonderful, but being judicious with
your time is key.
What business book or blog do have you found most
interesting or useful in the past year?
I read many business books during my post-graduate
studies. Some books have potent and seductive
hypotheses, such as Matthew Syned’s 10,000 hours
theory, which have now been debunked.
They dodge muck-and-nettles issues such as performance
plateaus and continuous improvement, but that book
will just gather dust at WH Smiths or Easons at the
I like assessment books such as
‘Making Good Progress’
by Daisy Christodoulou, and
‘Embedded Formative Assessment’
by Dylan Wiliam. They are leading practitioners who
give clear, pragmatic paths to fixing problems.
However I can’t imagine either will have Danny Boyle
asking for a Hollywood screenplay soon!
Geoff is an assessment consultant with two decades of
international education experience. Geoff shares his
insights and approach with presentations, blogs and
other free resources on his website
and via Twitter
Geoff's consultancy bridges and solves shared issues
across exam owners, regulators and technology
He supports customers who need to change their
existing assessment and learning processes,
procedures, platforms and strategy.
Geoff helps exam owners with their assessment and
learning technology choices, and technology companies
with their strategic issues.
A published author on E-Assessment, Geoff is also
co-owner of trade publication,
World Exam Tech.
He holds contemporary qualifications in assessment,
education, IT service, project and bid management.
He is a member of international trade bodies including
Association for Education Assessment in Europe (AEA
Europe), Association of Test Publishers (ATP), British
Assisted Technology Association (BATA), E-Assessment
Association (EAA) and Education Policy Institute
To learn more about how online assessment can benefit
your organisation and see TestReach in action, contact
the team here:
Request a Demo.