On-Screen Question Authoring for Examinations, with Charlotte Warr
When you are creating and managing question items, there are undoubtedly many benefits to using an on-screen assessment authoring solution. From a project management point of view, when you have multiple writers and subject matter experts contributing to a paper, online question authoring helps give the process a structure. You have a central point where all the work is securely stored, and the ability to give feedback online enhances the ease of collaboration and speed of reviewing items. Being able to preview questions also enables question writers to see immediately how it will work in practice from the candidates’ point of view. The whole process is quicker, more secure and also helps writers to be more creative – especially when using an exam writing tool that gives a wide variety of question types and options to use rich media.
But what is the experience for an individual exam question author? Charlotte Warr, Managing Director at Sarnia Training Limited, is an experienced question writer, who is regularly authoring item types for exams run on TestReach. We asked her for some feedback on her personal experiences of moving from paper-based to on-screen question authoring…
A Personal Experience of On-Screen Question Authoring
by Charlotte Warr, Managing Director, Sarnia Training Ltd.
Geothe wrote that if you plunge into the thick of life and seize it where you will, it will always be interesting. I can say that he was right given that within the last 18 months I have gone from being an experienced paper-based assessment creator with no online question authoring experience, to a totally involved user of the TestReach question authoring system for two different clients.
To many people who have not had direct experience, online testing with auto-scoring means single answer multiple choice questions, which just assess knowledge but are useless for anything deeper. This could not be further from the truth, both in the variety of question styles, but also the ways in which those questions can be used creatively to ensure that the students are challenged to really think about their learning, apply their knowledge and analyse information to arrive at the correct result.
From my experience many of the subject matter expert question reviewers, who have been asked to cast their eye over question banks I have created, have been surprised - not only at the different styles of question that are possible, but also the depth of challenge that the questions can pose. Prior experience is always likely to colour your current perception of things and I must admit to feeling almost like a child in a sweet shop when I first saw the variety of question types I could “play” with.
Is it still as difficult to find that 4th good distractor in a multi choice question? – Yes, of course it is – some things have not changed with the move from paper to screen!
A change of medium is equally challenging to the question author as well. For me, that is part of the interest. I am not only developing my own skills but thinking hard about different ways to use online questioning to really draw out a student’s practical skills which are no less relevant than the academic knowledge. The ability to use images or extracts from documents within the stem of the question or as a free-standing resource is a powerful tool to bring real-life scenarios into play within the questioning process. This both engages the student, but also coupled with relevant e-learning, ensures a more focussed learning experience. This can be taken back and used immediately in the workplace rather than just being viewed as something that has to be endured, the book then shut and any take-aways forgotten rather than being used.
Pure academic learning (however it is delivered) is just one part of the learning journey for anyone, and that has not changed with the advent of online assessment. What has changed is how engaging we can make the testing process without in any way devaluing the robustness of the process, the rigour of the assessment and the value of any award that comes with success.
The other positive for the students is the flexibility that remote invigilation offers – if they want to take the exam at their kitchen table they can (subject to some requirements) but they are not forced to go to an exam centre. For one of my clients who used an alternative exam process, this involved having to actually travel to a different country.
More Questions & Answers on Item Writing with Charlotte Warr
What question type did you have the most fun with?
As a question writer I had the most fun with questions where I could use images and other resources as a basis for students to consider their answer. It was also fun for me considering how different question types can be used to best effect for certain types of questions. For example, questions that were mathematical in nature or questions asking students to match pieces of information.
Were there time savings for you by authoring directly into the tool?
There is a definite time saving both in authoring directly into the tool and also by having the independent review process conducted directly within the system. This not only allows a clear audit trail but reduces re-keying of data.
What are the interesting ways you have used images in questions?
So far, I have used images in questions where the students have to correctly identify various label positions. I’ve also used images in hotspot questions where, for example, students were asked to highlight the areas of a building which might pose a security risk. Going forward I envisage using images as resources to support a set of questions rather than just being in the stem of a single question.
Do you see many advantages to using categorisation in questions?
In my view there are two main advantages, namely the ability to test a wider area of the syllabus in a single question, and also to use different question forms as widely as possible.
What do you feel are the pros and cons of using many different item types in one paper as opposed to one item type?
For me, I think the main advantages are the fact that different question types suit different areas of testing better than a single choice MCQ might do. I also think different question types are more interesting for the student when taking the test. In my opinion the only downside would be catching the student out if they were anticipating only MCQ and were presented with different question types. However as long as they do the tutorial beforehand they should be in no doubt - and we also ensure our students are aware of the different types available!
See Online Question Authoring in Practice
If you would like to see how impactful on-screen question authoring can be, follow this link visit the TestReach demo page to request an online demonstration of our question authoring tool. This will give you a practical overview of the wide variety of item types possible, and also help you to understand how it streamlines the overall examination writing process.
If you’d like to read about how other organisations have used online assessment, you can take a look at some online assessment case studies here.
If you would like to contact Charlotte at Sarnia Training in connection with exam writing services, you can visit her website here: Sarnia Training
