Computer-Assisted Assessment

What is CAA or Computer-Assisted Assessment?

CAA is a timed, invigilated, grade bearing, exam, which uses objective question types. It takes place on campus, in a pre-booked PC Lab. The University uses software called QMARK to run these tests. 2,500 students were assessed in the University using CAA in 2009/2010.

There are 22 different question types available so not just multiple choice.

Caroline Gipps (2005) asserts that, "It seems likely that the use of 'simple' computer-assisted assessment (with multiple-choice and short-answer responses) will continue to develop, as will the use of multi-media materials for assessment tasks.'’

CAA is erroneously thought of as useful only for testing facts or knowledge. Questions can easily be written which require students to use skills of application, analysis, comparison, comprehension etc. Have a look at some examples of higher order cognitive skills questions.

Further background information can be found in the University's Guidelines for the Quality Assurance of Summative Computer Assisted Assessment.

Benefits of using Computer-Assisted Assessment

CAA can be one of the two assessment methods on any module. The benefits of using summative CAA are generally understood as:

• No marking! Automatic marking abolishes marking time and allows results to be known by the module leader immediately after the test finishes.
• No moderation
• It is a method which, in general, students like.
• No plagiarism.
• Wider coverage of the range of the module content than say in essay type questions – this ensures that students are acquainted with more than a few select topics in a module.
• Adds to the diversity of summative assessment methods
• Provides an objective and consistent means for summatively assessing
• Time effective in assessing very large modules
• Enables assessments to be constructed using multimedia
• Enhances the validity and the reliability of the assessment. QMARK provides you with a statistical analysis of the assessment, down to the level of the usefulness of each item in each question. This enables staff to continually improve the assessment and adjust it according to the data on performance on the questions.

The drawbacks of using CAA

• There is a significant investment in time in preparing your first set of questions for use in CAA.
• Writing questions that test ‘higher order skills’ requires up-front effort and time
• Cannot be used to assess written expression, practical skills or creativity
• High risk summative assessment
• Allocating grades and the relationship to the alphanumeric scale needs some thought.

How do I use it?

Please contact your School's Blended Learning Unit (BLU) Advisor, who can support you through the whole process.

In short:

• Check you don’t need to change your assessment type through SQEC
• Contact your BLU advisor for advice
• Read the University’s CAA Quality Assurance guidelines
• Tell BLU, ITS (ex2000) and Registry you are using CAA – when, where, with which module, class size etc
• Author your questions and trial with colleagues - bearing in mind Bloom!
• Get a colleague to check these questions and scoring scheme
• Enter the questions into the QMARK software – there are two ways of doing this
• Send the resulting question file (.qpack) to BLU
• Get secure log-in information from BLU
• Test that your assessment and all questions work as you desire
• Get there early on the day to log on all PCs with the ID/Password
• Get the marks sent to you by BLU  

There is a full detailed list of the steps which need to be followed are available within the Guidelines for the Computer Assisted Assessment - Scroll down to Appendix A Operational Guidelines for the use of summative CAA 2010 / 2011.

References

Bull, J (2005) Introduction to Computer-assisted Assessment

Gipps, C. (2005) What is the role for ICT-based assessment in universities?  Studies in Higher Education, Volume 30, Issue 2 April 2005 , pages 171 - 180

JISC InfoKit – Effective use of e-assessment

QMARK perception web site

QuestionMark Live – the place to enter your questions