What is CAA or Computer- Assisted Assessment?

CAA is a timed, invigilated, grade bearing, exam, which uses objective question types. It takes place on campus, in a pre-booked PC Lab. The University uses software called QMARK to run these tests. 2,500 students were assessed in the University using CAA in 2009/2010.

There are 22 different question types available so not just multiple choice.

CAA is erroneously thought of as useful only for testing facts or knowledge. Questions can easily be written which require students to use skills of application, analysis, comparison, comprehension etc. Have a look at some examples of higher order cognitive skills questions.

Further background information can be found in the University's Quality Assurance Guidelines for CAA - November 2013 (Word doc 33k). 

Benefits of using Computer-Assisted Assessment

CAA can be one of the two assessment methods on any module. The benefits of using summative CAA are generally understood as:

  • No marking! Automatic marking abolishes marking time and allows results to be known by the module leader immediately after the test finishes.
  • No moderation
  • It is a method which, in general, students like.
  • No plagiarism.
  • Wider coverage of the range of the module content than say in essay type questions – this ensures that students are acquainted with more than a few select topics in a module.
  • Adds to the diversity of summative assessment methods
  • Provides an objective and consistent means for summatively assessing
  • Time effective in assessing very large modules
  • Enables assessments to be constructed using multimedia
  • Enhances the validity and the reliability of the assessment. QMARK provides you with a statistical analysis of the assessment, down to the level of the usefulness of each item in each question. This enables staff to continually improve the assessment and adjust it according to the data on performance on the questions.

The drawbacks of using CAA

  • There is a significant investment in time in preparing your first set of questions for use in CAA.
  • Writing questions that test ‘higher order skills’ requires up-front effort and time
  • Cannot be used to assess written expression, practical skills or creativity
  • High risk summative assessment
  • Allocating grades and the relationship to the alphanumeric scale needs some thought.

How do I use it?

Please contact your School's e-Learning Advisor who can support you through the whole process.

In short:

  • Check you don’t need to change your assessment type through SQEC
  • Contact your e-Learning advisor for advice
  • Read the University’s Quality Assurance Guidelines for CAA - November 2013 (Word doc 33k)
  • Tell CETL, ITS (ex2000) and Registry you are using CAA – when, where, with which module, class size etc
  • Author your questions and trial with colleagues - bearing in mind Bloom!
  • Get a colleague to check these questions and scoring scheme
  • Enter the questions into the QMARK software – there are two ways of doing this
  • Send the resulting question file (.qpack) to CTEL
  • Get secure log-in information from CTEL
  • Test that your assessment and all questions work as you desire
  • Get there early on the day to log on all PCs with the ID/Password
  • Get the marks sent to you by CTEL

There is a full detailed list of the steps which need to be followed are available within the Quality Assurance Guidelines for CAA - November 2013 (Word doc 33k)