Government Technology

A new generation of young people is leaving school and meeting a squeeze on both sides. If they choose to pursue higher education they are faced by increasing demand and competition for university places, or alternatively they may decide to enter into the workforce, where they face a troubled job market with a similar imbalance between supply and demand.
    
As a consequence, those wishing to make themselves a more employable prospect are ever more urgently seeking some means of distinguishing themselves from everyone else, and one way in which they can do this is by producing some solid, recognisable evidence of what they have learned and achieved.
    
Training and learning opportunities are understandably in high demand, and developments in their delivery have kept pace with such demand. Conferences such as Learning Technologies serve as a showcase for some dazzling examples from the cutting edge, such as Virtual Learning Environments (VLEs), Learning Management Systems (LMS) and e-Learning platforms; and already demonstrate an impressive breadth of offerings from the education and training sectors.
    
In addition to these advanced learning approaches, assessment has also embraced the benefits of technology and this crucial measurement of learning is becoming increasingly important when candidates are taking their first steps into higher education or when seeking employment.
    
Nowadays, there is much more on offer from assessment. With the advent of new technology comes the opportunity to create items (questions or tasks to be assessed) that are accurate, stimulating, and a true measurement of the skills or knowledge of the candidate.
    
Modern types of items can demonstrate not just retention of facts, but can show accurate evidence of understanding concepts, or the application of knowledge to specific scenarios or job roles.
    
The goal of the continued development of testing technology is to improve measurement, either by measuring the same knowledge or understanding better than before, or by measuring something more than or different from before.
    
However, even with all of this technology at hand, there is one important caveat to remember, and that is not to let the tail wag the dog – technology must always be the servant, never the master. The process of developing your test to incorporate some of these innovative item types is a highly complex one, which should start with good test design and may involve consultation work with expert psychometricians. Following the test design phase, the process may require custom software development. Also, the more complex the item types being utilised, the more potential operational issues must be considered. Therefore, test creators must stay focused on the purpose of the test and what it is intended to measure, and the technology must simply be a means to that end, meaning that the most appropriate item type may actually be the simplest!
    
Item types can be classified as either Selected Response or Constructed Response. The former includes multiple-choice, but also much more, and the latter requires the candidate to create their own response without provided options.

Selected Response

• Multiple-choice. With the latest computer-based tests (CBT), multiple-choice can be constructed with more possible answers, both to reduce the opportunity to guess, and also to more directly assess traits. Psychometric analysis behind each item allows the test to assess the candidate, not only on whether or not they get the item right, but also on the answers they choose, to build up a picture of the cognitive process.
• Multiple-response. In a “select all that apply” item type, CBT can score more easily – it can mark dichotomously or with partial credit for getting some, but not all, correct answers.
• Ordered response / Select and classify. These types of items can measure a different type of cognitive task from what can be measured by multiple-choice items. CBT makes these less confusing – for example, the candidate can drag-and-drop historical events into correct order or prioritise a list of elements in rank order; or drag and drop on-screen elements into different categories.
• Hot spots. On an on-screen graphic, such as a map, spreadsheet, diagram or image, candidates can select areas. CBT gives a better, more direct and more accurate measurement of this.
  

• Entering quantitative, verbal or figural responses – such as giving a short answer to a verbal question, by marking or drawing an on-screen figure. This item type offers an increased cognitive challenge.
• Essays. Computer scoring can supplement and complement human markers. Essay marking software is extremely sophisticated today - it is not proposed to replace the human input altogether, particularly in more creative, arts-based disciplines, but advances have been made and have proven their validity in more mature CBT markets, such as the USA.
• Problem solving vignettes. The test-taker is asked to solve a problem though a series of steps or items. This can be structured: all test takers progress through items in the same order; or unstructured: test takers are taken down different branches of questioning depending on their item responses.