Description of BS ISO/IEC 23988:2007 2007

1.1 General

This Standard gives recommendations on the use of information technology (IT) to deliver assessments to candidates and to record and score their responses. Its scope is defined in terms of three dimensions: the types of assessment to which it applies, the stages of the assessment “life cycle” to which it applies and this Standard’s focus on specifically IT aspects.

1.2 Types of assessment

This Standard is relevant to a wide range of assessments, including:

• assessments used in education (both compulsory and post-compulsory), training and compliance (e.g. compliance with legislation relating to health and safety or financial services);

• assessments of knowledge, understanding and skills (i.e. “achievement tests”), but not psychological tests of aptitude and personality;

• high-stakes assessments and examinations and also low-stakes assessments used for feedback on progress, identification of learning needs, self-assessment and remediation;

• assessments which include feedback, as well as those which provide only a result;

• both fixed-date, test-windows and on-demand assessments;

• both items which can be scored by computer and the delivery (but not scoring) of items or tasks which are delivered using IT, but have to be referred to a human marker for scoring; however, the use of IT for scoring lengthy free-text responses (“essays” and similar) is excluded;

• a wide range of computer-scorable assessments, including not only “objective tests” (multiple-choice and other item types), but also assessments of keyboarding skills and software use;

  NOTE Items can include graphics, multimedia and access to data, case studies, etc. This Standard does not, however, cover the use of specialist simulations (e.g. of industrial processes or driving/piloting).

• assessments taken in universities, academies, colleges, high-schools, training centres, schools and assessment centres and also assessments taken in less formal settings, including learning centres and in the workplace;

• assessments which are set, taken and scored within a single organisation (for example a university or a company) and assessments taken in an organisation separate from the one responsible for the assessment content;

• assessments delivered in a variety of ways, including on-line, on local networks and on stand-alone computers.

Clause 9 of this Standard is also applicable to the use of IT for the transmission of candidates’ coursework.

1.3 Assessment life cycle

Although assessment procedures vary, the typical life cycle of assessment consists of the following steps:

1. identification of need to assess;

2. design of outcomes/assessment methodology;

3. preparation and calibration;

4. pre-registration (includes payment);

5. distribution¹⁾;

6. authentication (includes identification)¹⁾;

7. delivery¹⁾;

8. response return¹⁾;

9. scoring, result determination and/or feedback¹⁾;

10. data return¹⁾;

11. analysis;

12. appeals;

13. certification.

Preparation of assessment content is outside the scope of this Standard, but where pretesting is undertaken using IT for delivery, the relevant clauses of this Standard should be taken into consideration.

1.4 Focus on specifically IT aspects

The emphasis throughout this Standard is on the additional or different measures which should be taken as a result of the introduction of IT. Measures which are common to both paper and IT-delivered systems are either omitted or covered in only general terms.

This Standard does not cover the purely pedagogical aspects of assessment preparation or scoring, but does cover the use of IT to implement pedagogical decisions and the interface between assessment content and IT delivery. The quality of the assessment content and the scoring procedures are of crucial importance to the validity and reliability of the final result, but fall outside the scope of this Standard.

Compliance with this Standard does not indicate that the assessment is pedagogically sound. This Standard should not be interpreted as reducing the need for assessment sponsors to follow good practice in developing assessments, including standardisation and attention to validity and reliability.

This Standard does not cover scoring by human markers, but does cover:

• IT scoring which may be subject to later review by human markers;

• the IT transmission and delivery of assessments partly or wholly to be scored by human markers and the return of the resulting responses for scoring.

This Standard covers the use of IT to provide automatic feedback and instant results, but not result-determination requiring human decision-making or intervention.

1.5 Focus on principles

The aim of this Standard is to set out principles and good practice, but not the details of the means by which they are to be achieved. It is therefore possible to follow the recommendations using a variety of technological or procedural approaches. This Standard is not specific to any particular hardware or software platform.

In many areas the principles outlined in this Standard will be supplemented by the specific regulations of assessment distributors.

1.6 Compliance

Assessment sponsors, assessment distributors and assessment centres may claim compliance with this Standard if they comply with all the clauses or subclauses applicable to their role (see table below).

Notes to the clauses indicate the role(s) to which each clause or subclause is applicable.

This Standard is applicable to both high-stakes and low-stakes assessments, but some clauses or subclauses are applicable only to high-stakes assessments; this is indicated in Table 1.

Table 1 Application of ISO/IEC 23988

The scenarios given in annex A illustrate how different types of organisation might need to comply with different clauses of this Standard.

About BSI

BSI Group, also known as the British Standards Institution is the national standards body of the United Kingdom. BSI produces technical standards on a wide range of products and services and also supplies certification and standards-related services to businesses.