» Fact Sheet: Validation of assessment resources

Fact Sheet: Validation of assessment resources

November 1, 2021 Sukh Sandhu

News

Compliance requirements and guidelines:

This Fact Sheet has been developed and produced to assist Registered Training Organisations (RTOs) in understanding the compliant validation practices as part of the learner resources to comply with a number of clauses of the Standards for Registered Training Organisations (RTOs) 2015.

Clause 1.9

The RTO implements a plan for ongoing systematic validation of assessment practices and judgements that includes for each training product on the RTO’s scope of registration:

when assessment validation will occur
which training products will be the focus of the validation
who will lead and participate in validation activities
how the outcomes of these activities will be documented and acted upon.

Clause 1.10

For the purposes of clause 1.9, each training product is validated at least once every five years, with at least 50 per cent of products validated within the first three years of each five year cycle, taking into account the relative risks of all of the training products on the RTO’s scope of registration, including those risks identified by the VET regulator.

Clause 1.11

For the purposes of clause 1.9, systematic validation of an RTO’s assessment practices and judgements is undertaken by one or more persons who are not directly involved in the particular instance of delivery and assessment of the training product being validated, and who collectively have:

1. on or prior to 31 March 2019:

vocational competencies and current industry skills relevant to the assessment being validated
current knowledge and skills in vocational teaching and learning
the training and assessment credential specified in Item 1, or Item 2, or Item 4, or Item 5 of Schedule 1.

2. on or after 1 April 2019:

vocational competencies and current industry skills relevant to the assessment being validated
current knowledge and skills in vocational teaching and learning
the training and assessment credential specified in Item 2 or Item 5 of Schedule 1.

Industry experts may be involved in validation to ensure there is the combination of expertise set out in a) or b) above.

Source: https://www.asqa.gov.au/standards/training-assessment/clauses-1.8-to-1.12

Interpretation of the regulatory requirements

This Fact Sheet outlines the requirements and responsibilities of registered training organisations (RTOs) for conducting compliant validation practices.

According to the Standards of Registered Training Organisations (RTOs) 2015, you are required to implement a quality review process (Clauses 1.8, 1.9, 1.10 and 1.11).

Clause 1.8a requires that the RTO’s assessment systems comply with the assessment requirements of the relevant training packages or VET accredited courses.

Clause 1.8b requires RTOs to ensure that the evidence gathered is valid (one of the Rules of Evidence) and that assessment processes and outcomes are valid (one of the Principles of Assessment).

These requirements must be met and demonstrated in all assessment policies, procedures, materials and tools of the RTO. Clause 1.8 primarily relates to the development (or purchase) of the RTO’s assessment resources.

Assessment validation has been strengthened in the Standards for RTOs and the requirement is to:

Develop and implement a comprehensive plan for ongoing systematic validation of assessment that includes all training products on the RTO’s scope of delivery (Clause 1.9)
Validate the assessment practices and judgements for each training product at least once every five years with at least 50% of products to be validated within the first three years of each five-year cycle (Clause 1.10)
Ensure that validation is conducted by one or more suitably qualified persons, who are not directly involved in the delivery and/or assessment of the training product being validated. (Clause 1.11).
These clauses relate primarily to the actual delivery and outcomes of the RTO’s assessment systems, including the performance of the RTO’s assessors.

Important bits of information

Criteria	Explanation	Comments
Validate the assessment practices and judgements for each training product Note: In order to completely (100%) validate a training product, what is the validation time frame?	Five years after the training product is initially listed on the national register or included in the scope of the RTO.	It is an ongoing process and will renew after every five years if a training package is not being updated/superseded, or removed/deleted from the national register.
Validate the assessment practices and judgements for each training product Note: In order to partially (50%) validate a training product, what is the validation time frame?	Three years after the training product is initially listed on the national register or included in the scope of the RTO.	It is an ongoing process and will renew after every five years if a training package is not being updated/superseded, or removed/deleted from the national register.
Why do you need to do validation more frequently? (Best industry practice)	If you identified that risk indicators exist for validation to occur more frequently.	The risk indicators may include: The use of new assessment tools; Delivery of training products where safety is a concern; The level and experience of the assessor; Changes in technology, workplace processes, legislation, and licensing requirements; Qualifications identified by the regulator as ‘high-risk’; Assessment conducted under a third-party agreement; The number of locations where assessment is conducted; The number of students; The mode of assessment such as RPL or on-line; Feedback from clients, trainers and assessors; and Audit and validation history
How many people do you need to conduct the validation?	One or more suitably qualified persons, who are not directly involved in the delivery and/or assessment of the training product being validated. If a team approach is used, the assessors who are directly involved in the assessments being validated may participate in the process, however, they cannot contribute to meeting the team requirements of Clause 1.11. If an individual approach is used, validation must be undertaken by a person who meets all the requirements of Clause 1.11.	Validation is a collaborative process. The team must hold collectively: Vocational competencies and current industry skills relevant to the assessment being validated Current knowledge and skills in vocational teaching and learning The TAE40110 Certificate IV in Training and Assessment (or its successor) or the TAESS00001 Assessor Skills Set (or its successor). Validators can be employees of your RTO, or you can seek external validators. The trainer and assessor who delivered/assessed the training product being validated: Can participate in the validation process as part of a team Cannot conduct the validation on his/her own Cannot determine the validation outcome for any assessment judgements they made Cannot be the lead validator in the assessment team.
Do you need to keep the records of validation?	It is important to keep the records of all validation activities and validators as auditors might ask for it during audit activities and for managing continuous improvement processes at an RTO.	You need to keep for validators information for the following reasons: To demonstrate that the RTO’s assessment system can consistently produce valid assessment judgements. Validation is undertaken by one person or by a team of people. The RTO must ensure the review process is completed by people who collectively hold: vocational competencies and current industry skills relevant to the assessment being validated current knowledge and skills in vocational teaching and learning, and the TAE40110 Certificate IV in Training and Assessment (or its successor) or the TAESS00001 Assessor Skills Set (or its successor). A regulatory body such as Australian Skills Quality Authority (ASQA) may request evidence of it during audit activity.
Reasons why validation of assessment and learner resources should be systematic and ongoing.	A unit of competency needs to be regularly reviewed to ensure that it meets current industry and regulatory requirements, therefore, you must have a systematic and ongoing mechanism to track any changes and to ensure all your resources are up-to-date and current.
How to schedule validation	The first step is to develop a “validation schedule” used to validate each training product (AQF qualification, skill set, unit of competency, accredited short course and module) on your scope of registration. A validation schedule is a five-year plan and each training product must be reviewed at least once in that five-year period, and at least 50% of your training products must be validated in the first three years of the schedule. Your RTO might choose to validate its training products more often, for example, if risk indicators demonstrate that more frequent validation is required. Indicators of risk might include: the use of new assessment tools delivery of training products where safety is a concern the level and experience of the assessor, or changes in technology, workplace processes, legislation, and licensing requirements. Once you have your validation schedule you need to complete a validation plan with dates and timelines. The more detailed your plan is with regards to who (needs to be included), when (what date and for how long), and what (which units are being validated, what information will be needed, from where are we getting it) the better your outcomes for your validation will be.	It is important RTOs have a clear understanding of the terms being used before they commence developing assessment validation plans and schedules under Clauses 1.9 -1.11. For example, in the context of the Standards for RTOs: What is a training product? What is the intent of assessment validation? What is a statistically valid sample?
What is statistically valid sampling?	A statistically valid sample is one that is: large enough that the validation outcomes of the sample can be applied to the entire set of judgements, and taken randomly from the set of assessment judgements being considered. Use ASQA’s validation sample size calculator for more information. It can be found here: https://www.asqa.gov.au/news-publications/publications/fact-sheets/conducting-validation#validation-sample-size-calculator
Validation outcomes	You must keep all records in a soft-copy/scanned format or in hard copy. The records must include all the tools used to conduct validation such as assessment resources, validation forms and checklists, profiles of validators etc. The validation outcomes should identify recommendations for improvement to the assessment tool, assessment process or assessment outcome. Sometimes the validation outcome can identify critical issues in the collection of valid evidence. When this occurs, you may: increase the validation sample size to assist in identifying patterns of issue validate completed assessments from other units of competency to see if the issue is spread across the whole of the qualification, and look for patterns of error (for example, consider if it is one assessor making invalid judgements—this could indicate the assessor requires further training in competency-based assessment). Your validation plan must clarify how you will document and act on validation outcomes. For example, an assessment validation checklist addressing the principles of assessment and rules of evidence may be utilised to document the validation outcomes. Outcomes of validation may be acted upon through your RTO’s continuous improvement processes. Your RTO must have a records management process to retain the evidence of the validation. You should retain evidence of: the person/people leading and participating in the validation activities (including their qualifications, skills and knowledge) the sample pool the validation tools used all assessment samples considered, and the validation outcomes. If the validation outcomes recommend improvements to the assessment tool, you should implement these recommendations across all training products, not only those included in the sample. If you make changes to the assessment tool, complete quality checks and review the revised tool prior to implementation.	During an audit, the auditors are looking for evidence that you have established a system that assures the quality of assessment in your RTO. This will include: having processes and tools that enable validation at all stages of the lifecycle of a training product in your organisation; having a plan for validation that is implemented successfully in your organisation; being able to demonstrate that you have managed who participates in the validation process; being able to demonstrate that your assessors have made quality assessment decisions; and your assessment practice has improved as a result of validation activities.

Description:

Validation is a process of checking the assessment tools, methods, judgements, evidence and processes to ensure that the training product meets:

Principles of Assessment – i.e. valid, reliable, flexible and fair
Rules of Evidence – i.e. valid, authentic, current and sufficient
The judgment made by the trainer/assessor is benchmarked with colleagues or industry experts
There is sufficient evidence to support the judgment of the trainer/assessor
Whether the requirements of the Training Package or accredited course have been met.

Typical benchmarks used during the validation process include:

National training packages which are developed by Skills Service Organisations (SSOs)/ Industry Reference Committees (IRCs) and can be found on the training.gov.au website.
Units of competency consist of competency standards and need to be unpacked so that those validating the assessments can compare the actual competency against the tools being validated.
Industry standards and consultation will vary, and these standards form the basis of the skills and knowledge required to perform work roles.
AQF Guidelines and Framework
Information provided to candidates, assessors and third parties
Legislation relevant to the assessment such as privacy, health and safety, anti-discrimination, copyright law and so on.

Assessment system

Documents required for conducting an effective validation session, in the RTO’s assessment system, includes but is not limited to:

Validation related documents:

Validation plan
Validation schedule
Validation record or validation form
Validation register
Validation report form
Continuous improvement form
Continuous improvement register
Pre-assessment validation documents

Assessment resources:

Unit assessment pack/student pack
Trainer assessment pack/assessor pack
Mapping document
Assessment evidence according to a sample size

Other documents:

Training and assessment strategy
Feedback forms
Unit of competency
Companion volume/implementation guide
AQF framework
ACSF framework

You will be required to evaluate if the assessment resources meet:

Training package requirements (application, elements and performance criteria, foundation skills, performance evidence, knowledge evidence, assessment conditions)
Principles of assessment; fairness, flexibility, validity and reliability
Rules of evidence; valid, sufficient, authentic and current
The appropriate level of difficulties (AQF Level)
Provide sufficient and clear instructions
Record any appropriate adjustments

What should you consider?

There are a number of things that you must consider when planning the validation, they are:

The responsibilities of who is accountable for what should be discussed in detail.
- Awareness of when a training product gets superseded, deleted or removed and their implications
- The validation requirements applicable to each training product
Requirements for documentation
- Having a compliant validation plan/validation register and validation schedule
- Processes and tools used for validation at all stages of the lifecycle of a training product
- Evidence of the suitability of individuals part of validation panel
- How the assessment practices have been improved as a result of the validation activities