Assessment in South Africa: Finding Method in the Madness

Assessment in South Africa: Finding Method in the Madness

By: Elaine Saunders

Elaine Saunders is a consultant, and registered Industrial Psychologist with the Health Professional Council Of South Africa and can be contacted on ipcons@iafrica.com about competency-based assessment.

--------------------------------------------------------------------

1. Introduction

The topic of assessment generally, elicits suspicion and confusion in the minds of many. Assessment methodology has been the sacrificial lamb in terms of many assessment processes and outcomes. It has borne the brunt of the frustration of many who have been passed over for jobs, training opportunities and career development. This criticism is in many instances not unwarranted. The misuse of assessment by untrained people is significant and many serious injustices are performed in assessment processes.

This article looks at the role of assessment in business, and examines some of the ethical issues. It aims to provide something of a framework, and a set of recommendations to assist subscribers to conduct assessments fairly in the workplace, and to achieve value from assessment processes.

The article will look at assessment within the context of current labour legislation, and this discussion will reveal that there are different assessment situations, which need to be treated differently. It will also show that assessment is a vital tool in complying with the various demands of labour law.

A further section is dedicated to the issue of identifying competence since this is at the heart of fair assessment. Lastly, we will examine who should be implementing and managing assessment in the business environment, and what skills are involved in applying fair, valid and reliable assessment procedures.

This article is the first in a 4-part series on critical issues in the field of assessment in South Africa and 'finding method in the madness'.

2. Labour Legislation – Equity, Skills Development, and Employment Law

Current labour legislation defines the standards surrounding the application of assessment in organizations rather loosely. It says that all tests and assessment methods must be valid and reliable.

The problem lies in the fact that not many people have the skills required to establish whether a test is valid or reliable. The only source of information for this is the person who sells the test instrument themselves and that is not exactly an objective source. You have probably heard about the interesting things that can be done with statistics to make them say the things we want them to say!

The statement around testing in labour legislation suffers from a number of shortcomings. The statement that assessment methods should be valid and reliable represents a necessary but insufficient condition for the application of fair and scientifically valid assessment.

Issues, such as the qualifications of people who carry out assessment, are not discussed at all, neither are any standards for assessment methodology defined. Because of this it has been possible to take advantage of the lack of rigour and integrity in the legislation, and extend the realm of assessment into the hands of people, who are really far from qualified for the task at hand.

Assessment is an art in the hands of a skilled expert; it is a recipe for disaster when used badly. Assessment is not about the instrument or the methodology; it is about the skill of the person who uses that methodology.

Psychologists are trained professionals, registered with the Health Professions Council of South Africa. They are like doctors – would you tell your doctor which medical instrument or process to use on you to heal your illness? Probably not – the doctor as a qualified professional uses his or her knowledge and specialist expertise to apply the correct remedial process. An in-depth understanding of intelligence theory and the theory of human behaviour is prerequisite for the application of ethical assessment procedures.

I am not however suggesting that nobody other than a psychologist can be involved in assessment in industry. It is possible to train people in business to conduct assessments, but very often the training applied tends to focus on the application of a particular instrument, and it neglects to tell people how to address the ethical and human side of assessment. For example:

>> how to feed back assessment results, particularly negative ones;

>> how to choose appropriate assessment instruments for the specific assessment context;

>> how to evaluate an assessment instrument or process in terms of legal compliance, fairness, reliability and validity, are but a few of the skills that anyone involved in assessment in business must have as part of their toolkit.

Personality assessment must remain the realm of the psychologist, particularly where interpretation and feedback is concerned. Other types of trained assessors can handle the administrative side of personality assessment, but the interpretation of test results must remain the domain of the professional. The discussion on who can assess is expanded on in a later section of this article.

Let’s take a look at assessment within some of the legal contexts –

# Equity

The role of assessment in the achievement of equity goals has been largely overlooked, and the results of this oversight are now being felt in the corporate environment.

The policy has, thus far, largely been one of ‘plugging in’ people from so-called designated or target groups into positions in order to get the numbers right. Very little prior assessment of ability and suitability for these positions has been undertaken.

Six to twelve months down the line (or longer depending on when the equity initiative was implemented) performance issues are raising their heads, and of course, labour legislation makes it very difficult to reverse an appointment decision. It would have been so much better to have invested in a solid assessment process prior to making these appointments, than having to deal with performance problems, their related implications for productivity and their potential for unfair labour practice claims.

The identification of learning potential is crucial in the context of equity appointments. Some extremely valuable work has been done by behavioural scientists in this country to develop culture fair, valid and reliable methods of assessing learning potential in people from equity target groups.

This type of assessment will tell the employer whether, and to what extent, an individual will benefit from further training and development. It will also enable the employer to assess whether an individual has the cognitive ability to deal with the demands of the position to which they intend to appoint them. Prior assessment of learning potential will allow the organisation to avoid inappropriate appointments together with the costs associated with the negative outcomes of such appointments.

It is really worth doing - the available methodologies are legally compliant and designed specifically for our South African conditions. The ones that I have seen are based on sound cognitive and intelligence theory and are well researched, and validated right here in South Africa. They are also non-verbal assessments, which is a critical element of culture fairness. Non-verbal assessments minimise the disadvantage that occurs in language-based tests, where the test takers are not mother tongue English speakers (assuming that the test is in English).

Learning potential assessment methodology has been designed to elicit information on learning ability whilst minimising the effect of previous advantage. It is structured around the work of theorists such as Vygotsky and his theory of the ‘zone of proximal development’ (ZPD).

The ZPD indicates a range of ability as the word ‘zone’ suggests. It identifies a point at which an individual should be in terms of cognitive ability in relation to their age and past development and secondly the point at which they are able to reach if sufficient quality development and learning opportunities are provided to them. (Hence the concept of potential (i.e. a futuristic concept) rather than ability (focussing on current ability).

The measurement of learning potential moves away from traditional evaluation of cognitive ability which is based on measuring past learning, usually in a school environment, and tests the constructs of intelligence which relate to an individual’s ability to respond to learning opportunities. Because of this it is able to minimise the effect of poor education and economic disadvantage that many equity target groups have been exposed to.

The question that the Equity Act requires employers to ask when making appointments is: Does this person have the potential to do the job, not can he/she do the job right now?

Of course this needs to be seen in context – i.e. some positions require that the incumbent enter the job and ‘hit the floor running’ whilst other positions allow for more training time before optimal performance is reached.

It is in this latter context that the equity person fits. The crucial issue is that if you assess for learning potential then you must provide the training and development (learning opportunities) or you cannot expect the incumbent to achieve optimal levels of performance. The assessment of learning potential must always go hand in hand with development.

# Skills development

Once again the value of being able to assess learning potential is evident in the evaluation of people who are being considered for further development.

The ability to predict whether an individual is capable of benefiting from training is not only invaluable, but I would argue that it is essential when one considers the vast investments being made in skills development in organisations today. The assessment process supports a return on investment – there is simply no other way of knowing whether the money spent on a person’s training and development will yield rewards for the company in the long term.

# Assessment and the National Qualifications Framework (NQF)

If one talks about assessment and skills development in the context of the National Qualifications Framework then assessment takes on a slightly different nature.

We need to distinguish between assessment which tries to make predictions about the future – i.e. a person’s ability to perform in a job which they have not occupied before, the ability to benefit from training etc., - in these instances we are talking about psychometric or psychological assessment.

In the context of NQF assessment we are assessing the outcomes of training or the person’s ability to perform to certainly clearly laid out unit standards. There is a significant difference in these two types of assessment.

>> One requires the rigour of scientific theory, and the application of skilled professionals to make predictions about the unknown.

>> The latter, i.e. NQF type assessment, is much less complex in that it requires an assessor to observe an outcome against clearly laid out criteria, generally of a practical and technical nature. You can easily observe and make judgments about the outcomes in terms of a quality of a welding process for example.

It is a completely different situation, to assess someone’s ability to be a good welder, when they have never done the job before.

NQF based assessment relies mainly on observation in assessment and it is relatively easy for assessors to be trained in this kind of assessment. No leaps of inference have to be made from the test outcomes to a different domain, e.g. the prediction of behaviour.

In NQF assessment the work either conforms to the laid down standards or it doesn’t. It is thus a different and very specific form of assessment, which is well supported by standards and training for the people who work within it. However, the principles and ethics of assessment are generic to any form of assessment and these must be embedded in the training of NQF assessors.

I often wish that psychometric, or psychological assessment, was subjected to the same rigour and controls that NQF assessment is subjected to, but more on that later.

# Employment law

This section relates to the role of assessment in a recruitment context. It is here where employers are vulnerable to claims of unfair labour practices in their use of assessment technology. It is also here where employers are subjected to a barrage of consultants eager to offload the latest ‘flavour of the month’ in regard to assessment products.

It is often the lot of the recruitment professionals to make judgements about assessment processes and instruments, without the backing of knowledge and skills in how to make these judgements. In the section on the ‘Art of Assessment’ you will find some guidelines on how to evaluate an assessment instrument.

The essence of inculcating fairness into your recruitment process is to ensure that whatever knowledge, skills or behavioural attributes you assess, you can tie these firmly to the requirements of the job. Identifying competence is therefore the first step in an assessment process. There is a tendency to focus primarily on choosing the right instrument – that is not what is important – it is like getting the cart before the horse. You have to know firstly what it is you want to measure, before you can choose an instrument to measure it with. Once this has been done, then questions in regard to the fairness, reliability and validity of the instrument can be answered.

2. Identifying competence

As discussed earlier, if we are to assess only job related competencies, as the law requires, then we have to be able to identify the competencies in a particular job that we wish to assess.

Competency is defined as the knowledge, skills and behavioural attributes necessary to perform in a specific job.

>> Knowledge refers to prior formal education and work experience. This component of competence needs to be flexible and should be stated in such a way that it represents the absolute minimum requirement.

The reason for this is that the law states that we may not impose educational standards and experience – if a candidate without these can demonstrate competence in a position, then we may not discriminate against them. Therefore, it takes some careful thinking and flexible handling of this component, to ensure that no unfair and unjustified barriers are placed on the recruitment process.

As far as assessment is concerned the knowledge component is normally assessed by the scrutiny and validation of the candidate’s CV or biographical form and by careful referencing with past employers. Referencing is a critical part of the evaluation of a candidate simply because research has shown that past performance is an excellent predictor of future performance.

It is often somewhat controversial and recruiters sometimes shy away from what can be a potentially dangerous evaluation tool. However, carefully structured questions based on identified competence are very valuable and will not place referees in an awkward position, because they will not be asked to comment on issues unrelated to the job for which the candidate is being considered.

>> Skills are normally defined as those practical and/or technical skills required to perform a job. For example, typing skills, in an IT context it may be the ability to manage networks, or maintain a desktop computer.

The skills component is normally fairly easy to assess because it is observable. Work samples, i.e. small samples of the work to be performed presented in an assessment format are very useful for assessing skills – the classic example here would be a typing test, but this type of assessment can be extended to other types of technical skills, with careful design in the hands of a professional.

>> The behavioural component of competence is perhaps the most challenging and is also the area where culture fairness is really a significant obstacle. It is also the area where the skills of a psychologist – a specialist in behavioural evaluation – are essential to ensure that a fair, valid and reliable process is implemented.

You will find that the assessment of behaviour becomes more relevant the more senior the position. The competency profile of a senior manager will consist of a much larger behavioural component that those positions lower down in the organization, which tends to have more of a technical focus.

Management theory will tell you that the most important transition managers have to make as they climb the corporate hierarchy is to leave behind their comfort zone of technical competence and become the managers of people and strategy. In this sense the design of a competency profile can incorporate change initiatives and can be the benchmark for the development of a competent and effective manager.

The assessment of behavioural attributes involves the evaluation of personality and the prediction of future behaviour around certain competencies like leadership, interpersonal skills, stress tolerance, etc.

The components of competence must be selected through the scrutiny of the performance required in the job. Assessment normally involves the application of personality assessments by psychologists, or through the design and application of assessment centres which focus on the observation of behaviour in simulated settings. The latter also requires the assessor to have advanced skills both in observation and the interpretation of behaviour on which the prediction of future behaviour will be based.

It is not uncommon for line management, or HR people to be trained as assessors for assessment centres. However, this training must include all aspects of the assessment of behaviour, including the generic principles and ethics of assessment. A registered psychologist should be responsible for the design of the assessment process and the exercises used within that process. They should also be responsible for the training of the lay assessors and for the supervision of their assessments.

The identification of competence is not the end of the line. Once competency has been identified it is necessary to define the outcomes or results that people are required to achieve, utilising their specific set of competencies. For example, one can have very good leadership attributes, but if they are not applied effectively to achieve results through people, then they are of no benefit to the organization.

In identifying competence you are asking the question: What qualities must this person have? In identifying outcomes you are asking what must they achieve with these qualities: e.g. must have excellent interpersonal skills (the competence in question) in order to achieve optimal outcomes in negotiations with unions ((results in the case of an IR Manager)

You will find that you are assessing competence in recruitment and development situations, but you are assessing outcomes in performance management. All these are forms of assessment, utilising different methodologies for different purposes.

3. Who can assess

The Health Professions Council of South Africa (HPCSA) (used to be called the SA Medical and Dental Council) stipulates that only a psychologist may be involved in personality assessments or ‘C’ tests as are they are categorised.

‘C’ tests are defined as any assessment instrument that measures behaviour or personality related attributes. It does not mean that because a test has not been classified by the HPCSA that it necessarily is outside their jurisdiction. A test that measures personality is a ‘C’ test, classified or not, and is subject to the same restrictions in application that a formally classified ‘c’ test has to adhere to.

Although I do not entirely agree with the way in which test categorisation and control is implemented by the HPCSA there is much to be said for their policy of restricting the use of psychological assessment to the hands of qualified professionals. I say this because I have seen so much damage caused by the use of assessment instruments in the hands of lay assessors, who do not have the advanced level of skill and knowledge required to understand and interpret the personality of others.

So, to answer the question, who can assess?

All personality assessment should only be carried out in the hands of a qualified, registered psychologist. This condition applies whether psychologist tests/questionnaires are being used or, behaviour is assessed by observation in assessment centre exercises.

Psychometrists and trained lay assessors can however be involved in the administrative elements of personality assessment, such as test administration and the marking and recording of results. This is especially so in computer based testing.

Other types of assessment however, such as the technical and practical skills, which are observed in NQF types of assessment and in the evaluation of technical and practical skills in the work environment for selection purposes, can be carried out by lay assessors, providing they have completed an appropriate training program.

It is highly recommended however that such lay assessors are trained, supervised and mentored by a qualified, registered psychologist. I say this because the principles and ethics of assessment are the same regardless of what element of competence is being assessed.

4. The art of assessment

I refer to assessment as an ‘art’ to emphasise the need for specialist skills and ability in a process which requires analytical thinking, creative ability, application of complex knowledge and the ability to interpret facts and make predictions based on these interpretations.

As I have said previously fair, valid and reliable assessment is not about which instrument you choose to apply, it is about the way in which you go about the process, how you use the instruments and the integrity and ethics which you apply to the process.

The ‘art’ of assessment involves the correct choice of instrument for the assessment context, the application of fair procedures in assessment and the accurate interpretation of test data in relation to the prediction of job performance.

It is far more than just assessment methodology although this is an important part. It extends to the manner in which you treat the test candidates, the confidentiality of the process and test data, the provision of accurate feedback, the transparency of the process and a number of other ethical issues that are part and parcel of the psychologist’s toolkit.

Since this is not a perfect world and many of you are placed in the position where you will have to choose assessment methodologies for your organization, as best you can, without the advice and support of a psychologist. To assist you in this process here are some guidelines to apply when evaluating assessment products:

>> If you are considering purchasing a personality assessment instrument, ask who the supervising psychologist is. There should be one. Ask for contact details so that you can ask any questions that you have of the supervising psychologist.

>> Look at the language factor – is it a verbal or non-verbal test? If the test is language based (i.e. verbal) – consider the population group that you will use it on – are they proficient in the language of the test – proficiency would mean that the can read, write and speak the language of the test fluently.

>> Look at the instructions of the test – are they clear and easily understandable?

>> Insist on doing the test yourself – most test vendors will provide this experience free of charge as part of the product evaluation process.

>> Ask to see the technical manual – again, there should be one. The information in this document should allow you to view the results of validity and reliability studies.

>> Look and see if studies have been done on different cultures and what the group differences in scores are. This will give you an indication of possible culture bias.

>> If a technical manual is not available ask for information on validity, reliability and culture fair studies.

>> Ask to see the norms which have been developed for the test – check the population groups on which the norms were developed – correlate the characteristics of that population group with the kind of people you wish to use the test on.

>> Check on what training is provided and what the content of the training program is. The training should include assessment principles and ethics, as well as technical information on the administration of the test.

>> Check the report format – does it give you what you need?

>> Check on costs – some vendors offer different types of reports and different charges apply.

>> Where computerized tests are concerned, check on the nature of the back up and support that the vendor will provide.

>> Check on the knowledge base of the vendor in regard to assessment. Make sure that they can support you with information regarding the assessment process and that they are able to guide you in ensuring a fair, valid and reliable assessment process.

>> Ask whether the vendor will support you in the event that the assessment process is challenged from a legal perspective. They should be willing to do this.

5. Conclusion

There are many forms of assessment and many different contexts within which assessment occurs. Different assessment methodologies apply, different qualifications and skills are demanded of the assessors and process will depend on the purpose of the assessment.

All behavioural, and personality based assessment must be carried out by a registered psychologist with skills and experience in assessment in a cross-cultural context, as is the nature of the assessment environment in South Africa.

The identification of competence, and correlation of assessment with job requirements is essential for fair assessment. The application of non-verbal assessment methods is critical in minimising culture bias. The assessment of learning potential in equity appointments is crucial in maximising development opportunities for people within designated groups.

Assessment within the context of NQF as well as the assessment of practical and technical skills which involve purely the evaluation of defined, observable results can be undertaken by lay assessors, provided a high standard of training has been applied. The assessment processes should be monitored and controlled by a psychologist.

Assessment results directly and significantly affect people’s lives – it is a barrier through which they must pass before they can be afforded the opportunities that provide them with an improved quality of life. It is an extremely responsible process – handle it with care and place it within the hands of the professionals.