360-degree feedback: how to avoid a disaster

360-degree feedback: how to avoid a disaster

By Curtiss S. Peck who can be contacted at: cpeck@asiintl.com;cspeck@execpc.com; www.asi-intl.com

1. Introduction

Multi-rater assessments and 360-degree feedback are popular. They can be very effective for coaching, performance improvement, career development, identifying training needs, and determining the effectiveness of training.

However, many people continue to look for quick fixes and silver bullets. Some have looked at 360-degree assessments as another way to solve all their leadership and organizational problems with the stroke of a pen and feedback from employees—who will surely be open. 360-degree feedback will give people the information they need to make intelligent decisions. Some believe that once people get the open and honest feedback, the information will be embraced and everything will be better.

The saddest part of how absurd this may sound is that many people do not even give this much thought to their initiatives prior to implementing a 360-degree assessment process. The results, in many cases, are disasters, which occur for a multitude of reasons. Let me tell you about three examples.

The first occurred in a health care organization. The Vice President of Human Resources was rated by his staff as being a strong, positive, and effective leader. He rated himself similarly. However, his boss, the Corporate Senior Vice President, did not rate him as being effective. At a scheduled meeting with the SVP, the VP HR asked to discuss the differences in perceptions. He sincerely wanted constructive feedback and wanted to meet the expectations of his boss. Most of all, he wanted to have a meaningful discussion that would contribute to his development.

The issue of different perceptions was raised early in the meeting. The SVP promptly said, "Obviously, everyone else is right and I am wrong. So, we’ll just go along with what others have said."

The utility of the process was quickly destroyed. The VP HR never received constructive feedback or coaching from the SVP. Tension between the two people remained until the VP HR resigned four years later.

The second failure occurred when surveys, that included written comments, were returned to the target manager. The manager confronted specific employees about their comments. Needless to say, the integrity of the process was compromised. It did not take long for word to spread throughout the company about this incident. Trust became a serious issue and the process was never repeated again.

The third example involved the assessment of an entire sales organization, from vice president to the field sales force. The company executed the assessment and feedback processes very well. However, there was no accountability for development plans, there was no follow up after the initial feedback meetings, and no training was offered or provided for clearly identified weaknesses in a majority of the sales people. To the amazement of senior management when they suggested a reassessment two years later, the regional managers refused to participate and the field sales people thought the whole thing was a joke and a waste of time and money.

Did these situations really occur? Absolutely! Could these problems been avoided? Of course they could have.

2. Don’t become the next contributor to a 360-degree feedback disaster.

This paper offers useful ideas on how to ensure that you get the maximum value from your multi-rater assessment process. This is a process rather than a program. When implemented properly, it is a long-term process that emphasizes development, capitalizing on strengths and accountability.

The terms target manager, ratee, and appraisee are used in this paper. They all refer to the person being assessed. Also included in this paper are references to 360-degree feedback for sales people, teams, peers, and others.

360-degree assessment and feedback can work, it can be successful, and all employees can embrace it—if it is done properly.

3. The process in action

This paper addresses the ideas around planning and implementing a 360-degree feedback process that focuses on development, communication, trust, openness, and a long-term commitment. The issues covered include:

>> Forming a process committee

>> Clarifying the purpose of the process

>> Ensuring support at the top

>> Being prepared to follow up after the assessments

>> Identification of core competencies

>> Selecting assessment instrument(s)

>> Training managers, raters, ratees, and coaches

>> Selecting raters

>> Implementing the process

>> Providing feedback

>> Action planning

>> Follow through

>> Tracking change over time.

------------------------------

>> Form process committee

------------------------------

The first step in developing a multi-rater assessment process is to form a process committee. The name of the committee is not as important as its membership. However, it is important to think of the message being communicated when selecting a name or stating the initial purpose of the committee. Ideally, the committee should include at least one credible and respected person from the senior management level who is willing to be the sponsor and champion for this process. If no one is willing to do so, you might want to reconsider whether your company is ready for a 360-degree feedback process.

Embarking on instituting a multi-rater assessment process requires the commitment of resources that must be approved by the senior management team. This is not a program or a quick fix. If that is what people want, do not go any further. Senior management needs to commit to a minimum of five years. Some companies, like Weyerhaeuser, have been fine-tuning their processes for over ten years. They are committed to a quality process.

In addition, the committee should include representatives from human resources, training or organization development, and representatives from employee bargaining units if the process will affect their members directly or indirectly. It is better to invite them in the beginning and decide that their involvement is not critical, than to find that they create barriers later because they misunderstand your intentions.

--------------------------------

>> Clarify the purpose

-------------------------------

The Process Committee needs to develop a clear statement describing the purpose of the multi-rater process. This information should be communicated to all employees. It is important that the main purpose includes a commitment to develop employees.

Some companies wish to use multi-rater appraisals for their performance reviews, along with aiding decision about wages and promotions. You are encouraged not to do that—at least not for the first couple of years. Fear and anxiety levels will elevate once you announce this process. Trust may become an issued. These feelings may last as long as one year or more for some people. Even outstanding performers will become somewhat anxious, despite what they say. This is something new and there is always a fear associated with sailing uncharted waters. Nonetheless, some people are justified in their feelings. It is difficult to hide poor performance with a valid and reliable process.

Be aware of activities going on behind the scene that are intended to undermine this process. Address the behaviour immediately, however, do so in positive ways. Try to identify people’s perceptions and concerns. Clarify misperceptions or misunderstandings. Use these situations as opportunities to promote the value and benefits of the process. Ensure people that this is not going to be a witch-hunt.

There is another important issue to consider as you develop your statement of purpose. Do not use your multi-rater assessment and feedback process to identify reasons to terminate an employee or for downsizing. Actually, if you are anticipating any downsizing, do not initiate this process until after the changes have been made. If you have employees who should be terminated because they are not responding to coaching and/or training, make those decisions based on the information you have about his/her performance (or lack of). The integrity of your multi-rater process will be damaged if there is any perception that employment decisions are resulting from that assessment data.

------------------------------------------

>> Ensure support at the top

-----------------------------------------

If you have one or more senior managers who are sponsors and champions, you are more likely to have a process that will be successful. Egos can be very fragile, even at the top. If someone feels threatened by the process, for whatever reason, he or she can undermine the process and have profound effects on future efforts to develop employees. The size of the company, and age, gender, and experience of the managers do not seem to make a difference. You need to plan for the unexpected. The more support you have at the top of the organization, the less likely the process will falter.

As you gain the support of top management, make sure that there is a commitment to following through with training, coaching, and other development efforts. As part of your plan, ensure that there is support for a reassessment one to two years after the first assessment. Despite what some people might say, there is little to no value in conducting reassessments in less time. Ideally, the process is most manageable and has the most utility if you conduct reassessments every eighteen to twenty-four months.

-----------------------------------------

>> Be prepared to follow up

----------------------------------------

One of the biggest mistakes that companies make when initiating a multi-rater assessment process is failing to prepare to follow up after the assessments and support the training needs of the employees. Training is not the only solution to identified areas of weakness, yet it is one that needs to be thought through and budgeted for.

Depending on the number of competencies measured, you may need to prioritise the greatest needs after the data is analysed. Depending on the availability of resources, including money, time, staff, and facilities, you may be able to respond to a portion of the needs during the first year. It is more important to do an excellent job of facilitating the learning and development of employees in a few areas than to be moderately effective in all areas.

It is important that your plan includes opportunities for employees to receive performance coaching and training, that there is accountability throughout the organization, and that there will be a reassessment one to two years later. Reassessments allow you to measure changes in behavior, as well as the effectiveness of training. Reassessments also allow you to identify and track organizational issues that impact change in both positive and negative ways.

-------------------------------------------

>> Identify Core Competencies

-------------------------------------------

Having a clear purpose and support for this process are important. Now you need to build the process that best meets your needs. You need to identify the competencies that are necessary for success in specific areas. You need to consider each appraisal group separately. Examples are managers (top, middle, first level), peers, teams, and field sales people.

The Process Committee may wish to identify the core competencies or you may wish to form employee focus groups to assist with this process. Employee focus groups are a wonderful way to increase the involvement of more employees. As more and more employees become active participants, they generally become ambassadors for the process.

Ask for volunteers for the focus groups. Ensure that you have multiple representatives from each level of the organization (executive, middle, and first level) and that each major functional area is represented. You will want to include more people if you choose to develop the assessment instrument internally rather than purchase a standard instrument. More is shared below about designing survey(s) internally.

After you identify the core competencies, see if they cluster in a model that will be easy for people remember, that the interrelationships are apparent and logical, and that the results will be useable. When it comes to the assessment of competencies, most people want and need the measurement of behaviors that can be operationalised. Many inventories contain great categories and interesting questions, nevertheless, they severely lack the clarity needed to be operationalised. The utility of the assessment process increases when people can make direct transitions from the questions to decisions about their behavior, i.e., what is meeting the expectations of others and what needs to change.

--------------------------------------------------

>> Select An Assessment Instrument

--------------------------------------------------

There are basically two options available for selecting an assessment instrument, 1) a commercially produced standard inventory, or 2) develop a custom inventory. Having clearly defined competencies helps when looking for a standard inventory.

Some of the benefits of selecting a standard inventory include:

# Availability

# Validity and reliability

# A proven track record with other users

# Training for users

# Scoring software that is already developed

# Scoring software that can be licensed so reports can be produced in house, thus reducing costs.

# Support material already developed

# External support by experienced and reputable company

There are also disadvantages to choosing a commercially produced assessment. Some people make the mistake of choosing an inventory based on the slickness of the packaging or the size of the company and fail to closely evaluate the utility of the instrument and reports prior to making a buying decision.

Some companies provide inventories that measure fairly narrow competencies, and of course, only those competencies that they also provide training modules for. This is not bad; you just need to be aware of these possible limitations. Many training companies are not effective assessment companies and vice versa.

A standard inventory may not be an exact match to your needs. In other words, the inventory may not measure the competencies you have identified as being important. Some companies, however, are willing to work with you to develop a modified version of their standard inventory. This can often be done for a nominal fee. On the other hand, some companies are unwilling to adjust their surveys or allow you to license their scoring software.

In addition, costs vary significantly between service providers. Expensive does not mean better. Generally speaking, the average cost for complete multi-rater surveys and reports ranges from $125 to $175 per target manager, team or sales person. Costs can be reduced if you administer and score the inventories in house, or if you use your intranet for administration of the assessments.

If you choose to develop a multi-rater inventory internally, you can use or expand the focus groups that already exist. When choosing this route, you may initiate this part of the process prior to identifying categories of competencies. Instead, each focus group brainstorms a list of behaviors that they believe are important to success as part of each appraisal group—for example, managers, teams, field sales people, and peers mentioned above.

Cluster the behaviors according to common themes, e.g., Communication, Planning, Teamwork, etc. The Process Committee, possibly with the assistance of an outside consultant, should develop seven to ten questions for each theme or category. Each question should describe a single behavior. Compound questions lead to unreliable results.

This process is not easy, nor is it quick. Nevertheless, the time you spend in the beginning may prevent a major problem later.

Present the list of questions to the focus groups for review, clarification, and modification. Ask the focus group members to individually select the top three questions for each category. Identify the five or six questions that receive the highest ratings for each category. These should be the questions you include in your survey.

Choose a rating scale. Five- to seven-point scales are usually effective. Generally, a scale exceeding seven points does not add value and less than five points fails to offer adequate differentiation between values.

Field test the survey and perform statistical analysis on the results. Remove any questions that are shown to be unreliable. During general use, continue to monitor and evaluate the reliability and validity of the survey until you are certain the data are robust.

You also need to decide whether or not written comments will be solicited. The research by Dr. David Antonioni, University of Wisconsin-Madison, suggests that appraisees want written descriptive feedback from appraisers. Our research and experience shows that including written comments can lead to the demise of the survey process. Appraisers are often fearful of being identified by their written comments. There is also a greater tendency for appraisers to inflate their ratings, a conclusion supported by Dr. Antonioni’s research.

Experience has shown that appraisers may use offensive language when describing behavior or their feelings about the appraisee. If the statements are edited as they are entered in the scoring program, employees may lose respect for the integrity of the process. If the comments are not edited, people may be hurt emotionally.

Therefore, the author recommends, as a general rule, that written comments should not be included. If you have the right questions and the proper rating scales, there is more than sufficient information in the report from which to make decisions and formulate development plans.

An effective survey clarifies expectations of raters. The author’s research and Dr. Antonioni’s research found that simply measuring present behaviour does not provide appraisees with clear information about people’s expectations. Dr. Antonioni writes, "Without a clear understanding of the expectations, they (appraisees) are less able to take constructive action to change unsatisfactory behaviour."

An approach that the author has found to be effective is to have raters answer each question twice. If the rating scale is based on the frequency of behaviour, e.g., never to always, it is convenient for raters to respond to the frequency of both present and expected (desired) behavior. The larger the gap, the greater the need for change.

Interestingly, this approach occasionally reveals behavior that occurs too frequently. In those cases, the individual needs to reduce the emphasis or frequency of some behavior and redirect his or her energy toward increasing other behavior.

The benefits of developing survey instruments internally include the possibility of achieving greater employee acceptance and commitment to the 360-degree feedback process. You can contract for external support as needed.

The downsides of developing assessments internally include the time and costs involved, which initially are high. The process requires additional time to ensure that the assessment inventory is reliable and valid. Support material and scoring software also need to be developed.

-----------------------------------------------------------------

>> Train managers, raters, ratees, and coaches

-----------------------------------------------------------------

Each of the steps above contributes to educating all employees about the purpose, content, and process of a 360-degree feedback process. You also need to ensure that people are properly trained. The more prepared people are to fulfill their roles, the more successful this process will be.

The training of managers, at all levels, includes an understanding of their role in this process. They need to be supportive. They need to be prepared to meet with their direct reports to discuss the results of the assessments. They need to become actively involved in providing mentoring and coaching, resources and time needed for training, opportunities to capitalize on identified strengths, and eliminate or reduce organizational barriers that hinder top performance.

Raters need to know why they are completing the feedback instruments, how to properly complete the inventory, and why their honest feedback is needed. They also need to be assured that their individual responses (identity) will remain confidential. Of course, the responses of the ratee’s boss will not be confidential because there generally is only one person in this category.

The individuals receiving the feedback need to understand that they will be held accountable to developing an action plan and for following through on the plan. They should be informed that the assessments will identify both strengths and soft spots. Training and coaching will be available as needed. They will also be responsible for meeting with their respective manager to discuss their reports and to reach agreement on their development plans. Those plans must then be discussed with their direct reports and others who may have completed inventories.

Coaches are often people from human resources, training, or organization development. Some organizations also train peers and others to be coaches. It is important that those selected to be coaches are competent themselves. They need to be taught good coaching skills. They must be active listeners.

Above all else, coaches must respect each person’s privacy and the confidentiality of the reports and information discussed during coaching sessions.

--------------------

>> Select raters

-------------------

Selecting those who will complete the assessments is a process not to be taken lightly. In the case of a management survey, it is ideal to offer all direct reports the opportunity to complete the assessment. In some cases where a person’s span of control includes 20, 30, or more direct reports, the cost of including all people becomes prohibitive. In these cases, the recommendation is to include a minimum of four people or twenty-five percent of the possible contributors, whichever is greater. The larger the percentage, the more likely the data will reflect an accurate picture of the appraisee.

A few years ago 54 people, who were members of 13 different employment groups, completed assessments on a manager. The manager said, "I influence all of those people and I want and need feedback from all of them."

Interestingly, all 54 people returned their inventories without any prodding. Were the results glowing in all cases? No. However, they did help identify areas where the manager needed to place more emphasis.

There are different methods for selecting contributors. First, a word of advice. Do not include someone who was recently disciplined or who is about to be dismissed. Their responses are likely to skew the data and may dilute the utility of the report.

When there are a number of possible contributors and you want feedback from six to eight people, have the target manager submit the names of ten people to his/her manager. The manager reviews the list, removes the names of people believed to be biased, and distributes surveys to the remaining people.

The same holds true for surveys of sales people where customers are selected. Because the responses rate from outside contributors tends to be lower, distribute twice as many inventories as you would like returned.

For example, at Skill Corporation, now S-B Power Tool Company, surveys were mailed to fifteen customers, per sales person, in each of two categories. The process was so well managed that 64% of the customers returned the surveys, despite the absence of any incentive.

Prior to the surveys being mailed, the sales people contacted each customer, informed them of the nature of the survey and asked them to complete the survey once received. The surveys were mailed in company envelopes along with a personalised letter to the customer from the Vice President of Sales.

Customers were given a specific deadline for returning the surveys. Also included was a postage-paid envelope addressed to us (an outside processing centre). The Monday following the cut-off date, each sales person was told how many surveys had been returned. Because the surveys were anonymous, the sales people contacted each customer and asked for their cooperation in returning the completed surveys.

Another method is to ask for volunteers. A problem occurs when few or no people volunteer. This method is not recommended.

A third option involves the random selection of participants. A division of Eaton Corporation used such a process. We imported the names of all employees into a special computer program. Imported data included departments, reporting relationships, employee names, among other data for sorting. The program randomly generated the names of raters, printed cover letters, labels for surveys, and labels for mailing envelopes. Each rater also received a pre-addressed, postage-paid envelope to return the surveys to us for scoring.

----------------------------------

>> Implement the process

----------------------------------

There are logistical issues that have not been addressed, but, generally speaking, you should now be ready to implement the process. This includes ordering, labeling and distributing the surveys. The returned surveys need to be organized and scored. Follow up with people who have less than an acceptable rate of return.

--------------------------

>> Provide feedback

--------------------------

There are many ways to provide feedback to the ratee. The most ideal approach is to have a coach meet one-on-one with each ratee to discuss their feedback and their responses to the feedback. Coaches need to help each person avoid making excuses or finding blame. Coaches should help individuals develop an initial action plan, which will be discussed with the individual’s manager. Sensitivity and confidentiality are important issues.

Coaches may also need to work together with specific individuals and their managers to finalize action plans and prioritise developmental needs. Development plans and lists of priorities should be given to Human Resources and/or Training so training needs can be prioritised and initiated.

-----------------------------------

>> Track change over time

-----------------------------------

360-degree feedback is part of the process of developing employees and improving the effectiveness of the organization. It helps identify strengths that may be under-utilised, as well as soft spots or weaknesses that need strengthening. It also offers the opportunity to evaluate the effectiveness of training and other change initiatives.

Conducting reassessments every one to two years allows you to objectively measure change, both positive and negative. It is a wonderful way to clearly communicate that the organization is serious about developing people and that employees will be held accountable for their development.