1.What is feedback?
2.Why is feedback important?
3. How can someone in the workplace get feedback?
4. How can teams and organizations get feedback?
5. What is 360-degree feedback?
6. Where did the idea of 360-degree feedback come from?
7. In what formats is 360-degree feedback commonly available?
8. Why is 360-degree feedback considered more effective than single-source feedback?
9. What does formal 360-degree feedback provide that spontaneous person-to-person
feedback doesn't?
10. What are the best uses of 360-degree feedback?
11. Who are the best sources of 360-degree feedback?
12. How does the 360-degree feedback process work?
13. How often should 360-degree feedback be administered?
14. What are the major benefits of 360-degree feedback?
15. What are the most common concerns about using 360-degree feedback?
16. What are the most common misconceptions about 360-degree feedback?
17. How is 360-degree feedback different from personality assessment?
18. What’s the best way to compare 360-degree feedback systems?
19. What questions do you need to ask when evaluating a 360-degree feedback system?
20. How can you verify the validity and reliability of a 360-degree feedback
survey?
21. How important are national and industry norms?
1.
What is feedback?
You don’t see yourself as others see you. You may not
be aware that some of your workplace behavior is causing problems
for the people around you.
Feedback is input from others. It reflects the perceptions
of the people around you about behavior patterns, performance
outcomes, competence, ideas, proposals, etc. Whether formal
and highly structured or spontaneous and informal, the most
useful kind of feedback—and the hardest to accept—focuses
on areas needing improvement, about which you were unaware.
Sources. 360-degree feedback is consolidated
input about an aspect of workplace performance that comes from
more than one person. This information can come from bosses,
coworkers, peers, direct reports and customers.
For individuals. Feedback may be intended
for individuals, groups or organizations. Traditionally, feedback
for individuals has come in the form of performance appraisals,
one-on-one coaching from a supervisor and occasionally informal
one-on-one input from coworkers. 360-degree feedback is a systematic
way to gather and collect data and comments from a variety
of people about specific areas of performance.
For groups and organizations. Feedback to
groups, teams and organizations typically involves input from
stakeholders, constituents and customers. This can take the
form of consolidated 360-degree feedback data (e.g., surveys
administered by 20/20 Insight GOLD), suggestion programs and
climate surveys. Feedback for groups of people can be communicated
using computer networks, focus groups, brainstorming sessions,
strategic planning and business meetings.
Effectiveness. The impact of feedback depends
on the skills of the people giving it and receiving it. Effective
feedback isn’t critical, aggressive or emotional. It
describes specific, observable behavior, giving a realistic
balance of positive and constructive information. It’s
timely and focuses only a few issues, so that the person receiving
it can do something to improve the behavior
Copyright © 2004 Performance Support
Systems, Inc. All rights reserved.
2. Why is feedback important?
People need feedback. Most people don't see
themselves as others see them. For this reason, they often
don't understand the impact their actions have on others. They
have "blind spots." For example, people don't always
know when their work is appreciated, and they aren’t
always sure when they’re causing problems. Even well
intentioned and hard-working people rely on ingrained patterns
for success. Because they aren’t always conscious of
what comes naturally, they may be the only ones who don't know
that they’re adversely affecting the performance of their
group.
Performance improvement. Feedback is essential
to learning. If people don't fully appreciate their strengths,
how can they use them to their advantage? If they aren't sure
how their actions create problems, how will they know what
to change, and will they have the motivation to improve?
Motivation. People who take a professional
attitude toward their work want feedback. They want to know
what’s working and what isn't. They want to know if managers
are pleased with their performance. They want to contribute
to solutions, not be the cause of problems. They don't like
having blind spots, and they want to know how to improve. They’re
willing to invest in themselves to achieve better results,
because they know this will increase their value in the career
marketplace.
The challenge. As valuable and as desirable
as it is, constructive feedback is not a regular occurrence
in most workplaces. The two most common reasons:
• They usually find it uncomfortable to confront each
other about performance issues.
• Most people aren’t sure how to give feedback effectively.
• Very few people like accepting negative feedback.
Copyright © 2004 Performance Support
Systems, Inc. All rights reserved.
3. How can someone in
the workplace get feedback?
How are you doing at work? Here are some of the best ways
to find out.
The results of work. One of the best ways
to measure your performance is to look at your results. What
was expected of you? What individual and team goals were set?
What did you accomplish? Did you achieve a high degree of quality?
Customer comments. Customers are people who
benefit from your work. Who are they? What do they need and
expect? Are structured questionnaires given regularly to your
customers? What do they have to say about the quality of your
product or service?
Regular contact with your manager. Managers
can’t be everywhere, but good managers make contact regularly,
have a good idea about the quality of work and tell people
what they like and don't like. Do you know what your manager
expects of you? What does your manager have to say about specific
aspects of your performance?
Performance review. Most organizations have
periodic formal evaluations of performance, which are often
tied to decisions about development and compensation. For a
variety of reasons, most performance appraisal systems are
not very effective. However, they can include indicators of
performance and are best used as a catalyst for more thorough
discussions about performance between managers and staff.
Input from coworkers. Most peers and coworkers
are reluctant to hold a mirror up to the behavior or performance
of their peers. Either they don't know how to give effective
feedback, or they find it uncomfortable to do so. Typically,
they feel it’s the manager's responsibility to tell someone
about actions that don't measure up. Nevertheless, input from
the people who work around you can be extremely valuable; coworkers
are often more aware of strengths and shortcomings than are
managers.
360-degree feedback. Multi-source feedback
consolidates input from a variety of sources, such as bosses,
customers, peers, coworkers and direct reports. Managed by
computer, it organizes an extensive array of credible data
and narrative information, which should be analyzed for individual
development and human resource development planning.
Copyright © 2004 Performance Support
Systems, Inc. All rights reserved.
4. How can teams and organizations
get feedback?
Organizations, teams and larger groups of people need feedback
for the same reasons that individuals do. Groups of people
perform and achieve results, so they, too, need a mirror held
up to their performance to find out how effective they are.
Otherwise, they may not know how to improve.
This feedback comes from many sources. It
contains suggestions and ideas, not just opinions about performance.
As a source of new information, it helps an organization think,
learn and create. The systems that facilitate this feedback
are the organizational equivalent of a working brain.
Computer networks. Using a typical local
area network and computer workstations, people, groups or departments
can easily request opinions, suggestions or ideas from the
larger population they serve.
Customer satisfaction surveys. A traditional
means for gathering structured feedback from customers, these
surveys can gather targeted information that will tell an organization
how to change a product or service so that it is more appealing
to customers.
Focus groups. Another time-tested method
of sampling customer opinions, these highly structured, facilitated
meetings put customers in face-to-face contact with people
within the organization. The payoff is qualitative information
about preferences, needs, suggestions and ideas.
Brainstorming sessions. Under the guidance
of a skilled facilitator, these structured meetings mix people
from different functional areas in order to spark a rich outpouring
of new ideas, which can feed the production of concepts, analysis
and decisions.
Feedback in meetings. Many business meetings
are less than productive, but this need not be the case. A
savvy meeting leader can involve participants to share dialogue,
opinions, suggestions and ideas.
Climate surveys. If carefully constructed
and administered, computer-based surveys can determine how
employees think and feel about the systems of an organization.
The data can be used to make policy and structural changes
to improve effectiveness.
Suggestion programs. Employees know what's
working and what isn't. They get ideas for making an organization
more successful. If suggestions are welcome, easy to contribute
and rewarding to share, an organization can collect an amazing
number of ideas.
360-degree feedback. Multi-source feedback
is an efficient, computer-based way to measure areas of individual
and group performance from a variety of sources, such as bosses,
customers, peers, coworkers and direct reports. Data consolidated
from many groups can reveal much about an organization.
20/20 Insight GOLD is designed to support 360-degree feedback,
but as a flexible platform for multi-source feedback surveys,
it can be used to support many of the information processes
described above.
Copyright © 2004 Performance Support
Systems, Inc. All rights reserved.
5. What is 360-degree
feedback?
360-degree feedback is feedback from many sources. It’s
an efficient, computer-based mechanism for gathering structured,
highly objective measurements about areas of individual and
group competence from a variety of sources, such as bosses,
customers, peers, coworkers and direct reports.
Administration can be outsourced, using an external processing
service, or you can manage it yourself, using software like
20/20 Insight GOLD. External processing services use a variety
of data-collection methods, such as web-based input, paper
forms, and phone keypad. On-site programs may also use forms,
but most use paperless administration. Surveys may include
executive leadership, operational leadership, team interpersonal
behavior, team effectiveness, sales, service and instruction—aspects
of performance that are hard to measure.
Survey information is collected anonymously and entered into
a computer. Scores are averaged, and the results are reported
in a matrix of performance data. The feedback may be quantitative
(numerical scores) and qualitative (narrative comments)—much
more information than one usually finds in a performance appraisal.
People receiving multi-source feedback can focus quickly on
strengths and areas for improvement.
The process focuses on observable behaviors that have been
validated locally to relate to the workplace. Because it combines
scaled ratings from many sources, the data are as fine-tuned,
comprehensive, detailed and accurate as measurements of human
performance can be. Most 360 systems have extensive safeguards
for anonymity and confidentiality. These features assure participants
that 360-degree feedback is a safe way to share information
that is hard to measure and awkward to communicate.
While 360 is an extremely effective diagnostic tool, don’t
expect it to solve all your organization's performance improvement
problems. It is, however, the best possible input for what
should follow afterward: leadership and developmental programs.
360-degree feedback is a relatively new and quickly evolving
technology, so keeping up with new developments isn’t
easy. Even people who write articles about 360 aren’t
always familiar with the latest technology. You need to learn
what systems are available and what their capabilities are,
so you can determine which approach will best serve your organization.
Copyright © 2004 Performance Support
Systems, Inc. All rights reserved.
6. Where did the idea
of 360-degree feedback come from?
The idea of 360-degree feedback for individuals is a relatively
new concept. Forty years ago, some assessments experimented
with self-ratings compared with ratings from others. These
early tools borrowed from a number of traditions.
Peer evaluation. The service academies were
among the first to use peer ratings. Along with ratings from
tactical officers, cadets received ratings from other cadets
in their unit. These ratings were summarized by computer and
used for counseling. Decades later, organizations decided that
similar practices might work for them.
Personality testing. While 360 feedback describes
observable behavior, personality testing focuses on patterns
of mind and thought, which are inferred indirectly. Personality
testing in organizations began in the fifties and became a
popular practice in the seventies. From personality testing,
researchers learned the value of giving feedback to individuals
in the workforce.
Organizational surveys. A form of multi-source
feedback, climate surveys and employee opinion surveys have
been in use since the sixties. These tools create feedback
about organizations, not individuals. From organizational surveys,
researchers discovered how to use computers to collect, analyze
and report survey data.
Performance appraisal. Since the fifties,
most organizations large enough to have a personnel manager
have used some form of performance review. Most appraisals
are highly subjective in their formats and are linked to compensation
or personnel action. From this history, researchers have validated
the usefulness of periodic performance feedback.
The first attempts at 360-degree feedback were designed to
show leaders how their self-perceptions differed from the perceptions
of direct reports. Building on the above traditions, researchers
developed more comprehensive and credible surveys. These were
introduced in the 1970s. The first 360 instruments were focused
on executive management and leadership. Initially, managers
distrusted the concept of direct reports rating them. It seemed
strange and threatening. Nevertheless, managers experienced
the power of the feedback, and they began seeking input from
multiple sources. Today, tools are far more flexible, affordable
and easy to use than ever. Once used almost exclusively by
larger organizations to develop executives, today they are
used by all types of organizations and throughout the workforce
to assess a myriad of interpersonal skill areas.
Copyright © 2004 Performance Support
Systems, Inc. All rights reserved.
7. In what formats is
360-degree feedback commonly available?
No two multi-source feedback systems are exactly alike. They
have different approaches to administration, use different
media and feature different surveys. They are even priced differently.
This diversity of choices has occurred because not all 360
systems have kept pace with the evolution of computers and
software. Basically, 360 systems can be different in three
ways.
1. Mode of administration
• Central scoring. Administration is done by a vendor.
This method was most popular before onsite scoring and
report software was available. The most expensive approach,
it’s still an option for organizations that want
to outsource 360 processing, although an organization still
must handle many feedback process tasks.
• On-site administration. The publisher licenses an organization to use
scoring and reporting software on-site. This do-it-yourself approach costs much
less and takes less time. Some off-the-shelf programs feature "paperless" administration
and built-in customization, which dramatically increase flexibility and affordability.
2. Survey. 360 is best used to assess aspects
of work that are otherwise hard to measure. Not all 360s
survey the same things, and surveys of similar focus can
vary in content and quality of construction. Some programs
let an organization customize standard surveys or use locally
developed surveys. Individual 360 feedback surveys typically
focus on:
• Executive leadership and management
• Team leadership
• Team interaction and interpersonal communication
• Sales and service
• Presentation and facilitation
3. Media
• Scannable forms. This is the most cumbersome,
limiting and expensive way to embody 360. Paper surveys
are used by organizations that don't have computers.
• Diskettes. A form of "paperless" administration, they are easy
to use, permit unlimited flexibility, increase confidentiality and reduce cost.
• Networks. Presenting 360 on the local area network eliminates the need
for all hard media, including diskettes.
• World Wide Web. This method lets respondents give input anytime, anywhere,
as long as they have online access. It represents the ultimate paperless media.
• Phones. An alternative for organizations without computers, phone keypad
entry systems are expensive, cumbersome, limiting and hard to customize.
20/20 Insight GOLD supports both modes of administration,
all types of surveys and all media except scan forms and phone
keypad entry.
Copyright © 2004 Performance Support
Systems, Inc. All rights reserved.
8. Why is 360-degree feedback
considered more effective than single-source feedback?
One person's feedback is rarely enough to convince most people.
Whether the message is about strengths or areas for improvement,
the feedback recipient often wonders whether one person's opinion
is valid.
The most common example in the workplace is that of a supervisor
giving feedback to a direct report. Although managers are usually
the most responsible, capable and experienced people in their
units, it seems that some people don't welcome their feedback.
One problem is that most bosses are not in the best position
to observe day-to-day behavior. They have a lot to observe,
and they often spend a lot of time interacting with higher-level
managers.
Further, few managers are skilled at giving constructive feedback.
Effective feedback requires a different pattern of communication
than the one most people learned while growing up. To be effective,
a manager would have to learn special interpersonal skills
and exercise the discipline to make a pattern of them. In the
busy workplace, most managers don't bother. And when feedback
is perceived as criticism, most people are not likely to welcome
what a manager has to say.
In addition to day-to-day interpersonal communication, the
traditional medium for periodic feedback has been performance
appraisal. For many reasons, these formal evaluations of performance
are, at best, tolerated. Appraisal has carried the weight of
setting goals and tracking two aspects of performance—the
development of competence and the achievement of results. For
years, managers and employees everywhere have expressed their
dislike of performance appraisal and agree that it hasn't done
a very good job at either task.
Peers, coworkers, direct reports and customers usually have
more detailed information about how employees do their jobs.
Furthermore, these people care deeply about performance issues,
because when coworkers don't do their jobs, it affects their
work.
Getting feedback from many sources about a specific issue
may be desirable, but it’s impractical without 360-degree
feedback. Because of its structure, thoroughness and anonymity,
360 is much easier to analyze, believe and use than single-source
feedback. Using an advanced technology like 20/20 Insight GOLD
to focus on specific behaviors and ask for ratings and comments
from a wide range of people, multi-source feedback is a relatively
simple process.
Copyright © 2004 Performance Support
Systems, Inc. All rights reserved.
9.
What does formal 360-degree feedback provide that spontaneous
person-to-person feedback doesn't?
A frequent comment is that if people would just give each
other regular feedback, you wouldn't need 360. This mostly
valid notion overlooks certain realities of the workplace.
For one thing, effective informal person-to-person feedback
is relatively rare in the typical workplace. Most people don't
know how to give appropriate feedback. They find it uncomfortable,
and they’re concerned that what they say won’t
be well received. Also, they believe that it’s the manager's
job to handle this responsibility. Furthermore, the daily rush
of work usually doesn't permit giving one-on-one feedback the
time it needs.
Most of the time, spontaneous verbal feedback isn’t
delivered in a form that’s easy to make sense of. All
too often, it’s not specific, objective or tactfully
communicated. Even in the best case, not all the important
areas will be covered. Comments may be given piecemeal. Also,
everyday feedback tends to be from only one person, so the
recipient might be tempted to question its validity. Consistent
feedback from many people is much harder to discount.
Whether printed or saved in a computer file, the structured
reports of 360-degree feedback let you review specific aspects
of performance when you have the time to reflect on areas of
strength and high-priority areas for development. Without this
analysis, it wouldn’t be realistic to expect much developmental
planning or improvement in performance.
Spontaneous, informal feedback about a specific instance is
valuable because it’s timely and focused on a specific
issue. On the other hand, 360-degree feedback is a superior
option for diagnosing developmental needs. It systematically
surveys a carefully researched set of key behaviors and collects
feedback from a variety of sources. 20/20 Insight GOLD report
displays present ratings and comments in a way that makes interpretation
easy. Recipients can pick out the strong and weak areas, so
they can decide what to do about improving their performance.
Copyright © 2004 Performance Support
Systems, Inc. All rights reserved.
10.
What are the best uses of 360-degree feedback?
The best use of 360 is to identify priority areas for improvement
of individual performance. Because 20/20 Insight GOLD is a
highly customizable platform for 360-degree feedback, it has
a variety of mainstream uses:
1. Measuring hard-to-quantify aspects of performance. Many
aspects of work, especially business, technical and administrative
activities, are relatively easy to quantify and measure,
so it doesn't make sense to go to the trouble of surveying
a lot of opinions about them. Other key areas of performance
are hard to quantify. Typically, these involve the interpersonal
aspects of work, such as leadership, communication, sales,
service, negotiation and instruction. 20/20 Insight GOLD
lets you customize assessments for these applications:
• Executive coaching
• Leadership development
• Employee development
• Team development
2. Needs assessment. Most 360 systems will
aggregate individual data at group and organization levels,
providing the best possible performance-based needs assessment
data. Organizations can easily identify areas of skill strength
and deficiency.
3. Organizational surveys. 20/20 Insight
GOLD can be also be used to gather feedback for any kind
of group, department or other organizational entity:
• Climate surveys
• Customer satisfaction surveys
• Team effectiveness surveys
4. Performance review. 360 is best used
for measuring aspects of performance that are otherwise hard
to measure. While its primary usefulness is in gathering
data about competence for developmental feedback, it can
also be used to gather data about hard-to-quantify results
for performance feedback. For example, results such as project
completion, safety records and sales totals are easy to quantify,
and they make excellent goals and accountability measures.
A 360 assessment would not be needed for such areas. However,
important team outcomes such as morale, ethics, cooperativeness
and customer satisfaction are much harder to quantify. Surveys
can help a manager include these "soft" results
along with "hard" bottom-line results in the performance
review.
Copyright © 2004 Performance Support
Systems, Inc. All rights reserved.
11.
Who are the best sources of 360-degree feedback?
How do others see you? What are their impressions of your
work? These perceptions form the core information reported
by 360 feedback. Because of the administrative effort involved,
you probably won't seek this kind of input frequently. So when
you do, you should seek it from a variety of sources. Ideally,
you’ll involve people who are familiar with how you go
about your work. Four sources are most commonly used.
Managers. Also called leaders, coaches, supervisors
or bosses. These are people you look to for empowerment, leadership,
coordination, guidance and other forms of support. While they
may not be able to observe you all the time, their expectations
are important and you’ll be wise to discover their perceptions
of how well you do your job.
Direct reports. Also called employees, staff
or subordinates, these are people who look to managers for
leadership, coaching, coordination, and other forms of support.
In a real sense, these are the "customers" of leadership.
These people are profoundly affected by the qualify of a leader’s
performance, and they typically have a lot to say about how
well that person develops, inspires, empowers and encourages
people.
Coworkers. Also called peers, colleagues,
associates or team members, these people have the most detailed
knowledge of what you do every day. They’re probably
most interested in your ability to interact with them as a
team member.
Customers. Whether external or internal customers,
stakeholders or suppliers, these are people who benefit from
what you do. They’ll be eager to tell you how delighted
or how disappointed they are with your services.
If 20/20 Insight GOLD is your platform for 360, there’s
no limit to the number of respondent types or total respondents
you can include to give an individual feedback.
Copyright © 2004 Performance Support
Systems, Inc. All rights reserved.
12.
How does the 360-degree feedback process work?
Feedback processes vary from setting to setting. A number
of structured activities may take place before, during and
after assessment.
1. Before Assessment. It's never a good idea
to simply begin assessing people. Some of these preparatory
steps may be necessary.
• Educate people about 360
• Resolve issues related to 360
• Integrate 360 into the performance management system
• Develop customized competency lists
• Reinforce feedback skills
• Reinforce performance coaching skills
• Conduct orientation briefings
• Train administrators to use the 360 software
• Conduct pilot assessment projects
• Train internal facilitators to lead interpretation and planning sessions
2. During Assessment. Organizations that
want to take care of administration and scoring internally
can use the 20/20 Insight GOLD system software to manage all
aspects of the feedback process. If they want to outsource
these functions, they can get these services from Performance
Support Systems—the publisher of 20/20 Insight GOLD.
Assessment administration
• Set up a customized assessment project on the software
• Prepare and distribute rater diskettes (if used)
• Collect and transfer data to the system
• Customize, print and bind individual reports
• Customize and print needs assessment reports
Special application programs
• Level 3 training evaluations
• Customer surveys
• Employee opinion surveys
3. After Assessment. The payoff of 360 comes
in the follow-up.
• Facilitate group feedback sessions
• Facilitate supplemental feedback sessions
• Facilitate planning sessions
• Coach individual development planning
• Conduct developmental programs
Copyright © 2004 Performance Support
Systems, Inc. All rights reserved.
13.
How often should 360-degree feedback be administered?
20/20 Insight GOLD makes it easy and economical to collect
and report structured feedback and make it available to all
employees as often as they need it. However, most of the benefit
of 360 would be missed if 360 were limited to only one administration.
It’s not the kind of thing about which managers should
say, "We did it, we got what we could out of it, so let’s
move on."
To be sure, the first administration provides a wealth of
revelations about strengths and new areas for improvement—typically
more than a person can address in one cycle of development.
The feedback is believable, since it comes from many sources;
and the narrative comments are often perceived as the most
valuable input, since they are more descriptive than numbers
can be. Ideally, feedback recipients will focus on one or two
high-priority areas for improvement. If their development plans
are realistic, and if they implement them, major gains in personal
performance can be achieved.
People need time to experiment with new patterns and learn
from successes and frustrations. The ideal period between assessments
is between nine months and a year. It's enough time to make
changes and create new perceptions; yet, it's not so distant
in the future that the hard work of behavioral change can be
deferred.
The knowledge that a follow-up repeat assessment will be conducted
can increase a person's motivation to undertake the difficult
challenge of changing ingrained patterns. The people who want
to change will welcome a repeat assessment. They’d like
to transform weak areas into strengths, and they realize a
follow-up assessment will document their progress.
Furthermore, each cycle of assessment is an opportunity to
address additional skill areas. A flexible 360 tool like 20/20
Insight GOLD lets you do this.
Because 20/20 Insight GOLD may be used for a variety of applications,
the same tool you use for individual feedback can also be used
for customer satisfaction surveys, organization climate surveys,
needs assessment and level 3 evaluation of training.
Copyright © 2004 Performance Support
Systems, Inc. All rights reserved.
14.
What are the major benefits of 360-degree feedback?
People need feedback, and 360-degree feedback is the most
effective way to give them information about hard-to-measure
aspects of their performance. With a flexible feedback platform
like 20/20 Insight GOLD, you can address a wide variety of
survey and assessment needs.
Systematic performance evaluation. Day-to-day,
person-to-person feedback is exceedingly valuable. But feedback
based on one person's observations is unpredictable, and it’s
usually focused on only one aspect of performance. 360 lets
you ask for and get feedback from many sources about a comprehensive
array of closely related workplace behaviors. This input is
ideal for individual development planning.
Coworker feedback. Giving effect feedback
requires a certain amount of interpersonal skill. Most people
are uncomfortable in the feedback role—whether giving
positive or constructive feedback—and they’re happy
to leave this task to managers. Also, most people don't like
being negative, and they don't want to risk offending their
coworkers. 360-degree feedback gives people a safe vehicle
for giving their opinions about another person's work patterns.
Feedback from many sources. Feedback from
one source is better than no feedback at all. But even when
it’s from the boss, it’s still one person's opinion.
Consistent feedback from many sources is more convincing. Faced
with the hard work of changing behavior, people want feedback
to be thorough and credible.
Objective, quantified data about "soft" areas
of performance. Many important elements of performance
are hard to quantify. The most obvious examples are the interpersonal
aspects of work, such as leadership, team communication,
sales, service, negotiation and instruction. Because 360
combines scaled measurements from many sources focused on
detailed, researched aspects of behavior, the data are highly
objective.
Simplified feedback administration. While
administering 360 feedback used to be extremely complex and
cumbersome, advanced on-site 360 software like 20/20 Insight
GOLD makes survey administration as easy as using a word processing
program.
Data for individual development planning. The
wealth of objective feedback about areas of competence makes
it possible to accept the data and focus on priority areas
for self-improvement.
Data for needs assessment and HRD planning. The
data produced by collecting individual feedback can be aggregated
as averages for group and organizational analysis of strengths
and weaknesses.
Copyright © 2004 Performance Support
Systems, Inc. All rights reserved.
15.
What are the most common concerns about using 360-degree
feedback?
Readiness. To implement 360-degree feedback successfully,
managers and employees alike must understand what it is and
how it will be used. They need to see that adopting this technology
is more than an experiment. It’s going to become a beneficial
part of the culture. Ideally, an organization will have previous
experience with performance feedback, such as performance appraisal.
Confidentiality. Most people want to give fair and honest
feedback, but they don’t want to be punished for doing
so. If they suspect that people in authority will evaluate
their input, they may not want to give honest ratings and comments.
Ideally, feedback remains anonymous. Also, the people who receive
honest feedback don't want to be punished for receiving it.
You will want ratings and comments to be kept confidential.
Managers are responsible for coaching employees, but the organization
must decide what kind of summary information bosses actually
need.
Trust. If people don't trust the process, they’ll find
it nearly impossible to give honest ratings and comments. They
might even withdraw support or try to derail the assessment
process. Several aspects of 360-degree feedback require trust.
Feedback recipients are likely to learn where they need more
development. Who will handle or see this information? How will
it be stored? All parties must trust that the system guarantees
anonymity and confidentiality. If people are told that detailed
information will be given only to those who receive feedback,
these promises must be kept. If leaders say that feedback is
to be given for developmental purposes only, participants must
feel confident that it won’t be used later for pay or
personnel decisions.
Follow-up. The purpose of 360-degree feedback isn’t
self-awareness. It’s performance improvement. Organizations
that use 360 should be prepared to support these developmental
activities. 360-degree feedback can identify strengths and
areas for improvement, but it doesn’t go beyond that.
If an organization doesn't follow through with an individual
development program, the feedback may have no benefits. Even
worse, people may be upset that it was a futile exercise.
Impact on compensation. Although many managers believe that
360-degree feedback can help solve problems with performance
management, many of them don’t understand how to use
it properly. Most appraisal systems have been linked to compensation
and other types of personnel action. This is why validity and
fairness of ratings have been issues. It's difficult for people
to give objective ratings about someone else's performance
when they know it will have an impact on that person's pay
or career. For this reason, most organizations use 360-degree
feedback solely as a developmental tool.
Copyright © 2004 Performance Support
Systems, Inc. All rights reserved.
16.
What are the most common misconceptions about 360-degree
feedback?
"It can fix or replace a flawed performance
appraisal system." Managers often conclude
that 360 is a high-tech answer to problems with performance
appraisal. While 360-degree feedback probably should become
part of the solution, it has never been used effectively
to manage both performance development (competence) and
performance review (results). The main value of 360 feedback
is to give individuals information about the process of
performance—how their work gets done. It’s
not the best tool for evaluating performance results—what
gets done—which in most cases are quantifiable and
better measured by other means.
"It's only for managers and leaders." Twenty
or thirty years ago, the first 360s featured a fixed set of
measures focused on management or leadership. The primary market
was managers in fairly large organizations. In the early 1990s,
360s began to focused on team leadership and team interaction.
Today, the most effective 360 systems are like 20/20 Insight
GOLD: open platforms for general skills assessment, with software
that can set up and administer any locally validated assessment.
This means that everyone in an organization can benefit from
360-degree feedback.
"It's a program you do one time." Because
360 used to be so expensive, many organizations didn’t
think of it as a tool that could be used regularly. In the
past, a fair number of managers participated only reluctantly,
hoping to get it out of the way. 360-degree feedback is not
a program; it's a diagnostic tool. Therefore, it should be
administered periodically to track progress and create new
priorities for development.
"We can't afford it." Systems
that use paper forms and scanners were inherently expensive—all
that sorting, labeling, distributing, collecting, checking,
resorting and scanning, printing, collating and binding. Online
paperless systems eliminate most of the administrative burden,
producing significant savings. Using 20/20 Insight GOLD, it's
now possible to get the highest quality 360-degree feedback
for only a small fraction of the cost of older instruments.
"Installation of an on-site system involves
a major consulting project." The installation
of a 360 system that uses forms and scanners is complex,
and the software of such systems is typically difficult
to use, which equates to expensive installations and certifications.
However, 20/20 Insight GOLD is a state-of-the-art paperless
system that can be installed in a few minutes by the user.
"It's a hassle." This
is still true about paper-based assessments, but on-site "paperless" (network-
and web-based) systems like 20/20 Insight GOLD greatly simplify
administration.
Copyright © 2004 Performance Support
Systems, Inc. All rights reserved.
17.
How is 360-degree feedback different from personality assessment?
Both personality assessment and 360-degree feedback are survey
technologies commonly used in the workplace for human resource
development. They give individuals feedback for self-improvement,
but in some instances they have been used to make personnel
selection decisions.
Even though some employees may have the impression that the
two assessments are similar, they have quite different purposes,
and they gather remarkably different information. Even the
way validity is established is different.
Personality Assessment
Purpose: Describe cognitive or behavioral style and identify
psychological traits.
Method: Collect data from a single source—self or other.
Feedback: Constructs are the primary feedback. The questionnaire
items are used indirectly to infer these constructs.
Validity: Research tries to answer this question: Do the survey
items truly relate to survey constructs, and are the constructs
truly what they say they are—correlated to real-world
phenomena?
360-degree Feedback
Purpose: Diagnose skills by giving feedback about competence
and measure hard-to-quantify aspects of performance.
Method: Collect data from many sources.
Feedback: Survey items are the primary feedback, survey items
are used for this directly. Categories are not constructs;
they are used for sorting only.
Validity: Survey development tries to achieve this goal: Do
the survey items truly relate to the competence required to
achieve the goals of an organization, and do the survey items
address the most important aspects of competence?
Copyright © 2004 Performance Support
Systems, Inc. All rights reserved.
18.
What’s the best way to compare 360-degree feedback
systems?
Comparing 360 programs isn’t an easy task. It’s
even hard to find out which options are available. One book
written for this purpose was Feedback to Managers, by VanVelsor
and Leslie (Center for Creative Leadership, 1990). It does
a good job of comparing the options available at the time,
but most of the advanced 360 systems in use today didn’t
exist in the 1980s when this book was researched. The world
of 360 feedback today looks remarkably different from the alternatives
described in this book.
Also, today’s technologies look very different from
each other. Because each system was developed during different
stages of the evolution of the technology, most feature unique
capabilities and measure different kinds of things.
Most 360-degree feedback software is expensive, so it can
be a costly mistake to invest in a system without learning
what the technology does and checking out the options. It’s
important to ask the right questions. In the exploratory stage,
the task of gathering, analyzing and reporting the facts may
be delegated to a committee or a special project team. In addition,
a variety of stakeholders may want to be involved in the selection
process. All these people need to learn more about 360 feedback.
Discovering what's important is indeed a learning experience.
What counts most is hard information about capabilities and
price, not the length of time a system has been on the market,
ads in national magazines, booths at trade shows, or slick
brochures and catalogs, all of which say very little about
the quality of a 360 system.
We suggest a four-step approach to making the decision to
adopt a 360-degree system:
1. Contact 360 publishers for information.
2. Compare capabilities and price.
3. Evaluate samples and demonstrations.
4. Get the opinions of current users.
Step 1: Contact 360 publishers for
information
Each 360 feedback system is different, and comparing them
can seem like comparing apples and oranges. Getting information
about capabilities can be a daunting task.
To gather and screen information, you’ll need to communicate
with the publishers. When you talk to them, evaluate each interaction.
If the publishers you contact are not helpful during the information-gathering
stage, they aren’t likely to perform better later when
you need technical service and support.
• Did materials tell you what you wanted to know? Were
they easy to understand?
• Could you figure out what you get and how much it will cost?
• How easy was it to contact someone who could answer your questions?
• Did you get straight answers, especially about price?
Step 2: Compare capabilities and
price
The most efficient way to compare options is to consider capabilities
first. What capabilities are important to your organization?
Which systems have these capabilities? If your list of specifications
is lengthy, it’s possible that none of them do. Next,
with the "short list" of systems that best your needs,
focus on price. There are four price issues: (1) software,
(2) installation and training, (3) assessments, and (4) technical
support.
If you outsource administration services, there will be no
front-end installation costs. If you use an on-site system,
how much time and money will it cost to get the system ready
to conduct assessments? Get estimates that include these typical
expenses:
• Administration software Support materials
• Scanner (if required) Scanner interface software
• Installation services Administration training
• Technical support Upgrade/maintenance
Next, how much will it cost to assess people? Universally,
publishers charge for assessment services according to the
number of feedback recipients or respondents. This "pay-as-you-go" charge
covers the publisher's research, development, maintenance,
service, marketing and other costs of doing business. Typically,
centralized administration services are more expensive, because
more publisher services are involved.
Define the scope of your feedback program; then find out what
you’ll need from each publisher to make that happen.
Be sure to ask probing questions to uncover “hidden costs.” Then
calculate and compare the total costs, according to the publishers’ price
structures. Estimate costs over a three-year period. Heavy
use typically does not happen during the first year. Many publishers
have discount pricing for quantity orders. In addition to the
price per assessment, be sure to consider the cost of support
materials such as development planning software.
Some programs, such as 20/20 Insight, feature unlimited assessment
licenses for specific individuals, giving quantity discounts
based on the size of the company. When using 360-degree feedback
with large numbers of people, quantity discount licenses can
lower the unit cost of assessment significantly.
Step 3. Evaluate samples and demonstrations
It’s important to study what will be used in the assessment
process. When you ask for information, ask for product samples,
sample reports and online demonstrations. If these materials
don’t answer all your questions, it may be helpful to
have the publisher or a representative demonstrate the system.
You can learn a great deal about the publisher's ability to
support you by talking to the experts themselves. Most publishers
are willing to tailor their briefings to focus on content the
stakeholders need most. This kind of input can be very instructive,
but it can also be resource-intensive; be prepared to share
in the costs of travel and materials.
Better yet, conduct a pilot assessment project—an even
more revealing demonstration. Conducting an assessment project
with a small group can determine whether the instrument is
a good fit for your culture. Simply arrange for the assessment
services from the publisher. Using different groups, you can
pilot several systems at once.
Due to the significant expense of the on-site installation
of a paper-form/scanner-based system, it’s usually not
feasible to pilot test the hardware and software. Thus, the
demonstration will be limited to the assessment itself, using
centralized scoring services.
"Paperless" systems like 20/20 Insight GOLD are
an exception. If you’re considering an on-site program
that’s affordable enough and easy enough to install,
total expenses may be even less than central scoring services,
and you can evaluate all aspects of the system and process.
And at the end of the project you will already own the system,
so you can do other pilot projects at minimal expense or effort.
• Do the products look professional?
• Are they consistent with the best theory and practice?
• How effective are they?
• How easy are the materials to use?
Step 4: Get the opinions of current
users
At this point, you should already know which systems would
best meet your requirements. You may have already identified
a preferred option. The purpose of contacting current users
is to confirm the results of your analysis. Contact the references
provided by the publishers, and compare their experiences with
yours.
• Did the system you’re planning to use meet
their expectations?
• What challenges and problems did they encounter, and how did they resolve
them?
• Did they discover any unexpected costs?
• Have they received good technical support and service?
• How do they feel about working with the publisher?
Copyright © 2004 Performance Support
Systems, Inc. All rights reserved.
19. What questions do you need to ask when evaluating
a 360-degree feedback system?
Getting structured input about performance from the people
who work around you is profoundly beneficial. By the year 2000,
most organizations were using some form of 360-degree feedback.
But there are dozens of 360 options, and most of them are quite
different. The task of gathering basic information about the
capabilities and costs of these systems is a formidable one.
The first step is knowing which questions to ask.
360 has evolved considerably in the past ten years. It’s
fair to say that even though they are all classified as "360-degree
feedback systems," some are not systems, no two instruments
are alike, and the first-generation tools bear little resemblance
to the newer ones. One of the first issues many organizations
consider is whether to purchase one of these off-the-shelf
programs.
The challenge of developing your own 360 software.
The motivation to develop an in-house software program comes
from the desire to save money. It’s true that until recently,
the price of 360-degree feedback has been a limiting factor.
Some organizations have paid hundreds of thousands of dollars
a year for centrally scored services to assess only a small
fraction of their workforce. They may be able to justify this
expense for upper-level managers, but no organization can afford
centrally scored 360 services for employees at all levels,
where this kind of feedback is most needed. Until the mid-90s,
most small businesses, government agencies and non-profit organizations
could not afford 360-degree feedback at all.
To avoid the bother and expense of centralized scoring, organizations
turn to on-site scoring. However, many on-site systems are
fairly expensive to install, and a locally developed program
can seem like a reasonable option. The major argument is that
the program can be custom-designed to address the special needs
of the organization.
If external software engineers must be contracted to do the
job, the project usually proves to be outrageously expensive.
However, even if programming talent is available on staff,
creating a "home-grown" 360-degree feedback program
isn’t cheap. Typically, managers underestimate the time
needed to develop a program that provides for assessment set-up,
user interface, confidentiality protection, data entry, scoring,
database management and reporting. Even a rudimentary program
must address these basic issues and can take a team of developers
several months to complete, test and refine.
Organizations that take this approach eventually learn what
360 software publishers already know: users want a lot more
than a "quick-and-dirty" approach to assessment.
Managers are vocal about their needs, an amazing range of capabilities
are possible, and all of them require sophisticated programming.
That’s why 360-degree feedback programs are so different
and the technology is changing so rapidly. Unfamiliar with
the state of the art and under the pressure of time constraints,
in-house programmers often leave out features that turn out
to be essential. In spite of the investment, it is unlikely
that the finished product will satisfy an organization's needs.
In any case, creating your own 360 program doesn’t make
sense when a few thousand dollars can buy a state-of-the-art
system with the full range of capabilities, ongoing upgrades
and support.
The right questions
Once you know what your organization requires, you'll be able
to rule out many off-the-shelf programs immediately. You'll
know whether your organization can manage its own administration
on-site. You'll find out if 360-degree feedback is needed for
everyone, or just for upper-level managers. You can project
whether you’ll need applications beyond individual developmental
assessment, such as needs assessment and customer satisfaction
surveys. You will have learned about requirements for customization,
special report formats or developmental recommendations. Obviously,
not all systems will fulfill your requirements.
As you narrow your search, you need to ask the publishers
the right questions. The most important questions fall into
five groups:
1. What does it measure?
2. What costs are involved?
3. How easy is it to use?
4. How flexible is it?
5. How does it report the data?
Questions to ask when Comparing 360-Degree Feedback
Options— A Checklist
1. What does it measure?
• What skills or competencies are addressed?
• Exactly what is the focus of the assessment—what does it measure?
• What kind of measurement does the survey address—observable behaviors,
cognitive processes, attitudes, values, traits, style or other characteristics?
• Is a list of survey items available for review?
• Can the program assess more than one area, such as leadership and team
interaction skills?
• Can the system be used to assess any locally developed survey?
• How many items are included in each survey?
• Does it measure observable behaviors?
• Is each item an important part of the survey?
• Do the items begin with an action verb?
• Does each item represent only one observable behavior?
• Are the items written in language that any employee can understand?
• Does each item describe an important action?
• Does each item describe a specific action?
• Are items phrased in a consistently positive way?
• Are the items free of sexual, racial or cultural bias?
• Upon what research is the item set based?
• Is a written report of the research and development of the assessment
available?
• Upon what theory, model or set of assumptions is the standard item set
based?
• If the assessment is psychologically based, what research confirms validity?
2. What costs are involved?
• What are the on-site installation costs?
• How much do the administration software and site license cost?
• Will consulting services be needed to install an on-site system, and how
much will they cost?
• Will you have to purchase new equipment?
• Is certification training necessary, and how much does it cost?
• What are the per-assessment costs?
• What is the cost per person assessed?
• How many respondents can provide feedback at that price?
• What do you get with each assessment?
• Does the publisher charge extra for optional report formats?
• Does the publisher charge extra for consolidated group or organization
reports?
• Do comment reports cost extra? If so, how much?
• Are developmental recommendations a standard report option, or do they
cost extra?
• Are quantity discounts available?
• Are extended-use licenses available?
• Do these licenses apply indefinitely or for fixed periods?
• What are the costs for upgrades and other maintenance?
• What are the charges for technical support?
• Is a toll-free number available for technical support?
3. How easy is it to use?
• Will the publisher sell a license to use the administration
software?
• Is paperless administration an option?
• Can assessments be presented to respondents on paper forms?
• Can assessments be presented to respondents on diskettes?
• Can assessments be presented to respondents on a network?
• Can assessments be presented to respondents on the Web?
• Is a development planning guide or software available?
• Is it included in the cost of assessment?
• If sold separately, how much does it cost?
• Is it really a system? Are all support materials included?
• Is a software user's guide available?
• Are guidelines, lesson plans and program aids available for facilitators?
• Are developmental materials available for participants?
• If these resources are not included in the price of the system, how much
do they cost?
4. How flexible is it?
• Can the standard set of items be customized?
• Does the administration software permit customization by the user?
• Does the survey focus on only one area, such as leadership, or can more
than one set of survey items be used with the administration software?
• Can any conceivable set of survey items be entered into the software and
used for assessment?
• Can the scale be modified?
• Can the number of phases in the scale be changed?
• Can phase descriptors be changed?
• Can the item stem be changed?
• What will it cost?
• Can the types of respondent relationships be modified (e.g., Manager,
Peer, Coworker, etc.)?
• Can the system change the way respondent relationships are worded?
• Can new respondent relationships be added?
• If so, how many?
• How many uses does it have?
• Can it be used to develop assessment items?
• Can it be used to assess technical skills?
• Can it be used to assess skills in general?
• Can it be used for pre- and post-evaluation of training programs?
• Can it be used for customer satisfaction surveys?
• Can it be used for organizational climate surveys?
5. How does it report the data?
• What report formats are available?
• How many report formats are available?
• What do the available report formats look like?
• Do they meet your requirements?
• Can you select which report formats are to be used?
• Can you select which formats to use in a given report?
• Can you determine the sequence of the report sections?
• Does the system permit self-customization of report formats?
• Can you design original report formats that meet your requirements?
• Does it allow comments about each behavior?
• Is a comment report available?
• Does it provide for comments linked to and reported with specific behaviors?
• Does it report recommendations for development of specific behaviors?
• Are developmental recommendations available?
• Are specific recommendations available for each item in the survey?
• Do the recommendations include suggested readings?
Copyright © 2004 Performance Support
Systems, Inc. All rights reserved.
20.
How can you verify the validity and reliability of a 360-degree
feedback survey?
Is the instrument valid? Whether or not a publisher claims
an “instrument” is “valid” for any
kind of professional or for all organizations, you can’t
escape the necessity to establish local validity. The fact
that an assessment was validated in the context of other populations
doesn’t mean that it will be valid for your organization.
For this reason, customizable assessment platforms are the
current state of the art, because they can be adjusted to align
with local conditions. You can make them valid.
Questions about general validity are most important with instruments
that were developed in the tradition of psychological tests
with the purpose of measuring things that can’t be observed
directly, such as values, attitudes, styles and traits. The
issue is whether the assessment measures what it is supposed
to measure.
If you’re considering a 360 feedback tool that doesn’t
measure observable behaviors, the following questions about
validity may be important:
• Do dimension scores correlate (0.4 or higher) with
other credible measures of the dimension?
• Do the studies confirm meaningful correlations for all dimensions of the
survey?
• How many studies confirm these results?
These kinds of validity questions don’t apply to surveys
that give feedback about observable behaviors, which is the
most common case in 360-degree feedback. There is no question
that they measure what they say they measure. Data about the
items themselves—not inferred dimensions—are the
primary reason for feedback. The more important validity questions
are:
• Do the items describe important aspects of the work
of the people receiving feedback?
• Is the feedback desired by the organization?
This validity is established when stakeholders within the
organization have reviewed and revised the assessment and ultimately
declared it valid.
Copyright © 2004 Performance Support
Systems, Inc. All rights reserved.
21.
How important are national and industry norms?
360-degree feedback gives two kinds of information: (1) qualitative
(written comments), and (2) quantitative (average scores based
on ratings). Most participants affirm that comments contain
the most meaningful feedback. This isn’t surprising.
Scores may show which area of behavior is strong or weak, but
they can't explain why. The lesson is that scores shouldn't
be considered the main element of feedback. They should be
used to locate problem areas, so that the individual can get
clarification through written comments, supplemental in-person
feedback, reflection and coaching.
Set a goal based on comparison with own scores: When a person
examines feedback scores, the most natural question is, "What
do the scores mean?" Are they high or low? The only
way to answer is to compare them to something.
Other behavior scores: Is this score a relative strength or
a potential area for development?
Previous score for the same behavior: Has there been improvement
in the score over time?
Norms: How does this score compare with the same score for
other people?
• Group norms
• Organization norms
• Industry norms
• US national norms
If a person receives 360 feedback, analyzes scores, sets a
single development goal and actually improves behavior in that
area, the feedback process is an enormous success. It's not
easy to change patterns of behavior that have been ingrained
on a daily basis for decades. It's simply not realistic to
try to correct every performance problem in one campaign of
self-improvement. Recipients of 360 feedback should be encouraged
to set one major development goal, or two at the most.
Therefore, we can conclude two things about norms. First,
they aren't the most meaningful points of comparison. To determine
which areas of professional behavior need improvement, it's
more meaningful to compare low scores with high scores to discover
weak areas, or to compare present performance with past performance.
Comparing self with others doesn't indicate where someone needs
to improve. A person may score below the norm in three areas,
but even lower in three other areas. Or an individual may score
above the norm in all areas; that doesn't mean that there isn't
a need for improvement.
Local norms are more meaningful
However, even though norms are less meaningful, they may be
interesting to feedback recipients. If you wish to include
norms with feedback, the second conclusion about norms is that
comparisons with coworker scores are more meaningful than more
generalized populations. Comparing the scores of plant foremen
in a plastics factory with the scores of account managers in
a pharmaceuticals company is like comparing apples to oranges.
Comparing individual scores with national norms is like comparing
apples to fruit salad. It's far more meaningful for individuals
to know how they compared to other members of their work unit
or how they compared to other people in their organization.
National norms aren’t needed
The idea of comparing individual scores with industry or national
norms originated in the early years of 360-degree feedback,
when surveys were standardized instruments. Organizations everywhere
administered the same assessment, which typically contained
over 100 items established by research to apply to all managers,
regardless of industry, organizational culture or leadership
roles. This is an outdated approach, and today most organizations
don't use this kind of survey. They customize surveys to align
with the requirements of particular positions. In some organizations,
no two surveys may be the same. This provides targeted feedback,
but it also makes it impossible to compare with national norms.
Copyright © 2004 Performance Support
Systems, Inc. All rights reserved.
