What is a Likert scale?
When a psychologist or sociologist wants to understand, say, the character of their patient or how someone behaves in a group, they usually ask questions following a Likert scale. This scale captures the intensity of the respondent’s feelings for a specific item. This scale requires respondents to indicate their agreement with a statement or give a rating. For example:
Organizational theory is classified as a subfield of sociology, and that's perhaps why the Likert scale has permeated into the vast majority of assessments conducted in organizations these days. Employee engagement surveys, for example, are totally Likert. And so are many assessments of strategy implementations, IT processes, sales approaches, and supplier evaluations.
Disadvantages of Likert
Aggregating the disadvantages of the Likert scale makes you wonder why they are still in use in organizations. Here are the most frequently cited disadvantages in scientific literature and elsewhere.
Socially acceptable answers
If the survey asks ‘In our team, everyone gets a chance to voice their opinion,’ it is likely that respondents will be tempted to give socially acceptable answers if that survey also asks for the respondent's name or personnel number. Therefore, these assessments are often anonymous, which makes, for example, knowledge sharing an impossibility. A variation of socially acceptable answers is ‘acquiescence bias,’ where respondents agree with statements as presented, for instance, when they work in a company culture that encourages and incentivizes eagerness to please.
Blamestorming and emotion
So, the assessment is anonymous, and the can of worms may be opened. ‘I am paid adequately for my work’ (I completely disagree). ‘My manager gives me the attention I need’ (I completely disagree). ‘I always give my best effort’ (I completely agree). Then emotion may take over, and nuance disappears. My salary is important, so I immediately completely disagree.
Sample biases
A technical term. This means that in very small groups, there is not enough response to calculate an average opinion. I might score ‘In our team, everyone gets a chance to voice their opinion’ 6 out of 7 because my previous boss was some sort of a drill sergeant. My colleague, on the other hand, will score our same team 2 out of 7 because his previous team was a sort of loved-up hippy group. If there are not enough respondents, the average score may significantly differ from reality. In large groups, sample bias means that respondents of different backgrounds interpret the question/statement differently. For example, North Americans tend to score more extreme (an ‘extreme response style’), and respondents from Asia-Pacific more neutral (‘central tendency bias’).
Self-leniency
We know from psychology that 75% of people think they are above average. It's an unconfirmed rumor that 92% of managers think they are above average. Self-leniency is the tendency for respondents to round off their answers upwards.
Evaluator bias and -leniency
When managers or consultants interpret the Likert outcomes, there is often evaluator bias: an interpretation of how the respondents interpreted the questions/statements. Then, there is evaluator leniency: a tendency to ‘forgive’ low scores because of some external factors that are suddenly part of the equation.
PRAIORITIZE uses a so-called "Poll"-scale
The use of Likert scales for psychometric purposes is understandable: someone's character or attitude cannot be exactly measured or verified. In organizational settings, however, many aspects can be verified. Enter the Guttman-Poll scale. The Guttman-Poll scale shows the answers in an increasing level of ‘quality’: each next answer is better than the previous one. Here is an example:
The Guttman-Poll scale is a sort of continuum from wrong to right; from not-acceptable-anymore to the-target-that-needs-to-be-achieved. Note that in the example above, Answer 3 is not the best answer at all. Answer 4 could be ‘We have formal, SMART key performance indicators that are reviewed each month.’ So, it is up to the consultant to decide what is not good and what is. In that sense, the consultant replaces the numeric Likert scores (usually 1-to-5 or 1-to-7) with a description of what "1" is and what "5" or "7" is. PRAIORITIZE requires a minimum of two answers and allows a maximum of 5 answers.
Next, the answers are verifiable. Management can ask respondent 31 to present her SMART key performance indicators. Our own scientific research has shown that a Guttman-Poll scale with verifiable answers neutralizes socially acceptable answers, blamestorming & emotion, and self-leniency. Apparently, respondents find it difficult to lie structurally.
The combination of the continuum and verifiability neutralizes the sample biases and the evaluator bias and -leniency. Small sample sizes are not a problem. Answers are tallied, not averaged. If only one respondent out of 100 respondents has SMART key performance indicators, we're not dismissing this one respondent (he would be lost in the average). The verifiability of his answer would show he is indeed the only one with such SMART indicators and can serve as a poster boy and knowledge-sharing donor. In large sample sizes, every respondent will know what an answer represents as it is formally described and verifiable. Hence, there is no evaluator bias or -leniency as managers or consultants will not interpret outcomes but just tally.
Finally, the Guttman-Poll scale allows for setting a target. A statement with a Likert scale gives a snapshot of the actual situation but does not allow it to have a sensible ambition score or target. We can set the target of question 3 in the above-mentioned Website User Survey example to ‘Agree,’ but that makes no sense. It would mean that the responsible web designer should trial-&-error until one day there is an ‘Agree’-score. Let's take the following example from a project management assessment:
Does the project have a business case?
No
There is a draft available
Business case delivered, waiting for sign-off
Yes, formally signed off
We can now set a target. If the actual situation is ‘No’, we could set a target that in the next three months, "Business case delivered, waiting for sign-off" has to be achieved.
What parts does the business case have?
Costs versus benefits
Cost-benefits and various scenarios
Costs-benefits, scenario, and a risk assessment
Projects with a budget under, say, €100k. have as target ‘Costs versus benefits’; projects with a budget higher than €100k. have as target ‘Cost-benefits and various scenarios and projects with a budget higher than €250k. have as target ‘Costs-benefits, scenario's and a risk assessment.’