Scoring Analysis
Understand reviewer scoring patterns and how calibration affects reviewer-assigned scores across your organisation.
Introduction
Reviewer-assigned score groups can have different means and spreads. Those differences can reflect rating style, participant case mix, or both. Some groups use a wide score range, whilst others cluster tightly.
These observed differences make direct comparison harder, but the calculation cannot identify their cause.
What is calibration?
Calibration is the process of adjusting scores to account for these observed reviewer-group patterns under a selected comparison policy. Review adjusted results alongside raw scores, evidence, assignments, and case mix.
Scoring Analysis is the live Signals dashboard where authorised users compare reviewer scoring patterns, inspect distributions, apply this calibration method, and rank results for a selected reporting period. It recalculates from eligible submitted responses and is not a saved snapshot.
The Three Types of Reviews
Our system handles three distinct types of reviews:
Self-Reflection (iMatter)
Self-assessments completed by the person being reviewed. These scores are never adjusted because they represent an individual's own perception of their performance. There is no reviewer bias to correct. Self-Reflection is not eligible for Scoring Analysis rankings, distributions, tiers, adjustment summaries, or exports.
Manager Review (uMatter)
Reviews submitted by specific managers about the person being assessed. These are adjusted using observed reviewer and organisation patterns.
Feedback Workshop (WeMatter)
Reviews that represent a collaboratively agreed set of ratings based on dialogue between reviewer and participant. These use the same calibration calculation as uMatter.
Important
Each review type is processed independently because they serve different purposes and may have different patterns in the data.
The URL-backed Analysis response setting chooses Feedback Workshop or Manager Review as the preferred reviewer-backed source used throughout Rankings, Distribution, and Adjustment. When it is unavailable, the other reviewer-backed response is the only fallback. Participants without a scorable reviewer-backed response are excluded from the analysis and identified in the reporting-period preview. The setting survives reloads and shared links, but it is not saved as an organisation-wide preference.
Documentation Sections
Navigate to the section most relevant to your needs:
How It Works
Step-by-step explanation of the calibration process
Presets Guide
Understanding and choosing the right preset configuration
Interpreting Results
Making sense of adjusted scores and percentiles
FAQ
Common questions and concerns about calibration
Technical Reference
Statistical foundations, methods, and best practices
Quick Start
If you're new to Scoring Analysis, we recommend reading the sections in this order:
- How It Works - Understand the 8-step process
- Presets Guide - Choose the right configuration for your organisation
- Interpreting Results - Learn to read adjusted scores and percentiles
- FAQ - Address common concerns and questions
For technical stakeholders and administrators, the Technical Reference provides in-depth statistical detail and implementation guidance.
Last updated on