Methodology

Why Engagement Signals has a methodology page

Engagement Signals is not simply a dashboard that happens to contain some AI copy. The narrative layer is a core part of the report, so it needs to be explainable. This page sets out how WeMatter handles the engagement evidence, how anecdotal comments are processed, and why the final report is designed to be more disciplined than a generic summary of free text.

At its best, Engagement Signals helps leaders move from raw scorecard data to a clearer understanding of the organisation's engagement story. That only works if the written insight is grounded in the evidence rather than floating above it.

---

The evidence Engagement Signals works from

Engagement Signals starts with submitted iMatter scorecards. From those scorecards, the report draws on two connected types of evidence.

The first is structured evidence: overall engagement scores, area-level ratings, response counts, participation patterns, and period-on-period changes when a comparison window exists. The second is anecdotal evidence: the summary comments and area-level written feedback people leave when they complete their iMatter responses.

The structured data shows where the engagement signal sits. The written evidence helps explain what may be shaping that signal in practice. Engagement Signals is designed to read those two layers together.

In simple terms, the methodology is trying to protect three things at once:

• signal quality
• narrative coherence
• traceability back to the source evidence

---

How written feedback is prepared

The comment pipeline does not treat every response as equally useful. Before the narrative is generated, written feedback is cleaned and filtered so the report is built from comments that carry meaningful signal.

In practical terms, that means low-value or trivial responses are stripped out, comments are prepared for analysis, and stronger evidence is given more weight. This matters because engagement reporting becomes much less useful if the narrative is dominated by throwaway remarks, very short responses, or multiple versions of the same point.

---

How anecdotes become patterns

Once the comments are prepared, Engagement Signals enriches them with context. Each anecdote is linked back to the engagement picture around it, including the relevant score, the area it belongs to, whether it sits in the current or previous period, and any available breakdown context such as region, department, or manager.

That context is important because the same sentence means something different in a strong area than it does in a weak one, and something different again if it is appearing inside a shifting or low-sample cohort.

From there, the pipeline looks for repeated patterns across the evidence rather than reading every comment as a separate observation. Related anecdotes are clustered together so the final narrative reflects themes that recur across the engagement dataset, not just the most dramatic individual remarks.

This is one of the reasons the report feels more considered than a one-pass AI summary. It is looking for patterns, not just paraphrasing text.

---

Why the report does not use every comment equally

One of the strengths of Engagement Signals is that it curates the evidence set before it writes the report. Instead of passing every available comment straight through, it selects representative evidence from the strongest clusters and uses that as the basis for the narrative.

This helps the report stay balanced. It reduces repetition, limits the influence of noisy outliers, and makes it easier for the narrative to represent the shape of the engagement picture rather than echoing the same anecdote in multiple places.

The result is a more readable and more defensible report. When a theme appears, it is there because the evidence supports it, not because one memorable comment happened to stand out.

Representative evidence selection helps the report avoid a few familiar failure modes:

• repeating the same anecdote in several sections
• over-weighting one loud cluster of comments
• letting low-value remarks crowd out stronger evidence

---

How scores and comments are combined

The narrative in Engagement Signals is built from both the selected written evidence and the structured reporting context. That includes headline engagement scores, area-level scores, changes over time, response counts, cohort filters, and group comparisons.

This is what gives the report its shape. The writing is not asked to summarise comments in isolation. It is asked to explain the engagement picture using both the quantitative measures and the qualitative evidence that sits behind them.

That is also why the report can move beyond a general sentiment summary. It can connect a written theme to a rising area, a weakening area, a strong region, a struggling department, or a pattern that is concentrated in one part of the cohort rather than evenly shared across it.

---

How the narrative is structured

Before Engagement Signals produces the final report text, it shapes the story. The pipeline frames the overall snapshot angle, identifies the strongest narrative threads, and builds the sections that appear as Key themes, Risks, Opportunities, and Strategy.

This matters because a good engagement report should feel coherent. It should not read like a disconnected list of observations. The methodology is designed to help the final report tell a joined-up story about what is happening across engagement, why it matters, and where leadership attention may be most useful.

In the final report, that story is expressed through:

• Key themes
• Risks
• Opportunities
• Strategy

---

Quality checks and coherence review

The first draft is not treated as the final answer by default. Engagement Signals validates the output against a defined structure and reviews it for coherence issues such as duplication, overused evidence, and weak or repetitive section design.

That review step is important because it makes the report feel more intentional. It reduces the risk of a generic or noisy output and helps ensure that the final narrative remains grounded in the strongest available engagement evidence.

---

Why this is powerful for engagement reporting

Engagement data is often difficult to use well because the most important part of the story sits between the numbers and the comments. A score alone may tell you that a team feels weaker in one area, but it does not tell you what people are actually experiencing. A set of comments alone may sound vivid, but without the score context it is hard to tell whether that experience is widespread, concentrated, improving, or declining.

Engagement Signals is powerful because it brings those layers together. It treats anecdotal evidence as a serious part of the engagement signal, connects it back to the structured data, and produces a report that is meant to support real leadership judgement rather than passive observation.

---

What to keep in mind when reading it

Even with a strong methodology, good judgement still matters. Small samples need careful interpretation. Narrative quality is strongest when participants leave meaningful written comments. Anonymous public views intentionally suppress some identity and manager-level detail. And as the product evolves, the exact model or prompt configuration may change even when the evidence-led principles stay the same.