Document Type: Framework
Status: Canon
Authority: HeadOffice
Applies To: Data Brain, Product Brain, Experimentation Brain, Affiliate Brain, Strategy Brain
Parent: Data Brain Canon
Version: v1.0
Last Reviewed: 2026-05-03
Purpose
The Analytics Question Quality Framework defines how all analytical questions must be structured before analysis is performed within MWMS.
This framework exists to prevent:
- vague analysis
- incorrect conclusions
- wasted time
- misleading data interpretation
It ensures that all analytics work is:
- structured
- actionable
- relevant
- decision-driven
Core Principle
Bad questions produce bad insights.
The quality of output is determined by the quality of the question.
Role In MWMS System
This framework controls the entry point of:
- Data Brain analysis
- Experimentation Brain testing
- Product Brain investigation
- Affiliate Brain performance analysis
Analytics Question Objective
Every analytics question must aim to:
- Understand a specific problem
- Identify measurable behaviour
- Enable a clear decision
Question Quality Criteria
Every question must meet three core criteria:
1. Specificity
The question must clearly define:
- what is being analysed
- which metric
- which feature
- which segment
- which timeframe
Example
Weak:
- “Why is performance down?”
Strong:
- “Why did trial-to-paid conversion rate drop for US users in the last 14 days?”
2. Segmentation
The question must define:
- which users
- which cohort
- which segment
Example
Weak:
- “Are users converting?”
Strong:
- “Are new trial users converting within 7 days compared to returning users?”
3. Actionability
The question must lead to:
- a decision
- a change
- an action
Example
Weak:
- “What is our churn rate?”
Strong:
- “Which user segment has the highest churn rate and what behaviour predicts it?”
Question Structure Template
All analytics questions must follow:
What [metric] changed for [segment] during [timeframe], and what behaviour explains it?Question Types
1. Diagnostic Questions
Used to understand:
- why something happened
2. Comparative Questions
Used to compare:
- segments
- features
- time periods
3. Predictive Questions
Used to estimate:
- future outcomes
4. Validation Questions
Used to confirm:
- experiment results
- hypotheses
Question Quality Levels
Level 1 — Vague
- no segment
- no metric
- no timeframe
Level 2 — Basic
- includes metric
- lacks segmentation
Level 3 — Structured
- includes metric
- includes segment
- includes timeframe
Level 4 — Decision-Ready
- includes metric
- includes segment
- includes timeframe
- leads to clear action
Rule
Only Level 3 and Level 4 questions are allowed in MWMS.
Input Requirements
Before analysis begins, the question must include:
- metric definition
- segment definition
- timeframe
- expected outcome
- business context
Common Failure Modes
1. Broad Questions
“Why is revenue down?”
2. No Segmentation
“All users grouped together”
3. No Timeframe
No comparison period
4. No Action Path
Results cannot be used
5. Vanity Questions
Interesting but not useful
Enforcement Rule
No analysis is performed if:
- the question is unclear
- segmentation is missing
- no decision can be made
Data Brain Integration
Data Brain must:
- validate question quality
- reject weak questions
- refine questions before analysis
Experimentation Brain Integration
Experiments must start from:
- high-quality questions
- clear hypotheses
Product Brain Integration
Product decisions must be driven by:
- structured questions
- not assumptions
Affiliate Brain Integration
Performance analysis must use:
- segmented questions
- not aggregate data
Drift Protection
The system must prevent:
- random analysis
- curiosity-driven reporting
- non-actionable insights
- over-analysis without purpose
Operational Rules
Rule 1: Ask Before Analyse
Define the question first
Rule 2: Refine Questions
Improve clarity before running analysis
Rule 3: Reject Weak Questions
Do not proceed
Rule 4: Document Questions
Track analysis inputs
Architectural Intent
This framework ensures MWMS:
- focuses analysis
- improves decision-making
- reduces wasted effort
- increases clarity
Final Rule
If the question is unclear:
→ the analysis must not proceed
Change Log
Version: v1.0
Date: 2026-05-03
Author: HeadOffice
Change:
Created Analytics Question Quality Framework to control input quality for all Data Brain analysis.
Change Impact Declaration
Pages Created:
Data Brain Analytics Question Quality Framework
Pages Updated:
None
Pages Deprecated:
None
Registries Requiring Update:
MWMS Architecture Registry
Data Brain Page Registry
Canon Version Update Required:
No
Change Log Entry Required:
Yes