MWMS Search Scrape Summarise Evidence Pipeline Standard

System: MWMS
Document Type: Standard
Status: Draft For MCR
Authority: HeadOffice
Applies To: Research Brain, Affiliate Brain, Ads Brain, Content Brain, HeadOffice Intelligence, Data Brain, Experimentation Brain, Risk Brain, AI Employees, Deep Search Workflows, Future Client Facing AI Systems
Primary Location: MCR
Future Operational Destination: mwmsbrain.site, mwmsheadofficebrain.site, Research Brain, Affiliate Brain, Ads Brain, HeadOffice Dashboards, AI Employee Dashboards, Future Client Portals
Parent Page: HeadOffice
Source Of Truth: MCR
Related Frameworks: MWMS Research Planning And Query Rewriting Standard, MWMS Source Visibility And Evidence Display Standard, MWMS Deep Search Quality And Observability Framework, MWMS AI Guardrail And Preflight Check Standard, MWMS AI Work Session Persistence Standard, MWMS AI Usage And Cost Visibility Standard, MWMS AI Observability Metadata Standard, MWMS Agent Loop Control Framework, MWMS AI Employee Evaluation Scorecard Standard
Course Source: Matt Pocock AIhero Build DeepSearch In TypeScript
Absorption Status: Approved For Integration

Purpose

The purpose of this standard is to define how MWMS turns external research into structured, source-backed, reviewable evidence.

This standard governs the evidence pipeline:

Search
→ Inspect / Scrape
→ Summarise
→ Rate
→ Store
→ Display
→ Use In Final Answer

MWMS AI Employees must not treat search snippets as proof.

Search results are leads.

Inspected, summarised, rated, and stored evidence is what supports trusted MWMS decisions.

This standard ensures Research Brain, Affiliate Brain, Ads Brain, Content Brain, HeadOffice Intelligence, and future AI Employees can gather source evidence in a controlled, cost-aware, auditable, and reusable way.

Scope

This standard applies to any MWMS workflow where an AI Employee uses external information, source inspection, web research, page content, or evidence summaries to support an answer or decision.

This includes:

Deep Search workflows
Research Brain investigations
Affiliate Brain offer evaluations
Ads Brain compliance reviews
Content Brain topic research
HeadOffice Intelligence follow-up research
Newsletter Intelligence source expansion
Product and vendor research
Tool evaluation
AI trend monitoring
Market research
Competitor research
Policy and compliance checks
Source verification
Client-facing research systems
Future AIBS intelligence workflows

This standard does not define exact scraper code, search provider configuration, frontend component implementation, or specific API vendor usage.

It defines the MWMS evidence pipeline standard.

Core Rule

The core rule is:

Search results are leads. Inspected and summarised sources are evidence.

An AI Employee should not produce a trusted answer from search snippets alone when the task requires evidence.

Search finds possible sources.

Inspection verifies what those sources actually say.

Summarisation compresses the useful evidence.

Rating judges trust, freshness, and relevance.

Storage makes the evidence reusable.

Display makes the evidence reviewable.

Only then should the evidence support an important MWMS decision.

Definition Of The Evidence Pipeline

The Search Scrape Summarise Evidence Pipeline is the structured process used by MWMS to turn research queries into decision-ready evidence.

The pipeline includes:

Search for candidate sources
Select sources worth inspecting
Inspect or scrape source content
Summarise source content against the research goal
Rate source trust, freshness, and relevance
Store source and evidence records
Display sources to the operator
Use evidence in final answer
Evaluate evidence sufficiency
Route weak or conflicting evidence for review

The pipeline exists to prevent shallow, unsupported AI answers.

Why MWMS Needs This Pipeline

Without a structured evidence pipeline, AI Employees may:

rely on search snippets
use weak sources
miss official sources
ignore source freshness
overuse commercial sources
hide source conflicts
overload the model with raw content
lose evidence after the session
make unsupported recommendations
repeat research unnecessarily
create answers that are hard to audit
produce confident but fragile conclusions

With this pipeline, MWMS gains:

stronger research quality
better evidence discipline
better source reuse
better cost control
better source visibility
better HeadOffice oversight
better AI Employee evaluation
better client-facing trust
stronger Research Brain capability
better affiliate and ads decision-making

Pipeline Overview

The standard MWMS evidence pipeline is:

Research Plan Created
The system defines what evidence is needed.
Queries Generated
Search queries are produced from the research plan.
Search Executed
Candidate sources are discovered.
Sources Selected
The system chooses which sources deserve inspection.
Sources Inspected / Scraped
Page content is retrieved where allowed and useful.
Sources Summarised
Content is compressed into task-relevant evidence notes.
Sources Rated
Trust, freshness, relevance, and limitations are assessed.
Evidence Stored
Source records and summaries are saved.
Evidence Displayed
Operators can see the evidence behind the answer.
Final Answer Produced
The answer uses the evidence, with limitations and confidence.
Evaluation And Kaizen
Weaknesses become eval cases, improvements, or regression items.

Relationship To Research Planning

This pipeline begins after the MWMS Research Planning And Query Rewriting Standard has defined:

research goal
decision supported
missing information
assumptions to verify
freshness needs
source preferences
risk areas
query plan
stopping criteria

The Research Planner defines what the pipeline should look for.

The Evidence Pipeline executes that plan.

Relationship To Guardrail And Preflight

The MWMS AI Guardrail And Preflight Check Standard decides whether research should begin.

Preflight asks:

Is the request safe?
Is it clear?
Are sources required?
Is the cost justified?
Is the workflow allowed?
Is human review needed?

Only after preflight approves research should the evidence pipeline begin.

Relationship To Agent Loop Control

This pipeline may run as:

a deterministic workflow
an assisted workflow
part of a controlled agent loop
part of a Deep Search workflow

The Agent Loop Control Framework decides whether the AI Employee should use an adaptive loop or a fixed workflow.

For many research tasks, the pipeline should be partly deterministic.

Example:

Research plan
→ search
→ inspect top eligible sources
→ summarise
→ rate
→ answer

The AI does not need to decide every tiny step when the workflow pattern is predictable.

Stage 1: Search

Search is the process of finding candidate sources.

Search should be guided by the research plan.

Search should not be random.

Recommended search metadata:

Field	Description
query	Search query used
query_purpose	Why this query was run
research_goal	Research goal connected to query
preferred_source_type	Official, expert, commercial, user-generated, etc.
freshness_need	Historical, evergreen, recent, current
search_provider	Tool or provider used
result_count	Number of results returned
search_timestamp	When search occurred
useful_result_count	Number of potentially useful results
follow_up_needed	Whether another search is needed

Search Rule

Search should seek evidence, not vibes.

A good search query should help find a source that can verify, disprove, or clarify the research question.

Stage 2: Source Selection

Source selection decides which search results are worth inspecting.

The system should prefer sources based on:

relevance to research goal
source type
likely trustworthiness
freshness
directness
official status
uniqueness
usefulness
risk requirements

Source selection should avoid:

duplicate sources
low-quality reposts
thin AI-generated pages
irrelevant pages
outdated pages for current topics
purely promotional sources when neutral evidence is needed
user-generated sources as proof unless clearly marked as signal

Source Selection Rule

Not every search result deserves inspection.

MWMS should inspect the best evidence candidates, not blindly inspect everything.

Stage 3: Inspect Or Scrape

Source inspection means opening, crawling, scraping, reading, or otherwise retrieving the source content.

Inspection should capture:

Field	Description
source_url	URL inspected
source_title	Page title
source_status	Inspected, failed, blocked, ignored
retrieved_at	Retrieval time
content_available	Yes or no
extraction_status	Successful, partial, failed, blocked
content_length	Approximate extracted content size
source_owner	Vendor, platform, publisher, organisation
source_type	Official, expert, commercial, user-generated, unknown
inspection_error	Error if failed

Inspection Rule

Important decisions should use inspected source content, not search snippets alone.

If inspection fails, the failure should be recorded.

Stage 4: Summarise

Source summarisation compresses inspected content into task-relevant evidence.

A source summary should not be a generic summary of the page.

It should summarise the source in relation to the research goal.

A good source summary should include:

what the source says that matters
which research question it helps answer
key claims or facts
figures or details if relevant
source limitations
what the source does not prove
whether the source supports or weakens the final decision
whether the source is current enough

Summarisation Rule

Summaries must compress evidence without inventing evidence.

If the source does not answer a question, the summary should say that clearly.

Stage 5: Rate

Each source should be rated before it is used in final reasoning.

Recommended ratings:

Rating Type	Purpose
trust_rating	How reliable the source appears
freshness_rating	Whether the source is current enough
relevance_rating	How directly it supports the task
evidence_strength	How strong the extracted evidence is
bias_risk	Whether source has commercial or ideological bias
conflict_status	Whether it conflicts with other sources

Rating Rule

A source should not be treated as high-value evidence simply because it was found by search.

It must be rated.

Stage 6: Store

Source evidence should be stored where useful.

Storage makes research reusable.

Recommended source record fields:

Field	Description
source_id	Unique source record
research_session_id	Parent research/work session
query_used	Query that found the source
source_url	URL
source_title	Title
source_type	Official, expert, commercial, user-generated, unknown
source_owner	Organisation, vendor, publisher
publication_date	Published date if available
last_updated_date	Updated date if available
retrieved_at	Retrieval date
extraction_status	Successful, partial, failed
evidence_summary	Task-relevant source summary
key_claims	Important claims/facts
limitations	What source does not prove
trust_rating	High, medium, low, unknown
freshness_rating	Current, acceptable, outdated, unknown
relevance_rating	High, medium, low
used_in_final_output	Yes or no
conflict_detected	Yes or no
review_required	Yes or no

Storage Rule

If evidence supports an MWMS decision, it should be stored or linked to a persistent work session.

Stage 7: Display

Evidence should be visible to the operator.

Display should connect to the MWMS Source Visibility And Evidence Display Standard.

A source display should show:

source title
source URL
source type
trust rating
freshness rating
evidence summary
limitations
used in final answer
conflict status
inspection status

Display Rule

Evidence should not be hidden only in backend traces.

The operator should be able to review what the AI used.

Stage 8: Use In Final Answer

The final answer should use the evidence responsibly.

A final answer should:

cite or reference sources where needed
distinguish evidence from inference
include limitations
mention weak or missing evidence
reduce confidence where evidence is weak
avoid unsupported claims
show the decision supported by evidence
trigger review if evidence is insufficient

Final Answer Rule

The AI Employee should not make the answer stronger than the evidence allows.

Stage 9: Evaluate Evidence Sufficiency

Before final output, MWMS should judge whether the evidence is enough.

Evidence sufficiency ratings:

Rating	Meaning
Strong	Reliable, current, directly relevant evidence
Acceptable	Usable evidence with minor limitations
Weak	Incomplete, indirect, old, or limited evidence
Insufficient	Not enough evidence to trust the answer

Evidence sufficiency should consider:

source quality
source freshness
source relevance
number of sources
source diversity
official source availability
directness of evidence
conflicts
extraction failures
risk level

Evidence Sufficiency Rule

High-risk decisions require stronger evidence.

Low-risk internal tasks may allow lighter evidence.

Stage 10: Kaizen And Regression Capture

The pipeline should create improvement signals.

Kaizen notes may include:

query plan was weak
source selection was poor
official source was missed
source extraction failed
source summary was too vague
source freshness was unknown
evidence was insufficient
source conflict was not resolved
final answer overclaimed
too many sources were inspected
source inspection cost was too high

Serious failures may become regression cases.

Combined Search And Scrape Pattern

Some tools combine search and source retrieval into one action.

MWMS may use combined search/scrape workflows when they:

reduce unnecessary AI decisions
reduce latency
improve evidence availability
simplify implementation
make source inspection more consistent
reduce the chance of answering from snippets only

Combined Action Rule

If search and source inspection almost always belong together, MWMS may consolidate them into one controlled workflow action.

However, combined tools must still preserve:

query used
sources found
sources inspected
source summaries
trust ratings
freshness ratings
failures
usage cost

A combined API does not remove the need for evidence discipline.

Search Snippet Rule

Search snippets may help decide which sources to inspect.

Search snippets should not normally be treated as final evidence for important decisions.

Search snippets may be acceptable only when:

the task is low-risk
the answer is not decision-critical
source inspection is impossible
the output clearly states limitations
confidence is reduced

Rule

Search snippets are leads, not proof.

Source Summary As Evidence Compression

Source summarisation is an evidence compression layer.

It allows MWMS to:

inspect more sources
avoid context overload
preserve key evidence
reduce final prompt size
store reusable research notes
improve source review
improve final answer quality

Evidence compression should preserve truth, not create new claims.

Rule

The summariser must not fill gaps with outside knowledge.

If a source does not provide an answer, the summary should say so.

Source Diversity Rule

Important research should avoid relying on one source type only.

Where relevant, MWMS should seek a mix of:

official sources
expert sources
commercial sources
user-generated signals
competitor sources
policy sources
market sources

Not every workflow needs all source types.

The research plan should define source diversity requirements.

Official Source Preference Rule

For factual, policy, compliance, pricing, product, platform, and legal questions, official sources should be preferred.

Examples:

Google Ads policy
affiliate network terms
vendor product page
software documentation
government regulation
official changelog
official pricing page

If no official source is found, the final answer should state that limitation.

Commercial Source Caution Rule

Commercial sources may be useful but should be handled carefully.

Examples:

affiliate review sites
sales pages
comparison pages
agency pages
vendor marketing pages

Commercial sources may contain bias.

They should be corroborated where possible.

They should not be treated as neutral proof without caution.

User-Generated Source Rule

User-generated sources can reveal useful signals, but they are not proof by themselves.

Examples:

Reddit
forums
social media
comments
community discussions

Use them for:

complaint signals
sentiment signals
repeated user problems
emerging trends
real-world experience

Do not use them as sole proof for factual claims unless clearly framed as anecdotal signal.

Source Conflict Rule

If sources disagree, the conflict must be captured.

Conflict records should include:

what sources conflict
what they disagree about
which source appears stronger
whether conflict affects confidence
whether human review is required

Rule

Source conflict should not be hidden.

Unresolved conflict should reduce confidence or trigger review.

Source Failure Rule

Source inspection failure must be recorded.

Failures may include:

page blocked
page unavailable
extraction failed
content incomplete
source outdated
source irrelevant
source duplicate
source paywalled
source too thin

A failed source may still be useful as a note, but should not support final claims.

Evidence Pipeline By Workflow Type

Research Brain

Should use the full pipeline for important investigations:

Research plan
→ search
→ inspect
→ summarise
→ rate
→ store
→ answer

Affiliate Brain

Should use source-backed evidence for:

vendor/product page
affiliate terms
payout evidence
customer complaint signals
competitor evidence
compliance risk

Ads Brain

Should use evidence for:

official policy
claim review
product category rules
jurisdiction
compliance risk
allowed/prohibited wording

Content Brain

Should use evidence for:

topic authority
search intent
competitor content
data points
claims
freshness

HeadOffice Intelligence

Should use evidence for:

signal validation
business relevance
routing justification
risk assessment
action recommendation

Evidence Pipeline Output Standard

A completed evidence pipeline should produce:

{
  "research_goal": "",
  "decision_supported": "",
  "queries_run": [],
  "sources_found": 0,
  "sources_inspected": 0,
  "source_summaries": [],
  "evidence_sufficiency": "",
  "source_conflicts": [],
  "evidence_gaps": [],
  "recommended_confidence": 0,
  "human_review_required": false,
  "final_answer_ready": false,
  "kaizen_note": ""
}

This is conceptual only.

Exact implementation may vary.

Recommended Source Summary Object

{
  "source_id": "",
  "query_used": "",
  "source_url": "",
  "source_title": "",
  "source_type": "",
  "source_owner": "",
  "publication_date": "",
  "retrieved_at": "",
  "extraction_status": "",
  "evidence_summary": "",
  "key_claims": [],
  "limitations": [],
  "trust_rating": "",
  "freshness_rating": "",
  "relevance_rating": "",
  "evidence_strength": "",
  "bias_risk": "",
  "conflict_detected": false,
  "used_in_final_output": false,
  "review_required": false
}

This object can later support Research Brain source libraries and HeadOffice evidence panels.

Usage And Cost Control

The evidence pipeline must be cost-aware.

It should track:

search actions
source inspections
source summaries
failed inspections
model actions
tool actions
evaluator actions
total estimated cost
cost per useful source
cost per final answer

The pipeline should stop or escalate when:

max source inspection limit is reached
cost limit is reached
enough evidence exists
sources are too weak
source conflict requires review
official sources cannot be found

This connects to the MWMS AI Usage And Cost Visibility Standard.

Evaluation Requirements

The evidence pipeline should be evaluated.

Evaluation questions:

Did the search find relevant sources?
Were the best sources selected?
Were sources inspected successfully?
Were source summaries accurate?
Were trust and freshness ratings reasonable?
Were weak sources flagged?
Were conflicts captured?
Was evidence sufficient before answering?
Did the final answer match the evidence?
Was cost justified?

This connects to the MWMS AI Employee Evaluation Scorecard Standard.

Human Review Triggers

Human review should trigger when:

evidence is weak
official source is missing
source freshness is unknown
sources conflict
only commercial sources are available
only user-generated sources are available
source inspection failed
output affects campaign launch
output affects budget
output affects compliance
output is client-facing
confidence is low
cost is high

Minimum Starting Implementation

MWMS does not need the full pipeline immediately.

Minimum implementation:

search query
source URL
source title
retrieved date
source type
source inspection status
evidence summary
trust rating
freshness rating
used in final answer yes/no
evidence sufficiency
final confidence

This gives MWMS a useful evidence pipeline without overbuilding.

Future Enhancements

Future enhancements may include:

MWMS Deep Search Source Record Standard
MWMS Source Summary Record Schema
MWMS Research Brain Source Library
MWMS Evidence Sufficiency Scorecard
MWMS Source Conflict Resolution Standard
MWMS Source Display UI Component Specification
MWMS HeadOffice Evidence Review Panel Specification
MWMS Client Facing Evidence Display Standard
MWMS Search Provider Evaluation Standard
MWMS Source Cache And Reuse Standard

These should be created only when implementation pressure justifies them.

Drift Protection

This standard prevents the following drift:

using search snippets as proof
answering without inspected evidence
hiding weak sources
hiding source conflicts
ignoring source freshness
dumping raw pages into final prompts
overloading context windows
losing source summaries
failing to store evidence
showing final answers without source support
overusing source inspection without cost control
trusting commercial sources too strongly
treating user-generated signals as proof
failing to evaluate evidence sufficiency

If the evidence pipeline is weak, the final answer must be treated with caution.

Architectural Intent

The architectural intent of this standard is to make MWMS evidence-first.

MWMS is not building AI Employees that simply search and answer.

MWMS is building AI Employees that plan research, inspect sources, compress evidence, rate reliability, store evidence, show sources, and answer only when the evidence supports the answer.

This pipeline turns external information into operational intelligence.

It makes MWMS stronger because every important answer can be traced back to the evidence that supports it.

Change Log

v1.0 Initial Draft

Created the MWMS Search Scrape Summarise Evidence Pipeline Standard based on absorbed insights from Matt Pocock AIhero Build DeepSearch In TypeScript.

Integrated principles from course sections covering:

combined search and scrape workflows
search plus source inspection
search, scrape, and summarise evidence flow
source summarisation as evidence compression
source trust and freshness rating
evidence sufficiency
source storage
source display
usage and cost control
source-backed final answers
avoiding search snippets as proof

Established this standard as the MWMS governance page for turning external searches into structured, stored, reviewable, and decision-ready evidence.