Parbat Nepal

Working Theory

Ideas I'm working through.

A set of working hypotheses about how modern search behaves — drawn from information retrieval, knowledge representation, and the observable behavior of ranking and generative systems. These are models under revision, not conclusions.

← Back to Homepage

Hypotheses.

Each note states a thesis and the discussion around it — what the model predicts, what would falsify it, and what remains open.

Intent as a Property of the SERP, Not the Keyword

A keyword string underdetermines intent. The search engine's own result composition — which page types, entities, and formats it chooses to rank — is the most reliable public record of how it currently interprets a query.

This inverts the classical workflow. Instead of classifying keywords by their surface features (modifiers, length, syntax), the interesting move is to read interpretation off the SERP itself and treat the keyword taxonomy as derived data. It also predicts instability: when the engine revises its theory of a query, the SERP recomposes, and any intent label assigned from the string alone silently expires.

Information Gain as a Contribution Criterion

A document's marginal value to an index is what it adds beyond the corpus already retrieved for that topic — a criterion formalized in Google's information-gain patent and sharpened by generative systems that preferentially cite novel, verifiable material.

Taken seriously, this reframes content strategy as a contribution problem analogous to academic publishing: restating consensus has near-zero marginal value regardless of quality. It also sets the theoretical limit of automated content generation — a model sampling from its training distribution reproduces consensus by construction, so novelty must enter the pipeline from outside the model.

Topic Coverage as Graph Covering

A topic can be modeled as an entity graph — a central entity, its attribute space, and its relations to neighbors. A site's authority on the topic is a function of how much of that graph its pages cover and how faithfully its internal links mirror the graph's actual structure.

The model makes testable predictions: coverage of high-centrality attributes should matter more than raw page count, and internal links that contradict semantic structure should dilute rather than reinforce. Its open problem is saturation — identifying the point where additional coverage stops adding evidence of expertise and starts reading as redundancy.

Cannibalization as Query-Space Partitioning Failure

When two documents from one site occupy overlapping regions of query space, ranking evidence splits between them. Cannibalization is thus a partitioning defect in the site's information architecture, not a content-quality problem.

The diagnostic follows from the definition: cluster the queries each page actually surfaces for, and overlapping clusters reveal the defect independent of any tool. The repair is equally structural — redraw intent territories so each page claims a distinct region, then make the partition legible through titles, links, and canonicals.

Internal Linking as Evidence Routing

Internal links do double duty: they route discovered authority toward the pages that need it, and they assert semantic relations the engine can verify. A site's link graph is a claim about its own knowledge structure.

Viewed this way, anchor text is predicate selection — naming the relation between two documents — and orphaned pages are assertions the site itself declines to make. The optimization question becomes graph-theoretic: what link topology maximizes both reachability and semantic coherence under a constraint of editorial plausibility?

Structured Data as an Assertion Layer

Schema markup lets a site state, in machine-readable form, what entities it discusses and how they relate. It creates no authority by itself; it reduces the engine's uncertainty about what is being claimed, so verification can proceed.

The useful mental model is testimony: assertions are weighed against the visible content and external corroboration, and unverifiable or contradicted assertions are discounted. This predicts the observed asymmetry — markup helps most where ambiguity is highest (local entities, events, products with many near-duplicates) and least where the engine already has high confidence.

Extractability and the Passage Economy

Generative retrieval operates on passages, not pages. A claim that survives extraction from its surrounding context — self-sufficient, attributed, entity-grounded — is disproportionately likely to be quoted by systems that compose answers.

This adds a stylistic dimension to ranking theory: prose that relies on distant antecedents or document-level context loses value under passage-level selection, independent of its truth or quality. The academic analogy is the abstract — writing optimized for citation by readers who will never read the whole paper.

Taxonomy Design Between Two Ontologies

A category structure serves two masters: human navigation patterns and the engine's entity classification. E-commerce taxonomy is the practical case where the two ontologies visibly diverge, and design is the act of reconciling them.

Users cluster products by task and occasion; classification systems cluster by attribute and type. A taxonomy optimized purely for either audience measurably fails the other. The interesting theoretical question is whether the reconciliation is always possible, or whether some domains force a genuine trade-off that must be resolved by priority, not cleverness.

Structural theories.

Longer-form models of document structure and diagnostic reasoning — attempts to systematize why certain page structures are legible to semantic search systems and others are not.

Semantic Structure of Commercial Pages

7 concepts

A theory of how commercial documents are parsed by semantic search systems — and how their structure can carry meaning rather than merely style.

  • 1Lemmatization — commercial queries reduce to root forms; natural language on the page should map cleanly onto them
  • 2Commercialization — transactional signals are distributed evidence of a page's commercial intent, not decoration
  • 3Verbalization — Subject-Predicate-Object sentence structures give parsers a direct path from prose to knowledge-graph entries
  • 4Visual Semantics — layout units (tabs, cards, lists) function as semantic segmentation hints, not just design
  • 5Attribute Filtration — a page is a ranked selection from the topic's attribute space; ordering encodes priority
  • 6Centerpiece Annotation — visual emphasis marks which data points the document itself considers central
  • 7Informational Deferral — educational material belongs structurally subordinate to the page's primary intent

Diagnostic Reasoning for Search Problems

5 stages

A reasoning discipline for moving from a keyword or ranking problem to a structural explanation — run fresh each time rather than pattern-matched from templates.

  • 1Entity Identification — what is this thing, and who is asking about it?
  • 2Attribute Extraction — enumerate the topic's full attribute space before judging any part of it
  • 3Attribute Filtration — which attributes carry decision weight for this query, in this domain?
  • 4Language Calibration — every field has insider terminology that signals epistemic membership
  • 5Structural Assembly — document structure follows from the preceding analysis, never precedes it

Attribute Priority as Information Ordering

5 levels

A theory of position: where information appears in a document is itself a claim about its importance, read by both users and ranking systems.

  • 1Level 1 — the few attributes that dominate the reader's decision belong at the point of entry
  • 2Level 2 — variants and qualifiers subdivide the topic without redefining it
  • 3Level 3 — quantitative specifics (cost, time, terms) anchor the abstract claims above them
  • 4Level 4 — outcome and proof attributes justify the ordering retroactively
  • 5Level 5 — comparative and edge-case material completes coverage without competing for position