Inside AstraClaw AI: How the 5-Agent Search Pipeline Works

When you type a search query into KeepCloud's search bar and press Enter, you're triggering a multi-step process that involves five specialized agents working in sequence and in parallel. This post walks through exactly what happens — from the moment your raw query arrives to the moment ranked file results appear on your screen.

Step 1 — Query arrives at the Preprocessor

The first agent to receive your query is the QueryPreprocessorAgent. Its job is to transform messy, real-world input into a clean set of search terms that the downstream agents can work with effectively.

It starts by removing noise words — words that appear frequently in queries but carry no search meaning. These include standard English stop words ("find", "show", "please", "me", "my", "the") and their Hindi and Hinglish equivalents ("dhundhke", "batao", "dikhao", "mera", "meri", "chahiye"). A query like "please find my invoice from last month" becomes simply [ "invoice", "last", "month" ].

Next, the Preprocessor checks each remaining term against your personal vocabulary — a dictionary built from all your filenames. If a term looks like a typo of a word in your vocabulary (similarity above a carefully tuned threshold), it is auto-corrected. "invoce" becomes "invoice" if you have files with "invoice" in their name. The threshold is deliberately high to avoid false corrections — common words like "science" are never corrected to a superficially similar filename.

Finally, certain terms trigger expansion. A query containing a city abbreviation known to Indian Railways (like "JP" for Jaipur or "NDLS" for New Delhi) expands to include the full city name, so searches work whether you typed the abbreviation or the full name.

Step 2 — Metadata and content search run in parallel

The Orchestrator (Agent 5) takes the clean terms from the Preprocessor and fires two search operations simultaneously using async parallelism:

Metadata search checks filenames, description fields, folder names, and provider-formatted path strings for matches against the query terms. This is fast — it runs against the locally cached file index with no API calls needed.

Content search delegates to the ContentReaderAgent (Agent 2). For Google Drive files, this uses Google's own full-text content search API — which has already indexed your documents for free and requires no download. For Dropbox files, it uses Dropbox's dedicated search endpoint. For OneDrive files, it uses Microsoft Graph's search functionality. For files that have already been locally OCR-indexed, it searches the cached extracted text directly.

Running both in parallel cuts the time to first result without sacrificing coverage.

"Metadata search and content search run simultaneously. The Orchestrator waits for both, then merges the results — so you never pay the time cost of running them sequentially."

Step 3 — Semantic search adds document-type awareness

The SemanticEngine (Agent 3) runs as a third search source for files that have already been processed through semantic analysis. It searches against the semantic tag clouds generated for each classified document — not keyword search, but tag-set intersection. A query for "train ticket jaipur" doesn't just pattern-match against file text. It finds files whose semantic profile includes the tags "train", "ticket", and "jaipur" — which were generated by identifying the document as an IRCTC ticket and extracting its station codes.

This is what enables "find train ticket from Delhi to Jaipur" to work even when the IRCTC PDF only contains shorthand notations rather than the full city names.

Step 4 — Verification scores every candidate result

The VerificationAgent (Agent 4) receives the merged pool of candidates from all three search sources. For each candidate, it independently evaluates the evidence across five dimensions: filename match, metadata match, cached OCR text match, semantic tag match, and provider content-match flag.

It combines these into a single confidence score between 0 and 100. Results below a minimum threshold (around 30%) are discarded as false positives. Results where all query terms appear directly in the filename score near 100% and appear at the top of the list. Results where terms appear only in the semantic tag cloud score lower but still surface if confidence is above the minimum.

This scoring is what prevents Astra from flooding you with loosely-related results. Every file you see has been independently verified as likely relevant, not just superficially matching.

Step 5 — Orchestrator deduplicates and ranks

The final step is deduplication and ranking. Because the same file can appear in results from metadata search, content search, and semantic search simultaneously, the Orchestrator removes duplicates. Deduplication checks both file ID and filename — the same file retrieved from multiple search paths appears only once in the final results.

Results are then sorted by confidence score, descending. Files with multiple corroborating signals (name match + content match + semantic match) rank highest. Files matching on only one dimension rank lower.

The full journey of a single query

To make this concrete: you type "Jaipur train ticket". The Preprocessor produces [ "jaipur", "train", "ticket" ] and expands "jaipur" to also include "JP" and "NDLS–JP route". The Orchestrator fires metadata search and content search in parallel. Metadata search finds a PDF named "trip_may.pdf" in a "Travel" folder — weak match. Content search finds two files whose text content contains those terms. Semantic search finds three files tagged as train tickets with Jaipur-related station codes. All six candidates go to Verification. The PDF named "trip_may.pdf" scores 31% — just above the floor, it stays. The scan named "IMG_0042.jpg" that contains an IRCTC receipt for NDLS→JP scores 94% — it appears at the top. Total time: under two seconds.