This article is auto-synced from its in-app version in Tai.

Knowledge items give agents grounded, up-to-date content to draw on — help articles, procedures, FAQs, and other documents.

Knowledge lets an agent answer questions from your company's own content rather than relying on general training alone — for example, your cancellation policy so the Customer Care agent can cite it accurately, or a product FAQ so agents give consistent answers about your offering.

A body of knowledge is rarely one file. When you choose FILE, the picker takes as many documents as you select, and each one becomes its own knowledge item — so a ten-document handbook is one trip through the form, not ten.

If a file is too large or of an unsupported type, its row says so and it's left out — the rest still go through. If an upload fails, that row keeps the reason and the button becomes Retry, which picks up only what didn't land; the documents that already succeeded aren't created twice.

Writing a good description is fiddly, especially for a document or a site you haven't read end to end. A Generate button under the Description field summarises the source's own content into a short draft.

The draft lands in the field for you to review and edit, with an Undo next to it if you'd rather keep what you had. Nothing is saved until you save the item. If a draft had to be cut mid-sentence to fit the length limit, it ends in an ellipsis (…) so you can see where it was cut.

For a WEBSITE item, you're not limited to letting the crawler discover pages on its own. Under Explicit pages, list the specific URLs you want indexed — one per line. These pages are always crawled in addition to whatever the sitemap and on-page links turn up.

If you only want those exact pages and nothing else, turn off Follow links. With it off, the crawl reads the start URL and your listed pages only — it never consults the site's sitemap or follows a single link from any page it reads. This is the right choice for a large site where only a handful of pages actually matter to your agents, or where the site's navigation would otherwise pull in far more than you need.

Every explicit page must be a full https://… address on the same site, at or below the start URL's path, and outside any exclude patterns — the same rules that already govern which pages a crawl can reach. A page outside that scope is rejected when you save, with an explanation of why.

Maximum pages still applies, and it counts the start page too. If you list 300 explicit pages, set it to at least 301 — the form tells you the number to use, so a long list can't be quietly cut short halfway through.

For a WEBSITE item, click Test connection under the crawl settings. It fetches only the start page and reports back in a couple of seconds:

It also flags the two problems that are easiest to miss: a sign-in page, which returns a normal response but has nothing worth indexing — the crawler refuses to index a sign-in screen rather than filling your index with it (see below for what happens then); and a start URL that redirects out of scope — to a different address, or to one your exclude patterns drop — which leaves the crawl with nothing to read.

The test reads only the start page, so treat it as a strong signal rather than a guarantee: a site whose home page is mostly images can still crawl well from its other pages. Nothing is saved by testing, and a warning doesn't stop you saving — it's there so you can fix the address before the first crawl runs.

A crawl stops when it has read the whole site — or when it reaches one of its limits, whichever comes first. If a limit came first, the item shows a Partial index badge next to its embedding status, and the sync-history chart highlights the runs it happened on. This is a different fact from a Many sign-in screens badge (see below): Partial index means the crawl stopped early on a limit, while Many sign-in screens means a large share of what it did reach was refused as sign-in screens. The two are independent, and a source can carry both at once.

The badge tells you which limit was reached and what to do about it:

This matters because the page count on its own can't tell you the difference. Without the badge, a source that indexed the first 200 pages of a 5,000-page site looks exactly like one that indexed a 200-page site in full — and agents would answer from the smaller picture without anything saying so.

The badge covers limits the crawl ran into. Pages it skipped one at a time are reported separately — see below.

Under Index size, a website source reports what its last crawl kept and what it left out — for example "120 pages · 500 chunks · 44 skipped" — with a breakdown of the reasons underneath.

Skipping is normal. A crawl of a real site constantly passes over PDFs and images, dead links, and pages that turn out to have no readable text; a source that skipped nothing simply says nothing. The number is there for the times it isn't normal — a large count next to blocked by robots.txt, could not be fetched, or a sign-in screen usually means something changed on the site rather than in the crawler.

The reasons you may see:

The count is of pages the crawl actually reached. If the run also stopped at a limit — the Partial index badge — then pages beyond that limit were never reached at all, and aren't counted here.

Every re-sync — and every automatic re-crawl — is recorded on the item's detail page, so clicking Re-sync is never a shot in the dark.

Read the Notes column to see what a run actually did:

Clicking Re-sync shows up here immediately, badge and all — no page reload needed to see a run start. If a run is already queued for the item, clicking Re-sync again tells you that, rather than starting a second one.

The crawler never indexes a sign-in screen — an index whose content is "Sign in to continue" is worse than a smaller one. What happens next depends on the rest of the site:

Usually the start page is what gives it away: one that is itself a sign-in screen settles it for the whole site. Where the start page can't be read — it errors, isn't a web page, redirects to a sign-in service on another address, or has no readable text — the crawler judges the site by the pages it could read instead: if as many of them are unmistakably sign-in screens as are real content, the site is treated as gated and the sync fails the same way. A sign-in screen here means a sign-in title next to a password box, next to a "Continue with Google"-style button, next to a prompt that mails you a login link or asks for a passkey, or with nothing else readable on the page at all.

Where the start page redirects, the crawler reads the page it lands on too. A sign-in screen there weighs on the sign-in side of that balance — the site chose to send you to it, so it says something about getting in — and where nothing else could be indexed at all, the sync fails as a sign-in screen rather than as a redirect out of scope, which is the more useful of the two answers. An ordinary page there weighs on neither side: a bare domain that redirects to its www address or to a regional home page says nothing about whether the rest of the site is open.

A start page that opens fine doesn't exempt the rest of the site, but it isn't ignored either: it counts twice toward real content, so it takes that many more sign-in screens to tip the balance. So a public home page above a members area keeps its public pages and leaves the members area out of the index, while a single "How to log in" support article on an otherwise open site is still indexed as the content it is — as is an ordinary page that merely carries the sign-in form your site's template puts on every page. A newsletter signup and a "Reset your password" page collect an address and mail something back too, and neither is counted as a sign-in screen.

The page's own title is what decides that, so there is one case where a real article does count: a support page titled "Single sign-on" or "Log in with a passkey", under a template that also puts a sign-in button above it, reads as a sign-in screen however much is written on it. One or two are outnumbered by the rest of a help centre. A whole section of them is not — if that's your site, point the source at a section below it.

A site that moved behind a sign-in wall between crawls is the usual cause. It's also worth checking that the page really is a sign-in screen — a page titled something like "Log in to your account", sitting next to a sign-in form, a "Continue with…" button, or an "Email me a login link" prompt, or one that renders entirely in the browser and has nothing else on it yet, can occasionally be read as one. If that's what happened, point the source at a different page.

Content sources are re-checked periodically. If a check fails, the person who set the source up receives an email with the reason and a link to the item — for example the site could not be reached, its robots.txt now blocks crawling, or it sits entirely behind a sign-in screen.

A failed check doesn't take the source away from your agents. They keep answering from the last version that indexed successfully, and the item shows a Stale badge instead of its usual status — so what they know simply stops moving forward until an update succeeds. Re-checks keep running in the background, and a source that starts working again picks itself back up without you doing anything. If the reason needs a fix at your end — a new address, a crawlable URL — make it and re-sync.

Some sources have no earlier version to fall back on, so agents cannot use them at all until an index succeeds. These show Failed rather than Stale:

You'll get one message per failure, not one per retry — and, if it's checked again and still fails, another every few weeks until it recovers or you fix it. No further messages once the source updates successfully again.

Deleting a knowledge item or a collection that is still attached somewhere no longer fails. TAI first tells you what still uses it, by name, and asks you to confirm.

Confirming detaches the item from everything that referenced it, so those agents and skills keep working; they simply no longer have that source. Detach it by hand first if you would rather review the referrers one by one.

Before you rely on a knowledge item, search it the way an agent would.

This shows the top 8 matches — the same number an agent itself retrieves, so what you see here is what it would see. Each result names its source and says whether it matched on meaning (Semantic), on wording (Keyword), or both. Anything that matched on meaning also shows how relevant it is: a percentage — how close your question's meaning is to the passage's — and a bar comparing it against the other results in this search, so even closely-scored results are easy to tell apart at a glance. A long result can be expanded to read the full passage instead of just the first few lines.

When nothing comes back, the message tells you why:

That last case is what an agent sees too: rather than quoting loosely related content, it will say it doesn't know. If a question you expect it to answer lands here, the source is probably missing the subject your users are asking about — add it, or reword the source to use the terms they use. You don't need to match their exact phrasing: wording matches allow for word forms, so "cancellation policy" finds "cancellation policies".

Your sources don't all have to be in the same language, and neither do the questions.

If you add a source in a new language, use Re-sync on it so its wording matches are indexed for that language.