Ask a general-purpose model how to configure your product and it will answer confidently from whatever it saw during training, which is not your docs. Retrieval-augmented generation, usually shortened to RAG, is the standard fix. Before the model answers, your server searches your own documents for the passages most related to the question and puts them in the prompt. The model then answers from those passages and says which ones it used. Nothing is trained or fine-tuned; the knowledge stays in your docs and is looked up for every question.
This tutorial builds that for a docs site written with Nuxt Content 3.15 in a Nuxt 4.5 app: a chat page that answers from the Markdown files in content/docs/ and links to its sources. It uses AI SDK 7 (ai 7.0) with OpenAI models. Pages live in app/, and server/ stays at the project root.
How retrieval finds the right passages
Search by meaning needs embeddings. An embedding is a list of numbers (1,536 of them for OpenAI's text-embedding-3-small) that places a piece of text in a space where texts with similar meaning sit close together. "How do I add a route?" and a paragraph about files in app/pages/ end up near each other even though they share few words.
Closeness is measured with cosine similarity, which compares the direction of two vectors: close to 1 means very similar, close to 0 means unrelated. The chatbot uses both in four steps:
- At build time, split the docs into chunks and embed every chunk once.
- Store the chunks and their vectors in a JSON file the server can read.
- For each question, embed the question and rank the chunks by similarity.
- Send the best few chunks to the model with instructions to answer only from them and cite them.
Install the packages and keep the key on the server
The code needs ai for embeddings and generation, @ai-sdk/openai for OpenAI's models and @ai-sdk/vue for the chat page:
npm install ai @ai-sdk/openai @ai-sdk/vue
The OpenAI key must never reach the browser. Declare it as a private runtime config value: Nuxt fills it from the NUXT_OPENAI_API_KEY environment variable, and only server code can read it.
export default defineNuxtConfig({
modules: ['@nuxt/content'],
runtimeConfig: {
openaiApiKey: '',
},
})
The indexing script below reads the same variable. For development, put NUXT_OPENAI_API_KEY=… in .env. Nuxt reads that file in nuxt dev and nuxt build, but not when the built server runs, so on a host set the variable both for the build (the script needs it) and for the running server.
Chunk and embed the docs at build time
The docs are a Nuxt Content collection, so they already have pages and URLs:
import { defineCollection, defineContentConfig } from '@nuxt/content'
export default defineContentConfig({
collections: {
docs: defineCollection({ type: 'page', source: 'docs/**/*.md' }),
},
})
Nuxt Content turns content/docs/1.guide/2.pages.md into the path /docs/guide/pages: numeric prefixes are dropped and index.md maps to its folder. The indexing script reads the same files from disk, splits each one at its ## headings, and uses the same path rule so that every chunk can link back to its page.
import { mkdir, readdir, readFile, writeFile } from 'node:fs/promises'
import { join } from 'node:path'
import { createOpenAI } from '@ai-sdk/openai'
import { embedMany } from 'ai'
const openai = createOpenAI({ apiKey: process.env.NUXT_OPENAI_API_KEY })
const toPath = file => ('/docs/' + file.replace(/\.md$/, '').replace(/(^|\/)\d+\./g, '$1').replace(/(^|\/)index$/, '')).replace(/\/$/, '')
const chunks = []
for (const file of (await readdir('content/docs', { recursive: true })).filter(f => f.endsWith('.md'))) {
const raw = await readFile(join('content/docs', file), 'utf8')
const title = raw.match(/^title:\s*(.+)$/m)?.[1] ?? file
for (const section of raw.replace(/^---\n[\s\S]*?\n---\n/, '').split(/^(?=## )/m)) {
const heading = section.match(/^## (.+)$/m)?.[1] ?? 'Overview'
const text = section.trim().slice(0, 3000)
if (text.length > 80) chunks.push({ id: `${toPath(file)}#${chunks.length}`, path: toPath(file), title, heading, text })
}
}
const { embeddings, usage } = await embedMany({
model: openai.embedding('text-embedding-3-small'),
values: chunks.map(chunk => `${chunk.title}: ${chunk.heading}\n${chunk.text}`),
})
await mkdir('server/assets', { recursive: true })
await writeFile('server/assets/rag-index.json', JSON.stringify(chunks.map((chunk, i) => ({ ...chunk, embedding: embeddings[i] }))))
console.log(`Indexed ${chunks.length} chunks with ${usage.tokens} tokens`)
A section under one heading is a good chunk for docs: it is about one thing, and it has a title. The page title and the heading are embedded together with the text, so a short section still carries its context. Sections of 80 characters or fewer are skipped, and each section is cut at 3,000 characters. Text past that point is not indexed, so split very long sections in your docs.
embedMany sends the values in batches of at most 2,048, the OpenAI provider's limit for one call. It does not count tokens, and OpenAI rejects a request over 300,000 tokens in total, so on a large site embed the chunks a few hundred at a time.
Run the script before every build, so the index always matches the docs you deploy:
{
"scripts": {
"index:docs": "node --env-file-if-exists=.env scripts/build-rag-index.mjs",
"build": "npm run index:docs && nuxt build"
}
}
In development, run npm run index:docs once before npm run dev: without the file, the chat finds no sources.
Retrieve: embed the question and rank
Nitro bundles files in server/assets/ with the server and exposes them through useStorage('assets:server'). The helper lives in server/utils/, so server routes can call it without an import. It loads the index once per server process, embeds the question with the same model, and keeps the best matches:
import { createOpenAI } from '@ai-sdk/openai'
import { cosineSimilarity, embed } from 'ai'
export type Chunk = { id: string, path: string, title: string, heading: string, text: string, embedding: number[] }
let index: Chunk[] | undefined
export async function retrieve(question: string, apiKey: string, limit = 4) {
index ??= (await useStorage('assets:server').getItem<Chunk[]>('rag-index.json')) ?? []
const openai = createOpenAI({ apiKey })
const { embedding } = await embed({ model: openai.embedding('text-embedding-3-small'), value: question })
return index
.map(chunk => ({ chunk, score: cosineSimilarity(embedding, chunk.embedding) }))
.filter(hit => hit.score > 0.3)
.sort((a, b) => b.score - a.score)
.slice(0, limit)
.map(hit => hit.chunk)
}
The question must be embedded with the same model as the chunks. Vectors from two different models live in different spaces, and their similarity means nothing.
Comparing the question with every chunk sounds slow, but it is a few thousand multiplications per chunk, and a docs site with a few hundred chunks ranks in a millisecond or two.
The 0.3 cut-off drops chunks that are barely related. It is a starting point, not a law; the evaluation at the end tells you where to set it.
Answer with citations
The route takes the conversation from the chat, retrieves chunks for the latest question, numbers them, and tells the model to cite those numbers. The chunks also go to the browser as source-url parts: createUIMessageStream writes them first and then merges in the model's streamed answer, so the page can list them as links.
import { createOpenAI } from '@ai-sdk/openai'
import { convertToModelMessages, createUIMessageStream, createUIMessageStreamResponse, streamText, toUIMessageStream, type UIMessage } from 'ai'
export default defineEventHandler(async (event) => {
const { messages } = await readBody<{ messages: UIMessage[] }>(event)
const question = messages.at(-1)?.parts.map(part => (part.type === 'text' ? part.text : '')).join(' ').trim() ?? ''
if (!question || question.length > 500) throw createError({ statusCode: 400, statusMessage: 'Ask a question of up to 500 characters' })
const { openaiApiKey } = useRuntimeConfig(event)
const sources = await retrieve(question, openaiApiKey)
const context = sources.map((s, i) => `[${i + 1}] ${s.title}, ${s.heading}\n${s.text}`).join('\n\n')
const result = streamText({
model: createOpenAI({ apiKey: openaiApiKey })('gpt-5.4-mini'),
instructions: `Answer questions about our docs using only the sources below. Cite them as [1], [2]. If the sources do not answer the question, say you do not know.\n\n${context || 'No sources found.'}`,
messages: await convertToModelMessages(messages.slice(-6)),
maxOutputTokens: 700,
})
return createUIMessageStreamResponse({
stream: createUIMessageStream({
execute: ({ writer }) => {
sources.forEach((s, i) => writer.write({ type: 'source-url', sourceId: String(i + 1), url: s.path, title: `[${i + 1}] ${s.title}: ${s.heading}` }))
writer.merge(toUIMessageStream({ stream: result.stream }))
},
}),
})
})
Four details in the route matter:
- The numbering ties each claim to a page. Source
[2]in the model's answer is the second chunk in the instructions and the second link on the page. - The chunks go into
instructions, not into the user's message, because they are context from your server, not something the user said. - "Say you do not know" matters as much as the sources. Without it, a model that finds nothing useful in the context falls back on its training and answers anyway.
- Only the last six messages go to the model, which caps the cost of each request in a long conversation.
Retrieval uses only the latest question. For a follow-up such as "and on Windows?", that is often too little; including the previous question in the search text is a cheap improvement.
Show the answer and its sources
useChat from @ai-sdk/vue 4 talks to the route through a transport. Text parts are the answer; source-url parts are the citations, already numbered by the server:
<script setup lang="ts">
import { useChat } from '@ai-sdk/vue'
import { DefaultChatTransport, type SourceUrlUIPart, type UIMessage } from 'ai'
const input = ref('')
const { messages, sendMessage, status } = useChat({ transport: new DefaultChatTransport({ api: '/api/ask' }) })
const sourcesOf = (message: UIMessage) => message.parts.filter((part): part is SourceUrlUIPart => part.type === 'source-url')
function ask() {
if (!input.value.trim()) return
sendMessage({ text: input.value })
input.value = ''
}
</script>
<template>
<article v-for="message in messages" :key="message.id">
<template v-for="(part, index) in message.parts" :key="index">
<p v-if="part.type === 'text'">{{ part.text }}</p>
</template>
<ul v-if="sourcesOf(message).length">
<li v-for="source in sourcesOf(message)" :key="source.sourceId">
<NuxtLink :to="source.url">{{ source.title }}</NuxtLink>
</li>
</ul>
</article>
<form @submit.prevent="ask">
<label>Ask the docs <input v-model="input" type="text"></label>
<button type="submit" :disabled="status === 'submitted' || status === 'streaming'">Ask</button>
</form>
</template>
Each answer now ends with links to the exact pages it drew on, so a reader can open a source and check the claim.
Limits to plan for
Freshness. The index is a build artefact. A page edited after deploy is not in it until the next build, and switching embedding models means re-embedding everything.
Size. Each chunk adds roughly 20 to 30 KB of JSON for its 1,536 numbers, so a few hundred chunks make a file of a few megabytes, loaded once per server process. Past a few thousand chunks, move the vectors into SQLite with a vector extension or a dedicated vector database.
Cost. Embedding is cheap per token compared with generation, but re-embedding every chunk on every build adds up on a large site. Store a hash of each chunk's text next to its vector and only embed the chunks that changed.
Made-up answers. Retrieval reduces invention; it does not end it. The model can still blend in training knowledge or misread a passage, which is why the instructions ask for citations and the page shows them.
Prompt injection. Only index content you control. Text in the index ends up in the model's instructions, so a planted sentence there is a prompt injection. Last week's post on API security for AI features covers that, and the rate limits a public route such as /api/ask needs.
Evaluate with a handful of questions
Before you trust it, write down five to ten real questions and the page that should answer each. Then check that retrieval finds that page, which needs no generation at all:
import { readFile } from 'node:fs/promises'
import { createOpenAI } from '@ai-sdk/openai'
import { cosineSimilarity, embedMany } from 'ai'
const cases = [
{ question: 'How do I add a new page?', expect: '/docs/guide/pages' },
{ question: 'Can I get a refund after a year?', expect: '/docs/billing/refunds' },
{ question: 'Which Node version do I need?', expect: '/docs/install' },
]
const index = JSON.parse(await readFile('server/assets/rag-index.json', 'utf8'))
const openai = createOpenAI({ apiKey: process.env.NUXT_OPENAI_API_KEY })
const { embeddings } = await embedMany({ model: openai.embedding('text-embedding-3-small'), values: cases.map(c => c.question) })
cases.forEach((c, i) => {
const top = index.map(chunk => ({ path: chunk.path, score: cosineSimilarity(embeddings[i], chunk.embedding) }))
.sort((a, b) => b.score - a.score).slice(0, 4)
console.log(top.some(hit => hit.path === c.expect) ? 'PASS' : 'FAIL', c.question, top[0]?.path, top[0]?.score.toFixed(2))
})
After each change to chunking, rebuild the index with npm run index:docs and run node --env-file=.env scripts/eval-rag.mjs. A failing question usually means a chunk is too long and mixes topics, or the page uses different words from your readers. The printed scores also show where to set the 0.3 cut-off.
Then ask the same questions in the chat and read each answer against its cited sources. That catches what no retrieval score shows: a passage the model misread, or a claim cited to the wrong source.