Embedding Providers

This guide covers configuring each supported embedding provider for semantic search. You only need one provider; choose based on your requirements:

Provider	Runs locally	Requires GPU	Internet required	Install size	RAM during embedding
Ollama	Yes	No (CPU works fine)	No	~2 GB (model)	~2-4 GB (separate process)
FastEmbed	Yes	No	First run only (model download)	Small runtime + model	~1-2 GB (in-process)
OpenAI	No (API call)	N/A	Yes	Minimal	Negligible
Voyage AI	No (API call)	N/A	Yes	Minimal	Negligible

All four providers produce embeddings that enable the semantic and hybrid search modes in the search tool.

Ollama

Ollama runs embedding models locally. It's the recommended option for local, private embeddings: easy to set up and works well on CPU.

Install Ollama

macOSLinuxDocker

brew install ollama

curl -fsSL https://ollama.com/install.sh | sh

If your vault server runs in Docker and Ollama runs on the host, no Ollama install inside the container is needed — just point to the host.

Pull the embedding model

ollama pull nomic-embed-text

Verify it's available:

ollama list

You should see nomic-embed-text in the list.

Configure

MARKDOWN_VAULT_MCP_EMBEDDING_PROVIDER=ollama
OLLAMA_HOST=http://localhost:11434
MARKDOWN_VAULT_MCP_OLLAMA_MODEL=nomic-embed-text
MARKDOWN_VAULT_MCP_EMBEDDINGS_PATH=/path/to/store/embeddings

CPU-only mode: if you have a GPU but want to force CPU-only (such as to reserve the GPU for inference):

MARKDOWN_VAULT_MCP_OLLAMA_CPU_ONLY=true

Docker-to-host networking: if Ollama runs on the host and the vault server runs in Docker:

Docker Desktop (macOS/Windows)Linux (without Docker Desktop)

OLLAMA_HOST=http://host.docker.internal:11434

Add to your compose.yml:

services:
  markdown-vault-mcp:
    extra_hosts:
      - "host.docker.internal:host-gateway"

Then use:

OLLAMA_HOST=http://host.docker.internal:11434

Verify

# Test Ollama is reachable
curl http://localhost:11434/api/tags

# Test embedding generation
curl http://localhost:11434/api/embeddings -d '{
  "model": "nomic-embed-text",
  "prompt": "test embedding"
}'

You should get a JSON response with an embedding array. After starting the vault server, use hybrid search:

Search for "project planning" using hybrid mode

If embeddings are working, hybrid and semantic search modes will return results ranked by conceptual similarity.

---

FastEmbed

FastEmbed runs ONNX embedding models directly in Python, with no separate server needed.

Install

pip install markdown-vault-mcp[embeddings]

Or with uv:

uv pip install markdown-vault-mcp[embeddings]

The [all] extra includes FastEmbed as well.

Configure

MARKDOWN_VAULT_MCP_EMBEDDING_PROVIDER=fastembed
MARKDOWN_VAULT_MCP_FASTEMBED_MODEL=BAAI/bge-small-en-v1.5
MARKDOWN_VAULT_MCP_FASTEMBED_CACHE_DIR=/path/to/store/fastembed-cache
MARKDOWN_VAULT_MCP_EMBEDDINGS_PATH=/path/to/store/embeddings

No host URL or API key needed. The model downloads automatically on first use and is reused from cache after that.

First startup downloads the model

Set MARKDOWN_VAULT_MCP_FASTEMBED_CACHE_DIR to a persistent location. In Docker, the default compose layout stores this under /data/state/fastembed on the state-data named volume to avoid re-downloading on container recreation.

Memory usage: in-process vs out-of-process

FastEmbed runs the ONNX model inside the Python process, so the container itself bears the full inference memory cost. The default model (BAAI/bge-small-en-v1.5, 512-token context) keeps this manageable. If you switch to a long-context model such as nomic-ai/nomic-embed-text-v1.5 (8192-token context), you should reduce _FASTEMBED_ONNX_BATCH_SIZE in providers.py by a large amount; see issue #306.

By contrast, Ollama runs inference in a separate server process; the Python container only sends HTTP requests and receives float vectors, so its own memory footprint stays low. If memory is tight (such as on a small VPS), Ollama may be a better fit since its memory is isolated from the MCP server.

Verify

Start the server and test with a search:

Search for "meeting notes" using semantic mode

If FastEmbed is working, you'll get results ranked by semantic similarity even if the exact phrase doesn't appear in the documents.

---

OpenAI

Uses the OpenAI Embeddings API (text-embedding-3-small by default). Requires an API key and internet access. Lowest local resource usage, but sends document content to OpenAI.

Get an API key

1. Go to OpenAI API Keys
2. Create a new secret key
3. Copy it

Configure

MARKDOWN_VAULT_MCP_EMBEDDING_PROVIDER=openai
OPENAI_API_KEY=sk-your-api-key-here
MARKDOWN_VAULT_MCP_EMBEDDINGS_PATH=/path/to/store/embeddings

For any other OpenAI-compatible endpoint, override the base URL and model:

MARKDOWN_VAULT_MCP_EMBEDDING_PROVIDER=openai
OPENAI_API_KEY=your-provider-api-key
OPENAI_BASE_URL=https://api.siliconflow.cn/v1
OPENAI_EMBEDDING_MODEL=BAAI/bge-m3
MARKDOWN_VAULT_MCP_EMBEDDINGS_PATH=/path/to/store/embeddings

See OpenAI-compatible endpoints for the caveats that come with pointing this provider at a third-party service.

Privacy

Document content (titles, headings, body text) is sent to the configured OpenAI-compatible provider for embedding. Do not use this provider if your vault contains sensitive data you don't want to share with that provider. Use Ollama or FastEmbed for fully local, private embeddings.

Cost

OpenAI embeddings are inexpensive. text-embedding-3-small costs $0.02 per million tokens. A vault of 1,000 notes (~500K tokens) costs about $0.01 to embed. Reindexing only processes changed documents.

Verify

# Test your API key (replace $OPENAI_API_KEY with your key, or export it first)
curl https://api.openai.com/v1/embeddings \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"input": "test", "model": "text-embedding-3-small"}'

For OpenAI-compatible providers:

curl "$OPENAI_BASE_URL/embeddings" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"input\": \"test\", \"model\": \"$OPENAI_EMBEDDING_MODEL\"}"

You should get a JSON response with an embedding array. After starting the server, test hybrid search:

Search for "project ideas" using hybrid mode

---

Voyage AI

Uses Voyage AI embeddings (voyage-4 by default). Voyage serves its Embeddings API in the OpenAI request/response shape, so this provider is the openai one with the base URL pinned to https://api.voyageai.com/v1 and its own key and model variables.

Get an API key

1. Go to the Voyage dashboard
2. Create an API key
3. Copy it

Configure

MARKDOWN_VAULT_MCP_EMBEDDING_PROVIDER=voyage
VOYAGE_API_KEY=pa-your-api-key-here
MARKDOWN_VAULT_MCP_EMBEDDINGS_PATH=/path/to/store/embeddings

To pick a different model:

MARKDOWN_VAULT_MCP_VOYAGE_MODEL=voyage-4-large

The default is voyage-4, the balanced member of the family; voyage-4-large trades cost for retrieval quality and voyage-4-lite trades quality for cost. The voyage-4 and voyage-3.5 families take 32,000 tokens of input and return 1024-dimensional vectors by default. Changing the model re-embeds the vault once on the next startup.

Queries and documents are embedded differently

Voyage's models are tuned for retrieval, and the vendor asks callers to say which side of a search each text is on. The server does: notes are embedded as input_type: document and search queries as input_type: query, which makes Voyage prepend its own retrieval prompt to each. Nothing to configure: it is how the voyage provider embeds.

Voyage vaults re-embed once on upgrade

Vaults embedded before this behavior existed hold vectors Voyage produced with no input_type at all. Those sit in a different space from the typed ones, so the vector sidecar's identity check fails on the first startup after the upgrade and the vault re-embeds itself once, exactly as it does after a model change. The rebuild is automatic; it costs one pass of Voyage API calls over the vault.

Voyage must be selected explicitly

Unlike the other three, voyage is not in the auto-detection chain. A VOYAGE_API_KEY exported for some other tool would otherwise take over an existing index and force a full re-embed.

Privacy

Document content (titles, headings, body text) is sent to Voyage AI for embedding. Use Ollama or FastEmbed for fully local, private embeddings.

You could always do this by hand

Setting MARKDOWN_VAULT_MCP_EMBEDDING_PROVIDER=openai with OPENAI_BASE_URL=https://api.voyageai.com/v1 still works and is wire-identical. The voyage name exists so the endpoint, the key variable, and a sensible model default are discoverable, and so the vector sidecar records voyage as the provider rather than openai.

Verify

# Test your API key (replace $VOYAGE_API_KEY with your key, or export it first)
curl https://api.voyageai.com/v1/embeddings \
  -H "Authorization: Bearer $VOYAGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"input": "test", "model": "voyage-4"}'

You should get a JSON response with an embedding array.

---

OpenAI-compatible endpoints

MARKDOWN_VAULT_MCP_EMBEDDING_PROVIDER=openai is a generic client, not a binding to OpenAI the company. Any service that serves POST /v1/embeddings in the OpenAI request and response shape works by pointing OPENAI_BASE_URL at it. Jina, Mistral, SiliconFlow, LiteLLM, vLLM, and Text Embeddings Inference all serve that shape, as do self-hosted gateways in front of another model.

The recipe

MARKDOWN_VAULT_MCP_EMBEDDING_PROVIDER=openai
OPENAI_API_KEY=your-provider-api-key
OPENAI_BASE_URL=https://api.example-vendor.com/v1
OPENAI_EMBEDDING_MODEL=the-vendor-model-name
MARKDOWN_VAULT_MCP_EMBEDDINGS_PATH=/path/to/store/embeddings

The base URL includes the version prefix and omits the trailing /embeddings: the SDK appends the path. Both OPENAI_BASE_URL and OPENAI_EMBEDDING_MODEL also accept the MARKDOWN_VAULT_MCP_-prefixed spelling, which wins over the bare one when both are set.

Confirm the endpoint answers before starting the server:

curl "$OPENAI_BASE_URL/embeddings" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"input\": \"test\", \"model\": \"$OPENAI_EMBEDDING_MODEL\"}"

A JSON response carrying an embedding array means the configuration will work.

A worked example

Voyage AI is the endpoint this pattern was verified against end to end (#949):

MARKDOWN_VAULT_MCP_EMBEDDING_PROVIDER=openai
OPENAI_API_KEY=pa-your-voyage-key     # a Voyage key in the OpenAI variable
OPENAI_BASE_URL=https://api.voyageai.com/v1
OPENAI_EMBEDDING_MODEL=voyage-4

That still works, and it is what the voyage provider does internally. Prefer the named provider for Voyage: it pins the base URL, gives the key its own variable, supplies a model default, and records voyage as the index identity. The example is here because it shows the shape for vendors that have no named provider.

What the server sends

Each request carries only model and input. The server never sends dimensions or user, and it leaves encoding_format to the openai SDK, whose base64 default is widely accepted. This narrow request shape is why strict endpoints work: Voyage answers HTTP 400 to dimensions, to user, and to encoding_format="float". An endpoint that requires a field outside model and input cannot be driven this way.

The one addition is vendor-specific and reached only through a named provider: voyage also sends input_type. Driving Voyage through this openai recipe instead sends no input_type, so it does not get the query/document asymmetry. That is one more reason to prefer the named provider.

Caveats worth knowing

Changing only the base URL is invisible to the index

The vector sidecar records the provider name, the model name, and how that provider embeds, and a mismatch on startup re-embeds the vault automatically. Under this recipe the first two stay openai and your configured model string no matter which vendor serves them, and the third is always empty, so aiming OPENAI_BASE_URL at a different vendor while keeping the same model name goes undetected: vectors from two different models end up in one index, and search quality degrades quietly. Force the rebuild yourself by deleting the two sidecar files beside EMBEDDINGS_PATH (the .npy matrix and the .json metadata) and restarting.

Unknown context length falls back to a 1500-character chunk cap

The chunk cap derives from the model's context length. That is known only for the models the server ships a table for, so a third-party model falls back to 1500 characters. If your model's context is smaller than roughly 536 tokens, set MARKDOWN_VAULT_MCP_MAX_CHUNK_CHARS explicitly, or the endpoint will reject or truncate oversize chunks. See MAX_CHUNK_CHARS.

Auto-detection reacts to the key alone

With MARKDOWN_VAULT_MCP_EMBEDDING_PROVIDER unset, an OPENAI_API_KEY in the environment selects the openai provider, including a key you exported for something else. Set the provider explicitly when a vault's backend matters.

Privacy

Document content (titles, headings, body text) is sent to whichever endpoint you configure. Use Ollama or FastEmbed for fully local embeddings.

When an endpoint deserves its own provider name

Named presets are reserved for vendors whose API has behavior the generic transport cannot express. Voyage qualifies on its strict request-shape rejections and its query/document asymmetry, Ollama on being a local runtime with no key. For every other OpenAI-compatible endpoint the recipe above is the supported answer, and it costs nothing to run. If you think an endpoint clears that bar, open an issue describing the specific behavior the generic client cannot reach.

---

Auto-detection

If you don't set MARKDOWN_VAULT_MCP_EMBEDDING_PROVIDER, the server tries providers in this order:

1. OpenAI: if OPENAI_API_KEY is set
2. Ollama: if OLLAMA_HOST is reachable
3. FastEmbed: if the package is installed

voyage is never auto-detected; select it explicitly.

Set MARKDOWN_VAULT_MCP_EMBEDDING_PROVIDER explicitly to avoid surprises when your environment changes (setting OPENAI_API_KEY for another tool will cause the server to switch from Ollama to OpenAI).

Common to all providers

Regardless of which provider you choose:

• MARKDOWN_VAULT_MCP_EMBEDDINGS_PATH is required to enable semantic search. Without it, only keyword search is available.
• Embeddings are built automatically on first startup when a provider is configured. Subsequent starts load the persisted index from disk and only process changed files.
• Use mode="hybrid" in search for best results; it combines keyword (BM25) and semantic (cosine similarity) scores using Reciprocal Rank Fusion.

Large vaults

The initial embedding build uses two levels of batching to keep memory bounded:

1. Vault level — 4 chunks per provider call by default, configurable via MARKDOWN_VAULT_MCP_EMBEDDING_BATCH_SIZE
2. ONNX level (FastEmbed only) — 32 chunks per inference call (_FASTEMBED_ONNX_BATCH_SIZE in providers.py)

The ONNX batch size is tuned for the default BAAI/bge-small-en-v1.5 model (512-token context). ONNX self-attention scales as O(batch × seq_len²) in RAM; long-context models require a much smaller batch size to avoid OOM — see issue #306.

For very large vaults (thousands of notes), the first startup may take several minutes. If the process is interrupted mid-build, it will rebuild from scratch on the next startup — partial indices are never persisted.

Notes with no body

A note with no body is indexed for keyword search but is not embedded. This covers zero-byte files, notes that are nothing but frontmatter, and notes whose body is only whitespace. These notes hold no text to embed, so they carry no semantic signal either way.

The server skips these notes deliberately. Embedding providers reject an empty input string with HTTP 400 (Voyage reports Input cannot contain empty strings or empty lists), and the rejection fails the whole request, so a single body-less note used to stop a batch of unrelated notes from being embedded as well. On version 3.1.0 it could hold the entire index stale until the file was excluded. See issue #1087.

Nothing is required of you. The notes still appear in keyword search and in list_documents, and they gain vectors as soon as they get a body. If you set MARKDOWN_VAULT_MCP_EMBED_CONTEXT or MARKDOWN_VAULT_MCP_SEARCHABLE_FIELDS, the enriched input carries the title and frontmatter values, so those notes are embedded as before.

Slow or CPU-only backends

Each embedding request has a wall-clock budget, MARKDOWN_VAULT_MCP_EMBED_TIMEOUT_S (default 30.0 seconds). It applies to the network providers, OpenAI and Ollama. FastEmbed runs in-process with no network call and ignores it.

The budget and the batch size interact, and the default pairing is tight on modest hardware. On a CPU-only Ollama running a large model such as bge-m3, a single chunk can take 4 to 11 seconds, so the default batch of 4 chunks may need 20 to 44 seconds against a 30-second budget. The result is intermittent timeouts under ordinary writing load rather than an outright failure, which makes it easy to misread (issue #954).

If you see embeddings request failed: Request timed out, adjust one or both:

# give each request more room
export MARKDOWN_VAULT_MCP_EMBED_TIMEOUT_S=120

# and/or send fewer chunks per request, so each one finishes sooner
export MARKDOWN_VAULT_MCP_EMBEDDING_BATCH_SIZE=2

Raising the timeout to 60 to 120 seconds and dropping the batch to 1 or 2 covers most CPU-only setups. Note that the two settings pull in opposite directions on throughput: a smaller batch means more round-trips, so prefer raising the timeout first and shrink the batch only if individual requests still overrun.

Chunk sizing and the embedding context

The chunker (shared by keyword and semantic search) bounds every chunk by a word cap (MARKDOWN_VAULT_MCP_MAX_CHUNK_WORDS, default 400) and a character cap (MARKDOWN_VAULT_MCP_MAX_CHUNK_CHARS). The character cap exists because a chunk that fits the word cap can still exceed the embedding model's token limit; token-dense content (tables, code, CJK) packs far more tokens per word. Without it, such a chunk would abort the build (Ollama returns HTTP 400) or be silently truncated (FastEmbed), producing degraded embeddings.

When you don't set MAX_CHUNK_CHARS explicitly, the default is min(1500, round(context_length × 2.8)): retrieval quality peaks at ~256 to 512 tokens per chunk regardless of the model's context, and the 1500-char ceiling keeps the fastembed/ONNX path clear of the out-of-memory regime seen with oversize chunks (issue #306). The result:

• bge-small-en-v1.5 (512-token context) → ~1,434 chars (below the ceiling, used as is)
• a shorter-context model (below ~536 tokens) uses its own smaller value, round(context_length × 2.8)
• a longer-context model (2048, 8192, …) is capped at the 1500-char ceiling
• unknown context (no provider, or Ollama unreachable at startup) → the 1500-char ceiling

Set a positive value to force an exact cap. Set -1 to opt into unbounded context-scaling (round(context_length × 2.8) with no ceiling, or 1500 when the model context is unknown). This reproduces the pre-bounded context-scaling and can OOM the host on the fastembed/ONNX path with a long-context model, so use it only with Ollama (out-of-process) or a remote provider.

MARKDOWN_VAULT_MCP_CHUNK_OVERLAP_WORDS (default 40, 0 disables) adds a few words from the end of the previous fragment to the start of each fragment when a section is too large for the caps and gets split on paragraph, line, or word boundaries. This improves retrieval recall at those arbitrary split points. An overlapped fragment can exceed the word or character cap by up to the overlap word count. Overlap applies to new and re-indexed notes, so run reindex to apply it across an existing vault; it does not force a rebuild on its own.

Changing the embedding model triggers a one-time cold rebuild

Because the char cap is derived from the model's context, the chunk boundaries themselves depend on the embedding model. Changing the embedding model (or setting/changing MAX_CHUNK_CHARS) re-chunks the FTS index, not just the embeddings, so on the next startup the server automatically rejects the warm-restart short-circuit and does a background cold rebuild (keyword search returns first, semantic search once embeddings finish). No manual reindex is needed. The same one-time rebuild happens when an embedding-enabled vault is upgraded from a release before this behavior existed.

Long-context models are opt-in

The defaults (BAAI/bge-small-en-v1.5 for FastEmbed, nomic-embed-text for Ollama) are memory-light. Long-context models (nomic-ai/nomic-embed-text-v1.5 at 8192 tokens for FastEmbed, or bge-m3:latest for Ollama, using the name:tag form Ollama requires) give larger chunks but cost more memory: FastEmbed runs ONNX in-process and its self-attention is O(batch × seq_len²) in RAM (see the memory note under FastEmbed and issue #306); Ollama needs the model to fit GPU VRAM at the larger context. Prefer the defaults unless you have the headroom.