Bring your own embeddings

Aetherfy stores and searches vectors. It does not produce them. There is no endpoint, parameter or SDK method that turns text into an embedding — you compute the vector yourself and hand Aetherfy a plain list of floats.


export AETHERFY_API_KEY="afy_live_your_key_here"

What Aetherfy does and does not do with embeddings

	Who does it
Turning text into a vector	You, with an embedding model of your choice
Storing the vector	Aetherfy
Indexing and searching the vector	Aetherfy
Storing the original text alongside it	Aetherfy, as a payload field

There is no server-side embedding in Aetherfy today. In the vector SDK, Point.vector is a required field. In the memory API, omitting vector raises EmbeddingNotSupportedError client-side, before any request is sent — it is a programming error, not a transient failure, and retrying will not help.

Aetherfy ships no integration with any embedding provider. There is no provider argument, no API-key passthrough, and no adapter package. That is not a limitation to work around: because the SDK only ever receives a list of floats, any model from any provider — hosted or running locally — works without Aetherfy knowing anything about it.

The Aetherfy embedding seam

Write one function. Everything else on this page calls it.


from typing import List
 
def embed(text: str) -> List[float]:
    """Return the embedding for `text`.
 
    Replace the body with a call to your embedding model. The only
    contract Aetherfy cares about: return a list of floats whose length
    equals the `size` you gave the collection, every time.
    """
    raise NotImplementedError


async function embed(text: string): Promise<number[]> {
  // Replace the body with a call to your embedding model. The only
  // contract Aetherfy cares about: return an array of numbers whose
  // length equals the `size` you gave the collection, every time.
  throw new Error('not implemented');
}

Three properties are what matter, and none of them involve Aetherfy:

Property	Why
Fixed length	The collection’s `size` is fixed at creation and every vector must match it
Same model for writes and queries	A query vector from a different model is not comparable to your stored vectors, and Aetherfy cannot detect this — you get plausible-looking, meaningless results
Same preprocessing for writes and queries	Casing, truncation and normalisation must match, for the same reason

Matching the Aetherfy collection dimension

size is set when you create the collection and cannot be changed afterwards. It must equal the output dimension of your embedding model.

Aetherfy imposes no cap on the dimension — a 384-, 768-, 1536- or 3072-dimension model is equally acceptable. What Aetherfy does enforce is consistency: every vector written to a collection must have exactly the declared length.

Read the dimension from the model itself rather than hard-coding it, so the two cannot drift apart.

Python


from aetherfy_vectors import AetherfyVectorsClient
from aetherfy_vectors.models import VectorConfig, DistanceMetric
 
client = AetherfyVectorsClient()
 
# Derive the dimension from the model, once.
DIM = len(embed("dimension probe"))
 
client.create_collection(
    "notes",
    VectorConfig(size=DIM, distance=DistanceMetric.COSINE),
    description=f"Notes embedded at {DIM} dimensions",
)
 
print(client.get_collection("notes").config.size)  # DIM

JavaScript


import { AetherfyVectorsClient, DistanceMetric } from 'aetherfy-vectors';
 
const client = new AetherfyVectorsClient();
 
// Derive the dimension from the model, once.
const DIM = (await embed('dimension probe')).length;
 
await client.createCollection(
  'notes',
  { size: DIM, distance: DistanceMetric.COSINE },
  `Notes embedded at ${DIM} dimensions`,
);
 
console.log((await client.getCollection('notes')).config.size); // DIM

Changing to a model with a different output dimension means creating a new collection and re-embedding everything. There is no in-place migration.

Choosing an Aetherfy distance metric for your model

Use the metric your embedding model was trained for. Aetherfy does not know or infer this and will happily index vectors under the wrong metric, returning results that look ranked but are not meaningful.

Enum member	Wire value sent	Value Aetherfy stores and returns
`DistanceMetric.COSINE`	`Cosine`	`Cosine`
`DistanceMetric.EUCLIDEAN`	`Euclidean`	`Euclid`
`DistanceMetric.DOT`	`Dot`	`Dot`
`DistanceMetric.MANHATTAN`	`Manhattan`	`Manhattan`

EUCLIDEAN does not round-trip: Aetherfy normalises distance names case-insensitively and stores both euclid and euclidean as Euclid, so a collection created with DistanceMetric.EUCLIDEAN reads back config.distance == "Euclid" and an equality check against 'Euclidean' silently fails. The other three come back unchanged.

Consult your model’s documentation. Aetherfy takes no position on which is correct for it.

A complete Aetherfy write-and-search flow in Python

Fill in embed() and this runs end to end.


import hashlib
from typing import List
 
from aetherfy_vectors import AetherfyVectorsClient
from aetherfy_vectors.models import VectorConfig, DistanceMetric, Point
 
 
def embed(text: str) -> List[float]:
    """REPLACE ME with your embedding model.
 
    This placeholder is deterministic so the example runs, but it is not
    semantic — similar sentences will not be near each other.
    """
    h = hashlib.sha256(text.encode()).digest()
    return [h[i % len(h)] / 255.0 for i in range(384)]
 
 
client = AetherfyVectorsClient()
 
DIM = len(embed("dimension probe"))
if not client.collection_exists("notes"):
    client.create_collection("notes", VectorConfig(size=DIM, distance=DistanceMetric.COSINE))
 
documents = [
    (1, "The mitochondria is the powerhouse of the cell."),
    (2, "Postgres uses MVCC for concurrency control."),
    (3, "Cosine similarity ignores vector magnitude."),
]
 
# Embed and write. Keep the original text in the payload — Aetherfy stores
# the vector, but the vector is not readable back as text.
client.upsert("notes", [
    Point(id=doc_id, vector=embed(text), payload={"text": text})
    for doc_id, text in documents
])
 
# Query with a vector from the SAME embed() function.
results = client.search("notes", query_vector=embed("how does Postgres handle concurrency?"), limit=2)
for r in results:
    print(r.id, r.payload["text"])
 
client.close()

Always store the source text in the payload. A vector cannot be turned back into text, so without the payload a search result gives you an id and nothing a human or a model can read.

A complete Aetherfy write-and-search flow in JavaScript


import { createHash } from 'node:crypto';
import { AetherfyVectorsClient, DistanceMetric } from 'aetherfy-vectors';
 
async function embed(text: string): Promise<number[]> {
  // REPLACE ME with your embedding model. Deterministic, not semantic.
  const h = createHash('sha256').update(text).digest();
  return Array.from({ length: 384 }, (_, i) => h[i % h.length] / 255);
}
 
const client = new AetherfyVectorsClient();
 
async function main() {
  const DIM = (await embed('dimension probe')).length;
  if (!(await client.collectionExists('notes'))) {
    await client.createCollection('notes', { size: DIM, distance: DistanceMetric.COSINE });
  }
 
  const documents: Array<[number, string]> = [
    [1, 'The mitochondria is the powerhouse of the cell.'],
    [2, 'Postgres uses MVCC for concurrency control.'],
    [3, 'Cosine similarity ignores vector magnitude.'],
  ];
 
  const points = await Promise.all(
    documents.map(async ([id, text]) => ({
      id,
      vector: await embed(text),
      payload: { text },
    })),
  );
  await client.upsert('notes', points);
 
  const results = await client.search(
    'notes',
    await embed('how does Postgres handle concurrency?'),
    { limit: 2 },
  );
  for (const r of results) console.log(r.id, r.payload.text);
 
  client.dispose();
}
 
main();

Embedding in batches for an Aetherfy import

Most embedding models expose a batch call that is far cheaper per item than one call per document. Embed in batches, then upsert in batches; the two batch sizes are independent.


from typing import List
 
def embed_batch(texts: List[str]) -> List[List[float]]:
    """REPLACE ME. Return one vector per input text, in input order."""
    return [embed(t) for t in texts]
 
 
EMBED_BATCH = 64
UPSERT_BATCH = 100
 
texts = [f"document number {i}" for i in range(1000)]
pending: List[Point] = []
 
for start in range(0, len(texts), EMBED_BATCH):
    chunk = texts[start:start + EMBED_BATCH]
    vectors = embed_batch(chunk)
    assert len(vectors) == len(chunk)  # order and count must line up
 
    for offset, (text, vector) in enumerate(zip(chunk, vectors)):
        pending.append(Point(id=start + offset, vector=vector, payload={"text": text}))
 
    while len(pending) >= UPSERT_BATCH:
        client.upsert("notes", pending[:UPSERT_BATCH])
        pending = pending[UPSERT_BATCH:]
 
if pending:
    client.upsert("notes", pending)


async function embedBatch(texts: string[]): Promise<number[][]> {
  // REPLACE ME. Return one vector per input text, in input order.
  return Promise.all(texts.map((t) => embed(t)));
}
 
const EMBED_BATCH = 64;
const UPSERT_BATCH = 100;
 
const texts = Array.from({ length: 1000 }, (_, i) => `document number ${i}`);
let pending: Array<{ id: number; vector: number[]; payload: { text: string } }> = [];
 
for (let start = 0; start < texts.length; start += EMBED_BATCH) {
  const chunk = texts.slice(start, start + EMBED_BATCH);
  const vectors = await embedBatch(chunk);
  if (vectors.length !== chunk.length) throw new Error('embedder returned a different count');
 
  chunk.forEach((text, offset) => {
    pending.push({ id: start + offset, vector: vectors[offset], payload: { text } });
  });
 
  while (pending.length >= UPSERT_BATCH) {
    await client.upsert('notes', pending.slice(0, UPSERT_BATCH));
    pending = pending.slice(UPSERT_BATCH);
  }
}
 
if (pending.length) await client.upsert('notes', pending);

The order guarantee is yours to uphold. Aetherfy pairs the id, vector and payload exactly as you assembled them and cannot detect a batch whose vectors were returned out of order.

Embeddings and the Aetherfy memory API

The memory layer has the same requirement, enforced more loudly: a namespace or thread write without a vector raises EmbeddingNotSupportedError before the request is built.


import hashlib
from aetherfy_memory import MemoryClient
from aetherfy_memory.exceptions import EmbeddingNotSupportedError
 
def embed(text: str) -> list[float]:
    h = hashlib.sha256(text.encode()).digest()
    return [h[i % len(h)] / 255.0 for i in range(384)]
 
memory = MemoryClient()
memory.create_namespace("customer-42")
ns = memory.namespace("customer-42")
 
# Correct: text plus its vector.
ns.add(text="Lives in NYC", vector=embed("Lives in NYC"))
 
# Incorrect: no vector. Raises immediately, no request is sent.
try:
    ns.add(text="Lives in NYC")
except EmbeddingNotSupportedError as e:
    print("Aetherfy does not embed for you:", e)


import { createHash } from 'node:crypto';
import { MemoryClient, EmbeddingNotSupportedError } from 'aetherfy-vectors';
 
function embed(text: string): number[] {
  const h = createHash('sha256').update(text).digest();
  return Array.from({ length: 384 }, (_, i) => h[i % h.length] / 255);
}
 
const memory = new MemoryClient();
 
async function main() {
  await memory.createNamespace('customer-42');
  const ns = await memory.namespace('customer-42');
 
  // Correct.
  await ns.add({ text: 'Lives in NYC', vector: embed('Lives in NYC') });
 
  // Incorrect. Throws immediately, no request is sent.
  try {
    await ns.add({ text: 'Lives in NYC' });
  } catch (e) {
    if (e instanceof EmbeddingNotSupportedError) {
      console.log('Aetherfy does not embed for you:', e.message);
    } else {
      throw e;
    }
  }
}
 
main();

DEFAULT_VECTOR_SIZE in the Aetherfy memory layer is 384. If your model produces a different dimension, use it consistently — the constant is a default, not a constraint on what you may store.

Common Aetherfy embedding mistakes

Mistake	Symptom	Fix
Query embedded with a different model than the documents	Results come back ranked but irrelevant; no error	Use one `embed()` for both paths
Different preprocessing at query time	Same as above	Put normalisation inside `embed()`, not at the call site
Collection `size` does not match the model	The write is rejected	Derive `size` from `len(embed(...))` at startup
Wrong distance metric for the model	Plausible but poor ranking; no error	Use the metric your model documents
Source text not stored in the payload	Search returns ids you cannot interpret	Always store the text under a payload key
Expecting Aetherfy to embed for you	`EmbeddingNotSupportedError`, or a missing `vector` field	Compute the vector before calling Aetherfy