Skip to Content
Raw

Filtering in the Aetherfy vector database

How filters travel through Aetherfy

A filter narrows a search, count, scroll, or delete to the points whose payload satisfies it. Aetherfy forwards a valid filter to Qdrant unchanged: it never rewrites one.

Filters are validated; an unrecognised key is refused with INVALID_FILTER, naming the key and the nearest valid one. The engine answers a misspelled key in its own way. Most it refuses with its own 400 or 422, in its own wording, but some it serves without a word: a range with only a misspelled bound (gtee for gte) has no bound and so matches every point, and a key that sits beside the right ones in a condition is ignored. A search then returns unfiltered results, and a delete by filter deletes more than you meant, up to everything. Aetherfy checks every clause key and condition key, at any depth, in every request that takes a filter (search and query, scroll, count, a delete or payload or vector write by filter, and every prefetch), and refuses every unrecognised one the same way, with 400, before anything is read or changed:

{ "error": { "code": "INVALID_FILTER", "message": "Unrecognised filter key 'must_nott' at filter.must_nott. Did you mean 'must_not'?", "field": "filter.must_nott", "key": "must_nott", "suggestion": "must_not" } }

field is the JSON path of the offending key in your request body (filter.must_not[0].mach for a misspelled match in the first must_not condition), key is the key as you sent it, and suggestion is the nearest valid key, present when one is close. An object that mixes two kinds of condition, such as a field condition with a must of its own, is refused the same way: the engine would keep one and drop the other. Only keys are checked. The members of the request around a filter are checked too: a misspelled fitler, limitt or prefetch is served by the engine as an unfiltered or unbounded request, so it is refused with INVALID_REQUEST_FIELD, with the path and the nearest member, the same way. This includes consistency, timeout, wait and ordering sent in a body: they are query parameters, have no effect there, and are refused. A delete naming both points and filter is refused too, since the engine would keep one and drop the other. A value of the wrong type is still the engine’s own 400, and a range that no value can satisfy (gt above lt) is a valid filter that matches nothing.

The SDKs send a filter as you wrote it and leave the judging to the server, so an SDK, a plain HTTP caller and the CLI all get the same answer. A refusal arrives as InvalidFilterError, carrying .field, .key and .suggestion. The vocabulary below is the whole contract.

Filter vocabulary in the Aetherfy vector database

An Aetherfy filter is an object with up to three clause arrays. Each clause array holds condition objects.

ClauseMeaningLogical operator
mustEvery condition in the array holdsAND
shouldAt least one condition in the array holdsOR
must_notNo condition in the array holdsNOT

The wire key for exclusion is must_not, with an underscore. In Python you write that key directly; in JavaScript you write mustNot and the SDK translates it. See the SDK spelling section below.

A filter may also carry min_should: {"conditions": [...], "min_count": 2} holds when at least min_count of its conditions hold. In Python it is the min_should key (or Filter.min_should); in JavaScript it is minShould, sent as min_should.

Conditions come in two shapes:

ConditionShapeUse
Match{"key": "<field>", "match": {"value": <value>}}Exact equality on a payload field
Range{"key": "<field>", "range": {"gte": ..., "lte": ..., "gt": ..., "lt": ...}}Numeric bounds; supply any subset of the four bounds

In the Aetherfy Python SDK you may pass either a plain dictionary to query_filter= or the Filter dataclass, whose to_dict() emits exactly the must / must_not / should wire keys. In the Aetherfy JavaScript SDK you pass a plain object to queryFilter.

Matching one value with Aetherfy

The simplest Aetherfy filter is a single equality condition under must.

import os from aetherfy_vectors import AetherfyVectorsClient client = AetherfyVectorsClient(api_key=os.environ["AETHERFY_API_KEY"]) results = client.search( collection_name="articles", query_vector=[0.1, 0.2, 0.3, 0.4], limit=10, query_filter={"must": [{"key": "lang", "match": {"value": "en"}}]}, ) for hit in results: print(hit.id, hit.score, hit.payload) client.close()
import { AetherfyVectorsClient } from 'aetherfy-vectors'; const client = new AetherfyVectorsClient({ apiKey: process.env.AETHERFY_API_KEY }); const results = await client.search('articles', [0.1, 0.2, 0.3, 0.4], { limit: 10, queryFilter: { must: [{ key: 'lang', match: { value: 'en' } }] }, }); for (const hit of results) { console.log(hit.id, hit.score, hit.payload); } client.dispose();

The same filter over the Aetherfy REST API:

curl -s -X POST https://vectors.aetherfy.com/api/v1/workspaces/default/collections/articles/points/query \ -H "Authorization: Bearer $AETHERFY_API_KEY" \ -H "Content-Type: application/json" \ -d '{"query":[0.1,0.2,0.3,0.4],"limit":10, "filter":{"must":[{"key":"lang","match":{"value":"en"}}]}}'

Filtering by which agent wrote the point in Aetherfy

Every point the Aetherfy vector database returns carries __aetherfy_agent_id in its payload — the attested author, set by Aetherfy from the credential that performed the write. It is the agent’s id when an Aetherfy-injected key made the write and null when one of your own keys did. The full contract is on the REST API reference.

It filters like any other payload key. Everything one agent wrote:

curl -s -X POST https://vectors.aetherfy.com/api/v1/workspaces/default/collections/articles/points/scroll \ -H "Authorization: Bearer $AETHERFY_API_KEY" \ -H "Content-Type: application/json" \ -d '{"limit":20,"with_payload":true, "filter":{"must":[{"key":"__aetherfy_agent_id","match":{"value":"6f1c…"}}]}}'

Everything written by any agent, as opposed to by you, is must_not an is_empty match on the same key, which leaves the points whose author is set. Use is_empty, not is_null: the vector store’s payload merge cannot store a null, so a point you wrote may hold no author key at all, and is_empty matches a missing key and a null alike.

curl -s -X POST https://vectors.aetherfy.com/api/v1/workspaces/default/collections/articles/points/scroll \ -H "Authorization: Bearer $AETHERFY_API_KEY" \ -H "Content-Type: application/json" \ -d '{"limit":20,"with_payload":true, "filter":{"must_not":[{"is_empty":{"key":"__aetherfy_agent_id"}}]}}'

Everything written with one of your own keys is the same condition under must:

curl -s -X POST https://vectors.aetherfy.com/api/v1/workspaces/default/collections/articles/points/scroll \ -H "Authorization: Bearer $AETHERFY_API_KEY" \ -H "Content-Type: application/json" \ -d '{"limit":20,"with_payload":true, "filter":{"must":[{"is_empty":{"key":"__aetherfy_agent_id"}}]}}'

Every point also carries __aetherfy_deployment_id, the deployment that made the write (for a task agent, the run) or null when one of your own keys did, so everything one deployment wrote is:

curl -s -X POST https://vectors.aetherfy.com/api/v1/workspaces/default/collections/articles/points/scroll \ -H "Authorization: Bearer $AETHERFY_API_KEY" \ -H "Content-Type: application/json" \ -d '{"limit":20,"with_payload":true, "filter":{"must":[{"key":"__aetherfy_deployment_id","match":{"value":"9a2e…"}}]}}'

Aetherfy does not create a payload index for either key, so a filter on one is scanned rather than looked up. If you filter by author or deployment routinely, create the index yourself through the index route.

Seeing a collection’s payload indexes in the Aetherfy dashboard

A filter on a payload field with an index is looked up; on any other field it is scanned. A collection’s page in the Aetherfy dashboard lists its payload indexes, field and type, on its SCHEMA view, as built in the region named above the list. A collection with none reads: No payload indexes: filtering on a field scans every point.

Indexes are added in code, where they are repeatable: with the SDK’s create_field_index (Python) or createFieldIndex (JavaScript), described on the SDK reference, or with afy index create. The dashboard lists them and does not create or delete them.

Requiring several conditions at once with Aetherfy

Put every condition in the must array. Aetherfy forwards it as a conjunction: a point is returned only if all of them hold.

import os from aetherfy_vectors import AetherfyVectorsClient client = AetherfyVectorsClient(api_key=os.environ["AETHERFY_API_KEY"]) results = client.search( collection_name="articles", query_vector=[0.1, 0.2, 0.3, 0.4], limit=10, query_filter={ "must": [ {"key": "lang", "match": {"value": "en"}}, {"key": "status", "match": {"value": "published"}}, ] }, ) print([hit.id for hit in results]) client.close()
import { AetherfyVectorsClient } from 'aetherfy-vectors'; const client = new AetherfyVectorsClient({ apiKey: process.env.AETHERFY_API_KEY }); const results = await client.search('articles', [0.1, 0.2, 0.3, 0.4], { limit: 10, queryFilter: { must: [ { key: 'lang', match: { value: 'en' } }, { key: 'status', match: { value: 'published' } }, ], }, }); console.log(results.map((hit) => hit.id)); client.dispose();

Accepting any of several alternatives with Aetherfy

should is the disjunction. A point qualifies if at least one condition in the array holds. Aetherfy passes the clause through unchanged, so the engine’s own should semantics apply.

import os from aetherfy_vectors import AetherfyVectorsClient client = AetherfyVectorsClient(api_key=os.environ["AETHERFY_API_KEY"]) results = client.search( collection_name="articles", query_vector=[0.1, 0.2, 0.3, 0.4], limit=10, query_filter={ "should": [ {"key": "lang", "match": {"value": "en"}}, {"key": "lang", "match": {"value": "de"}}, ] }, ) print([hit.payload for hit in results]) client.close()
import { AetherfyVectorsClient } from 'aetherfy-vectors'; const client = new AetherfyVectorsClient({ apiKey: process.env.AETHERFY_API_KEY }); const results = await client.search('articles', [0.1, 0.2, 0.3, 0.4], { limit: 10, queryFilter: { should: [ { key: 'lang', match: { value: 'en' } }, { key: 'lang', match: { value: 'de' } }, ], }, }); console.log(results.map((hit) => hit.payload)); client.dispose();

Excluding points with Aetherfy

Exclusion uses must_not — underscore, not camel case. Every condition in the array must fail for the point to be returned.

In the Aetherfy Python SDK you can write the dataclass or the dictionary; both produce must_not on the wire.

import os from aetherfy_vectors import AetherfyVectorsClient from aetherfy_vectors.models import Filter client = AetherfyVectorsClient(api_key=os.environ["AETHERFY_API_KEY"]) # Dataclass form — Filter.to_dict() emits the must / must_not / should wire keys. exclude_french = Filter(must_not=[{"key": "lang", "match": {"value": "fr"}}]) print(exclude_french.to_dict()) results = client.search( collection_name="articles", query_vector=[0.1, 0.2, 0.3, 0.4], limit=10, query_filter=exclude_french, ) print([hit.id for hit in results]) # Dictionary form — identical on the wire. results = client.search( collection_name="articles", query_vector=[0.1, 0.2, 0.3, 0.4], limit=10, query_filter={"must_not": [{"key": "lang", "match": {"value": "fr"}}]}, ) print([hit.id for hit in results]) client.close()

In the Aetherfy JavaScript SDK write mustNot, the camelCase spelling the Filter interface declares. The SDK translates it to the wire key must_not for you, the same way it translates withPayload and scoreThreshold.

import { AetherfyVectorsClient } from 'aetherfy-vectors'; import type { Filter } from 'aetherfy-vectors'; const client = new AetherfyVectorsClient({ apiKey: process.env.AETHERFY_API_KEY }); const excludeFrench: Filter = { mustNot: [{ key: 'lang', match: { value: 'fr' } }], }; const results = await client.search('articles', [0.1, 0.2, 0.3, 0.4], { limit: 10, queryFilter: excludeFrench, }); console.log(results.map((hit) => hit.id)); client.dispose();

How each Aetherfy SDK spells the exclusion clause

The wire key is must_not. Each SDK reaches it from its own vocabulary, and in both cases you write the SDK’s spelling, not the wire’s:

SDKWhat you writeWhat Aetherfy sends
Pythonmust_not in a dict, or Filter(must_not=...)must_not
JavaScript / TypeScriptmustNotmust_not — the SDK translates it

Each Aetherfy SDK rejects the other’s spelling, loudly and by name. Both validate filter clauses before sending and raise on anything outside their own three, with a hint naming the sibling SDK:

You writePythonJavaScript
must / shouldacceptedaccepted
must_notacceptedthrows — “Write the clause as mustNot; the SDK translates it to the wire key must_not.”
mustNotthrows — “Write the clause as must_not; mustNot is the JavaScript SDK’s spelling.”accepted
anything elsethrowsthrows

The server refuses an unrecognised clause as well (INVALID_FILTER), so a caller on plain HTTP is covered: the wire key is must_not, and a request that sends mustNot is refused with suggestion: "must_not". The SDKs’ check is a faster, local refusal that also names the sibling SDK’s spelling. A caller who guesses the wrong idiom — and arriving from the other language’s docs is the obvious way to guess wrong — gets an error that names the right spelling, never results that include rows they meant to exclude. In JavaScript write mustNot; in Python write must_not.

In Python this applies to plain dicts. A Filter dataclass could never carry the wrong spelling — its fields are must, must_not, should — so passing a Filter sidesteps the question entirely. Every filter-taking method accepts either form, including count(count_filter=...).

Clause order does not matter to you, but Aetherfy fixes it (must, must_not, should) before sending, so two callers writing the same clauses in different orders share one server cache entry instead of two.

Filtering by numeric range with Aetherfy

A range condition carries any subset of gte, lte, gt, and lt under the range key. Aetherfy forwards it unchanged; a bound it does not know, such as gtee, is refused as an unrecognised key.

import os from aetherfy_vectors import AetherfyVectorsClient client = AetherfyVectorsClient(api_key=os.environ["AETHERFY_API_KEY"]) results = client.search( collection_name="articles", query_vector=[0.1, 0.2, 0.3, 0.4], limit=10, query_filter={"must": [{"key": "year", "range": {"gte": 2020, "lte": 2025}}]}, ) print([hit.payload for hit in results]) # Ranges work on count and scroll too. print(client.count("articles", count_filter={"must": [{"key": "year", "range": {"gt": 2024}}]}, exact=True)) client.close()
import { AetherfyVectorsClient } from 'aetherfy-vectors'; const client = new AetherfyVectorsClient({ apiKey: process.env.AETHERFY_API_KEY }); const results = await client.search('articles', [0.1, 0.2, 0.3, 0.4], { limit: 10, queryFilter: { must: [{ key: 'year', range: { gte: 2020, lte: 2025 } }] }, }); console.log(results.map((hit) => hit.payload)); // Ranges work on count and scroll too. console.log( await client.count('articles', { countFilter: { must: [{ key: 'year', range: { gt: 2024 } }] }, exact: true, }), ); client.dispose();

Combining all three clauses in an Aetherfy filter

The three clauses compose in a single filter object: everything in must holds, at least one thing in should holds, and nothing in must_not holds. Aetherfy sends the whole object as one unit.

import os from aetherfy_vectors import AetherfyVectorsClient client = AetherfyVectorsClient(api_key=os.environ["AETHERFY_API_KEY"]) results = client.search( collection_name="articles", query_vector=[0.1, 0.2, 0.3, 0.4], limit=10, query_filter={ "must": [ {"key": "status", "match": {"value": "published"}}, {"key": "year", "range": {"gte": 2020}}, ], "should": [ {"key": "lang", "match": {"value": "en"}}, {"key": "lang", "match": {"value": "de"}}, ], "must_not": [ {"key": "archived", "match": {"value": True}}, ], }, ) for hit in results: print(hit.id, hit.score, hit.payload) client.close()
import { AetherfyVectorsClient } from 'aetherfy-vectors'; import type { Filter } from 'aetherfy-vectors'; const client = new AetherfyVectorsClient({ apiKey: process.env.AETHERFY_API_KEY }); const combined: Filter = { must: [ { key: 'status', match: { value: 'published' } }, { key: 'year', range: { gte: 2020 } }, ], should: [ { key: 'lang', match: { value: 'en' } }, { key: 'lang', match: { value: 'de' } }, ], mustNot: [{ key: 'archived', match: { value: true } }], }; const results = await client.search('articles', [0.1, 0.2, 0.3, 0.4], { limit: 10, queryFilter: combined, }); for (const hit of results) { console.log(hit.id, hit.score, hit.payload); } client.dispose();

The same object over the Aetherfy REST API, where the clause keys are the only spelling that exists:

curl -s -X POST https://vectors.aetherfy.com/api/v1/workspaces/default/collections/articles/points/query \ -H "Authorization: Bearer $AETHERFY_API_KEY" \ -H "Content-Type: application/json" \ -d '{"query":[0.1,0.2,0.3,0.4],"limit":10, "filter":{ "must":[{"key":"status","match":{"value":"published"}}, {"key":"year","range":{"gte":2020}}], "should":[{"key":"lang","match":{"value":"en"}}, {"key":"lang","match":{"value":"de"}}], "must_not":[{"key":"archived","match":{"value":true}}] }}'

Where filters apply across the Aetherfy vector database

The same filter object is accepted by four Aetherfy operations, under four different parameter names.

OperationPython parameterJavaScript optionREST body key
Searchquery_filterqueryFilterfilter
Countcount_filtercountFilterfilter
Scrollscroll_filterscrollFilterfilter
Delete pointspoints_selector (pass the filter object)pointsSelector (pass the filter object)filter

Engine-level query tuning is a separate concern from filtering and travels in its own field — see search tuning. The full method surface is on the SDK reference.

Last updated on