afy collections, index, and points

The Aetherfy CLI’s vector commands

Three command groups of the Aetherfy CLI read and manage the Aetherfy vector database, with the same API key as every other afy command. They cover inspecting collections, creating and deleting them, managing payload indexes, and reading points. Loading points is not among them: the Python and JavaScript SDKs do that, with the chunking and retries a bulk load needs (see /vectors/sdk).

The collections group also answers to collection.

Flags every Aetherfy vector command takes

Each of the nine commands takes these four flags, in addition to the global flags on /cli.

How the Aetherfy CLI chooses the vectors endpoint

The Aetherfy CLI picks the endpoint in the same order as the Aetherfy SDKs:

1. --vectors-url, when given.
2. AETHERFY_VECTORS_URL, when set. It also wins over an API region, with a warning on stderr: Aetherfy sets this variable on every agent machine.
3. --api-region, else AETHERFY_VECTORS_API_REGION. The region is turned into a URL by asking GET /api/v1/regions on https://vectors.aetherfy.com.
4. https://vectors.aetherfy.com.

An API region outside the three above is refused before any request, with exit code 2. The API region only chooses which Aetherfy endpoint answers; it is not where a collection’s data lives, which afy collections create --regions sets.

How the Aetherfy CLI chooses the workspace

Aetherfy collection names are scoped to a workspace: articles in workspace research and articles outside any workspace are two different collections. The Aetherfy CLI uses --workspace when it is given, else AETHERFY_WORKSPACE, else no workspace, which is the SDKs’ workspace="auto" default. Aetherfy sets AETHERFY_WORKSPACE only on an agent that belongs to a workspace. --workspace "" forces no workspace even where the variable is set.

Every --json object says which endpoint and workspace the command used:

{
  "endpoint": "https://vectors.aetherfy.com",
  "workspace": "research",
  "collections": []
}

workspace is null when no workspace was used.

Listing and inspecting Aetherfy collections

afy collections list lists the collections in the chosen workspace, or the collections outside any workspace when none is chosen. The two are separate: neither list includes the other.

afy collections list
afy collections list --workspace research --json

Each entry of collections in the --json output carries name, description (or null), size, distance, status, points_count and created_at. An empty result is "collections": [].

afy collections get <name> shows one collection. Its --json output holds one collection object with the same fields plus regions and updated_at.

afy collections get articles --json

Creating and deleting Aetherfy collections

afy collections create <name> creates a collection for vectors of one size, compared with one distance.

afy collections create articles --size 1536 --distance cosine
afy collections create articles --size 768 --distance dot --regions us-east-1,eu-central-1 --json

The --json output’s collection holds name, size, distance and the regions Aetherfy placed the collection in. Creating a collection that already exists with the same settings succeeds and changes nothing.

afy collections delete <name> deletes a collection and every point in it. It asks you to type the collection’s name first.

When stdin is not a terminal, nobody can answer the question, so without --yes the Aetherfy CLI deletes nothing and exits 2. A script passes --yes:

afy collections delete articles --yes --json

Aetherfy payload indexes: afy index create waits until the index is built

afy index create <collection> <field> creates a payload index on one field and exits 0 only once the index is built, so a filter or an ordered scroll on the field works as soon as it returns.

The default wait is ten minutes; these two commands wait the same way:

afy index create articles thread_id --type keyword
afy index create articles thread_id --type keyword --timeout 600

afy index create articles ts --type integer --timeout 300 --json
afy index create articles body --type '{"type":"text","tokenizer":"word","lowercase":true}'

This is the Aetherfy SDKs’ create_field_index behaviour, step for step:

A --timeout that is not a number of seconds above 0 is refused before any request, naming the value, with exit code 2.

afy index delete <collection> <field> drops the index on one field in a single request, allowed as long as a create, because Aetherfy may hold a delete the same way. Dropping an index the field does not have succeeds; a collection that does not exist is an error.

afy index delete articles thread_id

Reading Aetherfy points: count, get, and search

The Aetherfy CLI’s points commands only read. --filter takes a filter as a JSON object, in the shape the SDKs send (see /vectors/filtering).

afy points count <collection> counts the points exactly. Without --json it prints the bare number, so $(afy points count articles) is the count.

afy points count articles
afy points count articles --filter '{"must":[{"key":"lang","match":{"value":"en"}}]}' --json

afy points get <collection> <id>... reads points by id, with their payloads and without their vectors. An id made only of digits is sent as a number, anything else as a string, such as a UUID. An id that does not exist is left out of the answer rather than reported.

afy points get articles 17 42 --json

afy points search <collection> returns the points nearest to a vector.

afy points search articles --vector '[0.12, -0.03, 0.88]' --limit 5
afy points search articles --vector @query.json --json

In --json output each entry of points carries id and payload, and a search hit carries score as well.

Aetherfy vector command errors and exit codes

Nothing goes to stdout on a failure. On stderr, the Aetherfy vectors API’s own status, code and message are copied through unchanged. With --json stderr holds one object, for every failure except a command line that does not parse, which is a plain Error: line because parsing stopped before --json was necessarily read:

{"error": {"status": 404, "code": "NOT_FOUND", "message": "Collection 'articles' not found"}}

Without --json it is one line, Error: Collection 'articles' not found (NOT_FOUND). A failure that got no answer from the API has no status or code. The codes themselves are listed on /vectors/errors.