Skip to content
MilvusNode SDK
Sections

Collection Management

Markdown

Copy for agent

Collections are the primary data structure in Milvus, similar to tables in traditional databases. This guide covers all collection management operations.

import { MilvusClient, DataType } from '@zilliz/milvus2-sdk-node';
const schema = [
{
name: 'id',
data_type: DataType.Int64,
is_primary_key: true,
autoID: true,
},
{
name: 'vector',
data_type: DataType.FloatVector,
dim: 128,
},
];
await client.createCollection({
collection_name: 'my_collection',
fields: schema,
});
await client.createCollection({
collection_name: 'my_collection',
fields: schema,
consistency_level: 'Bounded', // 'Strong', 'Session', 'Bounded', 'Eventually'
});
await client.createCollection({
collection_name: 'my_collection',
fields: schema,
properties: {
'collection.ttl.seconds': 3600, // Collection-level TTL in seconds
},
});

Entity-level TTL lets each row carry its own expiration time. Create a nullable Timestamptz field and set the collection property ttl_field to that field name.

import { CollectionProperties, DataType } from '@zilliz/milvus2-sdk-node';
await client.createCollection({
collection_name: 'entity_ttl_collection',
fields: [
{ name: 'id', data_type: DataType.Int64, is_primary_key: true },
{ name: 'vector', data_type: DataType.FloatVector, dim: 128 },
{ name: 'expire_at', data_type: DataType.Timestamptz, nullable: true },
],
properties: {
[CollectionProperties.TTL_FIELD]: 'expire_at',
},
});
await client.insert({
collection_name: 'entity_ttl_collection',
data: [
{
id: 1,
vector: [
/* ... */
],
expire_at: null,
}, // never expires
{
id: 2,
vector: [
/* ... */
],
expire_at: '2099-12-31T00:00:00Z',
},
{
id: 3,
vector: [
/* ... */
],
expire_at: '2000-01-01T00:00:00Z',
}, // expired
],
});

Expired entities are excluded from query and search results. You can add a TTL field to an existing collection with addCollectionField() and then set CollectionProperties.TTL_FIELD with alterCollectionProperties().

await client.createCollection({
collection_name: 'my_collection',
fields: schema,
num_partitions: 8, // Number of partitions
});

For collections whose primary key field uses autoID: true, inserts normally omit the primary key. If you need to import or backfill rows with explicit primary keys, enable the allow_insert_auto_id collection property.

await client.alterCollectionProperties({
collection_name: 'my_collection',
properties: {
allow_insert_auto_id: 'true',
},
});
await client.insert({
collection_name: 'my_collection',
data: [
{
id: 10001, // accepted even when the primary key field has autoID: true
vector: [
/* ... */
],
},
],
});

Use this only when you need to preserve existing IDs. For normal writes to an AutoID collection, keep omitting the primary key field.

Check if a collection exists:

const exists = await client.hasCollection({
collection_name: 'my_collection',
});
console.log('Collection exists:', exists.value);

List all collections:

const result = await client.showCollections();
console.log(
'Collections:',
result.data.map((collection) => collection.name)
);
// Output: ['collection1', 'collection2', 'my_collection']

List collections with details:

const result = await client.showCollections({
type: ShowCollectionsType.All, // 'All' or 'InMemory'
});
result.data.forEach((collection) => {
console.log(`Collection: ${collection.name}`);
console.log(`ID: ${collection.id}`);
console.log(`Loaded: ${collection.loadedPercentage}%`);
});

Get detailed information about a collection:

const result = await client.describeCollection({
collection_name: 'my_collection',
});
console.log('Schema:', result.schema);
console.log('Collection ID:', result.collectionID);
console.log('Consistency Level:', result.consistency_level);

Modify collection properties:

await client.alterCollectionProperties({
collection_name: 'my_collection',
properties: {
'collection.ttl.seconds': 7200,
},
});

Rename a collection:

await client.renameCollection({
collection_name: 'old_name',
new_collection_name: 'new_name',
});

Remove specific properties:

await client.dropCollectionProperties({
collection_name: 'my_collection',
properties: ['collection.ttl.seconds'],
});
Milvus 3.0

Add a single field:

await client.addCollectionField({
collection_name: 'my_collection',
field: {
name: 'new_field',
data_type: DataType.Int64,
description: 'A new field',
},
});

Add multiple fields:

await client.addCollectionFields({
collection_name: 'my_collection',
fields: [
{
name: 'field1',
data_type: DataType.Int64,
},
{
name: 'field2',
data_type: DataType.VarChar,
max_length: 256,
},
],
});

Drop a field by name:

await client.dropCollectionField({
collection_name: 'my_collection',
field_name: 'field1',
});

You can also pass field_id instead of field_name. Get the field ID from describeCollection({ cache: false }) and provide exactly one identifier.

Modify field properties:

await client.alterCollectionFieldProperties({
collection_name: 'my_collection',
field_name: 'my_field',
properties: {
max_length: 512,
},
});

Get collection statistics:

const stats = await client.getCollectionStatistics({
collection_name: 'my_collection',
});
console.log('Row count:', stats.data.row_count);
console.log('Statistics:', stats.stats);

Load a collection into memory for search:

// Synchronous load (waits until loaded)
await client.loadCollectionSync({
collection_name: 'my_collection',
});
// Asynchronous load (returns immediately)
await client.loadCollectionAsync({
collection_name: 'my_collection',
});
const progress = await client.getLoadingProgress({
collection_name: 'my_collection',
});
console.log('Loading progress:', progress.progress);
const state = await client.getLoadState({
collection_name: 'my_collection',
});
console.log('Load state:', state.state); // 'LoadStateNotExist', 'LoadStateNotLoad', 'LoadStateLoading', 'LoadStateLoaded'

Refresh the loaded collection:

await client.refreshLoad({
collection_name: 'my_collection',
});

Release a collection from memory:

await client.releaseCollection({
collection_name: 'my_collection',
});

Create an alias for a collection:

await client.createAlias({
collection_name: 'my_collection',
alias: 'my_alias',
});

Get information about an alias:

const result = await client.describeAlias({
collection_name: 'my_collection',
alias: 'my_alias',
});
console.log('Collection name:', result.collection);
console.log('Alias:', result.alias);

List all aliases for a collection:

const result = await client.listAliases({
collection_name: 'my_collection',
});
console.log('Aliases:', result.aliases);

Change which collection an alias points to:

await client.alterAlias({
collection_name: 'new_collection',
alias: 'my_alias',
});

Remove an alias:

await client.dropAlias({
alias: 'my_alias',
});

Trigger compaction:

const result = await client.compact({
collection_name: 'my_collection',
});
console.log('Compaction ID:', result.compactionID);

Check compaction status:

const state = await client.getCompactionState({
compactionID: 12345,
});
console.log('State:', state.state);

Get detailed compaction plans:

const plans = await client.getCompactionStateWithPlans({
compactionID: 12345,
});
console.log('Plans:', plans.mergeInfos);

Get replica information:

const { collectionID } = await client.describeCollection({
collection_name: 'my_collection',
});
const replicas = await client.getReplicas({ collectionID });
console.log('Replicas:', replicas.replicas);

Pass the collection ID to include shard node details:

const replicas = await client.getReplicas({
collectionID,
with_shard_nodes: true,
});
replicas.replicas.forEach((replica) => {
console.log('Resource group:', replica.resource_group_name);
console.log('Outbound nodes:', replica.num_outbound_node);
console.log('Shard replicas:', replica.shard_replicas);
});

getReplicas() accepts either collection_name/db_name or a collectionID, depending on what identifiers are available in your application.

const pkField = await client.getPkFieldName({
collection_name: 'my_collection',
});
console.log('Primary key field:', pkField);
const pkType = await client.getPkFieldType({
collection_name: 'my_collection',
});
console.log('Primary key type:', pkType);
const pkFieldSchema = await client.getPkField({
collection_name: 'my_collection',
});
console.log('Primary key schema:', pkFieldSchema);

Drop a collection (this permanently deletes all data):

await client.dropCollection({
collection_name: 'my_collection',
});
  1. Naming: Use descriptive collection names
  2. Schema Design: Plan your schema carefully before creating collections
  3. Loading: Load collections before performing searches
  4. Aliases: Use aliases for easier collection management and zero-downtime updates
  5. Compaction: Regularly compact collections to optimize performance
  6. Release: Release collections from memory when not in use to free resources

Remove all data from a collection while keeping its schema and indexes:

await client.truncateCollection({
collection_name: 'my_collection',
});

This is faster than dropping and recreating the collection when you only need to clear data.

Describe multiple collections in a single call:

const result = await client.batchDescribeCollections({
collection_names: ['collection_a', 'collection_b', 'collection_c'],
});
result.responses.forEach((col) => {
console.log(`Collection: ${col.collection_name}`);
console.log(`Fields: ${col.schema.fields.map((f) => f.name)}`);
});

Modify collection-level properties:

await client.alterCollectionProperties({
collection_name: 'my_collection',
properties: {
'collection.ttl.seconds': 7200,
},
});

Modify field-level properties:

await client.alterCollectionFieldProperties({
collection_name: 'my_collection',
field_name: 'text',
properties: {
max_length: 1024,
},
});

Remove collection properties:

await client.dropCollectionProperties({
collection_name: 'my_collection',
properties: ['collection.ttl.seconds'],
});

External collections reference data stored outside Milvus. Use external_source and external_spec at collection creation time, and map Milvus fields to source fields with external_field. Milvus supplies the primary key for the external collection.

await client.createCollection({
collection_name: 'external_products',
external_source: 's3://bucket/path/products/',
external_spec: JSON.stringify({ format: 'parquet' }),
fields: [
{
name: 'id',
data_type: DataType.Int64,
external_field: 'product_id',
},
{
name: 'name',
data_type: DataType.VarChar,
max_length: 256,
external_field: 'title',
},
{
name: 'vector',
data_type: DataType.FloatVector,
dim: 128,
external_field: 'embedding',
},
],
});

Refresh an external collection when the source changes:

const refresh = await client.refreshExternalCollection({
collection_name: 'external_products',
external_source: 's3://bucket/path/products-v2/', // optional
external_spec: JSON.stringify({ format: 'parquet' }), // optional
});
const progress = await client.getRefreshExternalCollectionProgress({
job_id: refresh.job_id,
});
const jobs = await client.listRefreshExternalCollectionJobs({
collection_name: 'external_products',
});

Snapshots capture a collection at a point in time. You can describe, restore, pin, and remove snapshots with the snapshot APIs.

await client.createSnapshot({
collection_name: 'my_collection',
snapshot_name: 'my_collection_snapshot_001',
description: 'Before schema migration',
compaction_protection_seconds: 3600,
});
const snapshots = await client.listSnapshots({
collection_name: 'my_collection',
});
const snapshot = await client.describeSnapshot({
collection_name: 'my_collection',
snapshot_name: 'my_collection_snapshot_001',
});

Restore a snapshot into another collection and check the restore job:

const restore = await client.restoreSnapshot({
snapshot_name: 'my_collection_snapshot_001',
source_collection_name: 'my_collection',
target_collection_name: 'my_collection_restored',
});
const state = await client.getRestoreSnapshotState({
job_id: restore.job_id,
});
const restoreJobs = await client.listRestoreSnapshotJobs({
collection_name: 'my_collection_restored',
});

Pin snapshot data while copying it out, then release the pin when finished:

const pin = await client.pinSnapshotData({
collection_name: 'my_collection',
snapshot_name: 'my_collection_snapshot_001',
ttl_seconds: 3600,
});
await client.unpinSnapshotData({ pin_id: pin.pin_id });
await client.dropSnapshot({
collection_name: 'my_collection',
snapshot_name: 'my_collection_snapshot_001',
});

Export and Restore a Portable Snapshot Milvus 3.0.6

Section titled “Export and Restore a Portable Snapshot Milvus 3.0.6”

Export a snapshot to an object-storage path to create a self-contained bundle. Use the completed export’s metadata URI to restore into a new collection.

import { ExportSnapshotState } from '@zilliz/milvus2-sdk-node';
const job = await client.exportSnapshot({
collection_name: 'my_collection',
snapshot_name: 'my_collection_snapshot_001',
target_s3_path: 'exports/my_collection_snapshot_001',
});
const { info } = await client.getExportSnapshotState({ job_id: job.job_id });
if (info.state === ExportSnapshotState.ExportSnapshotCompleted) {
const restore = await client.restoreExternalSnapshot({
target_collection_name: 'my_collection_from_export',
snapshot_metadata_uri: info.snapshot_metadata_uri,
});
const restoreState = await client.getRestoreSnapshotState({
job_id: restore.job_id,
});
}

A relative destination uses the cluster’s configured storage. Export and restore accept an optional external_spec JSON string for configuring external object storage. Poll the job methods until the state is completed or failed. listSnapshots() also exposes snapshot descriptions and Unix creation times in snapshot_summaries when the server provides them.

Collection functions are auto-processing pipelines that transform data at insert and query time. The most common use case is BM25 full-text search, where a function automatically converts text to sparse vectors.

Milvus 3.0

Use addFunctionField() to atomically add a function output field, the function schema, and its bound index. The wrapper supports BM25 and MinHash functions.

import {
DataType,
FunctionType,
IndexType,
MetricType,
} from '@zilliz/milvus2-sdk-node';
await client.addFunctionField({
collection_name: 'my_collection',
field: {
name: 'sparse_vector',
data_type: DataType.SparseFloatVector,
is_function_output: true,
},
function: {
name: 'text_bm25',
type: FunctionType.BM25,
input_field_names: ['body'],
output_field_names: ['sparse_vector'],
params: {},
},
index_name: 'text_bm25_index',
extra_params: {
index_type: IndexType.SPARSE_INVERTED_INDEX,
metric_type: MetricType.BM25,
},
});

MinHash functions generate binary signatures for set or shingle similarity:

await client.addFunctionField({
collection_name: 'my_collection',
field: {
name: 'minhash_vector',
data_type: DataType.BinaryVector,
dim: 512,
is_function_output: true,
},
function: {
name: 'body_minhash',
type: FunctionType.MINHASH,
input_field_names: ['body'],
output_field_names: ['minhash_vector'],
params: { num_hashes: 16, shingle_size: 3 },
},
index_name: 'minhash_index',
extra_params: {
index_type: IndexType.MINHASH_LSH,
metric_type: MetricType.MHJACCARD,
params: { mh_lsh_band: 8 },
},
});

The lower-level alterCollectionSchema() API exposes the same schema mutation when you need to build the complete request yourself.

await client.dropFunctionField({
collection_name: 'my_collection',
function_name: 'text_bm25',
});

dropFunctionField() also removes the function output field and bound index.

For more details on BM25 functions, see Full-Text Search.

After modifying a loaded collection’s schema (e.g., adding fields or functions), refresh the load to pick up changes:

await client.refreshLoad({
collection_name: 'my_collection',
});

In SDK 3.0.4+, addFunctionField() returns after the schema change is accepted. Milvus asynchronously backfills existing sealed segments through schema-bump compaction. The wrapper supports BM25 → SparseFloatVector and MINHASH → BinaryVector; provide an explicit bound index_type other than AUTOINDEX.

const stats = await client.getCollectionStatistics({
collection_name: 'documents',
});
if (stats.status.error_code !== ErrorCode.SUCCESS) {
throw new Error(stats.status.reason);
}
console.log({
consistent: stats.data.schema_version_consistent_segments,
total: stats.data.schema_version_total_segments,
});

For an existing collection with sealed segments, poll these values with a timeout until total > 0 and consistent === total after numeric conversion. Missing counters do not prove completion. Also check getIndexState({ collection_name, index_name }) reaches Finished (stop on Failed) before loading and searching the new field. Empty collections need no historical backfill.