Collection Management

Collections are the primary data structure in Milvus, similar to tables in traditional databases. This guide covers all collection management operations.

Creating Collections

Basic Collection Creation

import { MilvusClient, DataType } from '@zilliz/milvus2-sdk-node';

const schema = [
  {
    name: 'id',
    data_type: DataType.Int64,
    is_primary_key: true,
    autoID: true,
  },
  {
    name: 'vector',
    data_type: DataType.FloatVector,
    dim: 128,
  },
];

await client.createCollection({
  collection_name: 'my_collection',
  fields: schema,
});

Collection with Consistency Level

await client.createCollection({
  collection_name: 'my_collection',
  fields: schema,
  consistency_level: 'Bounded', // 'Strong', 'Session', 'Bounded', 'Eventually'
});

Collection with Properties

await client.createCollection({
  collection_name: 'my_collection',
  fields: schema,
  properties: {
    'collection.ttl.seconds': 3600, // Collection-level TTL in seconds
  },
});

Entity-Level TTL

Entity-level TTL lets each row carry its own expiration time. Create a nullable Timestamptz field and set the collection property ttl_field to that field name.

import { CollectionProperties, DataType } from '@zilliz/milvus2-sdk-node';

await client.createCollection({
  collection_name: 'entity_ttl_collection',
  fields: [
    { name: 'id', data_type: DataType.Int64, is_primary_key: true },
    { name: 'vector', data_type: DataType.FloatVector, dim: 128 },
    { name: 'expire_at', data_type: DataType.Timestamptz, nullable: true },
  ],
  properties: {
    [CollectionProperties.TTL_FIELD]: 'expire_at',
  },
});

await client.insert({
  collection_name: 'entity_ttl_collection',
  data: [
    {
      id: 1,
      vector: [
        /* ... */
      ],
      expire_at: null,
    }, // never expires
    {
      id: 2,
      vector: [
        /* ... */
      ],
      expire_at: '2099-12-31T00:00:00Z',
    },
    {
      id: 3,
      vector: [
        /* ... */
      ],
      expire_at: '2000-01-01T00:00:00Z',
    }, // expired
  ],
});

Expired entities are excluded from query and search results. You can add a TTL field to an existing collection with addCollectionField() and then set CollectionProperties.TTL_FIELD with alterCollectionProperties().

Collection with Partition Configuration

await client.createCollection({
  collection_name: 'my_collection',
  fields: schema,
  num_partitions: 8, // Number of partitions
});

Allow Insert with AutoID

For collections whose primary key field uses autoID: true, inserts normally omit the primary key. If you need to import or backfill rows with explicit primary keys, enable the allow_insert_auto_id collection property.

await client.alterCollectionProperties({
  collection_name: 'my_collection',
  properties: {
    allow_insert_auto_id: 'true',
  },
});

await client.insert({
  collection_name: 'my_collection',
  data: [
    {
      id: 10001, // accepted even when the primary key field has autoID: true
      vector: [
        /* ... */
      ],
    },
  ],
});

Use this only when you need to preserve existing IDs. For normal writes to an AutoID collection, keep omitting the primary key field.

Checking Collection Existence

Check if a collection exists:

const exists = await client.hasCollection({
  collection_name: 'my_collection',
});

console.log('Collection exists:', exists.value);

Listing Collections

List all collections:

const result = await client.showCollections();

console.log('Collections:', result.collection_names);
// Output: ['collection1', 'collection2', 'my_collection']

List collections with details:

const result = await client.showCollections({
  type: ShowCollectionsType.All, // 'All' or 'InMemory'
});

result.data.forEach((collection) => {
  console.log(`Collection: ${collection.name}`);
  console.log(`ID: ${collection.id}`);
  console.log(`Loaded: ${collection.loadedPercentage}%`);
});

Describing Collections

Get detailed information about a collection:

const result = await client.describeCollection({
  collection_name: 'my_collection',
});

console.log('Schema:', result.schema);
console.log('Collection ID:', result.collectionID);
console.log('Consistency Level:', result.consistency_level);

Modifying Collections

Alter Collection Properties

Modify collection properties:

await client.alterCollectionProperties({
  collection_name: 'my_collection',
  properties: {
    'collection.ttl.seconds': 7200,
  },
});

Rename Collection

Rename a collection:

await client.renameCollection({
  collection_name: 'old_name',
  new_collection_name: 'new_name',
});

Drop Collection Properties

Remove specific properties:

await client.dropCollectionProperties({
  collection_name: 'my_collection',
  property_names: ['collection.ttl.seconds'],
});

Dynamic Schema

Adding Fields to Existing Collection

Add a single field:

await client.addCollectionField({
  collection_name: 'my_collection',
  field: {
    name: 'new_field',
    data_type: DataType.Int64,
    description: 'A new field',
  },
});

Add multiple fields:

await client.addCollectionFields({
  collection_name: 'my_collection',
  fields: [
    {
      name: 'field1',
      data_type: DataType.Int64,
    },
    {
      name: 'field2',
      data_type: DataType.VarChar,
      max_length: 256,
    },
  ],
});

Alter Field Properties

Modify field properties:

await client.alterCollectionFieldProperties({
  collection_name: 'my_collection',
  field_name: 'my_field',
  properties: {
    'field.max_length': 512,
  },
});

Collection Statistics

Get collection statistics:

const stats = await client.getCollectionStatistics({
  collection_name: 'my_collection',
});

console.log('Row count:', stats.row_count);
console.log('Statistics:', stats.stats);

Collection Loading

Load Collection

Load a collection into memory for search:

// Synchronous load (waits until loaded)
await client.loadCollectionSync({
  collection_name: 'my_collection',
});

// Asynchronous load (returns immediately)
await client.loadCollectionAsync({
  collection_name: 'my_collection',
});

Check Loading Progress

const progress = await client.getLoadingProgress({
  collection_name: 'my_collection',
});

console.log('Loading progress:', progress.progress);

Check Load State

const state = await client.getLoadState({
  collection_name: 'my_collection',
});

console.log('Load state:', state.state); // 'LoadStateNotExist', 'LoadStateNotLoad', 'LoadStateLoading', 'LoadStateLoaded'

Refresh Load

Refresh the loaded collection:

await client.refreshLoad({
  collection_name: 'my_collection',
});

Release Collection

Release a collection from memory:

await client.releaseCollection({
  collection_name: 'my_collection',
});

Collection Aliases

Create Alias

Create an alias for a collection:

await client.createAlias({
  collection_name: 'my_collection',
  alias: 'my_alias',
});

Describe Alias

Get information about an alias:

const result = await client.describeAlias({
  alias: 'my_alias',
});

console.log('Collection name:', result.collection);
console.log('Alias:', result.alias);

List Aliases

List all aliases for a collection:

const result = await client.listAliases({
  collection_name: 'my_collection',
});

console.log('Aliases:', result.aliases);

Alter Alias

Change which collection an alias points to:

await client.alterAlias({
  collection_name: 'new_collection',
  alias: 'my_alias',
});

Drop Alias

Remove an alias:

await client.dropAlias({
  alias: 'my_alias',
});

Collection Compaction

Compact Collection

Trigger compaction:

const result = await client.compact({
  collection_name: 'my_collection',
});

console.log('Compaction ID:', result.compactionID);

Get Compaction State

Check compaction status:

const state = await client.getCompactionState({
  compactionID: 12345,
});

console.log('State:', state.state);

Get Compaction Plans

Get detailed compaction plans:

const plans = await client.getCompactionStateWithPlans({
  compactionID: 12345,
});

console.log('Plans:', plans.mergeInfos);

Collection Replicas

Get replica information:

const replicas = await client.getReplicas({
  collection_name: 'my_collection',
});

console.log('Replicas:', replicas.replicas);

Include shard node details or query by collection ID when needed:

const replicas = await client.getReplicas({
  collection_name: 'my_collection',
  db_name: 'default',
  with_shard_nodes: true,
});

replicas.replicas.forEach((replica) => {
  console.log('Resource group:', replica.resource_group_name);
  console.log('Outbound nodes:', replica.num_outbound_node);
  console.log('Shard replicas:', replica.shard_replicas);
});

getReplicas() accepts either collection_name/db_name or a collectionID, depending on what identifiers are available in your application.

Primary Key Information

Get Primary Key Field Name

const pkField = await client.getPkFieldName({
  collection_name: 'my_collection',
});

console.log('Primary key field:', pkField);

Get Primary Key Field Type

const pkType = await client.getPkFieldType({
  collection_name: 'my_collection',
});

console.log('Primary key type:', pkType);

Get Primary Key Field Schema

const pkFieldSchema = await client.getPkField({
  collection_name: 'my_collection',
});

console.log('Primary key schema:', pkFieldSchema);

Dropping Collections

Drop a collection (this permanently deletes all data):

await client.dropCollection({
  collection_name: 'my_collection',
});

Best Practices

1. Naming: Use descriptive collection names
2. Schema Design: Plan your schema carefully before creating collections
3. Loading: Load collections before performing searches
4. Aliases: Use aliases for easier collection management and zero-downtime updates
5. Compaction: Regularly compact collections to optimize performance
6. Release: Release collections from memory when not in use to free resources

Truncate Collection

Remove all data from a collection while keeping its schema and indexes:

await client.truncateCollection({
  collection_name: 'my_collection',
});

This is faster than dropping and recreating the collection when you only need to clear data.

Batch Describe Collections

Describe multiple collections in a single call:

const result = await client.batchDescribeCollections({
  collection_names: ['collection_a', 'collection_b', 'collection_c'],
});

result.responses.forEach((col) => {
  console.log(`Collection: ${col.collectionName}`);
  console.log(`Fields: ${col.schema.fields.map((f) => f.name)}`);
});

Alter Collection Properties

Modify collection-level properties:

await client.alterCollectionProperties({
  collection_name: 'my_collection',
  properties: {
    'collection.ttl.seconds': 7200,
  },
});

Alter Field Properties

Modify field-level properties:

await client.alterCollectionFieldProperties({
  collection_name: 'my_collection',
  field_name: 'text',
  properties: {
    max_length: 1024,
  },
});

Drop Collection Properties

Remove collection properties:

await client.dropCollectionProperties({
  collection_name: 'my_collection',
  properties: ['collection.ttl.seconds'],
});

External Collections Milvus 3.0

External collections reference data stored outside Milvus. Use external_source and external_spec at collection creation time, and map Milvus fields to source fields with external_field.

await client.createCollection({
  collection_name: 'external_products',
  external_source: 's3://bucket/path/products/',
  external_spec: JSON.stringify({ format: 'parquet' }),
  do_physical_backfill: false,
  file_resource_ids: ['resource-id-1'],
  fields: [
    {
      name: 'id',
      data_type: DataType.Int64,
      is_primary_key: true,
      external_field: 'product_id',
    },
    {
      name: 'name',
      data_type: DataType.VarChar,
      max_length: 256,
      external_field: 'title',
    },
    {
      name: 'vector',
      data_type: DataType.FloatVector,
      dim: 128,
      external_field: 'embedding',
    },
  ],
});

Refresh an external collection when the source changes:

const refresh = await client.refreshExternalCollection({
  collection_name: 'external_products',
  external_source: 's3://bucket/path/products-v2/', // optional
  external_spec: JSON.stringify({ format: 'parquet' }), // optional
});

const progress = await client.getRefreshExternalCollectionProgress({
  job_id: refresh.job_id,
});

const jobs = await client.listRefreshExternalCollectionJobs({
  collection_name: 'external_products',
});

Collection Snapshots Milvus 3.0

Snapshots capture a collection at a point in time. You can describe, restore, pin, and remove snapshots with the snapshot APIs.

await client.createSnapshot({
  collection_name: 'my_collection',
  snapshot_name: 'my_collection_snapshot_001',
  description: 'Before schema migration',
  compaction_protection_seconds: 3600,
});

const snapshots = await client.listSnapshots({
  collection_name: 'my_collection',
});

const snapshot = await client.describeSnapshot({
  collection_name: 'my_collection',
  snapshot_name: 'my_collection_snapshot_001',
});

Restore a snapshot into another collection and check the restore job:

const restore = await client.restoreSnapshot({
  snapshot_name: 'my_collection_snapshot_001',
  source_collection_name: 'my_collection',
  target_collection_name: 'my_collection_restored',
});

const state = await client.getRestoreSnapshotState({
  job_id: restore.job_id,
});

const restoreJobs = await client.listRestoreSnapshotJobs({
  collection_name: 'my_collection_restored',
});

Pin snapshot data while copying it out, then release the pin when finished:

const pin = await client.pinSnapshotData({
  collection_name: 'my_collection',
  snapshot_name: 'my_collection_snapshot_001',
  ttl_seconds: 3600,
});

await client.unpinSnapshotData({ pin_id: pin.pin_id });
await client.dropSnapshot({
  collection_name: 'my_collection',
  snapshot_name: 'my_collection_snapshot_001',
});

Collection Functions

Collection functions are auto-processing pipelines that transform data at insert and query time. The most common use case is BM25 full-text search, where a function automatically converts text to sparse vectors.

Add a Function

Milvus 3.0

Milvus 3.0’s collection schema alteration support includes adding and dropping functions on existing collections. In the Node SDK, use the collection function APIs below.

import { FunctionType } from '@zilliz/milvus2-sdk-node';

await client.addCollectionFunction({
  collection_name: 'my_collection',
  function: {
    name: 'text_bm25',
    type: FunctionType.BM25,
    input_field_names: ['body'],
    output_field_names: ['sparse_vector'],
    params: {},
  },
});

Drop a Function

await client.dropCollectionFunction({
  collection_name: 'my_collection',
  function_name: 'text_bm25',
});

For more details on BM25 functions, see Full-Text Search.

Refresh Load

After modifying a loaded collection’s schema (e.g., adding fields or functions), refresh the load to pick up changes:

await client.refreshLoad({
  collection_name: 'my_collection',
});

Next Steps

• Learn about Partition Management
• Explore Index Management
• Check out Data Operations