Insert and Upsert Entities
This guide covers inserting and updating data in Milvus collections.
Inserting Data
Section titled “Inserting Data”Basic Insert
Section titled “Basic Insert”Insert data into a collection:
await client.insert({ collection_name: 'my_collection', data: [ { id: 1, vector: [0.1, 0.2, 0.3, 0.4], text: 'Hello Milvus', }, { id: 2, vector: [0.5, 0.6, 0.7, 0.8], text: 'Vector database', }, ],});Insert with AutoID
Section titled “Insert with AutoID”If your collection uses autoID, omit the ID field:
await client.insert({ collection_name: 'my_collection', data: [ { vector: [0.1, 0.2, 0.3, 0.4], text: 'Hello Milvus', }, { vector: [0.5, 0.6, 0.7, 0.8], text: 'Vector database', }, ],});Insert into Specific Partition
Section titled “Insert into Specific Partition”Insert data into a specific partition:
await client.insert({ collection_name: 'my_collection', partition_name: 'partition_1', data: [ { vector: [0.1, 0.2, 0.3, 0.4], text: 'Data for partition 1', }, ],});Batch Insertion
Section titled “Batch Insertion”Insert large amounts of data efficiently:
const batchSize = 1000;const allData = [ /* large array of data */];
for (let i = 0; i < allData.length; i += batchSize) { const batch = allData.slice(i, i + batchSize); await client.insert({ collection_name: 'my_collection', data: batch, });}Data Format
Section titled “Data Format”Vector Data Formats
Section titled “Vector Data Formats”FloatVector
Section titled “FloatVector”{ vector: [0.1, 0.2, 0.3, 0.4, /* ... */], // Array of numbers}BinaryVector
Section titled “BinaryVector”{ binary_vector: [0b10101010, 0b01010101], // Two bytes for dim = 16}SparseFloatVector
Section titled “SparseFloatVector”Multiple formats supported:
// Array formatconst arrayRow = { sparse_vector: [1.0, undefined, undefined, 2.5, undefined],};
// Dictionary formatconst dictionaryRow = { sparse_vector: { 0: 1.0, 3: 2.5 },};
// CSR formatconst csrRow = { sparse_vector: { indices: [0, 3], values: [1.0, 2.5], },};
// COO formatconst cooRow = { sparse_vector: [ { index: 0, value: 1.0 }, { index: 3, value: 2.5 }, ],};Float16Vector / BFloat16Vector
Section titled “Float16Vector / BFloat16Vector”{ vector: [0.1, 0.2, 0.3, 0.4], // Array of numbers (will be converted)}Scalar Data Formats
Section titled “Scalar Data Formats”{ id: 123, // Int64 age: 25, // Int64 score: 98.5, // Float is_active: true, // Bool name: 'John Doe', // VarChar metadata: { // JSON category: 'tech', tags: ['ai', 'ml'], },}Upserting Data
Section titled “Upserting Data”Upsert updates existing entities or inserts new ones:
await client.upsert({ collection_name: 'my_collection', data: [ { id: 1, vector: [0.9, 0.8, 0.7, 0.6], // Update existing entity text: 'Updated text', }, { id: 999, vector: [0.1, 0.2, 0.3, 0.4], // Insert new entity text: 'New text', }, ],});Partial Update
Section titled “Partial Update”Update only specific fields:
await client.upsert({ collection_name: 'my_collection', partial_update: true, data: [ { id: 1, text: 'Updated text only', // Only update text field // vector field not included, will remain unchanged }, ],});Partial Array Upsert Milvus 3.0
Section titled “Partial Array Upsert Milvus 3.0”For Array fields, use field_ops to append values to or remove values from the existing array instead of replacing the whole field.
import { FieldPartialUpdateOpType } from '@zilliz/milvus2-sdk-node';
await client.upsert({ collection_name: 'my_collection', data: [ { id: 1, tags: ['new-tag'], // Appended to the existing tags array scores: [3], // Removed from the existing scores array }, ], field_ops: [ { field_name: 'tags', op: FieldPartialUpdateOpType.ARRAY_APPEND }, { field_name: 'scores', op: FieldPartialUpdateOpType.ARRAY_REMOVE }, ],});You can also pass operation names as strings:
await client.upsert({ collection_name: 'my_collection', data: [{ id: 1, tags: ['urgent'] }], field_ops: [{ field_name: 'tags', op: 'ARRAY_APPEND' }],});Notes:
field_opsis supported only byupsert().ARRAY_APPENDandARRAY_REMOVEapply to Array fields.- The primary key must be included in each upsert row.
- Fields not included in the upsert row remain unchanged.
ARRAY_APPENDmust not make the array exceed itsmax_capacity.ARRAY_REMOVEremoves matching values from the existing array.- When
field_opsis provided, the SDK automatically sendspartial_update: true.
Replace a Value by Path Milvus 3.0.6
Section titled “Replace a Value by Path Milvus 3.0.6”Use PATH_REPLACE to replace one value in an existing row. Each Array or StructArray operand contains one element. A JSON operand is the new value itself.
await client.upsert({ collection_name: 'my_collection', data: [ { id: 1, scores: [100], items: [{ label: 'updated' }], settings: false }, ], field_ops: [ { field_name: 'scores', op: 'PATH_REPLACE', path: '[0]' }, { field_name: 'items', op: 'PATH_REPLACE', path: '[0][label]' }, { field_name: 'settings', op: 'PATH_REPLACE', path: '["enabled"]' }, ],});Use zero-based indexes like [0]. For a whole StructArray element, supply all its children. For a single child, use [0][child] and supply only that child. Quote JSON object keys, as in ["settings"][0]["enabled"]. The primary key and the selected value must already exist.
With HttpClient, use collectionName and pass operations as fieldOps: [{ fieldName: 'scores', op: 'PATH_REPLACE', path: '[0]' }].
Insert Transformers
Section titled “Insert Transformers”For Float16 and BFloat16 vectors, use transformers:
import { DataType, f32ArrayToF16Bytes } from '@zilliz/milvus2-sdk-node';
await client.insert({ collection_name: 'my_collection', data: [ { vector: [0.1, 0.2, 0.3, 0.4], // f32 array }, ], transformers: { [DataType.Float16Vector]: f32ArrayToF16Bytes, },});Insert Response
Section titled “Insert Response”The insert operation returns mutation results:
const result = await client.insert({ collection_name: 'my_collection', data: [ /* ... */ ],});
console.log('IDs:', result.IDs); // Array of inserted IDsconsole.log('Insert count:', result.insert_cnt);console.log('Delete count:', result.delete_cnt);console.log('Upsert count:', result.upsert_cnt);console.log('Timestamp:', result.timestamp);Data Validation
Section titled “Data Validation”The SDK validates data before insertion:
- Field existence: All fields must exist in the collection schema
- Data types: Values must match field data types
- Vector dimensions: Vector length must match field dimension
- Binary vectors: Length must be dimension / 8
- VarChar length: String length must not exceed max_length
Error Handling
Section titled “Error Handling”Handle insertion errors:
try { await client.insert({ collection_name: 'my_collection', data: [ /* ... */ ], });} catch (error) { if (error.message.includes('field does not exist')) { console.error('Invalid field name'); } else if (error.message.includes('dimension')) { console.error('Vector dimension mismatch'); } else { console.error('Insert failed:', error); }}Best Practices
Section titled “Best Practices”- Batch inserts: Insert data in batches (1000-10000 entities per batch)
- Flush after insertion: Call
flush()after inserting data to ensure persistence - Use autoID: Let Milvus generate IDs unless you have specific requirements
- Validate data: Validate data before insertion to avoid errors
- Handle errors: Always wrap insertions in try-catch blocks
- Use upsert for updates: Use
upsert()instead of delete + insert for updates
Complete Example
Section titled “Complete Example”import { MilvusClient, DataType } from '@zilliz/milvus2-sdk-node';
const client = new MilvusClient({ address: 'localhost:19530',});
await client.connectPromise;
// Create collectionawait client.createCollection({ collection_name: 'my_collection', fields: [ { name: 'id', data_type: DataType.Int64, is_primary_key: true, autoID: true, }, { name: 'vector', data_type: DataType.FloatVector, dim: 128, }, { name: 'text', data_type: DataType.VarChar, max_length: 256, }, ],});
// Insert dataconst result = await client.insert({ collection_name: 'my_collection', data: [ { vector: Array.from({ length: 128 }, () => Math.random()), text: 'First document', }, { vector: Array.from({ length: 128 }, () => Math.random()), text: 'Second document', }, ],});
console.log('Inserted IDs:', result.IDs);
// Flush to ensure data is persistedawait client.flushSync({ collection_names: ['my_collection'],});await client.closeConnection();Next Steps
Section titled “Next Steps”- Learn about Query Operations
- Explore Delete Operations
- Check out Data Management