# Insert and Upsert Entities

Insert and upsert Milvus entities from Node.js, prepare rows that match the collection schema, and handle write results.

SDK: `@zilliz/milvus2-sdk-node@3.0.10` · Tested with Milvus 3.0.6

Canonical documentation: https://milvus-io.github.io/milvus-sdk-node/operations/data-operations-insert/

## Section links

- [Inserting Data](https://milvus-io.github.io/milvus-sdk-node/operations/data-operations-insert/#inserting-data)
- [Basic Insert](https://milvus-io.github.io/milvus-sdk-node/operations/data-operations-insert/#basic-insert)
- [Insert with AutoID](https://milvus-io.github.io/milvus-sdk-node/operations/data-operations-insert/#insert-with-autoid)
- [Insert into Specific Partition](https://milvus-io.github.io/milvus-sdk-node/operations/data-operations-insert/#insert-into-specific-partition)
- [Batch Insertion](https://milvus-io.github.io/milvus-sdk-node/operations/data-operations-insert/#batch-insertion)
- [Data Format](https://milvus-io.github.io/milvus-sdk-node/operations/data-operations-insert/#data-format)
- [Vector Data Formats](https://milvus-io.github.io/milvus-sdk-node/operations/data-operations-insert/#vector-data-formats)
- [FloatVector](https://milvus-io.github.io/milvus-sdk-node/operations/data-operations-insert/#floatvector)
- [BinaryVector](https://milvus-io.github.io/milvus-sdk-node/operations/data-operations-insert/#binaryvector)
- [SparseFloatVector](https://milvus-io.github.io/milvus-sdk-node/operations/data-operations-insert/#sparsefloatvector)
- [Float16Vector / BFloat16Vector](https://milvus-io.github.io/milvus-sdk-node/operations/data-operations-insert/#float16vector--bfloat16vector)
- [Scalar Data Formats](https://milvus-io.github.io/milvus-sdk-node/operations/data-operations-insert/#scalar-data-formats)
- [Upserting Data](https://milvus-io.github.io/milvus-sdk-node/operations/data-operations-insert/#upserting-data)
- [Partial Update](https://milvus-io.github.io/milvus-sdk-node/operations/data-operations-insert/#partial-update)
- [Partial Array Upsert Milvus 3.0](https://milvus-io.github.io/milvus-sdk-node/operations/data-operations-insert/#partial-array-upsert-milvus-30)
- [Replace a Value by Path Milvus 3.0.6](https://milvus-io.github.io/milvus-sdk-node/operations/data-operations-insert/#replace-a-value-by-path-milvus-306)
- [Insert Transformers](https://milvus-io.github.io/milvus-sdk-node/operations/data-operations-insert/#insert-transformers)
- [Insert Response](https://milvus-io.github.io/milvus-sdk-node/operations/data-operations-insert/#insert-response)
- [Data Validation](https://milvus-io.github.io/milvus-sdk-node/operations/data-operations-insert/#data-validation)
- [Error Handling](https://milvus-io.github.io/milvus-sdk-node/operations/data-operations-insert/#error-handling)
- [Best Practices](https://milvus-io.github.io/milvus-sdk-node/operations/data-operations-insert/#best-practices)
- [Complete Example](https://milvus-io.github.io/milvus-sdk-node/operations/data-operations-insert/#complete-example)
- [Next Steps](https://milvus-io.github.io/milvus-sdk-node/operations/data-operations-insert/#next-steps)

This guide covers inserting and updating data in Milvus collections.

## Inserting Data

### Basic Insert

Insert data into a collection:

```javascript
await client.insert({
  collection_name: 'my_collection',
  data: [
    {
      id: 1,
      vector: [0.1, 0.2, 0.3, 0.4],
      text: 'Hello Milvus',
    },
    {
      id: 2,
      vector: [0.5, 0.6, 0.7, 0.8],
      text: 'Vector database',
    },
  ],
});
```

### Insert with AutoID

If your collection uses autoID, omit the ID field:

```javascript
await client.insert({
  collection_name: 'my_collection',
  data: [
    {
      vector: [0.1, 0.2, 0.3, 0.4],
      text: 'Hello Milvus',
    },
    {
      vector: [0.5, 0.6, 0.7, 0.8],
      text: 'Vector database',
    },
  ],
});
```

### Insert into Specific Partition

Insert data into a specific partition:

```javascript
await client.insert({
  collection_name: 'my_collection',
  partition_name: 'partition_1',
  data: [
    {
      vector: [0.1, 0.2, 0.3, 0.4],
      text: 'Data for partition 1',
    },
  ],
});
```

### Batch Insertion

Insert large amounts of data efficiently:

```javascript
const batchSize = 1000;
const allData = [
  /* large array of data */
];

for (let i = 0; i < allData.length; i += batchSize) {
  const batch = allData.slice(i, i + batchSize);
  await client.insert({
    collection_name: 'my_collection',
    data: batch,
  });
}
```

## Data Format

### Vector Data Formats

#### FloatVector

```javascript
{
  vector: [0.1, 0.2, 0.3, 0.4, /* ... */], // Array of numbers
}
```

#### BinaryVector

```javascript
{
  binary_vector: [0b10101010, 0b01010101], // Two bytes for dim = 16
}
```

#### SparseFloatVector

Multiple formats supported:

```javascript
// Array format
const arrayRow = {
  sparse_vector: [1.0, undefined, undefined, 2.5, undefined],
};

// Dictionary format
const dictionaryRow = {
  sparse_vector: { 0: 1.0, 3: 2.5 },
};

// CSR format
const csrRow = {
  sparse_vector: {
    indices: [0, 3],
    values: [1.0, 2.5],
  },
};

// COO format
const cooRow = {
  sparse_vector: [
    { index: 0, value: 1.0 },
    { index: 3, value: 2.5 },
  ],
};
```

#### Float16Vector / BFloat16Vector

```javascript
{
  vector: [0.1, 0.2, 0.3, 0.4], // Array of numbers (will be converted)
}
```

### Scalar Data Formats

```javascript
{
  id: 123,                    // Int64
  age: 25,                    // Int64
  score: 98.5,                // Float
  is_active: true,             // Bool
  name: 'John Doe',            // VarChar
  metadata: {                  // JSON
    category: 'tech',
    tags: ['ai', 'ml'],
  },
}
```

## Upserting Data

Upsert updates existing entities or inserts new ones:

```javascript
await client.upsert({
  collection_name: 'my_collection',
  data: [
    {
      id: 1,
      vector: [0.9, 0.8, 0.7, 0.6], // Update existing entity
      text: 'Updated text',
    },
    {
      id: 999,
      vector: [0.1, 0.2, 0.3, 0.4], // Insert new entity
      text: 'New text',
    },
  ],
});
```

### Partial Update

Update only specific fields:

```javascript
await client.upsert({
  collection_name: 'my_collection',
  partial_update: true,
  data: [
    {
      id: 1,
      text: 'Updated text only', // Only update text field
      // vector field not included, will remain unchanged
    },
  ],
});
```

### Partial Array Upsert Milvus 3.0

For Array fields, use `field_ops` to append values to or remove values from the existing array instead of replacing the whole field.

```javascript
import { FieldPartialUpdateOpType } from '@zilliz/milvus2-sdk-node';

await client.upsert({
  collection_name: 'my_collection',
  data: [
    {
      id: 1,
      tags: ['new-tag'], // Appended to the existing tags array
      scores: [3], // Removed from the existing scores array
    },
  ],
  field_ops: [
    { field_name: 'tags', op: FieldPartialUpdateOpType.ARRAY_APPEND },
    { field_name: 'scores', op: FieldPartialUpdateOpType.ARRAY_REMOVE },
  ],
});
```

You can also pass operation names as strings:

```javascript
await client.upsert({
  collection_name: 'my_collection',
  data: [{ id: 1, tags: ['urgent'] }],
  field_ops: [{ field_name: 'tags', op: 'ARRAY_APPEND' }],
});
```

Notes:

- `field_ops` is supported only by `upsert()`.
- `ARRAY_APPEND` and `ARRAY_REMOVE` apply to Array fields.
- The primary key must be included in each upsert row.
- Fields not included in the upsert row remain unchanged.
- `ARRAY_APPEND` must not make the array exceed its `max_capacity`.
- `ARRAY_REMOVE` removes matching values from the existing array.
- When `field_ops` is provided, the SDK automatically sends `partial_update: true`.

### Replace a Value by Path Milvus 3.0.6

Use `PATH_REPLACE` to replace one value in an existing row. Each Array or StructArray operand contains one element. A JSON operand is the new value itself.

```typescript
await client.upsert({
  collection_name: 'my_collection',
  data: [
    { id: 1, scores: [100], items: [{ label: 'updated' }], settings: false },
  ],
  field_ops: [
    { field_name: 'scores', op: 'PATH_REPLACE', path: '[0]' },
    { field_name: 'items', op: 'PATH_REPLACE', path: '[0][label]' },
    { field_name: 'settings', op: 'PATH_REPLACE', path: '["enabled"]' },
  ],
});
```

Use zero-based indexes like `[0]`. For a whole StructArray element, supply all its children. For a single child, use `[0][child]` and supply only that child. Quote JSON object keys, as in `["settings"][0]["enabled"]`. The primary key and the selected value must already exist.

With `HttpClient`, use `collectionName` and pass operations as `fieldOps: [{ fieldName: 'scores', op: 'PATH_REPLACE', path: '[0]' }]`.

## Insert Transformers

For Float16 and BFloat16 vectors, use transformers:

```javascript
import { DataType, f32ArrayToF16Bytes } from '@zilliz/milvus2-sdk-node';

await client.insert({
  collection_name: 'my_collection',
  data: [
    {
      vector: [0.1, 0.2, 0.3, 0.4], // f32 array
    },
  ],
  transformers: {
    [DataType.Float16Vector]: f32ArrayToF16Bytes,
  },
});
```

## Insert Response

The insert operation returns mutation results:

```javascript
const result = await client.insert({
  collection_name: 'my_collection',
  data: [
    /* ... */
  ],
});

console.log('IDs:', result.IDs); // Array of inserted IDs
console.log('Insert count:', result.insert_cnt);
console.log('Delete count:', result.delete_cnt);
console.log('Upsert count:', result.upsert_cnt);
console.log('Timestamp:', result.timestamp);
```

## Data Validation

The SDK validates data before insertion:

- **Field existence**: All fields must exist in the collection schema
- **Data types**: Values must match field data types
- **Vector dimensions**: Vector length must match field dimension
- **Binary vectors**: Length must be dimension / 8
- **VarChar length**: String length must not exceed max_length

## Error Handling

Handle insertion errors:

```javascript
try {
  await client.insert({
    collection_name: 'my_collection',
    data: [
      /* ... */
    ],
  });
} catch (error) {
  if (error.message.includes('field does not exist')) {
    console.error('Invalid field name');
  } else if (error.message.includes('dimension')) {
    console.error('Vector dimension mismatch');
  } else {
    console.error('Insert failed:', error);
  }
}
```

## Best Practices

1. **Batch inserts**: Insert data in batches (1000-10000 entities per batch)
2. **Flush after insertion**: Call `flush()` after inserting data to ensure persistence
3. **Use autoID**: Let Milvus generate IDs unless you have specific requirements
4. **Validate data**: Validate data before insertion to avoid errors
5. **Handle errors**: Always wrap insertions in try-catch blocks
6. **Use upsert for updates**: Use `upsert()` instead of delete + insert for updates

## Complete Example

```javascript
import { MilvusClient, DataType } from '@zilliz/milvus2-sdk-node';

const client = new MilvusClient({
  address: 'localhost:19530',
});

await client.connectPromise;

// Create collection
await client.createCollection({
  collection_name: 'my_collection',
  fields: [
    {
      name: 'id',
      data_type: DataType.Int64,
      is_primary_key: true,
      autoID: true,
    },
    {
      name: 'vector',
      data_type: DataType.FloatVector,
      dim: 128,
    },
    {
      name: 'text',
      data_type: DataType.VarChar,
      max_length: 256,
    },
  ],
});

// Insert data
const result = await client.insert({
  collection_name: 'my_collection',
  data: [
    {
      vector: Array.from({ length: 128 }, () => Math.random()),
      text: 'First document',
    },
    {
      vector: Array.from({ length: 128 }, () => Math.random()),
      text: 'Second document',
    },
  ],
});

console.log('Inserted IDs:', result.IDs);

// Flush to ensure data is persisted
await client.flushSync({
  collection_names: ['my_collection'],
});
await client.closeConnection();
```

## Next Steps

- Learn about [Query Operations](https://milvus-io.github.io/milvus-sdk-node/operations/data-operations-query/)
- Explore [Delete Operations](https://milvus-io.github.io/milvus-sdk-node/operations/data-operations-delete/)
- Check out [Data Management](https://milvus-io.github.io/milvus-sdk-node/operations/data-management/)
