# Flush, Compact, and Import Data

Flush and compact Milvus collections, inspect their segments, and track bulk import jobs with the Node.js SDK.

SDK: `@zilliz/milvus2-sdk-node@3.0.10` · Tested with Milvus 3.0.6

Canonical documentation: https://milvus-io.github.io/milvus-sdk-node/operations/data-management/

## Section links

- [Flush Operations](https://milvus-io.github.io/milvus-sdk-node/operations/data-management/#flush-operations)
- [Flush Collection](https://milvus-io.github.io/milvus-sdk-node/operations/data-management/#flush-collection)
- [Flush Synchronously](https://milvus-io.github.io/milvus-sdk-node/operations/data-management/#flush-synchronously)
- [Get Flush State](https://milvus-io.github.io/milvus-sdk-node/operations/data-management/#get-flush-state)
- [Flush All Collections](https://milvus-io.github.io/milvus-sdk-node/operations/data-management/#flush-all-collections)
- [Compaction](https://milvus-io.github.io/milvus-sdk-node/operations/data-management/#compaction)
- [Trigger Compaction](https://milvus-io.github.io/milvus-sdk-node/operations/data-management/#trigger-compaction)
- [Get Compaction State](https://milvus-io.github.io/milvus-sdk-node/operations/data-management/#get-compaction-state)
- [Get Compaction Plans](https://milvus-io.github.io/milvus-sdk-node/operations/data-management/#get-compaction-plans)
- [Bulk Import](https://milvus-io.github.io/milvus-sdk-node/operations/data-management/#bulk-import)
- [Import Data](https://milvus-io.github.io/milvus-sdk-node/operations/data-management/#import-data)
- [List Import Tasks](https://milvus-io.github.io/milvus-sdk-node/operations/data-management/#list-import-tasks)
- [Load Balancing](https://milvus-io.github.io/milvus-sdk-node/operations/data-management/#load-balancing)
- [Segment Information](https://milvus-io.github.io/milvus-sdk-node/operations/data-management/#segment-information)
- [Get Query Segment Info](https://milvus-io.github.io/milvus-sdk-node/operations/data-management/#get-query-segment-info)
- [Get Persistent Segment Info](https://milvus-io.github.io/milvus-sdk-node/operations/data-management/#get-persistent-segment-info)
- [Publish Segment Manifest Updates](https://milvus-io.github.io/milvus-sdk-node/operations/data-management/#publish-segment-manifest-updates)
- [Metrics](https://milvus-io.github.io/milvus-sdk-node/operations/data-management/#metrics)
- [Best Practices](https://milvus-io.github.io/milvus-sdk-node/operations/data-management/#best-practices)
- [Next Steps](https://milvus-io.github.io/milvus-sdk-node/operations/data-management/#next-steps)

This guide covers data management operations including flushing, compaction, bulk import, and segment management.

## Flush Operations

Flush ensures data is persisted to disk.

### Flush Collection

Flush a collection asynchronously:

```javascript
const result = await client.flush({
  collection_names: ['my_collection'],
});

console.log('Flush segment IDs:', result.coll_segIDs);
```

### Flush Synchronously

Wait until flush completes:

```javascript
const result = await client.flushSync({
  collection_names: ['my_collection'],
});

console.log('Flush completed:', result.flushed);
```

### Get Flush State

Check if segments are flushed:

```javascript
const state = await client.getFlushState({
  segmentIDs: [12345, 12346],
});

console.log('Flushed:', state.flushed);
```

### Flush All Collections

Flush all collections in the cluster, or wait until the flush-all operation completes:

```javascript
const flushAll = await client.flushAll();

const state = await client.getFlushAllState({
  flush_all_tss: flushAll.flush_all_tss,
});

console.log('Flush-all completed:', state.flushed);
```

Use `flushAllSync()` to trigger flush-all and poll until completion:

```javascript
const result = await client.flushAllSync();
console.log('Flush-all completed:', result.flushed);
```

`flushAll()` and `getFlushAllState()` also accept the common request options such as `timeout`. The legacy `db_name` and `flush_all_ts` fields are still accepted for compatibility, but `flush_all_tss` is preferred.

## Compaction

Compaction merges small segments to improve query performance.

### Trigger Compaction

```javascript
const result = await client.compact({
  collection_name: 'my_collection',
});

console.log('Compaction ID:', result.compactionID);
```

Pass advanced compaction options when you need to target specific partitions, channels, or segments:

```javascript
const result = await client.compact({
  collection_name: 'my_collection',
  timetravel: 0,
  majorCompaction: true,
  partition_id: '12345',
  channel: 'by-dev-rootcoord-dml_0',
  segment_ids: [111, 222],
  l0Compaction: false,
  target_size: '536870912', // bytes
});
```

Common advanced fields:

| Field | Description |
| --- | --- |
| `timetravel` | Timestamp boundary for compaction |
| `majorCompaction` | Request major compaction |
| `partition_id` | Target a specific partition |
| `channel` | Target a specific DML channel |
| `segment_ids` | Compact specific segments |
| `l0Compaction` | Request L0 compaction |
| `target_size` | Target segment size in bytes |

### Get Compaction State

Check compaction status:

```javascript
const state = await client.getCompactionState({
  compactionID: 12345,
});

console.log('State:', state.state);
```

### Get Compaction Plans

Get detailed compaction information:

```javascript
const plans = await client.getCompactionStateWithPlans({
  compactionID: 12345,
});

console.log('Plans:', plans.mergeInfos);
```

## Bulk Import

Import data from files.

### Import Data

```javascript
const result = await client.bulkInsert({
  collection_name: 'my_collection',
  files: ['/path/to/data.json', '/path/to/data2.json'],
});

console.log('Task ID:', result.tasks[0]);
```

### List Import Tasks

Check import task status:

```javascript
const tasks = await client.listImportTasks({
  collection_name: 'my_collection',
});

tasks.tasks.forEach((task) => {
  console.log('Task ID:', task.id);
  console.log('State:', task.state);
  console.log('Imported rows:', task.row_count);
});
```

## Load Balancing

Balance data across nodes:

```javascript
await client.loadBalance({
  src_nodeID: 1,
  dst_nodeIDs: [2, 3],
  sealed_segmentIDs: [12345, 12346],
});
```

## Segment Information

### Get Query Segment Info

Get information about query segments:

```javascript
const info = await client.getQuerySegmentInfo({
  collectionName: 'my_collection',
  dbName: 'default',
});

info.infos.forEach((segment) => {
  console.log('Segment ID:', segment.segmentID);
  console.log('State:', segment.state);
  console.log('Level:', segment.level);
});
```

> The segment info APIs use proto field names: `collectionName` and `dbName`.

### Get Persistent Segment Info

Get information about persistent segments:

```javascript
const info = await client.getPersistentSegmentInfo({
  collectionName: 'my_collection',
  dbName: 'default',
});

info.infos.forEach((segment) => {
  console.log('Segment ID:', segment.segmentID);
  console.log('State:', segment.state);
  console.log('Level:', segment.level);
});
```

### Publish Segment Manifest Updates

`batchUpdateManifest()` publishes already-written Storage V3 manifest versions to Milvus metadata. Pass the segment ID and manifest version as decimal strings to preserve their full int64 precision. This API was verified with Milvus 3.0.6.

```typescript
await client.batchUpdateManifest({
  collection_name: 'my_collection',
  items: [
    { segment_id: segmentId, manifest_version: publishedManifestVersion },
  ],
});
```

## Metrics

Get collection metrics:

```javascript
const metrics = await client.getMetric({
  request: {
    metric_type: 'system_info',
  },
});

console.log('Metrics:', metrics);
```

## Best Practices

1. **Flush regularly**: Flush after bulk insertions
2. **Monitor compaction**: Check compaction state for large collections
3. **Use bulk import**: Use bulk import for large datasets
4. **Balance load**: Use load balancing for distributed deployments
5. **Monitor segments**: Check segment information for optimization

## Next Steps

- Learn about [Collection Management](https://milvus-io.github.io/milvus-sdk-node/management/collection-management/)
- Explore [Best Practices](https://milvus-io.github.io/milvus-sdk-node/advanced/best-practices/)
