Organize your JSON files with relationship data. Ensure each record includes lookup field references:

// Example record with relationships
{
  "name": "Customer Account 123",
  "revenue": 500000,
  "primarycontactid@odata.bind": "/contacts(abc123-4567-...)",
  "ownerid@odata.bind": "/systemusers(def456-7890-...)",
  "parentaccountid@odata.bind": "/accounts(ghi789-0123-...)"
}

Tip: Use the lookup format fieldname@odata.bind for relationships.

Before ingestion, identify duplicate records using the Analysis tab:

1. Go to Data Analysis tab
2. Select your target table and environment
3. Run query: $select=name,email&$top=1000
4. Visualize unique values to identify duplicate patterns

Use AI Support to generate deduplication logic:

1. Go to AI Support tab
2. Drag your target table to context
3. Use prompt: “Generate FetchXML to find duplicates in [table] based on fields [field1, field2, field3]”
4. The AI will generate deduplication queries

Use the Bulk Data Operations tab with this approach:

// Batch API payload with duplicate checking
{
  "requests": [
    {
      "id": "1",
      "method": "POST",
      "url": "/accounts",
      "headers": {"Prefer": "return=representation"},
      "body": {
        "name": "Unique Customer 1",
        "email": "unique1@example.com"
      }
    },
    {
      "id": "2", 
      "method": "GET",
      "url": "/accounts?$filter=email eq 'existing@example.com'"
    }
  ]
}

Process large datasets in manageable chunks:

1. Split 100K records into batches of 1000
2. Use Bulk Data Operations → Manual/File Ingestion
3. Enable AI-Powered Ingestion for deduplication
4. Monitor results in real-time

// Recommended batch structure
Batch 1: 1-1000 records (validate relationships)
Batch 2: 1001-2000 records (check for cross-batch duplicates)
Continue until all 100K processed

After ingestion, validate all relationships:

1. Use SQL Query tab with JOIN statements
2. Run: SELECT a.name, c.fullname FROM account a JOIN contact c ON a.primarycontactid = c.contactid
3. Check for broken relationships in results
4. Use visualization to spot data anomalies

Use the History tab to review all operations:

• Check for failed API calls
• Monitor performance metrics
• Identify patterns for optimization
• Adjust batch sizes based on performance