> For the complete documentation index, see [llms.txt](https://docs.solutioo.de/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.solutioo.de/datanorm-item-import/datanorm_en_us/importing-datanorm-files/duplicates-and-validation.md).

# Duplicates and Validation

## What counts as a duplicate

A duplicate exists when an imported article corresponds to one you already have. That is the normal case for every catalogue refresh – the supplier delivers the same articles with new prices.

The app checks each article against three areas:

<table><thead><tr><th width="290">Checked against</th><th>When</th></tr></thead><tbody><tr><td>Articles within the same file</td><td>always – catches conflicts inside the delivery itself</td></tr><tr><td>Existing items in Business Central</td><td>always</td></tr><tr><td>Not yet synchronised staging data from other sessions</td><td>only when <strong>Check Pending Staging for Duplicates</strong> is on</td></tr></tbody></table>

## The match field

<table><thead><tr><th width="240">Match field</th><th>Comparison</th><th>Suitable when</th></tr></thead><tbody><tr><td><strong>DATANORM Article No.</strong></td><td>The supplier's article number</td><td>Default case. The supplier assigns stable numbers.</td></tr><tr><td><strong>EAN/GTIN</strong></td><td>Barcode; the item's GTIN field is checked as well</td><td>You carry articles from several suppliers and want to unify them by barcode.</td></tr><tr><td><strong>Vendor Item No.</strong></td><td>Vendor item number; item-vendor entries are evaluated as well</td><td>You already maintain supplier numbers in Business Central.</td></tr></tbody></table>

The field is set through **Duplicate Match Field** in [Base Configuration](/datanorm-item-import/datanorm_en_us/getting-started/base-configuration.md).

## Matching per vendor

DATANORM article numbers are only unique within a given supplier. Two suppliers may use the same number for different articles.

With **Match Duplicates per Vendor** enabled, the app only compares within the same vendor, so articles from different suppliers no longer overwrite each other.

{% hint style="info" %}
This setting is on by default. Only turn it off if you deliberately want to unify articles across all suppliers.
{% endhint %}

## A common false positive: A and B records

A DATANORM 4 article consists of two lines sharing the same article number: the A record and the B record. Checked naively, the second line looks like a duplicate of the first.

The app merges both lines into **one** staging record and counts in **Merged Source Records** how many file lines went in. A value of 2 is the normal case for DATANORM 4 and not a sign of trouble.

{% hint style="warning" %}
If an import reports a suspicious number of duplicates, check these two things first:

* Was the same file read in twice by accident? Then the reports are correct.
* Is **Check Pending Staging for Duplicates** on while an older session is still unsynchronised? Then its records show up as matches.
  {% endhint %}

## Configuring the handling

<table><thead><tr><th width="290">Duplicate handling</th><th>Effect</th></tr></thead><tbody><tr><td><strong>Skip</strong></td><td>Known articles are ignored. Only new ones get through.</td></tr><tr><td><strong>Update</strong></td><td>Existing items are overwritten with the new data. For regular catalogue refreshes.</td></tr><tr><td><strong>Create New</strong></td><td>A new item is always created. Produces duplicates quickly – use deliberately.</td></tr><tr><td><strong>Mark for Manual Review</strong></td><td>Every match waits for your decision. The safest setting, especially at the start.</td></tr></tbody></table>

## Skipping the duplicate check

For very large files the check can be switched off entirely with **Skip Duplicate Check**, which speeds up the import considerably.

{% hint style="warning" %}
Without the duplicate check, importing the same file a second time creates duplicate items. Only use the option for a controlled initial import into an empty environment.
{% endhint %}

## Other validations

Besides duplicate detection, the app checks while reading:

* **Record structure** – lines whose layout does not match the record type are logged and skipped rather than aborting the import.
* **Completeness** – the end record confirms that the file is complete.
* **Mandatory values** – an article without an article number cannot be processed.
* **Deletion flag** – articles flagged as deleted are handled according to the **Delete Flag Handling** setting: mark for review, block the item, or end-date the price.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.solutioo.de/datanorm-item-import/datanorm_en_us/importing-datanorm-files/duplicates-and-validation.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.