Expected behavior of Document Intelligence classification with dynamic banner content

Hello @IT Cognity

Thank you for reaching out to Microsoft Q&A.

Based on the behavior you described, the current classification result is more likely related to the overall combination of layout/template differences rather than the dynamic banner content alone.

In Azure AI Document Intelligence, classification models primarily learn from both structural layout features and visual/textual cues across the entire document page, including:

• Relative positioning of sections and fields

• Text distribution and alignment

• Visual structure and spacing

• Repeating layout patterns

• Headers, branding areas, tables, and section organization

From the details shared, small graphic swaps such as dynamic marketing banners can influence classification, but it is usually the combination of:

• Banner/image changes

• Branding/theme/color differences

• Section styling modifications

• Spacing and alignment changes

• Field positioning differences

• General template presentation changes

that pushes the document toward a different classification boundary.

In your case, it is likely that both the banner variation and the broader template/layout differences are contributing, with the broader layout changes probably having a larger impact than the banner alone.

Regarding the banner specifically:

If the banner always appears in approximately the same location, with similar dimensions/layout, and only the banner image/content changes dynamically, this alone would typically not be expected to completely break classification in a well-generalized model.

However, classification models do not automatically ignore those regions by default. The model still evaluates visual and textual signals across the full page. Because of this, entirely unseen banner graphics or highly dynamic promotional content can still affect classification confidence, especially when combined with additional template/layout variations.

If the banner is the only changing element and the classifier has been trained using enough representative banner variations, the model can usually generalize successfully. However, with completely unseen graphics, classification accuracy may still degrade unless those variations are represented during training or the region is excluded from analysis.

General recommendations for this scenario are:

1. Train with representative variations Include multiple real-world document variants during classifier training, especially samples containing:

Different banner versions, Seasonal/promotional content changes, Branding/theme variations, Known template variants used in production

This helps the classifier generalize better across expected visual changes.

Focus on stable layout regions If possible, maintain consistency in the core business-relevant document structure across versions, including key field positions and section organization.

Consider masking or excluding highly dynamic visual regions (optional) If the banner area is highly dynamic (large image changes, promotional graphics, major color/theme shifts, variable text blocks, etc.), preprocessing the document to mask, blur, or exclude that region before classification can improve consistency.

This approach is especially useful when:

• The banner appears in fixed coordinates/regions

• Marketing content changes frequently

• The stable document structure is otherwise highly consistent

• The banner introduces significant visual noise unrelated to document identity

Use a hybrid approach for best robustness In many production implementations, customers achieve the best results by combining both approaches:

• Masking or suppressing high-variance graphic regions AND

• Training the classifier with multiple representative real-world banner variations

This typically provides stronger robustness against future marketing/content changes.

Avoid grouping substantially different templates into a single class without sufficient representation If the documents differ significantly beyond the banner area, additional representative training samples may be required, or the documents may need to be modeled as separate template variants/classes depending on the business requirement.

Please refer this: Custom classification models in Document Intelligence https://learn.microsoft.com/azure/ai-services/document-intelligence/train/custom-classifier

I Hope this helps. Do let me know if you have any further queries.

Thank you!

Hello @SRILAKSHMI C **,
**Just to confirm the recommended implementation: if we choose to mask or blur the dynamic banner area, should this preprocessing be applied only to the training documents, or do you recommend applying the same preprocessing to every production document before classification as well?

Hello IT Cognity

Greetings! Thanks for your follow-up question!

That's a great and very practical clarification to ask and the answer is important to get right. Yes, you must apply the same masking or blurring preprocessing to both your training documents AND every production document at inference time. This is not optional it is a fundamental requirement for the approach to work correctly.

Here's why this matters and how to implement it properly:

Why Both Training and Production Must Be Treated the Same

The Document Intelligence classification model learns patterns from the documents you train it on. If your training documents have the banner area masked but your production documents still contain the dynamic banner, the model will encounter visual features during inference that it never saw during training. This mismatch will actually make classification worse, not better the model will be confused by the unexpected content in the region it learned to treat as blank or neutral.

The golden rule is: whatever preprocessing you apply to training documents must be applied identically to every production document before it is sent to the classifier.

Step 1: Define the Banner Region as Fixed Coordinates

Since you confirmed the banner always appears in the same section with approximately the same position and dimensions, define the banner region as a fixed bounding box (for example, x, y, width, height in pixels or percentage of page). Document this as a constant in your preprocessing pipeline so it is applied consistently every time.

Step 2: Apply Masking in Your Preprocessing Pipeline

Before sending any document whether training or production to the Document Intelligence classifier, apply the mask to the defined banner region. You can do this using standard image processing libraries such as Pillow in Python or any image manipulation library available in your Salesforce integration layer. Replace the banner area with a solid neutral color (white or light grey works well) so the model sees a consistent blank region regardless of what the original banner contained.

Step 3: Retrain the Classifier with Masked Training Documents

Once your preprocessing pipeline is ready, go back and apply the masking to all your existing training documents and retrain the custom classification model from scratch using the masked versions. Do not mix masked and unmasked training samples the entire training set should be preprocessed consistently.

Step 4: Apply the Same Preprocessing to Every Production Document

In your Salesforce-based onboarding flow, every document that arrives for classification must pass through the same masking step before being sent to the Document Intelligence API. This should be a mandatory step in your pipeline, not an optional one.

Step 5: Validate Before Going Live

After retraining with masked documents, test the classifier with a sample set of production documents that have also been masked especially documents with banner variations that were not in the training set. Confirm that classification confidence improves and that the model is no longer affected by banner content changes before deploying to production.

In short masking only the training data without masking production data will not help and may make things worse. The preprocessing must be symmetric across both phases. This is the correct and recommended implementation.

If this answer helps you kindly accept the answer which will help others who have similar questions.

Best Regards,

Jerald Felix.

Hi @IT Cognity

Thank you for the follow-up question.

If you decide to use masking or blurring of the dynamic banner region, the recommended approach is to apply the same preprocessing consistently to both the training documents and the production documents used for classification.

The reason is that the classifier learns patterns from the training data. If the banner is masked during training but remains visible during inference (production), the model will encounter visual content that it was not trained to interpret, which can negatively impact classification accuracy. Similarly, if the banner is visible during training but masked during production, the model may rely on information that is no longer available at inference time.

For best results, the preprocessing pipeline should be consistent across the entire lifecycle:

• Training documents --> banner masked/blurred

• Validation/testing documents --> banner masked/blurred

• Production documents --> banner masked/blurred

This ensures the model learns and evaluates documents using the same visual representation.

That said, if the banner occupies a relatively small area and you can provide sufficient representative banner variations during training, masking may not be necessary. In many cases, training with diverse real-world banner examples is the preferred first approach because it allows the model to generalize while preserving the full document content.

Therefore, our general recommendation would be:

First, train and evaluate the classifier using representative samples that include the expected banner variations.

If banner variability continues to impact classification performance, consider introducing masking/blurring.

If masking/blurring is adopted, apply it consistently to both training and production documents.

This approach typically provides the most reliable and predictable classification behavior.

Thank you!