Dynamics 365 Customer Insights – Journey

Custom Form Submission Validation in Dynamics 365 Customer Insights – Journeys: Inside the Validation Pipeline and the “ms_captcha_solution” Error

While implementing custom form submission validation (server side validation) for Dynamics 365 Customer Insights – Journeys Real-Time Marketing forms, we came across the following error,

“Required params cannot be null or empty –
ms_captcha_solution
ms_captcha_type
ms_captcha_flow_id”

after enabling the data-validate-submission attribute on the form as shown below.

Setting this attribute to true caused the platform to invoke the msdynmkt_validateformsubmission Custom API, which ultimately resulted in the error.

After reviewing Microsoft’s documentation, plugin trace logs, and decompiling the Microsoft assemblies, we were able to understand exactly how the validation pipeline works.

When does this error occur?

• data-validate-submission=”true” is enabled.
• Microsoft CAPTCHA isn’t configured.
• No custom validation plugin overwrites the default validation response.

Understanding the Validation Pipeline

Customer Insights invokes the following Custom API:

msdynmkt_validateformsubmission

The msdynmkt_validateformsubmission Custom API is implemented by Microsoft’s Microsoft.Dynamics.Cxp.Forms.Plugins.Plugins.ValidateFormSubmissionPlugin, which performs the default CAPTCHA validation and initializes the msdynmkt_validationresponse.

We can then register our own plugin steps on the same message.

Microsoft recommends registering custom validation plugins with an Execution Order of 20, allowing them to execute after the out-of-the-box Microsoft.Dynamics.Cxp.FormsReCaptcha.Plugins.ReCaptchaValidationPlugin (Execution Order 10) and overwrite the validation response if required.

Check the flow below to get more details –

Why does the error occur?

The VerifyCaptchaChallenge service expects the following fields:

ms_captcha_solution
ms_captcha_type
ms_captcha_flow_id

If any are missing, it returns IsValid = false.

if (solution == null ||
    string.IsNullOrEmpty(captchaType) ||
    string.IsNullOrEmpty(flowId))
{
    return new VerifyCaptchaResponse
    {
        IsValid = false
    };
}

How our custom validation plugin resolves the issue

Our custom Honeypot custom plugin performs its own validation and overwrites the validation response.

SetValidationResponse(context, true, null);

// or

SetValidationResponse(context, false, “Form validation failed.”);

What about the ReCaptchaValidationPlugin?

The ReCaptchaValidationPlugin checks for g-recaptcha-response. If it isn’t present, it simply returns.

if (field == null)
{
    tracing.Trace("g-recaptcha-response field was not present in form submission");
    return;
}

Plugin Trace Logs

Our trace logs confirmed:

1. ValidateFormSubmissionPlugin executes.
2. Our Honeypot plugin overwrites the validation response.
3. ReCaptchaValidationPlugin executes afterwards and exits because g-recaptcha-response isn’t present without throwing any exception.

Conclusion

Although the error appears to be a configuration issue, it’s actually the expected behaviour of the default validation pipeline. The default implementation expects Microsoft’s CAPTCHA fields. If we’re implementing our own custom plugin for form submission validation, it should overwrite the validation response after performing its own server-side validation.

Get more details –

https://learn.microsoft.com/en-us/dynamics365/customer-insights/journeys/real-time-marketing-form-customize-submission-validation

https://www.ameyholden.com/articles/recaptcha-v3-cloudflare-turnstile-for-customer-insights-journeys-forms

Check the previous posts –

Implementing a Simple Honeypot in Dynamics 365 Customer Insights – Journeys Real-time Marketing Forms

Honey Pot Validation (Server-side)

Hope it helps..

Advertisements

Implementing Server-Side Honeypot Validation for Dynamics 365 Customer Insights – Journeys Forms

In our previous post, we implemented a simple Honeypot for Dynamics 365 Customer Insights – Journeys Real-Time Marketing forms using JavaScript.

Although that prevents most automated submissions, the validation only runs in the browser. Anyone can bypass the JavaScript and submit requests directly to the server.

Fortunately, Customer Insights – Journeys provides a server-side validation pipeline through the msdynmkt_validateformsubmission message, allowing us to validate every form submission before a Lead, Contact, or custom record is created.

When a form with the data-validate-submission attribute is submitted, Microsoft first executes its built-in validation plugin, followed by any custom plugins registered on the same message.

Update the HTML of the marketing form and add the attribute –

One important behavior to be aware of is that if our form doesn’t contain Microsoft’s built-in CAPTCHA fields, the default Microsoft validation plugin sets IsValid = false, causing the submission to fail. Our custom plugin must overwrite this response by returning IsValid = true when our validation succeeds, or IsValid = false if it fails.
In this post, we’ll use this pipeline to validate the Honeypot field that we created in the previous article.

Previous article:
Implementing a Simple Honeypot in Dynamics 365 Customer Insights – Journeys Real-Time Marketing Forms

Register the Validation Plugin

The submitted form values are available in the msdynmkt_formsubmissionrequest input parameter.

Next, as described in the Microsoft Learn documentation, we can overwrite the response by returning a new ValidateFormSubmissionResponse with IsValid = true.

Read the Honeypot Field

var request = Deserialize<FormSubmissionRequest>(                (string)context.InputParameters["msdynmkt_formsubmissionrequest"]);
 var fields = request?.Fields ?? new List<FormField>();
 string honeypotValue = fields.FirstOrDefault(f => f.Key == HoneypotFieldName)?.Value;

Reject Bot Submissions

if (!string.IsNullOrWhiteSpace(honeypotValue))
{
    SetValidationResponse(context, false, "Form validation failed.");
    return;
}

Return the Validation Response

Customer Insights expects the plugin to return a ValidateFormSubmissionResponse. We also include the Honeypot field in ValidationOnlyFields, so it isn’t mapped to the target record.

On Successful Form submission, we can see our validation plugin triggering

If server side validation fails (i.e. honey pot field has a value in it) we get our custom error message and submission fails.

Complete Plugin

using Microsoft.Xrm.Sdk;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Json;
using System.Text;

namespace DI.D365.Plugins.Marketing
{
    /// <summary>
    /// Plugin Registration
    ///
    /// Message: msdynmkt_validateformsubmission
    /// Stage: PostOperation
    /// Mode: Synchronous    
    /// </summary>
    public class ValidateHoneypot : IPlugin
    {
        private const string HoneypotFieldName = "hp_check";
        private const string ErrorMessage = "Form validation failed.";

        public void Execute(IServiceProvider serviceProvider)
        {
            var context = (IPluginExecutionContext)serviceProvider.GetService(
                typeof(IPluginExecutionContext)
            );

            var tracing = (ITracingService)serviceProvider.GetService(typeof(ITracingService));

            if (!context.InputParameters.Contains("msdynmkt_formsubmissionrequest"))
            {
                tracing.Trace("Form submission request not found.");
                return;
            }

            var request = Deserialize<FormSubmissionRequest>(
                (string)context.InputParameters["msdynmkt_formsubmissionrequest"]
            );

            var fields = request?.Fields ?? new List<FormField>();

            string honeypotValue = fields.FirstOrDefault(f => f.Key == HoneypotFieldName)?.Value;

            tracing.Trace($"Honeypot Value: {honeypotValue}");

            if (!string.IsNullOrWhiteSpace(honeypotValue))
            {
                tracing.Trace("Honeypot validation failed.");

                SetValidationResponse(context, false, ErrorMessage);

                return;
            }

            tracing.Trace("Honeypot validation passed.");

            SetValidationResponse(context, true, null);
        }

        private void SetValidationResponse(
            IPluginExecutionContext context,
            bool isValid,
            string error
        )
        {
            var response = new ValidateFormSubmissionResponse
            {
                IsValid = isValid,
                ValidationOnlyFields = new List<string> { HoneypotFieldName },
                Error = error
            };

            context.OutputParameters["msdynmkt_validationresponse"] = Serialize(response);
        }

        private static T Deserialize<T>(string json)
        {
            using (var stream = new MemoryStream(Encoding.UTF8.GetBytes(json)))
            {
                return (T)new DataContractJsonSerializer(typeof(T)).ReadObject(stream);
            }
        }

        private static string Serialize<T>(T value)
        {
            using (var stream = new MemoryStream())
            {
                new DataContractJsonSerializer(typeof(T)).WriteObject(stream, value);

                return Encoding.UTF8.GetString(stream.ToArray());
            }
        }
    }

    #region Helper Classes

    [DataContract]
    public class FormSubmissionRequest
    {
        [DataMember(Name = "Fields")]
        public List<FormField> Fields { get; set; }
    }

    [DataContract]
    public class FormField
    {
        [DataMember(Name = "Key")]
        public string Key { get; set; }

        [DataMember(Name = "Value")]
        public string Value { get; set; }
    }

    [DataContract]
    public class ValidateFormSubmissionResponse
    {
        [DataMember(Name = "IsValid")]
        public bool IsValid { get; set; }

        [DataMember(Name = "ValidationOnlyFields")]
        public List<string> ValidationOnlyFields { get; set; }

        [DataMember(Name = "Error")]
        public string Error { get; set; }
    }
    #endregion
}

Conclusion

Adding a client-side Honeypot is a great first step, but validating it on the server makes the solution much more robust. Since every submission passes through the msdynmkt_validateformsubmission pipeline, bots can’t bypass the validation simply by skipping our JavaScript.

“The msdynmkt_validateformsubmission Custom API is implemented by Microsoft’s Microsoft.Dynamics.Cxp.Forms.Plugins.Plugins.ValidateFormSubmissionPlugin, which performs the default Microsoft CAPTCHA validation and initializes the msdynmkt_validationresponse. We can then register our own plugin steps on the same message. Microsoft recommends registering custom validation plugins with an Execution Order of 20, allowing them to execute after the out-of-the-box Microsoft.Dynamics.Cxp.FormsReCaptcha.Plugins.ReCaptchaValidationPlugin (Execution Order 10) and overwrite the validation response if required.”

References

Hope it helps..

Advertisements

Implementing a Simple Honeypot in Dynamics 365 Customer Insights – Journeys Real-time Marketing Forms

While working with Dynamics 365 Customer Insights – Journeys Real-time Marketing Forms, we wanted to explore a simple way of reducing bot submissions.

A common technique used on websites is a honeypot. The idea is straightforward:

Add a field that genuine users never see.
Hide it using JavaScript.
If the field contains a value when the form is submitted, assume it was completed by a bot and cancel the submission.

In this post, we’ll implement this using the supported Client-side Extensibility events provided by Dynamics 365.

Step 1: Add an Unbound Field

Add a new Short Text field to the form.

The field can be Unbound, as it is only used for validation and does not need to be stored in Dataverse.

We used hp_check for the field name as this avoids browser autofill heuristics that may populate well-known fields such as Middle Name, Email, or Phone.

Step 2: Hide the Field

The field should remain in the HTML so that automated bots can still discover it, but it should not be visible to genuine users.

In our implementation, the field is hidden using CSS, which avoids any brief flicker that could occur if it were hidden only after the form loads.

.textFormFieldBlock:has(input[name="hp_check"]) {
    position: absolute !important;
    left: -9999px !important;
    width: 1px !important;
    height: 1px !important;
    overflow: hidden !important;
}

Step 3: Validate During Form Submission

Using the supported d365mkt-afterformload and d365mkt-formsubmit events, configure the field and validate it before submission.

<script>
let honeypotField = null;

document.addEventListener("d365mkt-afterformload", function () {

    honeypotField = document.querySelector('[name="hp_check"]');

    if (!honeypotField) {
        return;
    }

    honeypotField.setAttribute("autocomplete", "off");
    honeypotField.setAttribute("tabindex", "-1");
    honeypotField.setAttribute("aria-hidden", "true");
});

document.addEventListener("d365mkt-formsubmit", function (event) {

    if (honeypotField && honeypotField.value.trim() !== "") {
        event.preventDefault();
    }

});
</script>

The d365mkt-formsubmit event is cancelable, so calling event.preventDefault() prevents the form from being submitted.

Testing

To test the implementation, we opened the browser Developer Tools and executed the following in the Console:

document.querySelector('[name="hp_check"]').value = "I am a bot";

When we clicked Submit, the d365mkt-formsubmit event was triggered and event.preventDefault() successfully cancelled the submission.

Things to consider

While implementing this, there were a few considerations that helped improve the solution:

Use a neutral field name such as hp_check instead of generic field name to reduce the chance of browser autofill populating the field.
Hide the field using CSS instead of JavaScript to avoid a brief flash of the field while the page is loading.
Keep the field unbound, as it is only used for validation.
A client-side honeypot helps reduce spam submissions, but it should be considered one layer of protection. For higher security requirements, additional server-side validation can also be implemented. For e.g. if it is bound field on lead than a plugin on pre create operation or if it is unbound we can have plugin against Form Submission table.

Conclusion

A honeypot is a simple but effective technique for reducing automated submissions to Dynamics 365 Customer Insights – Journeys Real-time Marketing Forms.

By using the supported Client-side Extensibility events, the implementation remains lightweight, requires no Dataverse customizations, and can be easily reused across multiple for

References

Real-time Marketing Form Client-side Extensibility

Hope it helps..

Advertisements

Capture UTM Parameters in Dynamics 365 Marketing Forms Using JavaScript (Dynamics 365 Customer Insights)

When running marketing campaigns, UTM parameters help us understand where our leads are coming from. They help us track whether a lead came from Google Ads, Facebook campaigns, email campaigns, or some other source.

Recently while working with a Dynamics 365 Marketing form, we used a small JavaScript snippet to automatically capture UTM parameters from the URL and store them directly into marketing form fields.

 <script>
document.addEventListener("d365mkt-afterformload", function () {
    const params = new URLSearchParams(window.location.search);
      const mappings = [
        { param: "utm_source", name: "custom_utm_source" },
        { param: "utm_medium", name: "custom_utm_medium" },
        { param: "utm_campaign", name: "custom_utm_campaign" },
        { param: "utm_term", name: "custom_utm_term" },
        { param: "utm_content", name: "custom_utm_content" },
        { param: "gclid", name: "custom_gclid" },
        { param: "gclsrc", name: "custom_gclsrc" },
        { param: "fbclid", name: "custom_fbclid" }
    ];
    mappings.forEach(function (m) {
        const field = document.querySelector(`[name="${m.name}"]`);
        const value = params.get(m.param);

        if (field && value) {
            field.value = value;        
        }
    });
});
</script>

Suppose the marketing form URL is opened like this:

https://contoso.com/form?utm_source=google&utm_medium=cpc

In the example below, the values from the URL are automatically populated into the form fields.

The script first waits for the Dynamics 365 Marketing form to fully load using the d365mkt-afterformload event. This is important because the fields may not yet exist when the page initially loads.

After that, the script reads the query string from the URL using URLSearchParams. So if the URL contains values like utm_source=google or utm_medium=cpc, those values become available to the script.

The mappings array is used to map URL parameters to marketing form fields. For example, utm_source maps to custom_utm_source and utm_medium maps to custom_utm_medium.

The script then loops through each mapping, finds the matching field inside the marketing form, and sets the value automatically.

Using this approach helps us capture campaign attribution data directly inside Dataverse during form submission

References –

https://paulinekolde.info/javascript-library-for-real-time-marketing-form-in-customer-insights

Extend Customer Insights – Journeys marketing forms using code

Hope it helps..

Advertisements

Step-by-Step: Configure Double Opt-In in Dynamics 365 Customer Insights – Journey (Real Time Marketing)

Double Opt-In is a two-step process for email subscription:

The contact fills out a subscription form (initial opt-in).
They receive an email with a confirmation link. Only after clicking that link is their subscription considered confirmed (final opt-in).

This process ensures that the person who signed up wants to hear from us—and that the email address is valid.

Enable the Double Opt-In Feature

Below is how we can enable it in Dynamics 365 Customer Insights – Journey

Go to Settings > Feature Switches > Compliance (Double Opt-In)

Create or Configure a Compliance Profile

The Compliance Profile is what governs how consent is collected and managed in real-time marketing.

Go to Settings > Customer engagement > Compliance profiles

Here we will enable it in the existing default compliance profile. We can have different compliance profiles for different regions, languages, or purposes.

Navigate to the Double opt-in tab.

A screenshot of a computer

AI-generated content may be incorrect.

As part of the setup, the system creates the Double opt-in email and the journey required as part of the process for that particular compliance profile.

We can see it configured after a few minutes.

Below is the email created by the system that will be used for the journey generated. Notice its state will be Ready to Send.

Similarly, we will have the journey created, with Status = Live.

Link Compliance Profile to the Form

Open an existing marketing form or create a new one.

In the form editor, drag a Consent field.

In the property pane for the Consent element, select the compliance profile (default one in our case), specify the purpose, and other required properties.

To see it in action let us submit the form using the standalone page.

Till this point, we do not see any contact point consent record created.

We can see the confirmation email received in our mailbox. From there we can confirm it.

A paper with a check mark and a green circle

AI-generated content may be incorrect.

After confirmation, we can see our consent record created with the appropriate consent status.

A green pen pointing to a white box

AI-generated content may be incorrect.

Get all details

Also check out this wonderful tutorial – Dynamics 365 Customer Insights Journeys: How Consent Management Actually Works | Deep Dive Tutorial by Nathan Rose.

Hope it helps..

Customize the Real-time Marketing Form form / Form Editor to add field (Dynamics 365 Customer Insights – Journey)

Suppose we want to add the Lead Type (a custom choice field) to the Marketing Form’s form / form settings (RTM).

Open the Form table for customization, here we need to add the field to below 2 forms.

Starting with the Form Settings form, drag the field to the form.

A screenshot of a computer

Description automatically generated

If we check the form now, it will ask us to add it to the main form.

Open the Information (Main) form, double click the field to add it. As the field will not render in the form designer, select the field from the Tree View, and Hide it.