Migrating Content from Dynamics to Drupal – A Data-Driven Approach

Migrating content from one system to another can be complex, but with the right planning and tools, it becomes manageable. In this project, we have moved content from Microsoft Dynamics to Drupal, using a structured approach to ensure accuracy, efficiency, and minimal manual intervention.

Here’s how we have tackled the challenge.

The Data: Understanding the Source

The content is divided into three types: web pages, knowledge articles, and web links.

With knowledge articles, the migration process was smoother. Since these are custom entities within Dynamics, they could be exported in JSON format. Because the number of returned items was quite large, we created a Drush command executed every 30 minutes, where we stored the corresponding JSON file locally with all the information. The standard used for communication and data retrieval with Microsoft Dynamics was OData (Open Data Protocol).

With web pages and web links, things were more complex, as these entities could not be exposed via an API like in the case of knowledge articles. After discussing with the Colchester team, the chosen solution was to obtain a ZIP file containing all the content using Microsoft Power Platform CLI with Power Pages, following this article.

This content (web pages and web links) is stored in Drupal following a structure that is easy to read later by related migrations, using a custom form created specifically for the website.

With all the information available, we were then able to extract the data we needed to map into Drupal:

• Microsoft Dynamics ID
• The body of each page
• Links for redirects
• References to parent/children elements

However, the content migration is not executed automatically with all available content. Editors wanted to define which content to migrate using a CSV file.

Admin Interface: Simplifying the Process

Using Migrate Source CSV, Migrate Source UI and a custom module, we created a simple form where editors could:

• Upload the CSV containing the content they wanted to migrate
• Specify the type of content they were migrating: web pages, knowledge articles, or web links
• Indicate whether they wanted to update existing content or not

All of this operates under the workspace umbrella, since the content migrated through this form had to go directly into the currently active workspace.

Migrating content using workspaces allows editors to verify that:

• The migrated content is correct
• There are no broken links in the content
• Redirects have been created correctly to avoid losing SEO positioning
• If the content maintains a parent or child relationship with other content, this relationship has been set up correctly
• The workspace can be shared with other departments before publishing the migrated content

Logging and Reporting

As part of the work, we created a specific log page where editors could identify some errors or see some notices about the migration process, example:

• Missing Microsoft Dynamics IDs where they are expected
• Errors related to the migration with details.
• Discrepancies between the CSV uploaded by editors and the information imported from Dynamics.
   

Logs were stored historically, accessible via a dedicated admin tab. This ensured transparency and allowed the teams to review past migrations.

Conclusion

By breaking the migration into clear steps—data extraction, mapping, admin interface, and logging—we ensure a smooth transition from Dynamics to Drupal. The team’s collaboration and structured approach minimising errors and streamlining the process, we were able to save the client months of manual work within 9 days of collaboration with a developer. They can furthermore continue without our intervention.