Skip to main content

How to Migrate Custom Entities To Paragraphs From Drupal 7 to Drupal 8 With Migrate API - Part One

2018-12-31

In a previous article, we explained how to perform a simple migration using the Drupal UI. Now we’re going to show you how to migrate custom entities to paragraphs from Drupal 7 to Drupal 8.

Why might you want to do this? Paragraphs is an excellent module that has won over vast swathes of the Drupal community, it excels at being able to encapsulate what you may have already been using entities for, and even panels, by giving customizable paragraph types that can be moved and placed, giving the end user more control on how they choose to display things.

Let’s imagine we have an entity type in Drupal 7 called Info Card, which we use to place little snippets of information along with a button, sort of like a CTA, with 3 placed in a row. We would have to create an entity, then, in turn, create a bundle, then we would need to display it somehow, so we could probably use Fieldable Panels Panes and create a new type, then we’d need to reference our entities, then we would need to add a template.

That’s quite a lot of steps, and expecting an end user to know how to create an entity, where to find them, what pages they are on, and tie all of this information together, can be quite confusing for them. Especially if you’ve already explained to them how content and content types work, or panels, now entities, now fieldable panels panes.

In this scenario, we’re going to have an Info Card content type, with a link and text snippet, built as an entity and bundle in Drupal 7. We’re going to write a new module in Drupal 8 to bring that over as a paragraph type, and then import the content, so we’ll also create 3 info cards.

We’re going to need a Drupal 7 site and a Drupal 8 site we can work off of. You can download both versions of Drupal here.

The Drupal 7 site will need to have ECK installed on it and the Link module (which is now enabled by default in Drupal 8).

The Drupal 8 site will need to have Paragraphs, Migrate, Migrate Plus, Migrate Tools, Drupal Migrate and Drupal Migrate UI installed.

 

Setting up the entity on Drupal 7

To create the entity on Drupal 7, we make sure the ECK and Links modules are downloaded and enabled, then go to Structure -> Entity Types in the admin menu.

Click + Add Entity Type to add a new entity.

Name the custom entity Info Card with the machine name info_card and select Title as a default property.

Add a Link field titled Link with the machine name field_info_card_link

Add a Long Text field titled Text with the machine name field_info_card_text

We should end up with something like this

 

Now let’s go ahead and create a couple of example cards by clicking the Entity List tab and the + Add Info Card.

 

 

Now that we have the Info Card entity defined and a handful of entities, we’re now going to get Drupal 8 setup to migrate these over.

 

Setting up the Paragraph Type

Before we do anything, ensure that you have Paragraphs installed and enabled on your Drupal 8 instance, as well as Migrate, Migrate Tools, Migrate Plus, Drupal Migrate and Drupal Migrate UI.

On the Drupal 8 site, we’re going to create a new Paragraph Type called Info Card.

Go to Structure -> Paragraph Types

Click + Add paragraph type

Here we recreate the fields from our Drupal 7 version, preparing it for a content migration.

 

 

This includes the Link field with the same machine name field_info_card_link and the Text field with the machine name field_info_card_text.

Configuring the migration filesNow we’ll descend into some code. We’ll create a new module, called migrate_info_card. Place it under the following file structure for the Drupal 8 site:

+ modules/
 + custom/
   - migrate_info_card/

We’ll also need the following files: migrate_info_card.module, migrate_info_card.info.yml

+ modules/
 + custom/
   + migrate_info_card/
     - migrate_info_card.module
     - migrate_info_card.info.yml

Add the following to the migrate_info_card.info.yml file.

name: Migrate Info Card
type: module
description: Migrates Info Cards Entities to Paragraphs from Drupal 7
core: '8.x'

And add the following to the migrate_info_card.module

<?php
/**
* @file
* migrate_info_card.module
*/

These stubs will register our module.

Before we set up our migration, we’ll want to create a migration group. Make sure that you have the Migrate Plus module installed and add the following directory to the migrate_info_card module along with a file titled migrate_plus.migration_group.info_card.yml our directory structure should look something like this now.

+ modules/
 + custom/
   + migrate_info_card/
     + config/
       + install/
         - migrate_plus.migration_group.info_card.yml
     - migrate_info_card.module
     - migrate_info_card.info.yml

This new files naming schema works as followed:

migrate_plus.migration_group.GROUP_MACHINE_NAME.yml we only replace the GROUP_MACHINE_NAME text, leaving the prefixed migrate_plus.migration_group portion and the .yml suffix.

In this file add the following.

# Group ID machine name.
id: info_card
# Label that will show up in the UI section when listing migration groups.
label: Info Cards
# Description that will show up in the UI section when listing migration groups.
description: Imports Info Cards.
# A textual tag to explain where the data is coming from.
source_type: Drupal 7 Site
# The 'key' portion lists out the database source this data comes from.
shared_configuration:
 source:
   key: migrate
# Modules that must be included to use this migration group.
dependencies:
 enforced:
   module:
   - paragraphs
   - migrate_info_card

This file declares our migration group, note, however, we are using a shared_configuration section in the YML file, with a source key of migrating we will need to add this as a database in our setting.php or local.settings.php file.

Edit either the local.settings.php or settings.php file now to add the database config of our Drupal 7 site, replace the capitalized data here with your own settings.

# Make sure this config ID stays as "migrate" to match migration group.
$databases['migrate']['default'] = array(
   'database' => '', # Add your database name here.
   'username' => '', # Add your username here.
   'password' => '', # Add your password here if you have one.
   'prefix' => '', # Add a table prefix if you're using one.
   'host' => '', # Add a host here, most likely localhost.
   'port' => '', # Add the MySQL port here, may vary i.e. MAMP.
   'namespace' => 'Drupal\Core\Database\Driver\mysql',
   'driver' => 'mysql',
);

Now that we’ve declared our source database and migration group, we’ll need to create a new migration config file. Add a file called info_card.yml to a directory called migrations in the root of the module. The directory structure will now look like this:

+ modules/
 + custom/
   + migrate_info_card/
     + config/
       + install/
         - migrate_plus.migration_group.info_card.yml
     + migrations/
       - info_card.yml
     - migrate_info_card.module
     - migrate_info_card.info.yml

Add the following to the new info_card.yml file. Refer to the comments for a description on what we’re doing here.

# The machine ID of the migration.
id: info_card
# A Readable label for the migration.
label: "Info Card"
# Use the "info_card" migration group defined previously.
migration_group: info_card
# Some tags to identify the migration.
migration_tags:
 - origin
 - entity
 - info_card
# Use a custom plugin titled "migrate_info_card" and the "migrate" database source
source:
 plugin: migrate_info_card
 key: migrate
# The field "processing" and mapping from the Drupal 7 Entity to the new Drupal 8 Paragraph type.
process:
 # Title maps to title
 title: title
 # Map the Paragraph Link URI to the link url field that will be pulled from the source database.
 "field_info_card_link/uri": field_info_card_link_url
 # Map the Paragraph Link Title to the link title field that will be pulled from the source database.
 "field_info_card_link/title":
   plugin: default_value
   default_value: "See More"
   source: field_info_card_link_title
 # Map the Paragraph Text Value to the text field value that will be pulled from the source database.
 "field_info_card_text/value": field_info_card_text_value
 # Map the Paragraph Text Format to the text field format that will be pulled from the source database.
 "field_info_card_text/format":
   source: field_info_card_text_format
   plugin: default_value
   default_value: full_html
# The destination is to create new Paragraph entities that can be referenced in blocks using Paragraphs.
destination:
 plugin: 'entity_reference_revisions:paragraph'
 default_bundle: info_card
migration_dependencies:
 required: {  }
 optional: {  }

 

Creating a source plugin

A Migrate source plugin will let Drupal 8 know where our data is coming from.

In some instances a Node source plugin might be used, which will speed up a lot of the work when pulling in data from Nodes on different Drupal instances, there might be a need to pull in migration data from an XML source or a CSV source or a JSON source.

For all scenarios, a source plugin can be written to handle the data extraction. On some occasions, there might need to be more work done depending on the origin of the source, however, for extracting data from our custom Info Card entities, we’re going to use the SQLBase source plugin that we can find in the core.

The SQLBase source plugin can be found here: core/modules/migrate/src/Plugin/migrate/source/SqlBase.php

We’ll extend this class to create our own class.

Do this now by creating a new file called InfoCard.php in our module directory under src/Plugin/migrate/source. The directory structure must look exactly like this.

+ modules/
 + custom/
   + migrate_info_card/
     + config/
       + install/
         - migrate_plus.migration_group.info_card.yml
     + migrations/
       - info_card.yml
     + src/
       + Plugin/
         + migrate/
           + source/
             + InfoCard.php
     - migrate_info_card.module
     - migrate_info_card.info.yml

Copy the following code to the new InfoCard.php file

<?php

namespace Drupal\migrate_info_card\Plugin\migrate\source;

use Drupal\migrate\Plugin\migrate\source\SqlBase;
use Drupal\migrate\Row;

/**
* Drupal 7 Info Card entity from database.
*
* @MigrateSource(
*   id = "migrate_info_card"
* )
*/
class InfoCard extends SQLBase {

 protected $moduleHandler;

 /*
  * {@inheritdoc}
  */
 public function query() {
   // Select node in its last revision.
   $query = $this->select('eck_info_card', 'ic')
     ->fields('ic', [
       'id',
       'type',
       'title',
     ])
     ->fields('l', [
       'revision_id',
     ])
     ->fields('t', [
       'revision_id',
     ])
     ->fields('lr', [
       'field_info_card_link_url',
       'field_info_card_link_title'
     ])
     ->fields('tr', [
       'field_info_card_text_value',
       'field_info_card_text_format'
     ]);

   $query->innerJoin('field_data_field_info_card_link', 'l', 'l.entity_id=ic.id');
   $query->innerJoin('field_data_field_info_card_text', 't', 't.entity_id=ic.id');

   $query->innerJoin('field_revision_field_info_card_link', 'lr', 'lr.entity_id=l.entity_id');
   $query->innerJoin('field_revision_field_info_card_text', 'tr', 't.entity_id=tr.entity_id');
   
   $query->condition('l.type', 'info_card', '=');
   $query->condition('t.type', 'info_card', '=');

   return $query;
 }

 /**
  * {@inheritdoc}
  */
 public function prepareRow(Row $row) {
   return parent::prepareRow($row);
 }

 /**
  * {@inheritdoc}
  */
 public function fields() {
   $fields = [
     'id' => $this->t('Entity ID'),
     'type' => $this->t('Entity Type'),
     'title' => $this->t('Info Card Title'),
     'field_info_card_link_url' => $this->t('Info Card Link URL'),
     'field_info_card_link_title' => $this->t('Info Card Link Title'),
     'field_info_card_text_value' => $this->t('Info Card Text Value'),
     'field_info_card_text_format' => $this->t('Info Card Text Value')
   ];
   return $fields;
 }

 /**
  * {@inheritdoc}
  */
 public function getIds() {
   $ids['id']['type'] = 'integer';
   $ids['id']['alias'] = 'ic';
   return $ids;
 }

}

Let’s break this down.

First, we set our namespace to migrate source Drupal\MODULE_NAME\Plugin\migrate\source

Second, we require SQLBase and Row.

Third, we declare the InfoCard class, extending the SQLBase class. The comment is important as it sets the id that we declared in our info_card.yml config file, this id must match so that it can identify and load this plugin.

<?php

namespace Drupal\migrate_info_card\Plugin\migrate\source;

use Drupal\migrate\Plugin\migrate\source\SqlBase;
use Drupal\migrate\Row;

/**
* Drupal 7 Info Card entity from database.
*
* @MigrateSource(
*   id = "migrate_info_card"
* )
*/
class InfoCard extends SQLBase {

 

Next, we set up the query.

Here were selecting our Info Card entities from the table ECK created; we select the id, type and title fields.

We then join the data fields for the Link and Text fields and then join the Revision tables for those fields.

Lastly, we set a condition to using the info_card entity type. This isn’t really necessary for our simple scenario, but we’ll do it anyway. Use your own due diligence when creating source plugins.

 /*
  * {@inheritdoc}
  */
 public function query() {
   // Select node in its last revision.
   $query = $this->select('eck_info_card', 'ic')
     ->fields('ic', [
       'id',
       'type',
       'title',
     ])
     ->fields('l', [
       'revision_id',
     ])
     ->fields('t', [
       'revision_id',
     ])
     ->fields('lr', [
       'field_info_card_link_url',
       'field_info_card_link_title'
     ])
     ->fields('tr', [
       'field_info_card_text_value',
       'field_info_card_text_format'
     ]);

   $query->innerJoin('field_data_field_info_card_link', 'l', 'l.entity_id=ic.id');
   $query->innerJoin('field_data_field_info_card_text', 't', 't.entity_id=ic.id');

   $query->innerJoin('field_revision_field_info_card_link', 'lr', 'lr.entity_id=l.entity_id');
   $query->innerJoin('field_revision_field_info_card_text', 'tr', 't.entity_id=tr.entity_id');
   
   $query->condition('ic.type', 'info_card', '=');
   $query->condition('l.type', 'info_card', '=');
   $query->condition('t.type', 'info_card', '=');

   return $query;
 }

 

For the last part, we declare the prepareRow function and simply call the parent method.

Next, we define the fields and their labels that we used in our mapping in the info_card.yml file, these are mapped to the Paragraph equivalents.

We also set a getIds function to specify which field is used to designate the unique value for our rows. In this case, it is the original ECK entity id.

/**
  * {@inheritdoc}
  */
 public function prepareRow(Row $row) {
   return parent::prepareRow($row);
 }

 /**
  * {@inheritdoc}
  */
 public function fields() {
   $fields = [
     'id' => $this->t('Entity ID'),
     'type' => $this->t('Entity Type'),
     'title' => $this->t('Info Card Title'),
     'field_info_card_link_url' => $this->t('Info Card Link URL'),
     'field_info_card_link_title' => $this->t('Info Card Link Title'),
     'field_info_card_text_value' => $this->t('Info Card Text Value'),
     'field_info_card_text_format' => $this->t('Info Card Text Value')
   ];
   return $fields;
 }

 /**
  * {@inheritdoc}
  */
 public function getIds() {
   $ids['id']['type'] = 'integer';
   $ids['id']['alias'] = 'ic';
   return $ids;
 }
 
}

Once this is done, we technically have a Migration function that will pull all of our content over and create Paragraph referenced entities.

If we were to install the module and run it, we would see something like this when using Migrate Tools from Drush after doing so.

Now the problem with this is that, although we can pull our Drupal 7 Info Card entities over to Drupal 8 Info Card paragraphs, by themselves we can do much with them.

We can’t reuse references that we’ve imported, so one thing we could do is migrates those references to a custom block type that references the Info Card Paragraph type.

For that, we will have to wait for Part Two of this tutorial!