Drupal 8 ships with the ability to migrate content from older versions of Drupal or from any other systems. When you set up a custom migration you may need to modify incoming data before it is saved to Drupal. Today, I’ll show you how to create your own process plugin for mapping incoming content.

This post is specifically about migration process plugins and I will assume you are familiar with configuring simple migrations. The migration modules included in Drupal core cover a wide range of topics. If you are new to migrations or need more information on creating custom migrations, please see the Migrate API documentation on drupal.org.

When you create a new migration, you can specify different process methods to use for mapping fields from your source. The default plugin is get which will copy over values verbatim. Other plugins may be used for more advanced mappings and many common process plugins are provided out-of-the-box by Drupal. For example, you can use the callback plugin to run incoming values through strtolower() or any other function before it is saved to the new database.

1
2
3
4
5
process:
  destination_field:
    plugin: callback
    callable: strtolower
    source: source_field

You can find a list of process plugins provided by Drupal core at https://www.drupal.org/node/2129651. Unfortunately, the process plugins provided by Drupal may not be sufficient for all your source data. There could be a case where you need some additional custom processing. You can perform your own processing in a custom migrate process plugin.

For this example, let’s assume you have some incoming Unix timestamps. Drupal 8 date fields require all dates to be in the W3C date format but, as of this writing, there are no process plugins to convert date values. Follow these steps to create a custom plugin that converts any incoming date into the correct W3C format.

1. Create a new module.

The new process plugin needs a place to live. Create a custom module called “Migrate Dates” and start with a simple info file (migrate_dates.info.yml). Custom modules should live in the /modules directory at the root of your Drupal installation.

1
2
3
4
5
6
7
name: 'Migrate Dates'
description: 'Migrates date fields to Drupal 8'
package: 'Migration'
type: 'module'
core: '8.x'
dependencies:
  - migrate

2. Create the process plugin class

Next, add a new plugin class called W3CDate in your module at src/Plugin/migrate/process/W3CDate.php. The plugin namespace should follow the standard Drupal naming conventions.

The @MigrateProcessPlugin annotation tells migrate about the new process plugin and the id key will be used to select this plugin in a migration’s mappings. The class should extend Drupal\migrate\ProcessPluginBase and override the transform() method. This method will be passed the incoming source value and is responsible for returning the modified destination value.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
<?php
/**
 * @file
 * Contains \Drupal\migrate_dates\Plugin\migrate\process\W3CDate.
 */
 
namespace Drupal\migrate_dates\Plugin\migrate\process;
 
use Drupal\migrate\MigrateException;
use Drupal\migrate\MigrateExecutableInterface;
use Drupal\migrate\ProcessPluginBase;
use Drupal\migrate\Row;
 
/**
 * Convert a date field to the W3C format.
 *
 * @MigrateProcessPlugin(
 *   id = "w3c_date",
 * )
 */
class W3CDate extends ProcessPluginBase {
 
  /**
   * {@inheritdoc}
   */
  public function transform($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) {
    return $value;
  }

}

This is the basic shell for a process plugin. To use this plugin in a migration mapping, just use the w3c_date id.

1
2
3
4
process:
  destination_date_field:
    plugin: w3c_date
    source: source_date_field

3. Write a unit test

You may have noticed that the the W3CDate::transform() method above simply returns $value. This isn’t very useful yet. You will implement the actual processing soon but first you should write a simple unit test make sure the plugin will work correctly. Drupal 8 ships with PHPUnit built in. Configuring and running PHPUnit tests is out of scope for this post but you can find more information on drupal.org about Drupal’s implementation of PHPUnit.

Create a new test case in tests/src/Unit/Plugin/migrate/process/W3CDateTest.php. The test case should extend Drupal\Test\migrate\Unit\process\MigrateProcessTestCase. To test plugin, run some example data through the W3CDate::transform() method and check the results.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
<?php
namespace Drupal\Tests\migrate_dates\Unit\Plugin\migrate\process;

use Drupal\migrate_dates\Plugin\migrate\process\W3CDate;
use Drupal\Tests\migrate\Unit\process\MigrateProcessTestCase;

/**
 * @coversDefaultClass \Drupal\acc_migrate\Plugin\migrate\process\W3CDate
 * @group acc_migrate
 */
class W3CDateTest extends MigrateProcessTestCase {

  /**
   * Tests date conversion to the W3C format.
   *
   * @dataProvider providerTestW3CDate
   */
  public function testW3CDate($value, $expected) {
    $plugin = new TestW3CDate($configuration);
    $actual = $plugin->transform($value, $this->migrateExecutable, $this->row, 'testproperty');
    $this->assertSame($expected, $actual);
  }

  /**
   * Tests date conversion with an invalid date string.
   */
  public function testInvalidDate() {
    $plugin = new TestW3CDate($configuration);

    // Prepend '?' to ensure an invalid date in the off chance that the
    // random string returns a valid date format.
    $value = '?' . $this->getRandomGenerator()->string();
    $this->setExpectedException('Drupal\migrate\MigrateException', 'Invalid source date');
    $plugin->transform($value, $this->migrateExecutable, $this->row, 'testproperty');
  }

  /**
   * Data provider for testW3CDate().
   */
  public function providerTestW3CDate() {
    return [
      // Tests D6 date module format.
      ['2009-11-19 17:00:00', '2009-11-19T17:00:00'],
      // Tests UNIX timestamp.
      [574087511, '1988-03-11T12:45:11'],
      // Tests negative UNIX timestamp.
      [-1483151235, '1923-01-01T21:32:45'],
      // Tests human readable date with no time component.
      ['July 4, 1776', '1776-07-04T00:00:00'],
      // Tests human readable date with 12hr time component.
      ['Nov 8, 1980 3:17 p.m.', '1980-11-08T15:17:00'],
    ];
  }

}

class TestW3CDate extends W3CDate {
  public function __construct($configuration) {
    $this->configuration = $configuration;
  }

}

The first test (testW3CDate()) uses the data provider method providerTestW3CDate() to run through a set of example date input and expected output. The second test (testInvalidDate()) makes sure the correct exception is thrown if an invalid date is encountered. Obviously, both of these tests will fail at first because W3CDate::transform() hasn’t actually been implemented. You can now go back and finish the process plugin.

4. Implement W3CDate::transform()

The transform method will use the standard PHP DateTime class to convert an incoming date into the appropriate format. The DateTime constructor only accepts human readable dates so you will need to check if the incoming date is a Unix timestamp. If so, you can use the DateTime::setTimestamp method to initialize the date object, otherwise just pass the incoming value to the DateTime constructor. Do not worry about timezones for this simple example.

If the incoming value cannot be parsed into a date then the DateTime object will throw an exception. You can catch this and rethrow it as a MigrateException with a more useful message (remember that the unit test checks for a MigrateException on invalid input). This will mark the source row as failed and save the message to the database.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
<?php
/**
 * {@inheritdoc}
 */
public function transform($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) {
  try {
    if (is_numeric($value)) {
      $date = new \DateTime();
      $date->setTimestamp($value);
    }
    else {
      $date = new \DateTime($value));
    }
    $value = $date->format('Y-m-d\TH:i:s');
    }
    catch (\Exception $e) {
      throw new MigrateException('Invalid source date.');
    }
    return $value;
  }

More information

I will be presenting at Drupalcamp Colorado 2016. Vote for and attend the session “Migration Season: Moving Your Stuff to Drupal 8” if you want to learn more about Drupal 8 migrations.