Agenda
- When do we need to migrate data?
- A quick history of how Drupal has tried
- Feeds overview
- Migrate 2.x overview
- Questions
Typical use cases
- Migration from a previous version of a site prior to launch
- Source may or may not be Drupal
- Periodic imports of data from an external source
- Sources: SQL, XML (e.g. RSS), CSV, HTML
A quick history
- Let's just write one quick script!
- Let's just use CSV! (Node Import)
- Let's just scrape other people's feeds! (FeedAPI; Feed Element Mapper)
- Let's just provide lots of hooks! (Migrate 1.x)
Why not just write a quick script?
- Avoid tweaking source data in your text editor
- Sensibly handle updates to source data and iterative migrations
- Re-import some or all source data as you debug your migration script
- Writing handlers means re-inventing the wheel
Feeds vs. Migrate 2.x
- Both map a variety of data sources to a Drupal data structure
- Update migrated data in place and add new data on subsequent iterations
- Feeds has 22k installs; Migrate has 4k
Feeds
- Sources, targets, mappings are defined via the web frontend
- Feed importers can be defined with less technical skill
- Data imports can be run by end users or on a schedule
- No Drush integration (patches are in development)
Feeds Overview
Fetchers (data sources)
- Core:
- File, HTTP, PubSubHubbub
- Contrib:
- Feeds SQL, LDAP, Directory fetcher (to find new files), etc
Feeds Overview
Parsers (data normalization)
- Core:
- RSS, CSV, OPML, Sitemap
- Contrib:
- XPath Parser, Flicker, YouTube, WordPress, LDAP, etc
Feeds Overview
Processors (data storage in Drupal)
- Core:
- Nodes, taxonomy terms, users, feed nodes
- Contrib:
- Comments, commerce products, database tables
Feeds Tips
feeds_dbg() prints to /tmp/feeds_example_org.log when the
feeds_debug variable is TRUE
Use a Feeds tamper 'Find replace REGEX' to import local images:
REGEX: @.*/(.*)@
Replacement: file:///path/to/feeds/images/$1
Feeds Tips
Use Feeds tamper's 'String to Unix timestamp' to import dates as node creation
or modification times
Associate nodes with their translations using hook_feeds_after_import()
Use the Feeds XPath Parser module with the Chrome XPath Helper extension
Migrate 2.x
- No field mapping via web frontend (must be done in code)
- Assumes experience with object-oriented programming
- Data imports:
- Can be run by end users from web frontend
- Typically run via drush
Migrate 2.x Overview
- Create a class which extends the Migration class (MyMigration)
- Extend this to handle your data types (MyTermMigration, MyUserMigration, MyNodeMigration)
Creating a Migrate Class
- Define a data source and its fields (SQL or XPath queries)
- Define a destination and set of field mappings
- Alter data or skip rows as needed in prepareRow()
- Alter Drupal nodes before they're saved in prepare()
- Alter Drupal nodes after they're saved in complete()
Creating a Migrate Class
- Specify dependencies (e.g., users before nodes)
- Mark unmapped source and destination fields
- Reference previously mapped fields
- Reference issues in your bugtracker
- Create placeholder nodes to allow for circular node references
Learning to use Migrate
- Creates taxonomies and content types
- Imports nodes, users, comments, terms from SQL & XML sample data