Contents

Introduction

  • Importing and merging advertiser's platforms' data
  • Requirements, installation & updates
  • Limitations
  • Changelog
  • License
  • Support

Tracking params, importing and merging

  • Google AdWords
  • Microsoft Bing Ads
  • Criteo
  • Facebook Ads
  • Taboola
  • Individual campaigns
  • Other advertising platforms and Piwik's default tracking params

API

  • getVisits
  • getEcommerceOrderWithVisits
  • getEcommerceOrdersWithVisits
  • getMarketingPerformance
  • getStatus

Extending AOM

  • Events

Quality assurance and monitoring

  • Logging
  • Monitoring

Plugin development

  • Tests

Piwik AOM Plugin Documentation

Integrate additional data like campaign names, ad impressions, costs etc. from advertising platforms (such as Google AdWords, Microsoft Bing, Criteo, Facebook Ads and Taboola) and individual campaigns (such as cost per view/click/acquisition and fixed price per months deals) into Piwik and combine that data with individual visits - create a whole bunch of new opportunities!

Importing and merging advertiser's platforms' data

To obtain data (campaign names, costs, ad impressions, etc.) from the advertising platforms, API access must be granted and configured within the settings of this plugin.

To map individual visits with the data from the advertising platforms, all links from the platforms must have additional params that supply mapping-related data like campaign and ad IDs to the Piwik AOM plugin. This data is stored in the database columns log_visit.aom_platform and log_visit.aom_ad_params. When a visitor returns with other tracking params, a new visit is started automatically.

After you have set up a cronjob ./console aom:process that gets executed every minute a copy of every single Piwik visit is written (and conversions are added) to aom_visits.

After you have set up auto archiving the following two tasks will be executed by AOM:

1. Import data from advertising platforms

Imported data is stored in the database table aom_{platformName}. Every advertising platform's data is aggregated on a daily basis in the timezone of the advertising platform's account. Merged data will only be correct, when your website's timezone matches the timezones of each advertising platform account.

2. Merge imported data with individual visits

The background job iterates over all imported advertising platform data and updates the visits in the aom_visits table with that data. If no real Piwik visits match the advertising platform's data, an artificial visit is being created in the aom_visits table to make sure that all cost is allocated.

The data in aom_visits contains both all Piwik visits and all costs. It is used in the marketing performance report in the Piwik UI and it can be used via the API endpoints provided by the Piwik AOM plugin (e.g. for further processing in a data warehouse).

Whenever a visit is added to or updated in the aom_visits table, an AOM.aomVisitAddedOrUpdated event is posted. You can listen to that event within your separate plugin to integrate Piwik AOM in your event oriented architecture (e.g. to publish those events to external systems or message brokers).

Manual importing, merging & visit reprocessing

You can (re)import data manually by executing ./console aom:import --platform={platformName} --startDate=YYYY-MM-DD --endDate=YYYY-MM-DD. Reimporting data deletes all previously imported data for the given platform and period, but it does not delete any data in aom_visits (it only adds and updates to that table). Adding --merge=true merges the newly imported data immediately after the import has finished.

You can merge imported data manually by executing ./console aom:merge --platform={platformName} --startDate=YYYY-MM-DD --endDate=YYYY-MM-DD.

Requirements, installation & updates

MySQL 5.0.3 or higher is required as we store more than 255 characters in VARCHAR. As we use MySQL multi row inserts and updates for a better performance make sure that your max_allowed_packet size is high enough (this roughly depends on how much visits you have).

You must be able to run cron jobs to set up both auto archiving and the background job ./console aom:process (execution of the latter every minute) that updates visits and conversions in the aom_visits table.

You can install the Piwik AOM plugin from the Piwik marketplace.

Set up auto archiving to automatically import and merge data from the advertiser's platforms. The auto archiving cron job executes the core:archive command which triggers Piwik's TaskScheduler and thus this plugin's tasks (see Tasks.php). You can trigger all of this plugin's background jobs manually by executing ./console core:run-scheduled-tasks --force.

Merging will only be correct, when your website's timezone matches the timezones of each advertising platform account.

Add the params that are required to assign imported data to individual visits to all links managed in the advertising platforms as described here.

Limitations

Advertising platforms

All data is being processed on a daily basis, i.e. variations of click costs during the day are not being considered.

Google AdWords

Visits of Google display network are assigned on AdGroup level.

Performance

This plugin has successfully been tested with up to 25k visitors a day for a period of more than two years, running on an AWS server with 4 GB RAM and a separate AWS RDS MySQL database. Single PHP processes might use up to 1 GB RAM and run for minutes. You might encounter performance issues during importing, merging, accessing the plugin's marketing performance report in the UI or using the plugin's API when you have more traffic or a an infrastructure with less performance.

Changelog

1.2.0 on 21 August 2017 and afterwards

1.1.0 on 24 July 2017 and afterwards

1.0.0 on 11 July 2017 and afterwards

  • Restructures the entire Piwik AOM plugin core from lots of batch jobs to an event oriented processing (and introduces the AOM.aomVisitAddedOrUpdated event for added and updated visits)
  • Breaking backwards compatibility: AdWords auto-tagging (gclid) is now the only supported AdWords tracking method
  • Breaking backwards compatibility: Renamed status API endpoint to getAomVisitsStatus(...)
  • Adds foundations for individual campaigns (still in development, cannot be used yet)
  • Rewrote various tests
  • The implementation of Facebook Ads is deprecated and does not work at the moment

0.6.* on 17 April 2017 and afterwards

  • Adds support for Taboola

0.5.* on 17 February 2017 and afterwards

  • Adds support for Piwik 3

0.4.* on 16 November 2016 and afterwards

  • Adds gclid as alternative AdWords tracking method
  • Updates AdWords and Facebook Ads API
  • Adds unique keys to avoid duplicates an imported data
  • Adds matchtype to AdWords data

0.3.* on 9 June 2016 and afterwards

  • Adds drill-downs into marketing performance reports (campaigns, ad groups)
  • Fixes [prefixed database table names](https://github.com/advanced-online-marketing/AOM/issues/19) and updates dependencies (libraries of AdWords and Facebook Ads)

0.2.* on 22 May 2016 and afterwards

  • Identifies AOM tracking params from referrer URL in addition to action URL
  • Integrates Piwik AOM data into Piwik UI
  • Fixes various bugs

License

GPL v3 / fair use

Support

Please report bugs and suggest features at github.

Leave comments or questions regarding this documentation below.

If you require intense consulting or like to sponsor the development of a specific features, get in touch!

Found a bug, got a questions or have a fabulous idea?

Contact

We hope you find this Piwik plugin useful.
Feel free to get in touch if you have any questions or suggestions.

Got questions about Piwik AOM or suggestions how to improve?

Feel free to create an issue at github or contact me.

André Kolell
Online Marketing and Web Development Enthusiast

Want to get more out of Piwik AOM?

You miss a feature or like intense support on your individual Piwik setup?
Get in touch with me and let's work out how we can make things going.

Daniel Stonies
Passionate Web Developer and IT Project Manager