matomo-org / matomo

Empowering People Ethically with the leading open source alternative to Google Analytics that gives you full control over your data. Matomo lets you easily collect data from websites & apps and visualise this data and extract insights. Privacy is built-in. Liberating Web Analytics. Star us on Github? +1. And we love Pull Requests!

Home Page:https://matomo.org/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[GDPR issue / feature update request] New disableCampaignParameters() function doesn't work as intended

SW-Vincent opened this issue · comments

disableCampaignParameters() function removes UTMs from the URL within Matomo tracking request.
This means the issue is double :

  • it (consent) cannot change during the visit
  • visits with / without consent cannot be differentiated.

Summary

Using this feature as it is now will blend Paid and Organic trafic sources without any way to tell them apart, which is a big issue for most of my customers.
The best case scenario would be :

  • to count UTM visits within campaign source, weither consent has been given or not; with a dummy campaign name, media and source ;
  • to treat consent as a visit-level dimension, which means it is stored within a cookie and the last value of the cookie within the visit will count for the whole visit (therefore if i consent after the pageview hit, UTM will still be tracked) ;

An other decent scenario would be to still flood Organic data with paid data but give a dedicated visit-level dimension to tell them apart.
There is no workaround with custom dimensions as it is now, because a visit-level custom dimension only considers its last value of the visit, while Matomo only handles UTM as the first pageview of the visit.
If this is not acceptable in regards to GDPR, then Piano Analytics is in trouble because it has been a feature for some times (called hybrid consent).

Why are the above elements required ?
Matomo pageview is expected to be sent before consent is given, as its main purpose in France is to be part of CNIL consent exemptions (which requires some of Matomo functions, such as e-commerce, Heatmaps and UTMs to be disabled or to require consent).
Therefore, any element that requires consent -such as UTMs- needs to be able to change during the visit (this can be done for heatmaps and session recordings, but this is a special case).
Delaying pageview until consent is given doesn't look like an acceptable workaround (plus it wouldn't solve tge "telling paid and organic apart" issue).

Your Environment

  • Matomo Version: Mostly Cloud
  • PHP Version:
  • Server Operating System:
  • Additionally installed plugins:

Hi @SW-Vincent.
Thanks for creating this issue.
The feature was meant for pages that would like to track fully without consent only. In that case it might be required to not track any campaigns at all.

It is currently not possible to update the referer after the first page view. Therefore you may need to request consent before the first page view and disable campaign parameters only if consent is not provided.

Using a dummy campaign instead of fully removing the parameters would be something we could implement. But I'm unable to say if that would comply with certain privacy laws in case no consent was given.

Hi,

Thanks for your answer.

The feature was meant for pages that would like to track fully without consent only. In that case it might be required to not track any campaigns at all.

GDPR requires to not track UTMs at all or require consent for them.
As things are now we can either be GDPR compliant or have an acceptable level of measurement, but not both as UTMs are consided a basic need for most of our 15 customers under Matomo (and some consent-exempted analytics tools have dedicated options that are satisfying).

It is currently not possible to update the referer after the first page view. Therefore you may need to request consent before the first page view and disable campaign parameters only if consent is not provided.

We are fully aware that it is not currently possible to update the referer after the first pageview, and this is precisely the feature we are askin for (in addintion for a way to differenciate campaign and organic trafic).
This is why i was asking for UTMs to be processed like a visit-level custom dimension, which would probably require significant changes to Matomo.

Using a dummy campaign instead of fully removing the parameters would be something we could implement. But I'm unable to say if that would comply with certain privacy laws in case no consent was given.

I can ask CNIL about that, but i do not have a dedicated contact so i expect a few months would pass-by before i get an answer.

@sgiehl I got the chance to have a phone call with @SW-Vincent .
So here is the situation, it is clearly written on CNIL website that the use of UTM parameters has to be under a consent mode.
Correct me if i am wrong @SW-Vincent but what we would like to achieve here with Matomo is this:

Three tracking modes:

  • Opt-in: Users agree to be tracked. By clicking on a consent banner saying explicitly yes.
  • Opt-out: Users refuse to be tracked. By clicking on a consent banner saying explicitely no.
  • Hybrid (consent exemption): The visitor will be tracked unless he opted out. In this hybrid mode all functionalities are available except for visit log, ecommerce, user ID, and UTMs cannot be collected. However, if they opt-in, then it should be available.

We need to find a way to:

  • indicate when the visitor is in hybrid mode or in opt-in mode.
  • as by default, the utm cannot be collected, how could it push back to the system if the visitor opt-in?

--> Did I sum it up well? @SW-Vincent

@sgiehl what do you think about it?

Thanks @Chardonneaur, that's exactly the overall idea !

If i may add / specify a few points :

  • i didn't find clear occurrences on the CNIL website that UTMs are overall considered personal data (although it is explicited on some specific cases like proxyfication), but CNIL explicitly told me so during email exchanges (find details* at the end of the comment) ;
  • some users may want to configure which features are / are not within exemption mode
  • heatmaps / session recordings / youtube video tracking may also be excluded of hybrid (consent exemption) mode

indicate when the visitor is in hybrid mode or in opt-in mode.

  • Hybrid and opt-in consent status must be tracked in order to know the sample proportion of data with consent and filter data with opt-in consent (otherwise some data wont make sense at all) ;
  • This is very Matomo-specific but how consent status must be aggregated within reports must be thought-of carefully (otherwise most basic informations will have to be accessed through custom reports) ;

From my point of view, this issue looks very compatible with how visit-level custom dimensions work, so i assume the "easiest" way to solve this would be to store both consent and any information that would require consent (within opt-in scope but not hybrid scope) within a cookie that Matomo would read as a visit-level dimension (i.e. value for the last hit of the session is value for the session).
It makes sense for UTMs at least but i'm not sure this is the perfect fit for visit log for example as i am not aware of the full technical process.

Translated CNIL message* :

The conditions under which audience measurement can be exempt from consent under Article 82 are defined in paragraphs 50, 51, and 52 of the CNIL guidelines on cookies and other trackers. These guidelines specifically state:
"These trackers must not enable the overall tracking of the person’s navigation across different applications or websites."
This is why, in its program to identify audience measurement solutions that can be used without consent, the CNIL has reiterated this criterion, specifying that "Any solution using the same identifier across multiple sites (e.g., cookies placed on a third-party domain loaded by multiple sites) to cross-reference, deduplicate, or measure a unified 'reach' of content is excluded."
Therefore, the use of UTM cannot be considered within the context of the consent exemptions of Article 82. While collecting the "referrer" at the "host" level without consent is permissible, any collection of such identifiers for business contribution measurement is excluded. It remains possible, but user consent will be required.

Thank you @SW-Vincent for creating this issue 🙌

I have a question regarding (and some consent-exempted analytics tools have dedicated options that are satisfying).

-> would you be able to share more information, how other tools have dealt with this issue? What makes their options satisfying? Is their model something we could follow also in Matomo? (I'm curious what it would look like)

Btw regarding this suggestion Hybrid and opt-in consent status must be tracked in order to know the sample proportion of data with consent and filter data with opt-in consent (otherwise some data wont make sense at all) ; -> this is (partially) being discussed in #16363

Regarding

From my point of view, this issue looks very compatible with how visit-level custom dimensions work, so i assume the "easiest" way to solve this would be to store both consent and any information that would require consent (within opt-in scope but not hybrid scope) within a cookie that Matomo would read as a visit-level dimension

As far as I understood, if it's not allowed to track the UTM parameters without consent, then similarly it may not be allowed to track the UTM parameters in a cookie... and if we can't store it in a cookie, then when the visitor consents later on another page, we've already lost the UTM parameters... That's why I'd be interested to see how other tools handle this 👍

Hi @mattab ,

Here are some examples of similar process's for other solutions.
Note that i do not know how it technically works for the solution, i can only describe how it works for whoever is in charge of implementation.
Additionnally, the main specificity for Matomo wouldn't be data collection but data agregation, considering a new dimension (with the value of consent) needs to be collected and agregated.

Lets start with the closest case scenario : Piano Analytics (i'll talk about the "old" consent mode as it has become more complex recently) :

  • there are 3 mode of consent (optin, optout, exempted) :
    • optin : all data is collected, consent value is stored
    • optout : no data is collected (consent value is not stored as nothing is)
    • exempted : data is collected without properties that would require consent (e.g. e-commerce, UTMs -it's beenn a thing for Piano for years apparently-, etc.), consent value is stored
  • The default behavior of the exempted mode is defined when initializing the tracker, before the first tag is sent (it would be defined within the configuration variable for Matomo)
  • Simple commands can switch from one mode to the other ;
  • The whole point is sending the exempted hit before consent is evaluated, additionnally it makes consent easier to manage as tags and consent evaluation dont need to be synchrone anymore in the asynchronous world that tag management is
  • When consent mode switch from "exempted" to "optin", a new hit is sent with all the informations and the same event ID. I assume event ID is the key to de-duplicate the hits.
  • Consent seems to have a page scope : if consent switch from exempted to granted at the 2nd page of the session, the corrective hits for the first page wont be sent (unless it is a single page application).

Then we have Google Consent Mode.
Whatever we think about its GDPR compliance, it has a similar functiunning (although it has a different purpose).

  • several purposes are set (analytics, marketing, etc.), with usually a default value of "denied". The request will be sent with a query parameter containing the consent status (G100 for no consent, G111, etc. if i am not mistaken).
  • all google tags (analytics, ads, conversion linker, etc.) will adapt the data sent based on the consent status they will automatically read
  • The default behavior has to be defined before any Google Tag is sent, but the "denied" version of tags will trigger even before consent is evaluated
  • Simple commands can change consent mode for each purpose ;
  • It makes consent easier to manage, as tags and consent evaluation dont need to be synchrone anymore in the asynchronous world that tag management is
  • When consent is granted, a new hit is sent with all the informations (and i identify some query parameters that looks like an event ID and does not change when consent is granted, so i assume it works the same way)
  • Consent seems to have a page scope : if consent switch from denied to granted at the 2nd page of the session, the corrective hits for the first page wont be sent (unless it is a single page application).

Then we have the Meta equivalent for consent mode.
It is slightly different as it is still binary (granted : full hit, denied : nothing), but it works in a similar way as thanks to it users can handle tags and consent asynchronously and switch from one consent mode to the other with a simple line of code.

Then we have Beyable.
It is quite peculiar, as it is composed of several modules, each one requiring a different tracking code (ex : analytics and ecommerce) and therefore each consent can be evaluated separately.
The Analytics module is not supposed to require consent but just as Matomo does not handle the UTMs problematic at the moment.

Let me know if you need some more details or if the details i provided are unclear.

Regarding

From my point of view, this issue looks very compatible with how visit-level custom dimensions work, so i assume the "easiest" way to solve this would be to store both consent and any information that would require consent (within opt-in scope but not hybrid scope) within a cookie that Matomo would read as a visit-level dimension

As far as I understood, if it's not allowed to track the UTM parameters without consent, then similarly it may not be allowed to track the UTM parameters in a cookie... and if we can't store it in a cookie, then when the visitor consents later on another page, we've already lost the UTM parameters... That's why I'd be interested to see how other tools handle this 👍

As only the last value of consent for the session would impact wether UTMs will be stored within Matomo or not (which would only be the case when consent is granted at the moment of the last action of the visit) i dont think this leads to a GDPR issue but that's ti be confirmed.

On the other hand, examples of the market (see my above message) have a page-level consent rather than a visit-level consent, which doesn't require any cookie.