tableau / connector-plugin-sdk

SDK for Developing Tableau Connector Plugins

Home Page:https://tableau.github.io/connector-plugin-sdk/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

How to match dayofweek DATEPART with Tableau display

aiguofer opened this issue · comments

About You:
Name: Diego Fernandez
Company: dbt Labs

Your question:
We're building a custom connector for the dbt Semantic Layer. We're a bit of a special case, because all aggregations and transformations are defined in the Semantic Layer, so we disable most transformations and aggregations in Tableau. We take the Tableau generated query and extract the desired metrics, dimensions, and transformations based on the SQL and then turn it into API calls for our Semantic Layer.

I'm working on supporting DATEPART type queries, but I can't get Tableau to display the right thing for dayofweek DATEPART. Here's what Tableau displays:
image

And here's the underlying data:
image

We're currently returning:

Sun = 0
Mon = 1
...
Sat = 6

I tried changing it on our end to return in ISO, but it displays the same thing:

Mon = 1
...
Sat = 6
Sun = 7

I tried adding a transformation to the underlying data by adding 1 to every value, and oddly enough this displays what we want:

Sun = 1
Mon = 2
...
Sat = 6

I also tried changing the Week start in Date Properties for the data source, but that only affects the display order:
image

We are currently making some changes on our end to always return DOW in ISO standard (we support multiple data warehouses, so we'll have to standardize the results from each of them to make sure it's always in the same format). We want to make sure that Tableau always displays this correctly, ideally without having to modify values specifically for tableau.

Are there any settings that we can set in either the manifest.xml or in dialect.tdd files to ensure Tableau treats our returned values correctly?

Internal tracking: W-14244290

It sounds like you may want to try adjusting the <start-of-week-offset value='...' /> in the connector's dialect definition. The default value is 0.

I'd also review the <date-function name='DATEPART'> elements, specifically <formula part='weekday'>, and examples (1, 2):

    <date-function name='DATEPART' return-type='int'>
      ...
      <formula part='weekday'>CAST(EXTRACT(DOW FROM %2) AS INTEGER)</formula> // example
      ...
      <argument type='localstr' />
      <argument type='datetime' />
    </date-function>

There is also TDVT test coverage for this function with test input and expected values based on our standard test data.

Thanks! I hadn't seen start-of-week-offset; I'll give that a try.

    <!--
      start-of-week-offset
      Tableau's start of week functions assume Sunday is 0.
      This is used to specify an offset. For example, if the
      database says Sunday is 1, the offset value should be 1.
      If Sunday is 6, the offset value should be either 6 or -1.
    -->

So in our case where we return ISO Standard (1 = Monday, ..., 7 = Sunday), we should set it to 7?

In our case the formula parts don't really matter... all we're doing is extracting the datepart and doing a mapping on the backend to ensure all platforms return ISO standard.

As for the tests, unfortunately they are impossible for us to use. A dbt Semantic Layer is tied to a dbt project and the semantic models defined within it. There's no way for us to map the test data to something that would make sense to test suite through a dbt project.

If you are unable to run the tests you can still refer to the expect values for the expected range of results (1-7) while experimenting with the function sample from the previous comment.

The start-of-week-offset will only get used with custom start of week functions, so it wouldn't apply to a <formula part='weekday'> used in the primary <date-function> definition.

The start-of-week-offset will only get used with custom start of week functions, so it wouldn't apply to a used in the primary definition.

Ahh ok thanks. So this doesn't really have much to do with my question.

If you are unable to run the tests you can still refer to the expect values for the expected range of results (1-7) while experimenting with the function sample from the previous comment.

I appreciate you linking me to tests but that also didn't answer my question of what Tableau expects. All I got from that it's that it expects numbers between 1 and 7, but nothing regarding which specific day of the week each number correlates to. I already knew what Tableau expects through experimentation, but my question was regarding settings I can make to change that.

So the actual answer to my question is:

  • There's no way to change Tableau's expectations regarding DATEPART('DAYOFWEEK')
  • Tableau expects Sun = 1, Mon = 2, ...., Sat = 7
  • Connector builders should modify the values that are returned by the DB to match using <formula part='weekday'>
  • If they can't use <formula part='weekday'>, they should modify the values to match using some other mechanism (in our case, in flight)

Based on what you're describing it sounds like something like <formula part='weekday'>1 + CAST(EXTRACT(DOW FROM %2) AS INTEGER)</formula> is the right implementation for your data source. You can also have more complex implementations as well like (CASE WHEN EXTRACT(WEEKDAY FROM %2) = 0 THEN 7 ELSE EXTRACT(WEEKDAY FROM %2) END) or something MOD(..., 7) based if that is appropriate.