`failed to populate column 'filter'` error when left joining `turbot_control` to `turbot_resource`

Question

`failed to populate column 'filter'` error when left joining `turbot_control` to `turbot_resource`

Joeturbot opened this issue 3 years ago · comments

Describe the bug
The filter column fails to populate with an unusual error.

Error: 2 connections failed: 
connection 'taurus': rpc error: code = Unknown desc = failed to populate column 'filter': FromQual transform can only be called if there is a singe equals qual for the given column
connection 'canis': rpc error: code = Unknown desc = failed to populate column 'filter': FromQual transform can only be called if there is a singe equals qual for the given column (SQLSTATE HV000)

Steampipe version

❯ steampipe --version
steampipe version 0.11.0

To reproduce

Run this query against any single or multiple Turbot workspaces. I'm running TE 5.37.21.

with workspace as (
    select workspace
    from turbot_resource
    where filter = 'resourceId:"tmod:@turbot/turbot#/" level:self'
),
     webhook_rotation_policy as (
         select value,
                workspace,
                filter
         from turbot_policy_setting
         where filter = 'policyTypeId:"tmod:@turbot/turbot#/policy/types/webhookSecretRotation"'
     )
select w.workspace as resource,
       e.filter,
       case
           when e.value is null then 'Skip'
           else e.value
           end     as webhook_rotation_setting,
       case
           when e.value like 'Enforce: Rotate webhook secret' then 'ok'
           else 'alarm'
           end     as status,
       case
           when e.value like 'Enforce: Rotate webhook secret'
               then 'Webhook Secrets in ' || w.workspace || ' are set to rotate.'
           else 'Webhook Secrets in ' || w.workspace || ' do not rotate.'
           end     as status
from workspace w
         left join webhook_rotation_policy e using (workspace)

I've tried running it against multiple workspace and against individual workspaces. There's no change in error.
I've tried the webhook_rotation_policy query alone and it works just fine. The workspace query returns properly too. My hunch is that filter error comes from joining the two queries.

Expected behavior
I get a list of workspaces with a column indicating whether the Webhook rotation policy has been set to Enforce or not.

Additional context
Working on a Turbot Workspace health mod. This query is a part of that effort.

Jon Udell · Answer 1 · Wed Dec 29 2021 08:34:08 GMT+0800 (China Standard Time)

This reproduces the error for me.

with workspace as (
  select 
    workspace
  from 
    turbot_resource
  where 
    filter = 'resourceId:"tmod:@turbot/turbot#/" level:self'
),

rotation_policy as (
  select 
    workspace,
    value,
    filter
  from 
    turbot_policy_setting
  where 
    filter = 'policyTypeId:"tmod:@turbot/turbot#/policy/types/webhookSecretRotation"'
  ),

joined as (
  select
    *
from
  workspace
left join
  rotation_policy
using
  (workspace)
)

select  * from joined

Adding an order by to the rotation_policy CTE resolves it.

One more reference for turbot/steampipe#968, probaby.

Joe Michela · Answer 2 · Fri Dec 31 2021 01:36:07 GMT+0800 (China Standard Time)

Thanks @judell I never would have thought to use order by. I'll give it a try.

Joe Michela · Answer 3 · Sat Jan 01 2022 08:28:31 GMT+0800 (China Standard Time)

Yep, use of order by solves the problem. Query works as expected now.

github-actions · Answer 4 · Fri Apr 29 2022 07:35:03 GMT+0800 (China Standard Time)

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 30 days.