HA 117 : template value should be a string for dictionary value @ data['attributes_template']
Mariusthvdb opened this issue · comments
please see community post
https://community.home-assistant.io/t/template-value-should-be-a-string-for-dictionary-value-data-attributes-template/238153
template value should be a string for dictionary value @ data['attributes_template']
how should we rewrite this:
- service: variable.set_variable
data:
variable: last_motion
attributes_template: >
{
"history_1":"{{states('variable.last_motion')}}",
"history_2":"{{state_attr('variable.last_motion','history_1')}}",
"history_3":"{{state_attr('variable.last_motion','history_2')}}",
"history_4":"{{state_attr('variable.last_motion','history_3')}}",
"history_5":"{{state_attr('variable.last_motion','history_4')}}",
"history_6":"{{state_attr('variable.last_motion','history_5')}}",
"history_7":"{{state_attr('variable.last_motion','history_6')}}",
"history_8":"{{state_attr('variable.last_motion','history_7')}}",
"history_9":"{{state_attr('variable.last_motion','history_8')}}",
"history_10":"{{state_attr('variable.last_motion','history_9')}}"
}
value: >
{{trigger.to_state.name|replace(' sensor motion','')}}:
{{as_timestamp(trigger.from_state.last_changed)|timestamp_custom('%X')}}
or, the CC should be adapted to use correct data type? not sure, so please have a look?
thanks!
The 0.117 beta adds a breaking change to templates that affects your component. Basically, they are changing it so templates output valid python data types. Up until now templates always output a string, now they will output: int, float, dict, etc.
This change causes a data type error with services that accept JSON in template fields, like mqtt.publish and rest_command.
Paulus and Frenck are working on solutions for this issue. However, based on their current tasks, it looks like Paulus is making changes to mqtt.publish to allow for JSON to be allowed.
Reference:
- 0.117 Draft Release Notes > Native Types Support for Templates
- 0.117 Draft Release Notes > Breaking Changes > Template - native types
- Error when return a json as payload_template in automation actions
#42203 home-assistant/core
home-assistant/core#42398 - Allow skip parsing template result #42401 home-assistant/core
yeah, thanks. Im following that.
Any thoughts on how this new solution would affect the Variable integration here?
No, it will definitely not solve it for this custom integration.
is there any way we could rewrite the above template using the new techniques? of course we all want to not have to use the legacy_templates: true
attributes_template here seems to be identical with payloads in the Mqtt and rest_command services.
No, it is not identical.
If I have a small moment of time left, I can see if I can do a PR. Basically, the whole JSON magic done here isn't needed anymore. Home Assistant now just does this all natively.
wow, that would be much appreciated Frenck, thanks!
looking forward to that, and especially to HA core doing this natively, that would really great news!
I don't think I can make a PR today, but... someone could try the following:
hass-variables/custom_components/variable/__init__.py
Lines 219 to 223 in fd9b44e
and change that to not use the JSON dumping:
attributes = attributes_template.async_render(
{"variable": current_state}
)Now you should be able to use native template results (e.g., real dictionaries).
As said in discord ^ is not working.
please elaborate on what is not working, and what you mean by that.
Have you tried Frenck's edit, and if yes, are there any errors, or results at all? Have you tested on HA 117.0b5?
Changed the lines to this:
try:
attributes = attributes_template.async_render(
{"variable": current_state}
)
All tested on 0.117b5 which fixed at least some other problems with my templates in my automations. But this didn't get fixed with this. Still getting:
File "/usr/src/homeassistant/homeassistant/components/automation/__init__.py", line 426, in async_trigger
await self.action_script.async_run(
File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 1010, in async_run
await asyncio.shield(run.async_run())
File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 245, in async_run
await self._async_step(log_exceptions=False)
File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 253, in _async_step
await getattr(
File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 460, in _async_call_service_step
await service_task
File "/usr/src/homeassistant/homeassistant/core.py", line 1402, in async_call
processed_data = handler.schema(service_data)
File "/usr/local/lib/python3.8/site-packages/voluptuous/schema_builder.py", line 272, in __call__
return self._compiled([], data)
File "/usr/local/lib/python3.8/site-packages/voluptuous/schema_builder.py", line 594, in validate_dict
return base_validate(path, iteritems(data), out)
File "/usr/local/lib/python3.8/site-packages/voluptuous/schema_builder.py", line 432, in validate_mapping
raise er.MultipleInvalid(errors)
voluptuous.error.MultipleInvalid: template value should be a string for dictionary value @ data['attributes_template']
well, it could still be a HA issue. I have a few other templates that after updating to ha 117.0b5 still don't work. Of course it might very well be I havent found the correct syntax yet, (making that a user error) but I have tried all of the suggested options, and still no luck unfortunately.
So, it might be best before PR'ing this integration that works very well still, we need to get the core HA issues out of the way.
Just a guess... replace attributes_template with just attributes in your service call (same for value_template -> value).
Yup. That actually did it!
ha, progress!
I take it this works in conjunction with the easier change Frenck suggested in the CC itself?
also, since the posted template above doesnt use a value_template, the only thing we need to change would be
- service: variable.set_variable
data:
variable: last_motion
attributes: >
{
"history_1":"{{states('variable.last_motion')}}",
"history_2":"{{state_attr('variable.last_motion','history_1')}}",
etcetc
"history_9":"{{state_attr('variable.last_motion','history_8')}}",
"history_10":"{{state_attr('variable.last_motion','history_9')}}"
}
value: >
{{trigger.to_state.name|replace(' sensor motion','')}}:
{{as_timestamp(trigger.from_state.last_changed)|timestamp_custom('%X')}}
?
Yup! Also you can make the suggested change in the init or just leave it out. Makes no difference.
so to conclude and for reference:
using Ha pre 117: don't change the CC and use attributes_template: in the service-call, and use legacy_templates: true
using HA 117.+: change the CC as Frenck suggests here and change to use attributes: in the service-call
with that we can close this issue I suppose.
Thanks for the support!
Actually:
- service: variable.set_variable
data:
variable: last_motion
attributes:
history_1: "{{ states('variable.last_motion') }}"
history_2: "{{ state_attr('variable.last_motion','history_1') }}"
history_9: "{{state_attr('variable.last_motion','history_8')}}"
history_10: "{{state_attr('variable.last_motion','history_9')}}"
value: >
{{trigger.to_state.name|replace(' sensor motion','')}}:
{{as_timestamp(trigger.from_state.last_changed)|timestamp_custom('%X')}}No need to fiddle with JSON structures anymore.
was too quick about closing this issue of course...
if we now can use attributes: instead of attributes_template:, isn't everything on attributes_template in the code of the CC no longer used? Iow, shouldn't this be taken out completely, as in rewritten....
well, the thing is, since Home Assistant 0.115, templates in service calls are pre-processed.
So, the processing done in this custom integration is actually useless. So attributes_template and value_template are just wasting CPU cycles since 0.115 and cause trouble as of 0.117 (because it tries to process a dictionary as a template string).
Well, that's clear ;-)
if only we had a HA native way of doing this, we wouldn't need to resort to the CC in the first place. Its really something we need without having to use all sorts of input_text's to store the information, if at all possible like that in the first place.
Variables should not be stored in the state machine like this. While helpful, the design of this integration just causes pollution in the state machine.
Something is either a variable or just an actual state. This mixes both.
Yeea. So I removed all the unnecessary template stuff and fixed the readme, examples etc. But idk if we want to stay on this repository since he seems to have abandoned it.
With all the removed stuff and leaving out the json structure it's still working just fine.
yes, I understand that, it has been stated like that before.
That's why I said, if only we could do this in a native core way. Not meaning the exact same implementation of course. I meant a valid Core way of (re/)storing and showing past states of an entity.
I understand the design of this integrity is not following HA design principles, so it would have to be a completely new way of doing things. Maybe in a python script, maybe a new core domain entity.
I would greatly appreciate any advice you'd give us for this functionality to be done better and in core.
Well for the time being this is the cropped down version with updated docs and the other PR's.
that's nice of course, thanks, but did you reach out to the author of this CC? Open source as it may be, it's still his project...
I just wrote him an E-Mail but no answer yet. For now it's just a fork with everything fixed.
just as a final note/question/issue on this:
I have tried this with the new CC, and adapted automation service with the format Frenck suggests here, but that results in:
2020-10-27 23:14:43 ERROR (MainThread) [homeassistant.components.automation.update_last_motion] While executing automation automation.update_last_motion
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/components/automation/__init__.py", line 426, in async_trigger
await self.action_script.async_run(
File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 1010, in async_run
await asyncio.shield(run.async_run())
File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 245, in async_run
await self._async_step(log_exceptions=False)
File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 253, in _async_step
await getattr(
File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 460, in _async_call_service_step
await service_task
File "/usr/src/homeassistant/homeassistant/core.py", line 1402, in async_call
processed_data = handler.schema(service_data)
File "/usr/local/lib/python3.8/site-packages/voluptuous/schema_builder.py", line 272, in __call__
return self._compiled([], data)
File "/usr/local/lib/python3.8/site-packages/voluptuous/schema_builder.py", line 594, in validate_dict
return base_validate(path, iteritems(data), out)
File "/usr/local/lib/python3.8/site-packages/voluptuous/schema_builder.py", line 432, in validate_mapping
raise er.MultipleInvalid(errors)
voluptuous.error.MultipleInvalid: expected dict for dictionary value @ data['attributes']
Changing the service to:
- service: variable.set_variable
data:
variable: last_motion
attributes: >
{
"history_1":"{{states('variable.last_motion')}}",
"history_2":"{{state_attr('variable.last_motion','history_1')}}",
"history_3":"{{state_attr('variable.last_motion','history_2')}}",
"history_4":"{{state_attr('variable.last_motion','history_3')}}",
"history_5":"{{state_attr('variable.last_motion','history_4')}}",
"history_6":"{{state_attr('variable.last_motion','history_5')}}",
"history_7":"{{state_attr('variable.last_motion','history_6')}}",
"history_8":"{{state_attr('variable.last_motion','history_7')}}",
"history_9":"{{state_attr('variable.last_motion','history_8')}}",
"history_10":"{{state_attr('variable.last_motion','history_9')}}"
}
value: >
{{trigger.to_state.name|replace(' sensor motion','')}}:
{{as_timestamp(trigger.from_state.last_changed)|timestamp_custom('%X')}}
makes the service run fine again, like before, and save the states properly.
if this is of importance to the way HA handles these templates, please note Frenck? Not asking you to fix this CC (Ive got it working now), but again, if this is of core HA relevance, just letting you know here.
How did your service looked when it didnt work?
as posted above, following Frenck advice here: #47 (comment)
of course filled out from 1-10 ;-)
Well because for me it works.
action:
- service: variable.set_variable
data:
variable: last_motion
attributes:
history_1: "{{states('variable.last_motion')}}"
history_2: "{{state_attr('variable.last_motion','history_1')}}"
history_3: "{{state_attr('variable.last_motion','history_2')}}"
history_4: "{{state_attr('variable.last_motion','history_3')}}"
history_5: "{{state_attr('variable.last_motion','history_4')}}"
history_6: "{{state_attr('variable.last_motion','history_5')}}"
history_7: "{{state_attr('variable.last_motion','history_6')}}"
value: >
{{trigger.to_state.name|replace('Motion Sensor','')}}
ok, let me try again, be right back.
heck, it does work now!....
what I might have forgotten to take out where the commas at the end of the lines in the variables. Not really sure.
Anyways, its working now.
must remember that both
- service: variable.set_variable
data:
variable: last_motion
attributes: >
{
"history_1":"{{states('variable.last_motion')}}",
"history_2":"{{state_attr('variable.last_motion','history_1')}}",
"history_3":"{{state_attr('variable.last_motion','history_2')}}",
"history_4":"{{state_attr('variable.last_motion','history_3')}}",
"history_5":"{{state_attr('variable.last_motion','history_4')}}",
"history_6":"{{state_attr('variable.last_motion','history_5')}}",
"history_7":"{{state_attr('variable.last_motion','history_6')}}",
"history_8":"{{state_attr('variable.last_motion','history_7')}}",
"history_9":"{{state_attr('variable.last_motion','history_8')}}",
"history_10":"{{state_attr('variable.last_motion','history_9')}}"
}
value: >
{{trigger.to_state.name|replace(' sensor motion','')}}:
{{as_timestamp(trigger.from_state.last_changed)|timestamp_custom('%X')}}
and
- service: variable.set_variable
data:
variable: last_motion
attributes:
history_1: "{{states('variable.last_motion')}}"
history_2: "{{state_attr('variable.last_motion','history_1')}}"
history_3: "{{state_attr('variable.last_motion','history_2')}}"
history_4: "{{state_attr('variable.last_motion','history_3')}}"
history_5: "{{state_attr('variable.last_motion','history_4')}}"
history_6: "{{state_attr('variable.last_motion','history_5')}}"
history_7: "{{state_attr('variable.last_motion','history_6')}}"
history_8: "{{state_attr('variable.last_motion','history_7')}}"
history_9: "{{state_attr('variable.last_motion','history_8')}}"
history_10: "{{state_attr('variable.last_motion','history_9')}}"
value: >
{{trigger.to_state.name|replace(' sensor motion','')}}:
{{as_timestamp(trigger.from_state.last_changed)|timestamp_custom('%X')}}
are identical in outcome, now must think of why and how that is
are identical in outcome, now must think of why and how that is
I think it's because they are both dictionaries, so they essentially are the same thing.
Thanks for doing the legwork on this one. I spent more time than I care to admit trying to resolve a similar issue before I finally thought to check the git to see if anyone else had the same problem.
My case was complicated by the fact that my attributes list is dynamic, but I got 'er all worked out with the fixes posted here.
- service: variable.set_variable
data:
variable: tts_saved_messages
value: "{{ states('variable.tts_saved_messages')|int + 1 }}"
attributes: >
{%- set count = namespace(value=0) -%}
{%- set found = namespace(value=0) -%}
{%- set msg = namespace(value='') -%}
{%- for attr in states.variable.tts_saved_messages.attributes -%}
{%- set count.value = count.value|int + 1 %}
{%- if state_attr('variable.tts_saved_messages',attr).split('@')[0]|trim != play_message %}
{% set msg.value = msg.value ~ '"msg' ~ count.value ~ '":"' ~ (state_attr('variable.tts_saved_messages',attr)) ~ '"' ~ ',' %}
{%- else -%}
{%- set found.value = count.value -%}
{%- endif -%}
{%- endfor -%}
{%- if found.value|int > 0 -%}
{%- set msg.value = '"msg' ~ count.value|int ~ '":"' ~ play_message ~ '@' ~ as_timestamp( now() )|timestamp_custom('%_I:%M %p',true) ~ '"' -%}
{%- elif count.value|int > 0 -%}
{%- set count.value = count.value|int + 1 %}
{%- set msg.value = msg.value ~ '"msg' ~ count.value ~ '":"' ~ play_message ~ '@' ~ as_timestamp( now() )|timestamp_custom('%_I:%M %p',true) ~ '"' -%}
{% else %}
{%- set msg.value = '"msg1":"' ~ play_message ~ '@' ~ as_timestamp( now() )|timestamp_custom('%_I:%M %p',true) ~ '"' -%}
{%- endif -%}
{{ '{ ' ~ msg.value ~ ' }' }}
replace_attributes: >
{%- set found = namespace(value=0) -%}
{%- for attr in states.variable.tts_saved_messages.attributes -%}
{%- if play_message == state_attr('variable.tts_saved_messages',attr).split('@')[0]|trim %}{%- set found.value = 1 -%}{%- endif -%}
{%- endfor -%}
{{ true if found.value|int == 0 else false }}
that's an interesting variable (template), and more over, you make me see something I've never seen before, the replace_attributes:
must explore the functionality of that, to see what it does exactly.
btw, closing this really now, as the original issue is solved, not in this repos documentation, but in HA core templating, and Frenck's instructions on how to write this Integration's config in yaml.