Database corruption on sensor rename

Question

Database corruption on sensor rename

wigster opened this issue 3 months ago · comments

System Health details

System Information

version	core-2024.2.5
installation_type	Home Assistant OS
dev	false
hassio	true
docker	true
user	root
virtualenv	false
python_version	3.12.1
os_name	Linux
os_version	6.6.16-haos
arch	aarch64
timezone	Europe/Prague
config_dir	/config

Home Assistant Community Store

GitHub API	ok
GitHub Content	ok
GitHub Web	ok
GitHub API Calls Remaining	4392
Installed Version	1.34.0
Stage	running
Available Repositories	1479
Downloaded Repositories	54

Home Assistant Cloud

logged_in	false
can_reach_cert_server	ok
can_reach_cloud_auth	ok
can_reach_cloud	ok

Home Assistant Supervisor

host_os	Home Assistant OS 12.0
update_channel	stable
supervisor_version	supervisor-2024.03.0
agent_version	1.6.0
docker_version	24.0.7
disk_total	113.9 GB
disk_used	40.1 GB
healthy	true
supported	true
board	odroid-n2
supervisor_api	ok
version_api	ok
installed_addons	Z-Wave JS UI (3.4.1), Check Home Assistant configuration (3.11.0), Samba share (12.3.1), Advanced SSH & Web Terminal (17.2.0), deCONZ (6.23.0), File editor (5.8.0), Bookstack (2.0.2), MariaDB (2.6.1), ESPHome (2024.2.2), Samba Backup (5.2.0), SQLite Web (4.1.2), LMS (Formerly Logitech Media Server) (8.4.0.0), NGINX Home Assistant SSL proxy (3.8.0), Studio Code Server (5.15.0), Browserless Chrome (2.2.0-5)

Dashboards

dashboards	1
resources	25
views	14
mode	storage

Recorder

oldest_recorder_run	5 March 2024 at 08:33
current_recorder_run	13 March 2024 at 21:33
estimated_db_size	2817.22 MiB
database_engine	sqlite
database_version	3.44.2

Spotify

api_endpoint_reachable	ok

Checklist

I have read the FAQ to see if there is a known solution to my problem.
I have enabled debug logging for my installation.
I have filled out the issue template to the best of my ability.
This issue only contains 1 issue (if you have multiple issues, open one issue for each issue).
This issue is not a duplicate issue of currently previous issues..

Describe the issue

I have Powercalc creating power/energy sensors for multiple things, but in particular my Hue bulbs running through deConz. I have recently been restructuring my lighting, moving the bulbs between rooms. WIth that I've been renaming and reassigning area to the individual bulbs.

Four times now I've had HA mark the database as corrupt and start a new one after I've renamed the entity_id's of the energy/power sensors to match the new names of the bulbs (the friendly names get changed automatically). I've not lost any data (sqlite3 .repair fixes the db), but this is clearly a very serious bug.

I apologise that I do not have the logs, but I am a little worried to blow my database again for the sake of this bug report. If you can't replicate it, I will submit a log.

It may be caused by the situation when I am renaming a bulb to the entity_id of another one which has recently been renamed. Maybe the Powercalc auto-generated entities somehow clash

Reproduction steps

Change name of bulb in deConz
Change the bulb entity name in HA
Reboot
Change the entity_id of the Powercalc energy/power sensors
Reboot
Find that the database is reset and the old one has been marked as corrupted.
...

Debug logs

Will try to do this if really needed.

Diagnostics dump or YAML config

create_utility_meters: true
utility_meter_types:
  - daily
  - yearly
power_sensor_naming: "{} Power"
energy_sensor_naming: "{} Energy"
create_domain_groups:
  # - light
ignore_unavailable_state: true
sensors:
  # Cooking group

  # Lights Group
  - create_group: Group Kids Sufit
    entities:
      - entity_id: light.kids_sufit_1
      - entity_id: light.kids_sufit_2
      - entity_id: light.kids_sufit_3

  - create_group: Group Shower Sufit
    entities:
      - entity_id: light.shower_sufit_1
      - entity_id: light.shower_sufit_2

  - name: "Light Kuchnia Reflektory"
    entity_id: light.kuchnia_reflektory
    power_sensor_id: sensor.light_kuchnia_reflektory_power
    energy_sensor_id: sensor.light_kuchnia_reflektory_electric_consumption_kwh

  - name: "Light Salon Table Right"
    entity_id: light.salon_table_right
    power_sensor_id: sensor.light_salon_table_right_power
    energy_sensor_id: sensor.light_salon_table_right_electric_consumption_kwh

  - name: "Light Salon Table Left"
    entity_id: light.salon_table_left
    power_sensor_id: sensor.light_salon_table_left_power
    energy_sensor_id: sensor.light_salon_table_left_electric_consumption_kwh

  - name: "Light Salon Spot"
    entity_id: light.salon_spot
    power_sensor_id: sensor.light_salon_spot_power
    energy_sensor_id: sensor.light_salon_spot_electric_consumption_kwh

  - name: "Light Kitchen Spot"
    entity_id: light.kitchen_spot
    power_sensor_id: sensor.light_kitchen_spot_power
    energy_sensor_id: sensor.light_kitchen_spot_electric_consumption_kwh

  - name: "Light Balcony Kulki"
    entity_id: light.balcony_kulki
    fixed:
      power: 19.1
    standby_power: 0

  - name: "Light WC Sufit"
    entity_id: light.wc_sufit
    power_sensor_id: sensor.light_wc_sufit_power
    energy_sensor_id: sensor.light_wc_sufit_energy

  - name: "Light Shower Mirror"
    entity_id: light.shower_mirror
    power_sensor_id: sensor.shower_mirror_power
    energy_sensor_id: sensor.shower_mirror_energy

  - name: "Light Bath Mirrors"
    entity_id: light.bath_mirrors
    power_sensor_id: sensor.bath_mirrors_power
    energy_sensor_id: sensor.bath_mirrors_energy

  - name: "Light Balcony Door"
    entity_id: light.balcony_door
    fixed:
      power: 9.5 # normalna żarówka 8.5 W chyba
    standby_power: 0.5

  - name: "Light Gabinet Bed"
    entity_id: light.gabinet_bed
    linear:
      attribute: brightness
      calibrate:
        - 3 -> 0.4
        - 25 -> 1.05
        - 50 -> 1.75
        - 100 -> 3.5
        - 150 -> 5.3
        - 200 -> 7.25
        - 255 -> 9.3
    standby_power: 0.4

  - name: "Light Gabinet Desk Strip"
    entity_id: light.gabinet_desk_strip
    linear:
      attribute: brightness
      calibrate:
        - 3 -> 0.4
        - 25 -> 1.1
        - 50 -> 2.1
        - 100 -> 4.4
        - 127 -> 5.6
        - 150 -> 6.8
        - 200 -> 9.5
        - 255 -> 12.4
    standby_power: 0.4

  - name: "Light Gabinet Sufit"
    entity_id: light.gabinet_sufit
    fixed:
      power: 25
    standby_power: 0.5

  - name: "Light Bedroom Bookcase"
    entity_id: light.bedroom_bookcase
    fixed:
      power: 4.5 # Intertech @ 0.7 + punktowe @ 1.9W x 2
    standby_power: 0.7

  - name: "Light Kuchnia Blat"
    entity_id: light.kuchnia_blat
    fixed:
      power: 40.5
    standby_power: 0.5

  - name: "Light Hall"
    entity_id: light.hall
    linear:
      attribute: brightness
      calibrate:
        - 3 -> 2.5
        - 39 -> 6.5
        - 129 -> 18.5
        - 255 -> 30.5
    standby_power: 0.5

Bram Gerritsen · Answer 1 · Sat Mar 16 2024 17:04:02 GMT+0800 (China Standard Time)

Thanks for you report, unfortunate you experience these issue.

I'm not using sqlite myself but using MariaDB in all my HA instances.
I have setup another virtual machine with HA OS running in Sqlite. Installed powercalc and did some renaming, but I am unable to reproduce.
I don't use Deconz so I cannot include that.
It's a very hard issue because first of all it's caused by something in HA core, and also it's installation specific.

I invested 1 hour already to try to reproduce, but I'm not going to put in any more efforts now.

When we have clear reproduction steps (without Deconz) in the mix I might have another look.

github-actions · Answer 2 · Tue Apr 16 2024 09:47:59 GMT+0800 (China Standard Time)

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

github-actions · Answer 3 · Sun Apr 21 2024 09:49:17 GMT+0800 (China Standard Time)

This issue was closed because it has been stalled for 5 days with no activity.