Update environment cache performance is slow

[Kevin-Delaney-BY]

In a Calm scenario with 100+ projects, updating or creating new Environments takes hours.
Calm updates projects and environments after every excecution. That means in a for_loop, the process takes a huge amount of time.
Is there a way to skip the cache update? or a better way to loop through multiple projects?

We also often see the error below. It does not break the loop, but slows the process down even more.

[2021-12-02 19:33:14] [INFO] [calm.dsl.cli.environments:309] Updating projects and environments cache ...
Traceback (most recent call last):
File "C:\Users**\calm-dsl\venv\lib\site-packages\peewee.py", line 2981, in execute_sql
cursor.execute(sql, params or ())
sqlite3.IntegrityError: UNIQUE constraint failed: environmentcache.name, environmentcache.uuid

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users**\calm-dsl\venv\Scripts\calm-script.py", line 11, in
load_entry_point('calm.dsl', 'console_scripts', 'calm')()
File "C:\Users**\calm-dsl\venv\lib\site-packages\click\core.py", line 829, in call
return self.main(*args, kwargs)
File "C:\Users**\calm-dsl\venv\lib\site-packages\click\core.py", line 782, in main
rv = self.invoke(ctx)
File "c:\users**\calm-dsl\calm\dsl\cli\utils.py", line 127, in invoke
return super().invoke(ctx)
File "C:\Users**\calm-dsl\venv\lib\site-packages\click\core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "c:\users**\calm-dsl\calm\dsl\cli\utils.py", line 116, in invoke
return super().invoke(ctx)
File "C:\Users**\calm-dsl\venv\lib\site-packages\click\core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "C:\Users**\calm-dsl\venv\lib\site-packages\click\core.py", line 1066, in invoke
return ctx.invoke(self.callback, ctx.params)
File "C:\Users**\calm-dsl\venv\lib\site-packages\click\core.py", line 610, in invoke
return callback(*args, kwargs)
File "c:\users**\calm-dsl\calm\dsl\cli\environment_commands.py", line 99, in _update_environment
update_environment_from_dsl_file(env_name, env_file, project_name)
File "c:\users**\calm-dsl\calm\dsl\cli\environments.py", line 311, in update_environment_from_dsl_file
Cache.sync_table(cache_type=CACHE.ENTITY.ENVIRONMENT)
File "c:\users**\calm-dsl\calm\dsl\store\cache.py", line 164, in sync_table
cache_table.sync()
File "c:\users**\calm-dsl\calm\dsl\db\table_config.py", line 873, in sync
cls.create_entry(
File "c:\users**\calm-dsl\calm\dsl\db\table_config.py", line 882, in create_entry
super().create(
File "C:\Users**\calm-dsl\venv\lib\site-packages\peewee.py", line 6122, in create
inst.save(force_insert=True)
File "C:\Users**\calm-dsl\venv\lib\site-packages\peewee.py", line 6311, in save
pk = self.insert(field_dict).execute()
File "C:\Users**\calm-dsl\venv\lib\site-packages\peewee.py", line 1796, in inner
return method(self, database, *args, kwargs)
File "C:\Users**\calm-dsl\venv\lib\site-packages\peewee.py", line 1867, in execute
return self._execute(database)
File "C:\Users**\calm-dsl\venv\lib\site-packages\peewee.py", line 2599, in _execute
return super(Insert, self)._execute(database)
File "C:\Users**\calm-dsl\venv\lib\site-packages\peewee.py", line 2336, in _execute
cursor = database.execute(self)
File "C:\Users**\calm-dsl\venv\lib\site-packages\peewee.py", line 2994, in execute
return self.execute_sql(sql, params, commit=commit)
File "C:\Users**\calm-dsl\venv\lib\site-packages\peewee.py", line 2988, in execute_sql
self.commit()
File "C:\Users**\calm-dsl\venv\lib\site-packages\peewee.py", line 2759, in exit
reraise(new_type, new_type(*exc_args), traceback)
File "C:\Users**\calm-dsl\venv\lib\site-packages\peewee.py", line 183, in reraise
raise value.with_traceback(tb)
File "C:\Users****\calm-dsl\venv\lib\site-packages\peewee.py", line 2981, in execute_sql
cursor.execute(sql, params or ())
peewee.IntegrityError: UNIQUE constraint failed: environmentcache.name, environmentcache.uuid

[Abhijeet Singh Kaurav]

fixed in release v3.4.0