Fresh install of the bakerydemo using pyenv generates 'KeyError at / <wagtail.images.models.Filter object at 0x10953ef40>'
birchcode opened this issue · comments
Fresh install of the demo and we're encountering some errors with image filters:
If we remove that line another error will occur for an image in another template so i assume it's not a template issue.
Is there something else I can debug?
Everything seemed to initiate as expected:
cp .env.example .env
(wagtailbakerydemo) rmp@rmps-Mac-Pro:~/projects/bakerydemo% ./manage.py migrate
./manage.py load_initial_data
./manage.py runserver
Operations to perform:
Apply all migrations: admin, auth, base, blog, breads, contenttypes, locations, recipes, sessions, simple_translation, taggit, wagtailadmin, wagtailcore, wagtaildocs, wagtailembeds, wagtailforms, wagtailimages, wagtailredirects, wagtailsearch, wagtailsearchpromotions, wagtailusers
Running migrations:
Applying contenttypes.0001_initial... OK
Applying auth.0001_initial... OK
Applying admin.0001_initial... OK
Applying admin.0002_logentry_remove_auto_add... OK
Applying admin.0003_logentry_add_action_flag_choices... OK
Applying contenttypes.0002_remove_content_type_name... OK
Applying auth.0002_alter_permission_name_max_length... OK
Applying auth.0003_alter_user_email_max_length... OK
Applying auth.0004_alter_user_username_opts... OK
Applying auth.0005_alter_user_last_login_null... OK
Applying auth.0006_require_contenttypes_0002... OK
Applying auth.0007_alter_validators_add_error_messages... OK
Applying auth.0008_alter_user_username_max_length... OK
Applying auth.0009_alter_user_last_name_max_length... OK
Applying auth.0010_alter_group_name_max_length... OK
Applying auth.0011_update_proxy_permissions... OK
Applying auth.0012_alter_user_first_name_max_length... OK
Applying wagtailcore.0001_squashed_0016_change_page_url_path_to_text_field... OK
Applying wagtailcore.0017_change_edit_page_permission_description... OK
Applying wagtailcore.0018_pagerevision_submitted_for_moderation_index... OK
Applying wagtailcore.0019_verbose_names_cleanup... OK
Applying wagtailcore.0020_add_index_on_page_first_published_at... OK
Applying wagtailcore.0021_capitalizeverbose... OK
Applying wagtailcore.0022_add_site_name... OK
Applying wagtailcore.0023_alter_page_revision_on_delete_behaviour... OK
Applying wagtailcore.0024_collection... OK
Applying wagtailcore.0025_collection_initial_data... OK
Applying wagtailcore.0026_group_collection_permission... OK
Applying wagtailcore.0027_fix_collection_path_collation... OK
Applying wagtailcore.0024_alter_page_content_type_on_delete_behaviour... OK
Applying wagtailcore.0028_merge... OK
Applying wagtailcore.0029_unicode_slugfield_dj19... OK
Applying wagtailcore.0030_index_on_pagerevision_created_at... OK
Applying wagtailcore.0031_add_page_view_restriction_types... OK
Applying wagtailcore.0032_add_bulk_delete_page_permission... OK
Applying wagtailcore.0033_remove_golive_expiry_help_text... OK
Applying wagtailcore.0034_page_live_revision... OK
Applying wagtailcore.0035_page_last_published_at... OK
Applying wagtailcore.0036_populate_page_last_published_at... OK
Applying wagtailcore.0037_set_page_owner_editable... OK
Applying wagtailcore.0038_make_first_published_at_editable... OK
Applying wagtailcore.0039_collectionviewrestriction... OK
Applying wagtailcore.0040_page_draft_title... OK
Applying wagtailcore.0041_group_collection_permissions_verbose_name_plural... OK
Applying wagtailcore.0042_index_on_pagerevision_approved_go_live_at... OK
Applying wagtailcore.0043_lock_fields... OK
Applying wagtailcore.0044_add_unlock_grouppagepermission... OK
Applying wagtailcore.0045_assign_unlock_grouppagepermission... OK
Applying wagtailcore.0046_site_name_remove_null... OK
Applying wagtailcore.0047_add_workflow_models... OK
Applying wagtailcore.0048_add_default_workflows... OK
Applying wagtailcore.0049_taskstate_finished_by... OK
Applying wagtailcore.0050_workflow_rejected_to_needs_changes... OK
Applying wagtailcore.0051_taskstate_comment... OK
Applying wagtailcore.0052_pagelogentry... OK
Applying wagtailcore.0053_locale_model... OK
Applying wagtailcore.0054_initial_locale... OK
Applying wagtailcore.0055_page_locale_fields... OK
Applying wagtailcore.0056_page_locale_fields_populate... OK
Applying wagtailcore.0057_page_locale_fields_notnull... OK
Applying wagtailcore.0058_page_alias_of... OK
Applying wagtailcore.0059_apply_collection_ordering... OK
Applying wagtailcore.0060_fix_workflow_unique_constraint... OK
Applying wagtailcore.0061_change_promote_tab_helpt_text_and_verbose_names... OK
Applying wagtailcore.0062_comment_models_and_pagesubscription... OK
Applying wagtailcore.0063_modellogentry... OK
Applying wagtailcore.0064_log_timestamp_indexes... OK
Applying wagtailcore.0065_log_entry_uuid... OK
Applying wagtailcore.0066_collection_management_permissions... OK
Applying wagtailcore.0067_alter_pagerevision_content_json... OK
Applying wagtailcore.0068_log_entry_empty_object... OK
Applying wagtailcore.0069_log_entry_jsonfield... OK
Applying wagtailcore.0070_rename_pagerevision_revision... OK
Applying wagtailcore.0071_populate_revision_content_type... OK
Applying wagtailcore.0072_alter_revision_content_type_notnull... OK
Applying wagtailcore.0073_page_latest_revision... OK
Applying wagtailcore.0074_revision_object_str... OK
Applying wagtailcore.0075_populate_latest_revision_and_revision_object_str... OK
Applying wagtailcore.0076_modellogentry_revision... OK
Applying wagtailcore.0077_alter_revision_user... OK
Applying wagtailcore.0078_referenceindex... OK
Applying wagtailcore.0079_rename_taskstate_page_revision... OK
Applying wagtailcore.0080_generic_workflowstate... OK
Applying wagtailcore.0081_populate_workflowstate_content_type... OK
Applying wagtailcore.0082_alter_workflowstate_content_type_notnull... OK
Applying wagtailcore.0083_workflowcontenttype... OK
Applying taggit.0001_initial... OK
Applying wagtailimages.0001_squashed_0021... OK
Applying wagtailimages.0022_uploadedimage... OK
Applying wagtailimages.0023_add_choose_permissions... OK
Applying wagtailimages.0024_index_image_file_hash... OK
Applying taggit.0002_auto_20150616_2121... OK
Applying taggit.0003_taggeditem_add_unique_index... OK
Applying taggit.0004_alter_taggeditem_content_type_alter_taggeditem_tag... OK
Applying base.0001_initial... OK
Applying blog.0001_initial... OK
Applying blog.0002_remove_blogindexpage_body... OK
Applying blog.0003_auto_20170329_0055... OK
Applying blog.0004_alter_blogpagetag_tag... OK
Applying blog.0005_use_json_field_for_body_streamfield... OK
Applying base.0002_auto_20170329_0055... OK
Applying base.0003_auto_20170823_1127... OK
Applying base.0004_auto_20180522_1856... OK
Applying base.0005_formfield_clean_name... OK
Applying base.0006_char_field_remove_null... OK
Applying base.0007_alter_formfield_choices_and_more... OK
Applying base.0008_use_json_field_for_body_streamfield... OK
Applying base.0009_alter_homepage_promo_text... OK
Applying base.0010_rename_people_person... OK
Applying base.0011_footertext_expire_at_footertext_expired_and_more... OK
Applying base.0012_person_expire_at_person_expired_and_more... OK
Applying base.0013_person_lockablemixin... OK
Applying base.0014_person_enable_default_workflow... OK
Applying base.0015_footertext_translatable... OK
Applying base.0016_footertext_bootstrap_translation... OK
Applying base.0017_footertext_enforce_not_null... OK
Applying base.0018_add_genericsettings_and_sitesettings... OK
Applying blog.0006_rename_blogpeoplerelationship_person... OK
Applying breads.0001_initial... OK
Applying breads.0002_remove_breadsindexpage_body... OK
Applying breads.0003_auto_20170329_0055... OK
Applying breads.0004_use_json_field_for_body_streamfield... OK
Applying breads.0005_breadtype_latest_revision... OK
Applying breads.0006_breadingredient_expire_at_breadingredient_expired_and_more... OK
Applying locations.0001_initial... OK
Applying locations.0002_remove_locationsindexpage_body... OK
Applying locations.0003_auto_20170329_0055... OK
Applying locations.0004_auto_20190912_1149... OK
Applying locations.0005_use_json_field_for_body_streamfield... OK
Applying locations.0006_alter_locationoperatinghours_day... OK
Applying recipes.0001_initial... OK
Applying sessions.0001_initial... OK
Applying simple_translation.0001_initial... OK
Applying taggit.0005_auto_20220424_2025... OK
Applying taggit.0006_rename_taggeditem_content_type_object_id_taggit_tagg_content_8fc721_idx... OK
Applying wagtailadmin.0001_create_admin_access_permissions... OK
Applying wagtailadmin.0002_admin... OK
Applying wagtailadmin.0003_admin_managed... OK
Applying wagtailcore.0084_add_default_page_permissions... OK
Applying wagtailcore.0085_add_grouppagepermission_permission... OK
Applying wagtailcore.0086_populate_grouppagepermission_permission... OK
Applying wagtailcore.0087_alter_grouppagepermission_unique_together_and_more... OK
Applying wagtailcore.0088_fix_log_entry_json_timestamps... OK
Applying wagtailcore.0089_log_entry_data_json_null_to_object... OK
Applying wagtailcore.0090_remove_grouppagepermission_permission_type... OK
Applying wagtailcore.0091_remove_revision_submitted_for_moderation... OK
Applying wagtaildocs.0001_initial... OK
Applying wagtaildocs.0002_initial_data... OK
Applying wagtaildocs.0003_add_verbose_names... OK
Applying wagtaildocs.0004_capitalizeverbose... OK
Applying wagtaildocs.0005_document_collection... OK
Applying wagtaildocs.0006_copy_document_permissions_to_collections... OK
Applying wagtaildocs.0005_alter_uploaded_by_user_on_delete_action... OK
Applying wagtaildocs.0007_merge... OK
Applying wagtaildocs.0008_document_file_size... OK
Applying wagtaildocs.0009_document_verbose_name_plural... OK
Applying wagtaildocs.0010_document_file_hash... OK
Applying wagtaildocs.0011_add_choose_permissions... OK
Applying wagtaildocs.0012_uploadeddocument... OK
Applying wagtailembeds.0001_initial... OK
Applying wagtailembeds.0002_add_verbose_names... OK
Applying wagtailembeds.0003_capitalizeverbose... OK
Applying wagtailembeds.0004_embed_verbose_name_plural... OK
Applying wagtailembeds.0005_specify_thumbnail_url_max_length... OK
Applying wagtailembeds.0006_add_embed_hash... OK
Applying wagtailembeds.0007_populate_hash... OK
Applying wagtailembeds.0008_allow_long_urls... OK
Applying wagtailembeds.0009_embed_cache_until... OK
Applying wagtailforms.0001_initial... OK
Applying wagtailforms.0002_add_verbose_names... OK
Applying wagtailforms.0003_capitalizeverbose... OK
Applying wagtailforms.0004_add_verbose_name_plural... OK
Applying wagtailforms.0005_alter_formsubmission_form_data... OK
Applying wagtailimages.0025_alter_image_file_alter_rendition_file... OK
Applying wagtailredirects.0001_initial... OK
Applying wagtailredirects.0002_add_verbose_names... OK
Applying wagtailredirects.0003_make_site_field_editable... OK
Applying wagtailredirects.0004_set_unique_on_path_and_site... OK
Applying wagtailredirects.0005_capitalizeverbose... OK
Applying wagtailredirects.0006_redirect_increase_max_length... OK
Applying wagtailredirects.0007_add_autocreate_fields... OK
Applying wagtailredirects.0008_add_verbose_name_plural... OK
Applying wagtailsearch.0001_initial... OK
Applying wagtailsearch.0002_add_verbose_names... OK
Applying wagtailsearch.0003_remove_editors_pick... OK
Applying wagtailsearch.0004_querydailyhits_verbose_name_plural... OK
Applying wagtailsearch.0005_create_indexentry... OK
Applying wagtailsearch.0006_customise_indexentry... OK
Applying wagtailsearch.0007_delete_editorspick... OK
Applying wagtailsearch.0008_remove_query_and_querydailyhits_models... OK
Applying wagtailsearchpromotions.0001_initial... OK
Applying wagtailsearchpromotions.0002_capitalizeverbose... OK
Applying wagtailsearchpromotions.0003_query_querydailyhits... OK
Applying wagtailsearchpromotions.0004_copy_queries... OK
Applying wagtailsearchpromotions.0005_switch_query_model... OK
Applying wagtailsearchpromotions.0006_reset_query_sequence... OK
Applying wagtailsearchpromotions.0007_searchpromotion_external_link_text_and_more... OK
Applying wagtailusers.0001_initial... OK
Applying wagtailusers.0002_add_verbose_name_on_userprofile... OK
Applying wagtailusers.0003_add_verbose_names... OK
Applying wagtailusers.0004_capitalizeverbose... OK
Applying wagtailusers.0005_make_related_name_wagtail_specific... OK
Applying wagtailusers.0006_userprofile_prefered_language... OK
Applying wagtailusers.0007_userprofile_current_time_zone... OK
Applying wagtailusers.0008_userprofile_avatar... OK
Applying wagtailusers.0009_userprofile_verbose_name_plural... OK
Applying wagtailusers.0010_userprofile_updated_comments_notifications... OK
Applying wagtailusers.0011_userprofile_dismissibles... OK
Applying wagtailusers.0012_userprofile_theme... OK
Copying media files to configured storage...
Awesome. Your data is loaded! The bakery's doors are almost ready to open...
Watching for file changes with StatReloader
Performing system checks...
System check identified no issues (0 silenced).```
Hi @birchcode - thanks for the report. Can you share the full traceback of the error, please? (From the 'traceback' section of the error page, "Switch to copy-and-paste view")
the full traceback of the error
Request Method: GET
Request URL: http://127.0.0.1:8000/
Django Version: 4.2.10
Python Version: 3.8.10
Installed Applications:
['bakerydemo.base',
'bakerydemo.blog',
'bakerydemo.breads',
'bakerydemo.locations',
'bakerydemo.recipes',
'bakerydemo.search',
'wagtail.embeds',
'wagtail.sites',
'wagtail.users',
'wagtail.snippets',
'wagtail.documents',
'wagtail.images',
'wagtail.search',
'wagtail.admin',
'wagtail.api.v2',
'wagtail.locales',
'wagtail.contrib.forms',
'wagtail.contrib.redirects',
'wagtail.contrib.routable_page',
'wagtail.contrib.table_block',
'wagtail.contrib.typed_table_block',
'wagtail.contrib.search_promotions',
'wagtail.contrib.settings',
'wagtail.contrib.simple_translation',
'wagtail.contrib.styleguide',
'wagtail',
'rest_framework',
'modelcluster',
'taggit',
'wagtailfontawesomesvg',
'debug_toolbar',
'django_extensions',
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'django.contrib.sitemaps']
Installed Middleware:
['debug_toolbar.middleware.DebugToolbarMiddleware',
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'wagtail.contrib.redirects.middleware.RedirectMiddleware']
Template error:
In template /Users/rmp/projects/bakerydemo/bakerydemo/templates/base/home_page.html, error at line 8
format-jpeg|fill-1920x900
1 : {% extends "base.html" %}
2 : {% load wagtailimages_tags wagtailcore_tags %}
3 :
4 : {% block content %}
5 : <div class="homepage">
6 :
7 : <div class="container-fluid hero">
8 : {% picture page.image format-{avif,webp,jpeg} fill-{800x650,1920x900} sizes="100vw" class="hero-image" alt="" %}
9 : <div class="hero-gradient-mask"></div>
10 : <div class="container">
11 : <div class="row">
12 : <div class="col-md-6 col-md-offset-1 col-lg-5 home-hero">
13 : <h1>{{ page.title }}</h1>
14 : <p class="lead">{{ page.hero_text }}</p>
15 : {% if page.hero_cta_link %}
16 : <a href="{% pageurl page.hero_cta_link %}" class="hero-cta-link">
17 : {{ page.hero_cta }}
18 : </a>
Traceback (most recent call last):
File "/Users/rmp/.virtualenvs/wagtailbakerydemo/lib/python3.8/site-packages/django/core/handlers/exception.py", line 55, in inner
response = get_response(request)
File "/Users/rmp/.virtualenvs/wagtailbakerydemo/lib/python3.8/site-packages/django/core/handlers/base.py", line 220, in _get_response
response = response.render()
File "/Users/rmp/.virtualenvs/wagtailbakerydemo/lib/python3.8/site-packages/django/template/response.py", line 114, in render
self.content = self.rendered_content
File "/Users/rmp/.virtualenvs/wagtailbakerydemo/lib/python3.8/site-packages/django/template/response.py", line 92, in rendered_content
return template.render(context, self._request)
File "/Users/rmp/.virtualenvs/wagtailbakerydemo/lib/python3.8/site-packages/django/template/backends/django.py", line 61, in render
return self.template.render(context)
File "/Users/rmp/.virtualenvs/wagtailbakerydemo/lib/python3.8/site-packages/django/template/base.py", line 175, in render
return self._render(context)
File "/Users/rmp/.virtualenvs/wagtailbakerydemo/lib/python3.8/site-packages/django/test/utils.py", line 112, in instrumented_test_render
return self.nodelist.render(context)
File "/Users/rmp/.virtualenvs/wagtailbakerydemo/lib/python3.8/site-packages/django/template/base.py", line 1005, in render
return SafeString("".join([node.render_annotated(context) for node in self]))
File "/Users/rmp/.virtualenvs/wagtailbakerydemo/lib/python3.8/site-packages/django/template/base.py", line 1005, in <listcomp>
return SafeString("".join([node.render_annotated(context) for node in self]))
File "/Users/rmp/.virtualenvs/wagtailbakerydemo/lib/python3.8/site-packages/django/template/base.py", line 966, in render_annotated
return self.render(context)
File "/Users/rmp/.virtualenvs/wagtailbakerydemo/lib/python3.8/site-packages/django/template/loader_tags.py", line 157, in render
return compiled_parent._render(context)
File "/Users/rmp/.virtualenvs/wagtailbakerydemo/lib/python3.8/site-packages/django/test/utils.py", line 112, in instrumented_test_render
return self.nodelist.render(context)
File "/Users/rmp/.virtualenvs/wagtailbakerydemo/lib/python3.8/site-packages/django/template/base.py", line 1005, in render
return SafeString("".join([node.render_annotated(context) for node in self]))
File "/Users/rmp/.virtualenvs/wagtailbakerydemo/lib/python3.8/site-packages/django/template/base.py", line 1005, in <listcomp>
return SafeString("".join([node.render_annotated(context) for node in self]))
File "/Users/rmp/.virtualenvs/wagtailbakerydemo/lib/python3.8/site-packages/django/template/base.py", line 966, in render_annotated
return self.render(context)
File "/Users/rmp/.virtualenvs/wagtailbakerydemo/lib/python3.8/site-packages/django/template/loader_tags.py", line 63, in render
result = block.nodelist.render(context)
File "/Users/rmp/.virtualenvs/wagtailbakerydemo/lib/python3.8/site-packages/django/template/base.py", line 1005, in render
return SafeString("".join([node.render_annotated(context) for node in self]))
File "/Users/rmp/.virtualenvs/wagtailbakerydemo/lib/python3.8/site-packages/django/template/base.py", line 1005, in <listcomp>
return SafeString("".join([node.render_annotated(context) for node in self]))
File "/Users/rmp/.virtualenvs/wagtailbakerydemo/lib/python3.8/site-packages/django/template/base.py", line 966, in render_annotated
return self.render(context)
File "/Users/rmp/.virtualenvs/wagtailbakerydemo/lib/python3.8/site-packages/wagtail/images/templatetags/wagtailimages_tags.py", line 203, in render
renditions = get_renditions_or_not_found(
File "/Users/rmp/.virtualenvs/wagtailbakerydemo/lib/python3.8/site-packages/wagtail/images/shortcuts.py", line 35, in get_renditions_or_not_found
return image.get_renditions(*specs)
File "/Users/rmp/.virtualenvs/wagtailbakerydemo/lib/python3.8/site-packages/wagtail/images/models.py", line 533, in get_renditions
for filter, rendition in self.create_renditions(*not_found).items():
File "/Users/rmp/.virtualenvs/wagtailbakerydemo/lib/python3.8/site-packages/wagtail/images/models.py", line 693, in create_renditions
filter = filter_map[existing.filter_spec]
Exception Type: KeyError at /
Exception Value: 'format-jpeg|fill-1920x900'
Hi @birchcode is this issue resolved for you?
I am getting the same error
@birchcode @carlotm Thanks for confirming. Does the error go away if you refresh the page, or does it happen consistently?
@ababic Any thoughts on what might be going on here? I can't replicate this locally, and for it to reach this line and fail, the filter spec would have to be returned in the "identify potential clashes" query, but not be in filter_map. But filter_map should contain everything in the filters list, so that doesn't make sense... My only hunch is that maybe it isn't safe to have multiple threads appending to to_create, and it's mixing up those results with other invocations of create_renditions - and we should be returning the Rendition instances and picking up the return value of executor.map instead. Not sure how we verify that, though...
@zerolab In https://github.com/wagtail/wagtail/pull/9550/files#r1246522485, you said: "if we change to source=self.file I had to reload a number of times until all renditions were generated as oppposed to all in one go with this implementation". Do you happen to remember what the "not all renditions being generated" failure case looked like - was it an error like this, by any chance?
@gasman no 500 like this. Just one or two renditions not being generated, if I recall correctly
@gasman it's consistent for me.
BTW I am not using pyenv
@birchcode Could I just check that you are using a virtual env of some kind and freshly installing all of the requirements? I think I've seen something like this after upgrading wagtail, but having an out-of-date version of pillow
@gasman I'm pretty sure this is what's happening...
generate_rendition_file()is silently failing whilst being run by the threadpool- Because of 1, the
to_createlist is empty - Because of 2, the
lookup_qcreated to find clashes doesn't actually apply any filtering - Because the current rendition queryset isn't being filtered, the line in question is looking in
filter_mapfor any old value, hence theKeyError
While I don't think we should be going to any lengths to support unstable builds, this issue did highlight that the code wasn't very easy to follow in it's current state, and could benefit from clearer exception handling (a limitation of the ThreadPoolExecutor.map() approach). As a result, I have submitted: wagtail/wagtail#11832
Hopefully addressed now in wagtail/wagtail#11832, which will be included in the upcoming Wagtail 6.1 release.