Room migration fails
herzkerl opened this issue · comments
While migrating the rooms from v2 to v3, I'm getting the following error:
rake aborted!
Errno::ENOENT: No such file or directory @ rb_sysopen - /usr/src/app/storage/Tw/eB/TweBjYAZTYDGkDYdE5ctyxrF
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/activestorage-5.2.8.1/lib/active_storage/service/disk_service.rb:36:in `binread'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/activestorage-5.2.8.1/lib/active_storage/service/disk_service.rb:36:in `block in download'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/activesupport-5.2.8.1/lib/active_support/notifications.rb:168:in `block in instrument'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/activesupport-5.2.8.1/lib/active_support/notifications/instrumenter.rb:23:in `instrument'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/activesupport-5.2.8.1/lib/active_support/notifications.rb:168:in `instrument'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/activestorage-5.2.8.1/lib/active_storage/service.rb:124:in `instrument'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/activestorage-5.2.8.1/lib/active_storage/service/disk_service.rb:35:in `download'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/activestorage-5.2.8.1/app/models/active_storage/blob.rb:165:in `download'
/usr/src/app/lib/tasks/migrations/migrations.rake:163:in `block (3 levels) in <main>'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/activerecord-5.2.8.1/lib/active_record/relation/batches.rb:70:in `block (2 levels) in find_each'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/activerecord-5.2.8.1/lib/active_record/relation/batches.rb:70:in `each'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/activerecord-5.2.8.1/lib/active_record/relation/batches.rb:70:in `block in find_each'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/activerecord-5.2.8.1/lib/active_record/relation/batches.rb:136:in `block in find_in_batches'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/activerecord-5.2.8.1/lib/active_record/relation/batches.rb:238:in `block in in_batches'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/activerecord-5.2.8.1/lib/active_record/relation/batches.rb:222:in `loop'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/activerecord-5.2.8.1/lib/active_record/relation/batches.rb:222:in `in_batches'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/activerecord-5.2.8.1/lib/active_record/relation/batches.rb:135:in `find_in_batches'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/activerecord-5.2.8.1/lib/active_record/relation/batches.rb:69:in `find_each'
/usr/src/app/lib/tasks/migrations/migrations.rake:133:in `block (2 levels) in <main>'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/usr/local/bin/bundle:23:in `load'
/usr/local/bin/bundle:23:in `<main>'
Tasks: TOP => migrations:rooms
(See full trace by running task with --trace)
I'm seeing the same error when migrating from 2.14.9 to 3.3.2 (previous migrations from 2.14.8.X to 3.2.X were successful however)
The first room (starting room of admin) is migrated successfully, but then the rake aborts. Full trace:
rake aborted!
Errno::ENOENT: No such file or directory @ rb_sysopen - /usr/src/app/storage/Co/ie/CoiepkWVBZygRzgviwrLh4qB
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/activestorage-5.2.8.1/lib/active_storage/service/disk_service.rb:36:in binread' /usr/src/app/vendor/bundle/ruby/2.7.0/gems/activestorage-5.2.8.1/lib/active_storage/service/disk_service.rb:36:in block in download'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/activesupport-5.2.8.1/lib/active_support/notifications.rb:168:in block in instrument' /usr/src/app/vendor/bundle/ruby/2.7.0/gems/activesupport-5.2.8.1/lib/active_support/notifications/instrumenter.rb:23:in instrument'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/activesupport-5.2.8.1/lib/active_support/notifications.rb:168:in instrument' /usr/src/app/vendor/bundle/ruby/2.7.0/gems/activestorage-5.2.8.1/lib/active_storage/service.rb:124:in instrument'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/activestorage-5.2.8.1/lib/active_storage/service/disk_service.rb:35:in download' /usr/src/app/vendor/bundle/ruby/2.7.0/gems/activestorage-5.2.8.1/app/models/active_storage/blob.rb:165:in download'
/usr/src/app/lib/tasks/migrations/migrations.rake:163:in block (3 levels) in <main>' /usr/src/app/vendor/bundle/ruby/2.7.0/gems/activerecord-5.2.8.1/lib/active_record/relation/batches.rb:70:in block (2 levels) in find_each'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/activerecord-5.2.8.1/lib/active_record/relation/batches.rb:70:in each' /usr/src/app/vendor/bundle/ruby/2.7.0/gems/activerecord-5.2.8.1/lib/active_record/relation/batches.rb:70:in block in find_each'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/activerecord-5.2.8.1/lib/active_record/relation/batches.rb:136:in block in find_in_batches' /usr/src/app/vendor/bundle/ruby/2.7.0/gems/activerecord-5.2.8.1/lib/active_record/relation/batches.rb:238:in block in in_batches'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/activerecord-5.2.8.1/lib/active_record/relation/batches.rb:222:in loop' /usr/src/app/vendor/bundle/ruby/2.7.0/gems/activerecord-5.2.8.1/lib/active_record/relation/batches.rb:222:in in_batches'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/activerecord-5.2.8.1/lib/active_record/relation/batches.rb:135:in find_in_batches' /usr/src/app/vendor/bundle/ruby/2.7.0/gems/activerecord-5.2.8.1/lib/active_record/relation/batches.rb:69:in find_each'
/usr/src/app/lib/tasks/migrations/migrations.rake:133:in block (2 levels) in <main>' /usr/src/app/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in block in execute'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in each' /usr/src/app/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in execute'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:219:in block in invoke_with_call_chain' /usr/src/app/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:199:in synchronize'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:199:in invoke_with_call_chain' /usr/src/app/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:188:in invoke'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:160:in invoke_task' /usr/src/app/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:116:in block (2 levels) in top_level'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:116:in each' /usr/src/app/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:116:in block in top_level'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:125:in run_with_threads' /usr/src/app/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:110:in top_level'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:83:in block in run' /usr/src/app/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:186:in standard_exception_handling'
/usr/src/app/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:80:in run' /usr/src/app/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/exe/rake:27:in <top (required)>'
/usr/src/app/vendor/bundle/ruby/2.7.0/bin/rake:23:in load' /usr/src/app/vendor/bundle/ruby/2.7.0/bin/rake:23:in <top (required)>'
/usr/local/lib/ruby/2.7.0/bundler/cli/exec.rb:63:in load' /usr/local/lib/ruby/2.7.0/bundler/cli/exec.rb:63:in kernel_load'
/usr/local/lib/ruby/2.7.0/bundler/cli/exec.rb:28:in run' /usr/local/lib/ruby/2.7.0/bundler/cli.rb:476:in exec'
/usr/local/lib/ruby/2.7.0/bundler/vendor/thor/lib/thor/command.rb:27:in run' /usr/local/lib/ruby/2.7.0/bundler/vendor/thor/lib/thor/invocation.rb:127:in invoke_command'
/usr/local/lib/ruby/2.7.0/bundler/vendor/thor/lib/thor.rb:399:in dispatch' /usr/local/lib/ruby/2.7.0/bundler/cli.rb:30:in dispatch'
/usr/local/lib/ruby/2.7.0/bundler/vendor/thor/lib/thor/base.rb:476:in start' /usr/local/lib/ruby/2.7.0/bundler/cli.rb:24:in start'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.1.4/libexec/bundle:46:in block in <top (required)>' /usr/local/lib/ruby/2.7.0/bundler/friendly_errors.rb:123:in with_friendly_errors'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.1.4/libexec/bundle:34:in <top (required)>' /usr/local/bin/bundle:23:in load'
/usr/local/bin/bundle:23:in `
Tasks: TOP => migrations:rooms
After a bit of checking the issue appears to be that the presentation file is missing in the FS (/storage). Which makes sense in my case, because our test environment has only test data for the database.
This issue was introduced in 04c7288. In the added code there is a check if the room has a presentation attached to it, but not if the presentation actually exists as file. The file on the drive is only called when r.presentation.blob.download is engaged, meaning a missing file causes "No such file or directory".
As for a potential fix, Ruby is not my strong side.
@danimo @farhatahmad
Would it be possible to add a check to skip presentations missing on file system? As of now 1 missing presentation file stops the migration process.
Can you try to modify the code like so?
if r.presentation.attached?
begin
params[:room][:presentation] = { blob: Base64.encode64(r.presentation.blob.download),
filename: r.presentation.blob.filename.to_s }
rescue Errno::ENOENT
p "Failed to locate '#{r.presentation.blob.filename.to_s}' in active storage, skipping."
end
endSorry for the late response. Your code does work, the migrations run successful!
Thank you very much!
@danimo Would you be able to send a PR for this please?
Done. Please merge.
New version released 2.14.10 that should solve this