Include run time to create/build factories in ActiveSupport::Notifications

[celvro]

Problem this feature will solve

Currently only 'factory_bot.run_factory' is instrumented. This doesn't include any of the time it takes to setup associations. In our project this is by far the slowest part by multiple magnitudes, and would be nice to have instrumented.

Desired solution

Include ActiveSupport::Notifications for build, build_stubbed, and create.

Ideally this would include object allocations to help debug highly nested factories.

[celvro]

Notice that test-prof needs to monkey patch factory_bot to add Notification for create manually

[celvro]

Oh and we'd need the after block.
eg) after(:build) { |user| user.big_object = create(:slow_factory)

[celvro]

Maybe all i need is for factory_bot to include the after hooks.. I added this in my user factory:

after(:create) do
  sleep(2)
end

But the event duration shows < 50 µs for run_factory

edit: also not tracking event allocations correctly. I would expect this to result in well over a million allocations (and it's quite slow) but the notification only records ~9000 before and after adding this block.

after(:build) do |u|
  1_000_000.times do
    u.title = Faker::Name.name
  end
end

[celvro]

This also measures 4 µs instead of 2 s, even when I put the sleep call inside after(:build)

module FactoryBot::Syntax::Methods
  old_create = instance_method(:create)

  define_method(:create) do |name, *traits_and_overrides, &block|
    ActiveSupport::Notifications.instrument('factory_bot.create') do
      old_create.bind(self).call(name, *traits_and_overrides, &block)
    end
  end
end

RSpec.configure do |config|
  config.include FactoryBot::Syntax::Methods
end

[celvro]

So given a completely clean environment with only this, I am not able to reproduce this issue:

require 'active_support'

ActiveSupport::Notifications.subscribe(/.*/) do |event|
    puts '*' * 20
    puts "Received event: #{event.name}"
    puts "      duration: #{event.duration}"
    puts "   allocations: #{event.allocations}"
    puts '*' * 20
end

class User
  attr_accessor :first_name
end


require 'factory_bot'

FactoryBot.define do
  factory :user do
    first_name { "John" }
    after(:build) { sleep(1) }
  end
end


require 'rspec'

RSpec.configure do
  include FactoryBot::Syntax::Methods
end

describe User do
  it 'builds a user' do
    expect(build(:user).first_name).to eq("John")
  end
end

Output:

$ rspec test.rb
********************
Received event: factory_bot.run_factory
      duration: 1001.3912379999965
   allocations: 362
********************

There must be some kind of configuration issue in my other project. Going to close this and update if I learn what caused it..