Dimension sizes not being correctly passed to OpenAI after original code removed in later PR

[Paul Danelli]

I previously committed a PR which fixed the issue of 512 vector dimensions not being supported when using "text-embedding-3-small".

The following PR reverted that and broke our production systems by forcing all text-embedding-3-small" and "text-embedding-3-large" models to only ever use their default dimensions of 1536, rather than allowing it to be set by the caller - obviously we'll have to stop mocking requests to OpenAI as this would have been caught by seeing 1536 vectors returned instead of 512.

This line prevents the dimensions passed to the method from being used unless they're 1536.

      if ["text-embedding-3-small", "text-embedding-3-large"].include?(model)
        parameters[:dimensions] = EMBEDDING_SIZES[model.to_sym] if EMBEDDING_SIZES.key?(model.to_sym)
      end

There were tests to ensure that 512 dimension sizes where supported - added in my original PR, but those were removed here: https://github.com/andreibondarev/langchainrb/pull/511/files#diff-e5cc0fcd8e724fd8ee9b499d8385a9a9f6bda124487df754d832336db0e2d5c5L110

The tests were added to ensure that this scenario was always accounted for and would have broken when the snippet above was added.

[Andrei Bondarev]

@prdanelli Thoughts on this approach? https://github.com/andreibondarev/langchainrb/compare/fix-openai%23embed?expand=1

If it's good -- I'll add specs and push it out.

[Paul Danelli]

@andreibondarev That looks ideal.

I did start to put up a PR together from a fork, but I haven't had time to write the specs as I'm not at work right now.

Thank you so much for jumping on this. 🎖️