Installing on Heroku
rodreegez opened this issue · comments
Before I begin, I would like to say:
- I'm really sorry, and,
- I'm weirdly a huge fan - I use a bunch of your other gems regularly. Thanks!
I've spent most of today trying to get OR Tools running on Heroku. I have little to no idea what I'm doing. It's been fun.
I've created a buildpack that seems to install OR Tools properly on a fresh heroku-18 stack, and I've created a new Rails app to try to utilise that buildpack and install the or-tools gem.
As per the README in the Rails app, I have:
- added the OR Tools buildpack (making sure it's run before the ruby/rails buildpack)
- set the BUNDLE_BUILD__OR_TOOLS to what I think is the correct path for OR Tools on the Heroku filesystem with
heroku config:set BUNDLE_BUILD__OR_TOOLS=/app/or-tools/or-tools_Ubuntu-18.04-64bit_v7.6.7691 - pushed the app up to Heroku.
The deploy is rejected because or-tools fails to install.
The pertinent bit of the installation log is as follows:
remote: -----> Ruby app detected
remote: -----> Installing bundler 2.0.2
remote: -----> Removing BUNDLED WITH version in the Gemfile.lock
remote: -----> Compiling Ruby/Rails
remote: -----> Using Ruby version: ruby-2.5.7
remote: -----> Installing dependencies using bundler 2.0.2
remote: Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment
remote: The dependency tzinfo-data (>= 0) will be unused by any of the platforms Bundler is installing for. Bundler is installing for ruby but the dependency is only for x86-mingw32, x86-mswin32, x64-mingw32, java. To add those platforms to the bundle, run `bundle lock --add-platform x86-mingw32 x86-mswin32 x64-mingw32 java`.
remote: Fetching gem metadata from https://rubygems.org/............
remote: Using rake 13.0.1
... standard rails gems omitted ...
remote: Using rails 6.0.3.1
remote: Using sass-rails 6.0.0
remote: Installing rice 2.2.0 with native extensions
remote: Fetching or-tools 0.1.5
remote: Installing or-tools 0.1.5 with native extensions
remote: Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
remote:
remote: current directory:
remote: /tmp/build_cf53ce24d0d4ec56170536db996a2607/vendor/bundle/ruby/2.5.0/gems/or-tools-0.1.5/ext/or-tools
remote: /tmp/build_cf53ce24d0d4ec56170536db996a2607/vendor/ruby-2.5.7/bin/ruby -r
remote: ./siteconf20200521-449-1fns9ab.rb extconf.rb
remote: checking for -lstdc++... yes
remote: creating Makefile
remote:
remote: current directory:
remote: /tmp/build_cf53ce24d0d4ec56170536db996a2607/vendor/bundle/ruby/2.5.0/gems/or-tools-0.1.5/ext/or-tools
remote: make "DESTDIR=" clean
remote:
remote: current directory:
remote: /tmp/build_cf53ce24d0d4ec56170536db996a2607/vendor/bundle/ruby/2.5.0/gems/or-tools-0.1.5/ext/or-tools
remote: make "DESTDIR="
remote: compiling ext.cpp
remote: cc1plus: warning: command line option ‘-Wimplicit-int’ is valid for C/ObjC but
remote: not for C++
remote: cc1plus: warning: command line option ‘-Wdeclaration-after-statement’ is valid
remote: for C/ObjC but not for C++
remote: cc1plus: warning: command line option ‘-Wimplicit-function-declaration’ is valid
remote: for C/ObjC but not for C++
remote: ext.cpp:2:10: fatal error: ortools/algorithms/knapsack_solver.h: No such file or
remote: directory
remote: #include <ortools/algorithms/knapsack_solver.h>
remote: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
remote: compilation terminated.
remote: Makefile:298: recipe for target 'ext.o' failed
remote: make: *** [ext.o] Error 1
remote:
remote: make failed, exit code 2
remote:
remote: Gem files will remain installed in
remote: /tmp/build_cf53ce24d0d4ec56170536db996a2607/vendor/bundle/ruby/2.5.0/gems/or-tools-0.1.5
remote: for inspection.
remote: Results logged to
remote: /tmp/build_cf53ce24d0d4ec56170536db996a2607/vendor/bundle/ruby/2.5.0/extensions/x86_64-linux/2.5.0/or-tools-0.1.5/gem_make.out
remote:
remote: An error occurred while installing or-tools (0.1.5), and Bundler cannot
remote: continue.
remote: Make sure that `gem install or-tools -v '0.1.5' --source
remote: 'https://rubygems.org/'` succeeds before bundling.
remote:
remote: In Gemfile:
remote: or-tools
I did find an issue on the Rice project that seemed to indicate that I might be stuffed due to the way Heroku uses a statically linked Ruby, however it seems someone has been digging into this more recently and figured out that you can in fact use Rice on Heroku, which seems to ring true as the error seems to be a failure to compile Knapsack, which is inside of OR Tools, so I presume it is at least trying to install or-tools with native extensions.
I don't really know where to go from here! Any ideas what to try next?
It would be wonderful to be able to run or-tools on Heroku from ruby/rails. And again, thanks for all your work both here and elsewhere.
Cheers.
Hey @rodreegez, thanks for sharing your approach. The main issue was Heroku runs buildpacks in a temporary directory, so it wasn't able to find OR-Tools.
I made a few improvements as a result of this:
- Improved the error message when OR-Tools isn't found
- Added an experimental
vendorbranch that downloads OR-Tools (will likely merge in the future)
With the 2nd one, you can just point your Gemfile to the branch and deploy to Heroku.
gem 'or-tools', github: 'ankane/or-tools', branch: 'vendor'OR-Tools is cached with the gem, which keeps things fast.
Wow, thanks so much. This seems to have worked perfectly.
Route for vehicle 0:
0 -> 5 -> 7 -> 1 -> 4 -> 3 -> 15 -> 11 -> 12 -> 0
Distance of the route: 2580m
Route for vehicle 1:
0 -> 9 -> 8 -> 6 -> 2 -> 10 -> 16 -> 14 -> 13 -> 0
Distance of the route: 2580m
Maximum of the route distances: 2580m
=> nil
irb(main):004:0>
I removed my buildpack and added the vendor branch of the gem, then deployed. Once deployed, I was able to run heroku console and call my VRPService class which is basically just a straight copy of the VRP example given in this gem's README.
Awesome, thank you so much.
I don't really understand why the buildpack approach didn't work. Is it because OR Tools isn't built as a binary like, say, imagemagick is, and therefore the stuff downloaded and extracted isn't available for rice to link to at bundle time?
Great, will push out a new release with it shortly. Here's my understanding of why the buildpack approach didn't work as is.
The OR-Tools library is needed in two situations:
- Compile time (headers and shared library)
- Run time (shared library)
Buildpacks run in /tmp/build_123456, so /app/or-tools doesn't exist. Since the build directory is different every time, you can't build against a static path (so shouldn't set any bundler config). You also can't link against a static path since the library is moved from /tmp/build_123456 to /app for run time (where /tmp/build_123456 won't exist).
To make it work, you'd need to use an approach similar to Heroku's Apt buildpack. https://github.com/heroku/heroku-buildpack-apt/blob/0357ec1e8863a803ada6c2d57f0850f2c5fa0366/bin/compile#L101-L122
This exports the proper environment to the next buildpack at compile time (/tmp/build_123456 path), as well as creates a profile script, so the library can be found at run time (/app path). LD_LIBRARY_PATH tells the app where to find the shared library at run time.
Does that make any sense?
Just pushed out 0.2.0, and will be deleting the vendor branch after a week, so please update your app as needed.
Thanks man - all updated to use the 0.2.0 gem from RG. Thanks again!