Authentication failure! csrf_detected(Rack 3.0.8)

Question

Authentication failure! csrf_detected(Rack 3.0.8)

viktor-shmigol opened this issue 8 months ago · comments

Hello,
I'm having an error: Authentication failure! csrf_detected on the Production env.
I identified the problem and found it happens when I use the latest gem rack v3.0.8.
However, if I downgrade it to v2.2.8, it's working without an issue.

ruby version: 3.2.2
rails version: 7.1.1
omniauth-rails_csrf_protection 1.0.1
omniauth-google-oauth2 version: 1.1.1

Thanks

Peter Gumeson · Answer 1 · Tue Oct 31 2023 04:21:11 GMT+0800 (China Standard Time)

Getting similar issue as well. Suspect may be related to rack/rack#2128

You didn't post a stack trace @viktor-shmigol but does this look like what you're getting?

Started POST "/users/auth/google_oauth2" for 127.0.0.1 at 2023-10-30 13:34:32 -0600
(google_oauth2) Request phase initiated.
(google_oauth2) Authentication failure! ActionController::InvalidAuthenticityToken: ActionController::InvalidAuthenticityToken, ActionController::InvalidAuthenticityToken
  
ActionController::InvalidAuthenticityToken (ActionController::InvalidAuthenticityToken):
  
omniauth-rails_csrf_protection (1.0.1) lib/omniauth/rails_csrf_protection/token_verifier.rb:34:in `call'
omniauth (2.1.1) lib/omniauth/strategy.rb:240:in `request_call'
omniauth (2.1.1) lib/omniauth/strategy.rb:193:in `call!'
omniauth (2.1.1) lib/omniauth/strategy.rb:169:in `call'
omniauth (2.1.1) lib/omniauth/builder.rb:44:in `call'
rack (3.0.8) lib/rack/tempfile_reaper.rb:20:in `call'
rack (3.0.8) lib/rack/etag.rb:29:in `call'
rack (3.0.8) lib/rack/conditional_get.rb:43:in `call'
rack (3.0.8) lib/rack/head.rb:15:in `call'
actionpack (7.1.1) lib/action_dispatch/http/permissions_policy.rb:36:in `call'
actionpack (7.1.1) lib/action_dispatch/http/content_security_policy.rb:33:in `call'
rack-session (2.0.0) lib/rack/session/abstract/id.rb:272:in `context'
rack-session (2.0.0) lib/rack/session/abstract/id.rb:266:in `call'
actionpack (7.1.1) lib/action_dispatch/middleware/cookies.rb:689:in `call'
activerecord (7.1.1) lib/active_record/migration.rb:653:in `call'

Samuel Williams · Answer 2 · Tue Oct 31 2023 04:49:00 GMT+0800 (China Standard Time)

Can you share the HTML for the key/value which might be causing the problem?

Peter Gumeson · Answer 3 · Tue Oct 31 2023 05:02:42 GMT+0800 (China Standard Time)

Sure @ioquatix, but I'm guessing it won't be too helpful as it's just the form params that get sent to google oauth endpoint.

<form data-turbo="false" action="/users/auth/google_oauth2" accept-charset="UTF-8" method="post"><input type="hidden" name="authenticity_token" value="...redacted..." autocomplete="off">
    <button class="btn btn-primary" type="submit">Login with Google</button>
</form>

Peter Gumeson · Answer 4 · Tue Oct 31 2023 05:04:44 GMT+0800 (China Standard Time)

I can also verify that downgrading rack back down to v2.2.8 (from v3.0.8) fixes the issue without changing any other code.

Samuel Williams · Answer 5 · Tue Oct 31 2023 05:57:36 GMT+0800 (China Standard Time)

name="authenticity_token"

Looks okay to me, is there some other part we are missing?

Prem Sichanugrist · Answer 6 · Tue Oct 31 2023 10:05:18 GMT+0800 (China Standard Time)

Quickly skim through Rack CHANGELOG and I couldn't spot anything that'd break. Also, doesn't look like Rails' ActionController::RequestForgeryProtection in which we piggyback from has any changes that would affect us either.

I'll investigate further. Thank you for the report.

Tomás Coêlho · Answer 7 · Tue Oct 31 2023 19:56:03 GMT+0800 (China Standard Time)

The same issue is also happening with me, with a identical backtrace. I'd like to highlight that it only happens in production environments, so I think it may be related to the middleware stack, but had no success debugging 😔.

Steve Hobbs · Answer 8 · Thu Nov 30 2023 20:53:48 GMT+0800 (China Standard Time)

@sikachu did you manage to find anything in your investigation? I maintain a library that uses this gem and one of our customers is reporting that they're experiencing this as well.

Scott Sherwood · Answer 9 · Fri Dec 01 2023 15:49:54 GMT+0800 (China Standard Time)

Following as I've just come across the following error when rack >3 is used with rails 7.1

Prem Sichanugrist · Answer 10 · Fri Dec 22 2023 18:16:45 GMT+0800 (China Standard Time)

Hello. Sorry for the wait, I just had time to look into this again.

I was trying to reproduce this using a newly-created Rails app, but I wasn't able to reproduce it. (The app lives here). I tried both in production and development environment, as reported, and I was able to get to do the full OAuth flow using Google provider.

You can see that I'm using Rack 3.0.8 here: https://github.com/sikachu/omniauth-rails_csrf_protection-issue-15/blob/main/Gemfile.lock#L192

Since I think I'm hitting a brick wall in investigating this, would you mind providing more information?

I created a special branch which will print a log message like this in the log, for params inspection:

D, [2023-12-22T18:54:54.231573 #30484] DEBUG -- omniauth: (google_oauth2) Request phase initiated.
D, [2023-12-22T18:54:54.232585 #30484] DEBUG -- omniauth: (token_verifier) {"authenticity_token"=>"..."}

Would you mind pointing your app to this branch, then checking the log for the params, if it contains the authenticity_token key?

gem "omniauth-rails_csrf_protection", github: "cookpad/omniauth-rails_csrf_protection", branch: "params-debug"

Would you mind posting the result of rails middleware in the environment that this problem occurs?
```
RAILS_ENV=production rails middleware
```
Would you mind letting me know the session store of the environment that this problem occurs?

Scott Sherwood · Answer 11 · Mon Jan 08 2024 18:46:37 GMT+0800 (China Standard Time)

Hi

@sikachu Thanks for looking into this. To answer your questions:

This was tested with with 'omniauth-auth0', '~> 3.1.1' rather than the Google version. (But I belive its the same issue as the google users above are seeing.)

After installing the debug branch, the error outputted in the logs is as follows:

App 3186006 output: D, [2024-01-08T10:34:10.830840 #3186006] DEBUG -- omniauth: (auth0) Request phase initiated. App 3186006 output: D, [2024-01-08T10:34:10.831325 #3186006] DEBUG -- omniauth: (token_verifier) {"authenticity_token"=>"h1KsqwTy5BlsBbFCBdM7dGfTin-DJ0AIyfWWfd-4k3KGkbw8vx-hYmRxB2RWcb1Rf0HS5NxEBQrRmBhMZE8E_w", "commit"=>"Login"} App 3186006 output: D, [2024-01-08T10:34:12.687879 #3186006] DEBUG -- omniauth: (auth0) Callback phase initiated. App 3186006 output: E, [2024-01-08T10:34:12.688333 #3186006] ERROR -- omniauth: (auth0) Authentication failure! csrf_detected: OmniAuth::Strategies::OAuth2::CallbackError, csrf_detected | CSRF detected

Session store:
Rails.application.config.session_store :cookie_store, key: '_sess', same_site: :lax

Thanks

futura · Answer 12 · Thu Feb 08 2024 23:58:55 GMT+0800 (China Standard Time)

Having the same issue. I just upgraded gems, which moved rack from v2.x to v3.0.8, and started getting ActionController::InvalidAuthenticityToken.

After visiting here, I explicitly set Gemfile to gem 'rack', '~> 2.0', which downgraded rack to 2.2.8, and it's working again.

$ rails middleware
use ActionDispatch::HostAuthorization
use Rack::Sendfile
use ActionDispatch::Static
use ActionDispatch::Executor
use ActionDispatch::ServerTiming
use ActiveSupport::Cache::Strategy::LocalCache::Middleware
use Rack::Runtime
use Rack::MethodOverride
use ActionDispatch::RequestId
use ActionDispatch::RemoteIp
use Sprockets::Rails::QuietAssets
use Rails::Rack::Logger
use ActionDispatch::ShowExceptions
use WebConsole::Middleware
use ActionDispatch::DebugExceptions
use ActionDispatch::ActionableExceptions
use ActionDispatch::Reloader
use ActionDispatch::Callbacks
use ActiveRecord::Migration::CheckPending
use ActionDispatch::Cookies
use ActionDispatch::Session::ActiveRecordStore
use ActionDispatch::Flash
use ActionDispatch::ContentSecurityPolicy::Middleware
use ActionDispatch::PermissionsPolicy::Middleware
use Rack::Head
use Rack::ConditionalGet
use Rack::ETag
use Rack::TempfileReaper
use OmniAuth::Builder
run Ulticonnector::Application.routes

futura · Answer 13 · Fri Feb 23 2024 10:05:46 GMT+0800 (China Standard Time)

So the error logged by @scottsherwood is occurring here?

https://github.com/omniauth/omniauth-oauth2/blob/master/lib/omniauth/strategies/oauth2.rb in callback_phase lines 86-87.

Not sure how this fits in, but hopefully it points in the right direction?

Prem Sichanugrist · Answer 14 · Fri Mar 15 2024 10:58:39 GMT+0800 (China Standard Time)

Sorry for the slow update.

The other day, @nevans opened #16 and reported an issue with thread safety, which could be the root cause of this mismatch (I was only tested this in web server with fork mode).

Would you mind pointing to main branch and test out if it solves the issue? If so, I'll cut a new release.

Scott Sherwood · Answer 15 · Sat Mar 16 2024 18:13:57 GMT+0800 (China Standard Time)

After a little more testing, I believe the issue that I was facing was related to phusion/passenger#2503

Updating passenger to the most recent version, which includes a fix, has resolved the issue.

Viktor Shmigol · Answer 16 · Thu Mar 21 2024 23:14:58 GMT+0800 (China Standard Time)

@scottsherwood I've upgraded the passenger docker image to phusion/passenger-ruby32:3.0.2.
I can confirm the issue is disappeared.
It's working even with rack 3.0.10.
Thank you for helping!

Prem Sichanugrist · Answer 17 · Fri May 10 2024 18:32:16 GMT+0800 (China Standard Time)

Since I think this issue was resolved by upgrading Passenger, I'm going to close this issue.

Thank you very much for everyone's input for nailing down to the affected service 🙇‍♂️