Github Actions don't set response.headers['Authorization'] correctly

Question

Github Actions don't set response.headers['Authorization'] correctly

davidwparker opened this issue 3 years ago · comments

Expected behavior

    response.headers['Authorization']

Should be the Authorization header.

(can also use request.env['warden-jwt_auth.token'] which also doesn't work).

Actual behavior

    response.headers['Authorization']

is set to ***.

Everything works fine if run locally. However, when running on Github Actions, it sets the incorrect thing.

Steps to Reproduce the Problem

Use rspec my request specs on Github Actions results in *** instead of the Bearer {JWT}

Debugging information

Provide following information. Please, format pasted output as code. Feel free to remove the secret key value.

Version of devise-jwt in use
devise-jwt (0.8.0)
Version of rails in use
6.1.1
Version of warden-jwt_auth in use
warden-jwt_auth (0.5.0)

Sorry if this isn't the proper place to ask this, but it's really hard to Google or Github search for "***".

I've searched all the files in this repo as well as the warden one, and couldn't find ***.

I'm at a loss for how to run these specs in Github Actions. Thanks!

If needed, can update my repo and output more things like what you have below.

Output of Devise::JWT.config
Output of Warden::JWTAuth.config
Output of Devise.mappings
If your issue is related with not getting a JWT from the server:
- Involved request path, method and request headers
- Response headers for that request

Marc Busqué · Answer 1 · Thu Feb 25 2021 12:48:26 GMT+0800 (China Standard Time)

It looks like your setup is masking sensitive information from the logs. I haven't had the chance to use GH actions yet, but there's an add_mask command which looks like the responsible. I don't think you have an issue here, as it only should affect what you see in the logs but not the actual test data. So, if your tests are failing it's for some other reason with the actual data. Definitely, it's not an issue from our end.

Parker Parker · Answer 2 · Thu Feb 25 2021 22:41:47 GMT+0800 (China Standard Time)

Here's my workflow for GH actions:

env:
  RUBY_VERSION: 3.0.0
  POSTGRES_USER: postgres
  POSTGRES_PASSWORD: postgres
  POSTGRES_DB: test_db
  STRIPE_SECRET_KEY: ${{ secrets.STRIPE_SECRET_KEY }}
  TEST_ENV_NUMBER: 1

name: Rails Specs
on: [push,pull_request]
jobs:
  rspec-test:
    name: RSpec
    runs-on: ubuntu-20.04
    services:
      postgres:
        image: postgres:latest
        ports:
        - 5432:5432
        env:
          POSTGRES_USER: ${{ env.POSTGRES_USER }}
          POSTGRES_PASSWORD: ${{ env.POSTGRES_PASSWORD }}
    steps:
      - uses: actions/checkout@v1
      - uses: actions/setup-ruby@v1
        with:
          ruby-version: ${{ env.RUBY_VERSION }}
      - name: Install postgres client
        run: sudo apt-get install libpq-dev
      - name: Install dependencies
        run: |
          gem install bundler
          bundler install
      - name: Create database
        run: |
          bundler exec rails db:create RAILS_ENV=test
          bundler exec rails db:migrate RAILS_ENV=test
      - name: Run tests
        run: bundler exec rspec spec/requests/projects*
      - name: Upload coverage results
        uses: actions/upload-artifact@master
        if: always()
        with:
          name: coverage-report
          path: coverage

It doesn't seem to have anything about masking, but that seems like it's a good starting point to look. I'll investigate it and let you know what I end up finding out. My hunch is you're correct but maybe it's hidden by default.

Parker Parker · Answer 3 · Thu Feb 25 2021 23:14:08 GMT+0800 (China Standard Time)

This is what I get when I log out the request.env in GH actions:

{"rack.version"=>[1, 3],
 "rack.input"=>#<StringIO:0x000055891b2e07a8>,
 "rack.errors"=>#<StringIO:0x000055891b2e0848>,
 "rack.multithread"=>true,
 "rack.multiprocess"=>true,
 "rack.run_once"=>false,
 "REQUEST_METHOD"=>"POST",
 "SERVER_NAME"=>"www.example.com",
 "SERVER_PORT"=>"80",
 "QUERY_STRING"=>"",
 "PATH_INFO"=>"/users/sign_in",
 "rack.url_scheme"=>"http",
 "HTTPS"=>"off",
 "SCRIPT_NAME"=>"",
 "CONTENT_LENGTH"=>"41",
 "rack.test"=>true,
 "REMOTE_ADDR"=>"127.0.0.1",
 "REQUEST_URI"=>"/users/sign_in",
 "HTTP_HOST"=>"www.example.com",
 "CONTENT_TYPE"=>"application/x-www-form-urlencoded",
 "HTTP_ACCEPT"=>
  "text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5",
 "HTTP_JWT_AUD"=>"test",
 "HTTP_COOKIE"=>"",
 "action_dispatch.parameter_filter"=>[:password],
 "action_dispatch.redirect_filter"=>[],
 "action_dispatch.secret_key_base"=>
  "d9b2485e7f66016be6497c99ac8b87d7a9e72e62c76aea643492ff72c8c970bf048540b87a5368507e26e124564974cc7cad0961d06251f8f7428c3f3e9a9ee1",
 "action_dispatch.show_exceptions"=>false,
 "action_dispatch.show_detailed_exceptions"=>true,
 "action_dispatch.logger"=>
  #<ActiveSupport::Logger:0x0000558919325530
   @default_formatter=
    #<Logger::Formatter:0x000055891932e108 @datetime_format=nil>,
   @formatter=
    #<ActiveSupport::Logger::SimpleFormatter:0x00005589193254b8
     @datetime_format=nil,
     @thread_key="activesupport_tagged_logging_tags:64140">,
   @level=0,
   @logdev=
    #<Logger::LogDevice:0x000055891932dbe0
     @binmode=false,
     @dev=
      #<File:/home/runner/work/useproducer-api/useproducer-api/log/test.log>,
     @filename=
      "/home/runner/work/useproducer-api/useproducer-api/log/test.log",
     @mon_data=#<Monitor:0x000055891932db18>,
     @mon_data_owner_object_id=54420,
     @shift_age=0,
     @shift_period_suffix="%Y%m%d",
     @shift_size=1048576>,
   @progname=nil>,
 "action_dispatch.backtrace_cleaner"=>
  #<Rails::BacktraceCleaner:0x00005589198d3900
   @filters=
    [#<Proc:0x00005589198d3018 /opt/hostedtoolcache/Ruby/3.0.0/x64/lib/ruby/gems/3.0.0/gems/activesupport-6.1.1/lib/active_support/backtrace_cleaner.rb:96>,
     #<Proc:0x00005589198d2e60 /opt/hostedtoolcache/Ruby/3.0.0/x64/lib/ruby/gems/3.0.0/gems/railties-6.1.1/lib/rails/backtrace_cleaner.rb:14>,
     #<Proc:0x00005589198d2e38 /opt/hostedtoolcache/Ruby/3.0.0/x64/lib/ruby/gems/3.0.0/gems/railties-6.1.1/lib/rails/backtrace_cleaner.rb:17>],
   @root="/home/runner/work/useproducer-api/useproducer-api/",
   @silencers=
    [#<Proc:0x00005589198d2f78 /opt/hostedtoolcache/Ruby/3.0.0/x64/lib/ruby/gems/3.0.0/gems/activesupport-6.1.1/lib/active_support/backtrace_cleaner.rb:100>,
     #<Proc:0x00005589198d2f00 /opt/hostedtoolcache/Ruby/3.0.0/x64/lib/ruby/gems/3.0.0/gems/activesupport-6.1.1/lib/active_support/backtrace_cleaner.rb:104>,
     #<Proc:0x00005589198d2de8 /opt/hostedtoolcache/Ruby/3.0.0/x64/lib/ruby/gems/3.0.0/gems/railties-6.1.1/lib/rails/backtrace_cleaner.rb:24>]>,
 "action_dispatch.key_generator"=>
  #<ActiveSupport::CachingKeyGenerator:0x00005589195f3370
   @cache_keys=
    #<Concurrent::Map:0x00005589195f3348 entries=3 default_proc=nil>,
   @key_generator=
    #<ActiveSupport::KeyGenerator:0x00005589195f33c0
     @iterations=1000,
     @secret=
      "d9b2485e7f66016be6497c99ac8b87d7a9e72e62c76aea643492ff72c8c970bf048540b87a5368507e26e124564974cc7cad0961d06251f8f7428c3f3e9a9ee1">>,
 "action_dispatch.http_auth_salt"=>"http authentication",
 "action_dispatch.signed_cookie_salt"=>"signed cookie",
 "action_dispatch.encrypted_cookie_salt"=>"encrypted cookie",
 "action_dispatch.encrypted_signed_cookie_salt"=>"signed encrypted cookie",
 "action_dispatch.authenticated_encrypted_cookie_salt"=>
  "authenticated encrypted cookie",
 "action_dispatch.use_authenticated_cookie_encryption"=>true,
 "action_dispatch.encrypted_cookie_cipher"=>nil,
 "action_dispatch.signed_cookie_digest"=>nil,
 "action_dispatch.cookies_serializer"=>:json,
 "action_dispatch.cookies_digest"=>nil,
 "action_dispatch.cookies_rotations"=>
  #<ActiveSupport::Messages::RotationConfiguration:0x00005589159b21e0
   @encrypted=[],
   @signed=[]>,
 "action_dispatch.cookies_same_site_protection"=>
  #<Proc:0x000055891b2eb950 /opt/hostedtoolcache/Ruby/3.0.0/x64/lib/ruby/gems/3.0.0/gems/railties-6.1.1/lib/rails/application.rb:629>,
 "action_dispatch.use_cookies_with_metadata"=>true,
 "action_dispatch.content_security_policy"=>nil,
 "action_dispatch.content_security_policy_report_only"=>false,
 "action_dispatch.content_security_policy_nonce_generator"=>nil,
 "action_dispatch.content_security_policy_nonce_directives"=>nil,
 "action_dispatch.permissions_policy"=>nil,
 "action_dispatch.routes"=>
  #<ActionDispatch::Routing::RouteSet:0x000055891a0483e8>,
 "ROUTES_60360_SCRIPT_NAME"=>"",
 "ORIGINAL_FULLPATH"=>"/users/sign_in",
 "ORIGINAL_SCRIPT_NAME"=>"",
 "rack.cors"=>
  #<Rack::Cors::Result:0x000055891b2eafa0
   @hit=false,
   @miss_reason="no-origin",
   @preflight=false>,
 "action_dispatch.request_id"=>"12fc6c7c-5755-4959-ad3f-fbc27ade834c",
 "action_dispatch.remote_ip"=>
  #<ActionDispatch::RemoteIp::GetIp:0x000055891b3133d8
   @check_ip=true,
   @ip="127.0.0.1",
   @proxies=
    [#<IPAddr: IPv4:127.0.0.0/255.0.0.0>,
     #<IPAddr: IPv6:0000:0000:0000:0000:0000:0000:0000:0001/ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff>,
     #<IPAddr: IPv6:fc00:0000:0000:0000:0000:0000:0000:0000/fe00:0000:0000:0000:0000:0000:0000:0000>,
     #<IPAddr: IPv4:10.0.0.0/255.0.0.0>,
     #<IPAddr: IPv4:172.16.0.0/255.240.0.0>,
     #<IPAddr: IPv4:192.168.0.0/255.255.0.0>],
   @req=
    #<ActionDispatch::Request POST "http://www.example.com/users/sign_in" for 127.0.0.1>>,
 "warden"=>
  Warden::Proxy:72680 @config={:default_scope=>:user, :scope_defaults=>{}, :default_strategies=>{:user=>[:jwt, :rememberable, :database_authenticatable]}, :intercept_401=>false, :failure_app=>DeviseCustomFailure},
 "warden-jwt_auth.revocation_manager"=>true,
 "warden-jwt_auth.token_dispatcher"=>true,
 "rack.attack.called"=>true,
 "action_dispatch.request.path_parameters"=>
  {:controller=>"sessions", :action=>"create"},
 "devise.mapping"=>
  #<Devise::Mapping:0x00005589192236a0
   @class_name="User",
   @controllers=
    {:confirmations=>"confirmations",
     :passwords=>"passwords",
     :registrations=>"registrations",
     :sessions=>"sessions"},
   @failure_app=Devise::FailureApp,
   @format=nil,
   @klass=#<Devise::Getter:0x0000558919222ed0 @name="User">,
   @modules=
    [:database_authenticatable,
     :rememberable,
     :recoverable,
     :registerable,
     :validatable,
     :confirmable,
     :jwt_authenticatable],
   @path="users",
   @path_names=
    {:registration=>"",
     :new=>"new",
     :edit=>"edit",
     :sign_in=>"sign_in",
     :sign_out=>"sign_out",
     :***"password",
     :sign_up=>"sign_up",
     :cancel=>"cancel",
     :confirmation=>"confirmation"},
   @path_prefix=nil,
   @router_name=nil,
   @routes=[:session, :password, :registration, :confirmation],
   @scoped_path="users",
   @sign_out_via=:delete,
   @singular=:user,
   @used_helpers=[:session, :password, :registration, :confirmation],
   @used_routes=[:session, :password, :registration, :confirmation]>,
 "action_controller.instance"=>#<SessionsController:0x000000000237f8>,
 "action_dispatch.request.content_type"=>
  #<Mime::Type:0x00005589158ab670
   @hash=-2752693084029049473,
   @string="application/x-www-form-urlencoded",
   @symbol=:url_encoded_form,
   @synonyms=[]>,
 "rack.tempfiles"=>[],
 "rack.request.form_hash"=>
  {"user"=>{"login"=>"User1", "password"=>"testtest"}},
 "rack.request.form_vars"=>"user[login]=User1&user[password]=testtest",
 "rack.request.form_input"=>#<StringIO:0x000055891b2e07a8>,
 "action_dispatch.request.request_parameters"=>
  {"user"=>{"login"=>"User1", "password"=>"testtest"}},
 "rack.request.query_string"=>"",
 "rack.request.query_hash"=>{},
 "action_dispatch.request.query_parameters"=>{},
 "action_dispatch.request.parameters"=>
  {"user"=>{"login"=>"User1", "password"=>"testtest"},
   "controller"=>"sessions",
   "action"=>"create"},
 "action_dispatch.request.formats"=>
  [#<Mime::Type:0x00005589158b7e98
    @hash=-1915605856198822225,
    @string="text/html",
    @symbol=:html,
    @synonyms=["application/xhtml+xml"]>],
 "devise.skip_timeout"=>true,
 "devise.allow_params_authentication"=>true,
 "rack.request.cookie_hash"=>{},
 "rack.request.cookie_string"=>"",
 "action_dispatch.cookies"=>
  #<ActionDispatch::Cookies::CookieJar:0x000055891b467ec8
   @committed=false,
   @cookies={},
   @delete_cookies={},
   @request=
    #<ActionDispatch::Request POST "http://www.example.com/users/sign_in" for 127.0.0.1>,
   @set_cookies={},
   @signed=
    #<ActionDispatch::Cookies::SignedKeyRotatingCookieJar:0x000055891b467d38
     @parent_jar=#<ActionDispatch::Cookies::CookieJar:0x000055891b467ec8 ...>,
     @verifier=
      #<ActiveSupport::MessageVerifier:0x000055891b467ab8
       @digest="SHA1",
       @on_rotation=nil,
       @options=
        {:digest=>"SHA1",
         :serializer=>ActiveSupport::MessageEncryptor::NullSerializer},
       @rotations=[],
       @secret=
        "\xFBrB\xE5\x99\x8AD\xC4\xD2\xD0$\x97V\xDB\xA7\xF2\xD3X\xDB\xD8\xE849\xE8>>\xE6w'\x11k\xFD\t\xF4o\xAD\x9C\x14:\xEAT\xEE\xB9\x02\x7Fx/O}=B\xC2j\x94\xE9\xF7\b\x7F\xBC\x9D\xBEY\xA9p",
       @serializer=ActiveSupport::MessageEncryptor::NullSerializer>>>,
 "rack.session"=>{},
 "warden-jwt_auth.token"=>
  "***"}

Notice the last line has the ***

Also this line: :***"password", (which is just this: :password=>"password",

It's not using Rails' Rails.application.config.filter_parameters += [:password] as that is [FILTERED].

Parker Parker · Answer 4 · Fri Feb 26 2021 05:52:30 GMT+0800 (China Standard Time)

Update:
I'm in contact with Github support to see if they can help provide context.

I've create a public repo here, if you want to look (no worries if you don't). If I get it fixed, I'll post the solution here and likely delete the repo:
https://github.com/davidwparker/programmingtil-rails-1/pull/2/checks?check_run_id=1982880533

Parker Parker · Answer 5 · Fri Feb 26 2021 08:49:36 GMT+0800 (China Standard Time)

Final update.

The problem was that I didn't have this set for GH Actions:

devise.rb

    jwt.secret = ENV['DEVISE_JWT_SECRET_KEY']

I thought I had, but then in my GH action file I needed to add it:

env:
  DEVISE_JWT_SECRET_KEY: ${{ secrets.DEVISE_JWT_SECRET_KEY }}

And I set the secret in my Github > Repo settings > Secrets.

The *** was a red herring.

But, GH does mask things they want, without telling users.

This is the feedback from them on that:

Thank you so much for that. We just asked the Actions product team and they mentioned that tokens that resemble the GITHUB_TOKEN also get masked, which is most likely why warden-jwt_auth.token is masked.
The same logic applies for the password text.
I've asked the team if there's a list of these keywords that get masked along with an option to opt out of masking these fields. Currently they mentioned there isn't a way to opt out, but they have received customer feedback for it and I can add your report to the list.
Again, I'm terribly sorry we don't have an immediate workaround. Currently the expected behaviour is to err on the side of caution, which is why we tend to mask any fields that resemble a password or token.

BTW, I added a workflow PR for your repo (but looks like you already use travis).

You can see it runs okay here:
https://github.com/davidwparker/devise-jwt/pull/1/checks?check_run_id=1983875591
Though on your main repo I don't see it run, probably because you have travis already. I'm not sure though:
#204

Feel free to ignore it if you don't want it. But just note that it works!

Florian Vallen · Answer 6 · Mon Nov 07 2022 18:14:04 GMT+0800 (China Standard Time)

Thank you so much! That helped me solve the same issue!
My CI pipeline is finally working. Also I managed to run the github actions workflow locally and reproduce the failed tests locally with act. Since services aren't supported in act, I had to install postgres in the same container (only for act). You can see my workflow file here