Agoo & Sidekiq::Web

[Miral]

I've been playing around with agoo on a project recently, and the need to add sidekiq came about, however I ran into an issue when it came to getting the Sidekiq UI to work. I'm not entirely sure if this is an agoo problem or a sidekiq problem, or a me problem, but this is where I'm at:

Server setup:

Agoo::Server.init(9393, "root", thread_count: 1, graphql: "/graphql")
Agoo::Server.handle(:GET, "/", Sidekiq::Web)
Agoo::Server.handle(:GET, "/status", Api::Status.new)
Agoo::Server.start
Agoo::GraphQL.schema(Api::Graphql::Schema.new) do
  Agoo::GraphQL.load_file("schema.graphql")
end
sleep

So when it came to adding the Sidekiq::Web route I found that it only worked at the root. If I set the path to /sidekiq I was getting 404 Not Found errors back from agoo. Having it at the root causes another problem, the Sidekiq UI loads, but none of the sections are accessible. Busy/Enqueued/Retry etc etc but maybe if I could get /sidekiq working this would be a non-issue

There's very little documentation for agoo so I'm not sure what to try next besides trying another web server, any suggestions? Thanks

[Peter Ohler]

I haven't tried Sidekiq myself. I assume Sidekiq::Web implements the #call() method. I'll double check but you make need a wildcard for paths such as /sidekick/busy. I'd be very glad to work with you on this and if you have some suggestions on what documentation would add value I'll get that in place too.

[Peter Ohler]

Will put together something in the examples tonight.

[Peter Ohler]

Fiddling around a little I came up with this:

Sidekiq::Web.set :sessions, false
Agoo::Server.handle(:GET, "/sidekiq", Sidekiq::Web)
Agoo::Server.handle(:GET, "/sidekiq/**", Sidekiq::Web)

If you turn up the logging with something like this:

Agoo::Log.configure(dir: '',
		    console: true,
		    classic: true,
		    colorize: true,
		    states: {
		      INFO: true,
		      DEBUG: false,
		      connect: true,
		      request: true,
		      response: true,
		      eval: true,
		      push: false,
		    })

you can see more of what is going on.

Sidekiq is getting the requests but it expects some authentication setup. Are you familiar with how to set up Sidekiq?

[Miral]

Hmm this leads me back to Not Found errors.

Here are the additions i made:

...
Sidekiq::Web.set :sessions, false

Sidekiq::Web.use Rack::Session::Cookie
Sidekiq::Web.use Rack::Auth::Basic do |username, password|
  username == 'hello' && password == 'world'
end

The first line sets a rack.session cookie which I believe clears up the authentication issue https://github.com/mperham/sidekiq/blob/93bfa9d40ebd711e88ac32043306b5e1aa41b6cb/lib/sidekiq/web/csrf_protection.rb#L69

Maybe there's some extra steps required here but this is how I've always setup Sidekiq auth in Sinatra based applications. Judging by the fact the error is back to being a 404 I guess auth is working.

[Peter Ohler]

When logging is turned up you can see that the Not Found is coming from Sidekiq. Is there some other Sidekiq setup required? I can look at some other apps to see what additional steps they take if you can suggest a few. Sinatra might be a place to start if you say it works.

[Miral]

Typically its plug and play, the only real thing you need to so is set a route for Sidekiq::Web, even the authentication can be skipped if you dont mind public access.

This is very old but it's how I've setup another sinatra application so I guess not much has changed in 5 years https://github.com/p8952/sinatra-sidekiq
This might be helpful too https://www.netguru.com/codestories/first-steps-with-sinatra-part-2 (Jump to sidekiq section)

I'll try and do some digging as well. It's also a bit weird that the Sidekiq webpage does show up when the route is / but not when it's /sidekiq

[Peter Ohler]

For me it didn't show up in either but I didn't have the auth set up correctly. I'll dig some more tonight. Day time job now. 😄

[Peter Ohler]

I added a branch named sidekiq-example. In examples/sidekiq/main.rb is the code I am playing with. Here is what I have so far. Stats work. The problem I am running into now is lack of middleware to get past the session check. If you know how to configure that, great. If not then a wrapper class might be needed.

require 'agoo'
require 'sidekiq/web'

# Setting the thread count to 0 causes the server to use the current
# thread. Greater than zero runs the server in a separate thread and the
# current thread can be used for other tasks as long as it does not exit.
Agoo::Log.configure(dir: '',
		    console: true,
		    classic: true,
		    colorize: true,
		    states: {
		      INFO: true,
		      DEBUG: false,
		      connect: true,
		      request: true,
		      response: true,
		      eval: true,
		      push: false,
		    })

class Stat
  def self.call(req)
    stats = Sidekiq::Stats.new
    workers = Sidekiq::Workers.new
    content = %|
<p>Processed: #{stats.processed}</p>
<p>Enqueued: #{stats.enqueued}</p>
<p>In Progress: #{workers.size}</p>
<p><a href='/'>Refresh</a></p>
<p><a href='/add_job'>Add Job</a></p>
<p><a href='/sidekiq'>Dashboard</a></p>
    |
    [ 200, { }, [ content ] ]
  end
end

Agoo::Server.init(6464, 'root', thread_count: 0)

Sidekiq::Web.set :sessions, false
Sidekiq::Web.use Rack::Session::Cookie
Sidekiq::Web.use Rack::Auth::Basic do |username, password|
  username == 'hello' && password == 'world'
end
Sidekiq::Web.use Rack::Auth::Basic do |username, password|
  [username, password] == [Settings.sidekiq.username, Settings.sidekiq.password]
end

Agoo::Server.handle(:GET, "/sidekiq", Sidekiq::Web)
Agoo::Server.handle(:GET, "/sidekiq/**", Sidekiq::Web)
Agoo::Server.handle(:GET, "/", Stat)

Agoo::Server.start()

[Peter Ohler]

Can this be closed?