Authentication
rashidkpc opened this issue · comments
Kibana authentication layer, phase 1
In the first phase of Kibana’s authentication system we’re focus on just that: authentication. That is, a clean way of allowing for login, logout and sessions that doesn’t require HTTP basic auth to be configured or a proxy to be setup. Authentication will be disabled by default, but enabled via kibana.yml
The login screen
Like any other web application this provides the ability to enter a username and password which will be sent to the server for validation. If validation succeeds the user is allowed into the application, if not they are returned to the login screen.
The session
Before even presenting the login screen we need to check if the user is already logged in. This means we need to keep a session.
Because Elasticsearch has no built in authentication mechanism we don’t consider it a secure location for session storage, thus the user’s client will need to be the authority on the session. Here we’ll use an encrypted cookie containing username:password:timestamp which will be encrypted by the server using a key provided in kibana.yml.
Potential pitfall: Session stealing
Because the cookie will contain everything the user needs to login, it will be vulnerable to session stealing like many other applications. To mitigate this concern we 2 lines of defense:
- Session timeouts Also configurable in
kibana.ymlshould be the session timeout. The server should validate the timestamp on the cookie and redirect the user to the login screen if it is older than the configured timeout allows. The timestamp on the cookie should be updated on every request. This could be infinite, however we should not recommend this. - Required SSL If the user is using authentication, SSL is required. The server should fail to start if the user does not have SSL turned on, but does have authentication turned on.
Authentication strategies
Regardless of if the user is coming with existing credentials in a cookie, or via a new login, the user will need to be authenticated server side in some way. The user will in fact be re-validated with every request, thus allowing for users to be quickly invalidated simply by removing them from the auth system. Initially we’ll provide a pluggable authentication system with 2 pre-built modules:
- .htpasswd This simple method works just like apache’s, or Shield’s user list. A file containing user names and hashed passwords. The kibana backend will check this file and either allow or deny the user.
- Basic auth proxy In this case, the user’s credentials will be relayed to Elasticsearch’s
/endpoint for authentication. If elasticsearch returns a 200, we let the user in, otherwise the user is logged out and returned to the login screen. In addition, the user’s credentials are included on every request to Elasticsearch from the backend. If at any point we receive a 401, the user will be logged out.
Implementing authentication strategies
The auth strategy module should contain 3 functions:
- init(server) Run to initialize the authentication strategy.
- authenticate(request, username, password) Run when attempting to establish the session for the first time, if the user does not have a cookie. It should return a promise that succeeds or fails. If it succeeds the user is issued a cookie, otherwise they are returned to the login screen.
- validate(request, session) (Optional) Run on every request, returns a promise that succeeds or fails. If it fails the user is logged out. The request can be modified here before being sent, including modifying cookies, headers, etc.
Logout
We should add a logout button to the navigation bar. When the user logs out they should be return to the login screen.
Some feedback:
- +1 for including a timestamp in the session key.
- add logging of events like login (successful or failed) and session timeouts.
- add support for both basic & digest authentication password formats in .htpasswd (see https://httpd.apache.org/docs/2.2/misc/password_encryptions.html)
+1 use passport for authentication https://www.npmjs.com/package/passport
All this exists already in the passport module.
- Using http basich auth / .htpasswd is not sufficent - epecially enterprise users might want LDAP/Kerberos and other methods.
- I would suggest to expose an API Hook (like you have already in auth.js) or even better integrate passport and read a configuration file (or .js file) for passport (which could be basic auth / .htpasswd as default)
- The complete list of authentication Strategies supported by passport:
https://github.com/jaredhanson/passport/wiki/Strategies
+1 for passport authentication.
Passport apport many authentication Strategies
Easy way to add functionalities without many effort.
👍
My two cents: Enable authentication by default.
a) Newbies who test deploy on internet servers do not expose data unintentionally
b) Pros will do configuration anyway, can simply switch it off
In case you wonder why: because humans make mistakes. And that would be bad press for Elastic.
Students find 40k unprotected MongoDB databases, 8 million telco customer records exposed
.htpasswd is in my opinion not a good way. Hard to manage and basically just a pain. Implementing this will be a waste of time because eventually other and better options will be made.
Basic auth proxy is a valid way because then the user can use whatever authentication method on the reverse proxy. Nginx, Apache, IISARR, F5, Citrix, anything at all. For many people means that they dont have to change a thing since many are already using authentication on the reverse proxy.
If this is used then SSL should be on the reverse proxy side and optional on the Kibana side.
But yes +1 for passport, people need LDAP/Kerberos.
Are there any plans now for ACLs?
Well, +1 for passport authentication.
+1 for passport. This would allow me to use SAML
+1for LDAP/AD authentication.
+1000 for auth-on-by-default.
+1 for LDAP/AD authentication integration, or at least for using a framework (Passport!) that allows integration with tons of providers.
+1024
great
+1 for LDAP/AD auth
+1 For LDAP/AD Auth
+1 For passport
Suggestion to use JWT for encryption
Will be happy to help for implementation for any of the functionality needed for authentication.
+1 for passport!
What about different user types?
Such as Basic Users (read-only), Priviledged users (people that can change stuff and create visualizations, etc) and Power Users/Admins (People that can manage who has access).
Having support for that would be awesome (on top of authorization).
What about OAuth? We already use that extensively elsewhere in our organization, and it's what we use in the wrapper auth proxy we have around kibana now.
I, for one, would really, really, really rather not have to support yet another custom, one-off authentication repository.
SAML/Getaccess would be nice too
@rashidkpc, are you working on this internally or smb can make a fork and make it happen?
Please support Passport. Since it already supports many strategies (including OAuth2), it will cover most use cases.
for CAS auth
+1 for LDAP
👍
+1 for LDAP
+1 for LDAP
+1 for LDAP
+1 for LDAP
+1
+1 for CAS
Does anyone noticed that when using a CAS with a Proxy the Kibana 4.2 stopped to work well?
I had a proxy authentincation (Apache) using Oracle Access Manager (CAS) working fine with Kibana 4.1.2 over a Cent OS.
Sadly, when I tried to make the same configuration on Kibana 4.2 I noticed that the integration doesn't work anymore because some new implementation conflicts with the cookie created by the authenticator. If a turn on the Oracle Access Manager to authenticate, the effect is that, besides using the same configuration used with Kibana 4.1.2, the Kibana 4.2 crashes and I get an error message like this: "{"statusCode":400,"error":"Bad Request","message":"Invalid cookie value"}"
+1
You should see on grafana as prior art. There is a lot of options there out of the box
+1
+1
I don't think anyone needs to +1 this issue; it's assigned and on the roadmap. You just notify 30+ other people with your +1.
What about adding a specific filter to all requests coming from a specific user? for example for api key filtering
Waiting for this feature. What is the planned release date for this?
For your inspiration: Grafana API (already available) - having pluggable authentication for integrations with 3rd parties
http://docs.grafana.org/reference/http_api/
Any plans to support Azure Active Directory authentication?
@tamilsweet I am wondering the same thing, is it available in any Kibana 5 snapshots?
Any news about passport auth ? is it plan to support it soon ?
Thanks a lot
+1
+1
@rashidkpc @lukasolson in Bitergia we plan to invest effort next weeks in authentication and basic authorization (read only and read and write). Right now we are using reverse proxy for auth but if we can help in this Authentication issue for Kibana (is there some proof of concept or some code already done?) it could be great.
We've folded support for authentication into Elastic's Shield plugin: https://www.elastic.co/guide/en/shield/current/kibana.html. Authentication against custom services can be accomplished through Shield support for custom realms: https://www.elastic.co/guide/en/shield/current/custom-realms.html
We decided to go in this direction because we wanted Elasticsearch fully protected and integrating with shield allows us to use the same auth system whether coming through Kibana or going directly to elasticsearch
@rashidkpc Makes sense, but will Shield still be behind enterprise support contracts or might we see a subset of Shield opensource/free with limited basic features that can be used for Kibana authentication?
Shield is only available with a subscription, I can't be sure what the future holds.
Elastic software is great, but I do not like that it is using security as a commercial selling point.
Like @elvarb suggests: make a subset of Shield opensource, so users can protect themselves and Elastic can say its taking security of its software and all its users very seriously.
Users should want a subscription by themselves. Not being forced to it.
Exactly @rtoma, Shield is in itself is a fantastic solution that goes so much further than what most people are doing already with reverse proxies. If we compare ELK to something like Graylog then Graylog has two features that ELK lacks, Authentication and Alerting. These two features are becoming more and more essential and I fear that some will move to other solution than ELK because these are lacking.
So what I would love to see Elastic try to keep ELK as close to the competition as possible when it comes to logging. When it comes to other use cases than logging for Elasticsearch and ELK as a whole then the full feature set of Shield and Watcher are unbeatable. So just some basic functionality of Shield and possibly Watcher would help Elastic immensely.
The logging use case of ELK is just the gateway to more business focused use cases.
Seconded @rtoma. Though anyone paying attention to this issue should've noticed by now that elastic.co isn't going to undermine their own product offerings. I'd rather they leverage some different aspect than security for this.
With kibana3 it was at least doable to hack your own very basic authentication/authorization framework on top by just doing URL filtering in a reverse proxy. This was enough for me to introduce ELK at my clients and later on help them towards a commercial support license, once the value of this tooling has been demonstrated. This is not possible anymore with kibana4, and kibana3/facets have been deprecated as of ES2.0 (with good reason).
Still this all feels very forced..
I have to agree with @rtoma and @pieterlange here - the fact that just basic access control is put behind a paywall with Shield struck me as sketchy from the moment I saw it (but I was lead to believe that via this issue some very fundamental authentication would be put in place). I don't need consulting / support, monitoring, alerting, etc. - all I need is for my client's data in Kibana not to be public to the world by default.
Yeah, I can lock it down using some reverse proxy or iptables rules, but I shouldn't have to do that.
Worse yet, you can't even purchase a license for Shield separately from a subscription (last I checked), and subscription fees aren't public - you'll have to get a quote first.
I had asked for the subscription and its just not afordable. When i asked it was around 25000€ / per year.
Hopefully this does not lead to a fork of Kibana which would split communities.
Would love to see a free plugin for that.
I couldn't agree more strongly with the above sentiments — putting authentication behind Shield makes it functionally inaccessible to 90+% of potential (and current!) Kibana users. We too asked for quotes for subscriptions, and were absolutely stunned at the cost — for our simple setup, purely used for logging, we'd have to spend nearly $50K a year, so that's the price of not wanting our Kibana data available on our entire network.
@rashidkpc, please PLEASE rethink this decision.
Sorry gang, not my decision to make
Why isn't a reverse proxy with oauth or any other auth a viable solution for Kibana 4? I set this up on my network and it seems to work fine. There is no ACL obviously. It sounds like for that you need a commercial license. Seems fair enough. What am I missing?
@rashidkpc, I understand. Its a company decision. Github is not the place to talk to a company, so I did this tweet: https://twitter.com/rtoma_/status/694897025440169984 - Let's see what happens.
@jhovell that is exactly the point. The only access control you can do with a reverse proxy is authentication and then method calls, limit PUT, POST, DELETE and so on but that does not work as well as it did in Kibana3 as far as I know.
The best we can do for now is limiting to methods on Kibana4 or at worst we will have to limit the methods between Kibana4 and Elastcsearch and run two instances of Kibana4 with different authentication profiles. One for read only and one for read and write.
I've been experimenting a little with on the fly request rewriting of
POSTed data. It's not a perfect ACL by any means, but does mean you can add
"should" filters to the various API calls (like _msearch, that Kibana uses
a lot) to restrict the result set.
This uses ngx_http_perl module in Nginx - it does effectively mean though, I'm double-proxying Kibana - once at the front to do user authentication, and once at the back to do the authorization elements.
If Kibana allowed a plugin to hook into the "mapUri" piece of the es proxy, then a plugin could hack in authentication to prevent access to certain areas of es. I've done that to limit access to certain indices to certain people and it has worked fairly well.
So, it's possible or not to enable authentication only in Kibana? I'm talking about the basic login/logout screen, user and password. @rashidkpc
Kibana forwards any basic HTTP auth, if one is required to access the elasticsearch backend.
So this enables you to protect a Kibana that is accessible on the Internet.
And your elasticsearch backend is only accessed through Kibana since v4, so it is not exposed.
However this alone does not perform any filtering on the requests send to elasticsearch.
For this, you'll have to write a proxy (using nginx with lua scripting, for instance) before elasticsearch, that filters and rewrites requests, and responses if necessary.
You can even load per credentials configuration to allow different users to access different indices/aliases, or views.
Basically this is like rewriting Shield, intercepting and checking every access.
If not your decision to make @rashidkpc, then whose?
hey, I answered similar questions in the context of Elasticsearch when we released shield. I highly recommend reading it: elastic/elasticsearch#664 (comment), it is valid to this discussion as well. I would also read the comments to what I wrote, I found them to be very constructive.
Nothing much changed since then. We still plan at one point to make parts of shield free, no concrete dates though. We do offer almost of all shield functionality as part of our Elasticsearch as a Service Standard offering (https://www.elastic.co/found) as its simpler there. One thing actually did change, we are now ~150 engineers :).
Thanks for the update @kimchy, glad that mini shield is on the plans. Would make everyone very happy if that will be available at the same time kibana authentication with big shield is released. One big announcement, useable for all.
@kimchy On that Elasticsearch as a Service page, Shield is explicitly called out as only part of the Premium service — are you saying that's not the case?
(And that page highlights another issue — it's easy to know how much we'd be in for if we went with the Standard service, but in order to find out more about how much Premium would cost, there's just "contact us". I'm sure I'm not the first, and not even the hundredth, to say that that in and of itself means you lose potential customers... it is much harder to consider a product when there are barriers to getting accurate pricing information at the time the research is being done.)
@delfuego in the context of Found (our Elasticsearch and Kibana as a Service), agreed, we need to better list the features you get in each lane. Certain aspects are embedded in the Found service regardless (message signing for example). We recently decided to offer most of the features of Shield in our Standard offering (so you can expect RBAC and so on to be part of it regardless of size, and Kibana session management). We are working on updating the site to properly reflect it.
Also, regarding Premium, which includes our dedicated support (see more here: https://discuss.elastic.co/t/typical-response-times-for-found-support/39925/2), we took our time (this is new!) to refine our pricing model for it (as you an imagine, a dedicated support engineer has cost associated with it). I believe now we are in a good place, so definitely considering publishing the pricing.
I wrote up a plugin that enables oauth2 authentication using Bell (https://github.com/hapijs/bell). The plugin is at https://github.com/trevan/oauth2. I believe I can also enhance it to do authorization (such as prevent editing of dashboards/visualizations or restrict which indices are available per user).
@trevan thank you for sharing it.
Others hitting this issue might also be interested in https://github.com/floragunncom/search-guard
We run Kibana behind the proxy and require users to be logged in.
Here is a Kibana plugin (https://github.com/kadishmal/kibana-express-session) that reads the session data from Redis saved previously using express-session in our main app.
The actual login process is handled by our main app outside Kibana.
@kimchy has there been any further clarity on when a free basic version of Shield would become available?
@johnnyshields no, nothing concrete yet.
Why it is a good idea to provide basic authentication and authorization. Yes, it's probably the admin's fault, but still the software is delivered with all ports wide open by default.
"Online databases dropping like flies, with >10k falling to ransomware groups"
http://arstechnica.com/security/2017/01/more-than-10000-online-databases-taken-hostage-by-ransomware-attackers/
Yes, it's probably the admin's fault, but still the software is delivered with all ports wide open by default.
To be fair, since Elasticsearch 2.0 the default is to only bind to localhost only.
You're still able to reach es through kibana...
any news?