`Accept: *` results in no media types.

[Natalie Wolfe]

Though * is not a valid value for an Accept header according to the HTTP spec, it is one I have encountered in the wild which caused problems for our service. The test case below shows the difference in the way negotiator handles */* and *. I would expect both values to be treated the same, the intent of * is obvious despite its divergence from spec.

accept.js

var Negotiator  = require('negotiator');
var restify     = require('restify');

var server = restify.createServer();

server.get('/', function(req, res, next){
    var neg = new Negotiator(req);
    res.json({mediaTypes: neg.mediaTypes()});
    next();
});

server.listen(9001);

test

$ node accept.js &
$ curl http://localhost:9001/ -H 'Accept: */*' && echo ""
{"mediaTypes":["*/*"]}
$ curl http://localhost:9001/ -H 'Accept: *' && echo ""
{"mediaTypes":[]}

This was originally reported to restify (restify/node-restify#1009) as an issue with their formatters and they pointed me to you as the source.

[Douglas Wilson]

I have marked it as a bug for now, but need to determine what specifically the specification says should happen (RFC 7231 for reference) before making a final decision.

[Natalie Wolfe]

Unfortunately, the spec does not mention anything about the Accept header having just * as the value. Its only mention of the asterisk as it relates to Accept is this paragraph:

The asterisk "" character is used to group media types into ranges,
with "/" indicating all media types and "type/" indicating all
subtypes of that type. The media-range can include media type
parameters that are applicable to that range.

This leads me to believe the following is the more relevant passage which states that the origin server can chose to either return a 406 or disregard the header.

A request without any Accept header field implies that the user agent
will accept any media type in response. If the header field is
present in a request and none of the available representations for
the response have a media type that is listed as acceptable, the
origin server can either honor the header field by sending a 406 (Not
Acceptable) response or disregard the header field by treating the
response as if it is not subject to content negotiation.

In the special case of * I believe the latter option is the better one as it more closely matches the clients intent.

[Douglas Wilson]

Right, and in fact, I just got done reading the spec and even testing Apache. Typically when the spec is not very clear, I look towards prior art, like Apache and nginx. Apache has the same behavior this module currently does: the default of */* is only applied if there was no header, otherwise whatever is in the header is used (in this case, the header does not contain any valid types, so .mediaTypes() return an empty array).

The .mediaTypes() method is just meant to return the types from the request. If the user of this module wants to disregard the header field, that's pretty simple: just don't perform negotiation when .mediaTypes().length === 0.

Alternatively, if the user of this module wanted to go the 406 route, then that's simple to do by sending a 406 when .mediaTypes().length === 0.

[Douglas Wilson]

Hi @NatalieWolfe, I mostly forgot about this issue and am trying to remember where we left off here. After re-reading through this, it seems like last I said was that the current behavior is expected and matches the other main implementation (Apache/nginx). Does that make sense?

Perhaps the issue may make more sense if you provided some more context on how you are using this module exactly, and why this is causing unexpected behavior. For example, I see in the initial post, you are just doing res.json({mediaTypes: neg.mediaTypes()});, which I assume was just for example, rather than showing how you are trying to make use of this module.

I looked at the linked Restify issue and, as it seems, this may ultimately be some kind of issue between this module as Restify, but I just don't know enough about Restify to really understand the issue (or really have time to investigate how Restify is calling into this module). Would it be possible to encourage someone from Restify to collaborate on a solution here?

[Natalie Wolfe]

Maintaining the Apache/nginx status quo does make sense, but is a little disappointing here. I did originally report this to Restify, however they would rather this be solved in the user space with a middleware that runs before anything else and scrubs the Accepts header manually. That is a solution I find even more disappointing than emulating Apache.

My use case was that an external service was hitting our server with the bad Accepts header. Restify internally uses Negotiator to determine the appropriate formatting for a response (e.g. json, xml, plain text, etc). In this case, because Negotiator was just dropping the, admittedly invalid, * value, Restify could not find a valid content formatter and thus would respond with a 406. If instead Negotiator substituted */* then Restify would use whatever formatter is configured as the default (usually json).

Whether or not Negotiator ignoring the * value is the best behavior is where we disagree. :)

Ultimately, if you are unwilling to step away from Apache's precedent then this ticket may be closed. I have already put in place the band-aid Restify suggested, so this is not a critical issue.

[Douglas Wilson]

Gotcha. I am not willing to depart from the way the other major implementations function. This is typically known as a de-facto standard. In addition, I think we are also behaving just as described in the actual RFC standard for this header. I don't see anything in the standard itself saying that "" should be treated as if it were "/*" so doing that would also be a deviation from the standard, on top of the other major implementations.

The reason I ask about Restify is because they don't have to send a 406 in this situation if they don't want, and this module provides enough information for Restify to make the decision it wants. Ultimately the behavior of treating this as a 406 is left up to Restify, while this module is just a mechanism to provide the information according to the standards.

I hope that makes sense. It doesn't sound to me like anything needs to be changed here from our side, though I understand you disagree. The change belongs in Restify.