Another case of my website not being compressed properly.

Question

Another case of my website not being compressed properly.

brstillwell opened this issue 6 years ago · comments

Hi, I have been reading the issues from others with the same exact problem and I have been doing everything they have, but I have still not been able to get my website to compress properly. I have deleted my modules and cleared the cache and reinstalled everything and still no luck. When I run DEBUG=compression node . I can see that there is some compression happening

  compression no compression: already encoded +3ms
  compression gzip compression +127ms
  compression no compression: already encoded +1ms
  compression gzip compression +66ms
  compression no compression: already encoded +0ms
  compression gzip compression +13ms
  compression no compression: already encoded +0ms
  compression gzip compression +4ms
  compression no compression: already encoded +0ms
  compression gzip compression +11ms
  compression no compression: already encoded +0ms
  compression gzip compression +7ms
  compression no compression: already encoded +1ms
  compression gzip compression +19ms
  compression no compression: already encoded +1ms
  compression gzip compression +48ms
  compression no compression: already encoded +1ms
  compression gzip compression +10ms
  compression no compression: already encoded +0ms
  compression gzip compression +5ms
  compression no compression: already encoded +0ms
  compression gzip compression +9ms
  compression no compression: already encoded +1ms
  compression gzip compression +22ms
  compression no compression: already encoded +0ms
  compression gzip compression +14ms
  compression no compression: already encoded +1ms
  compression gzip compression +46ms
  compression no compression: already encoded +0ms
  compression gzip compression +11ms
  compression no compression: already encoded +0ms
  compression gzip compression +97ms
  compression no compression: already encoded +0ms
  compression image/png not compressible +108ms
  compression no compression: filtered +0ms
  compression image/png not compressible +0ms
  compression no compression: filtered +0ms
  compression undefined not compressible +51ms
  compression no compression: filtered +0ms
  compression undefined not compressible +1ms
  compression no compression: filtered +0ms
  compression image/png not compressible +14ms
  compression no compression: filtered +0ms
  compression image/png not compressible +0ms
  compression no compression: filtered +0ms
  compression image/png not compressible +6ms
  compression no compression: filtered +1ms
  compression image/png not compressible +0ms
  compression no compression: filtered +1ms
  compression image/png not compressible +0ms
  compression no compression: filtered +1ms
  compression image/png not compressible +0ms
  compression no compression: filtered +0ms
  compression image/png not compressible +38ms
  compression no compression: filtered +0ms
  compression image/png not compressible +0ms
  compression no compression: filtered +0ms
  compression image/png not compressible +16ms
  compression no compression: filtered +0ms
  compression image/png not compressible +0ms
  compression no compression: filtered +0ms
  compression image/png not compressible +1ms
  compression no compression: filtered +0ms
  compression image/png not compressible +0ms
  compression no compression: filtered +1ms
  compression image/png not compressible +11ms
  compression no compression: filtered +1ms
  compression image/png not compressible +0ms
  compression no compression: filtered +0ms
  compression gzip compression +52ms
  compression no compression: already encoded +0ms
  compression gzip compression +135ms
  compression no compression: already encoded +0ms
  compression gzip compression +233ms
  compression no compression: already encoded +0ms
  compression undefined not compressible +916ms
  compression no compression: filtered +0ms
  compression undefined not compressible +0ms
  compression no compression: filtered +0ms
  compression undefined not compressible +1s
  compression no compression: filtered +0ms
  compression undefined not compressible +0ms
  compression no compression: filtered +0ms
  compression undefined not compressible +996ms
  compression no compression: filtered +1ms
  compression undefined not compressible +0ms
  compression no compression: filtered +0ms
  compression undefined not compressible +4ms
  compression no compression: filtered +0ms
  compression undefined not compressible +0ms
  compression no compression: filtered +0ms
  compression undefined not compressible +1s
  compression no compression: filtered +1ms
  compression undefined not compressible +0ms
  compression no compression: filtered +0ms
  compression undefined not compressible +994ms
  compression no compression: filtered +0ms
  compression undefined not compressible +0ms
  compression no compression: filtered +0ms
  compression undefined not compressible +1s
  compression no compression: filtered +0ms
  compression undefined not compressible +0ms
  compression no compression: filtered +0ms

but it does not have the header Content-Encoding: gzip

Here is my server.js

'use strict';
var loopback = require('loopback');
var boot = require('loopback-boot');
var compression = require('compression');
var bodyParser = require('body-parser');
var $ = require('jquery');
var app = module.exports = loopback();
var session = require('client-sessions');
var path = require('path');
app.engine('pug', require('pug').__express);
app.set('views', path.join(__dirname, '../views'));
app.set('view engine', 'pug');

app.use(bodyParser.json());
app.use(bodyParser.urlencoded({extended: true}));
app.use(compression({ threshold: 0 }));
app.use(loopback.static(__dirname+'../views'));

app.use(session({
  cookieName: 'session',
  secret: 'random_string_goes_here',
  duration: 30 * 60 * 1000,
  activeDuration: 5 * 60 * 1000,
}));

app.start = function() {
  // start the web server
  return app.listen(function() {
    app.emit('started');
    var baseUrl = app.get('url').replace(/\/$/, '');
    console.log('Web server listening at: %s', baseUrl);
    if (app.get('loopback-component-explorer')) {
      var explorerPath = app.get('loopback-component-explorer').mountPath;
      console.log('Browse your REST API at %s%s', baseUrl, explorerPath);
    }
  });
};
boot(app, __dirname, function(err) {
  if (err) throw err;
  if (require.main === module)
    app.start();
});

Douglas Wilson · Answer 1 · Fri Jan 12 2018 05:33:13 GMT+0800 (China Standard Time)

Thanks for the report @brstillwell . I can definitely debug through the code to figure out what is happening if you can help me get your code up and running on my machine. Can you provide all the following?

Version of Node.js you're running on
Version of this module and all the other modules you're requiring in your code there so I can get everything aligned like you have.
Instrucitons for the request to make to your app that you're expecting to see compressed.

Thanks!

Brandon Stillwell · Answer 2 · Fri Jan 12 2018 06:04:36 GMT+0800 (China Standard Time)

Node version: v6.11.2
Modules:

"body-parser": "^1.17.2",
    "browserify": "^14.4.0",
    "client-sessions": "^0.8.0",
    "compression": "^1.0.3",
    "cors": "^2.5.2",
    "helmet": "^1.3.0",
    "loopback": "^3.0.0",
    "loopback-boot": "^2.6.5",
    "loopback-cache": "^0.2.0",
    "loopback-component-explorer": "^4.0.0",
    "loopback-connector-postgresql": "^2.8.0",
    "pg-promise": "^7.3.3",
    "pug": "^2.0.0-rc.4",
    "serve-favicon": "^2.0.1",
    "strong-error-handler": "^2.0.0",
    "tilestrata-headers": "^0.3.0"

These are the dependencies that are being required or are in my middleware.

Is this what you mean by the instructions?

module.exports = function (app) {
  app.get('/', function (req, res) {
    //res.setHeader('Content-Encoding', 'gzip');
    res.render('leafletMap.pug');
  });
  };

Douglas Wilson · Answer 3 · Fri Jan 12 2018 06:09:49 GMT+0800 (China Standard Time)

By instructions I mean describe what you're doing to look for that Header. There have been many cases where how the user was looking for the header to be there was buggy (for example Chrome doesn't always show it if you're using the inspector and I don't know why). I installed those dependencies but I'm not able to get your app working:

$ node app.js
module.js:471
    throw err;
    ^

Error: Cannot find module 'jquery'
    at Function.Module._resolveFilename (module.js:469:15)
    at Function.Module._load (module.js:417:25)
    at Module.require (module.js:497:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> (app.js:6:9)
    at Module._compile (module.js:570:32)
    at Object.Module._extensions..js (module.js:579:10)
    at Module.load (module.js:487:32)
    at tryModuleLoad (module.js:446:12)
    at Function.Module._load (module.js:438:3)
    at Module.runMain (module.js:604:10)
    at run (bootstrap_node.js:389:7)
    at startup (bootstrap_node.js:149:9)
    at bootstrap_node.js:504:3

Douglas Wilson · Answer 4 · Fri Jan 12 2018 06:12:57 GMT+0800 (China Standard Time)

I installed the latest version of jquery and the app is running. I then made a request to http://localhost:3000 and I see compression happening:

$ curl -is --compress http://localhost:3000
HTTP/1.1 404 Not Found
X-Powered-By: Express
Content-Security-Policy: default-src 'self'
X-Content-Type-Options: nosniff
Content-Type: text/html; charset=utf-8
Vary: Accept-Encoding
Content-Encoding: gzip
Date: Thu, 11 Jan 2018 22:12:00 GMT
Connection: keep-alive
Transfer-Encoding: chunked

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>Cannot GET /</pre>
</body>
</html>

You can see the response header Content-Encoding: gzip in there. Not sure what the difference is between our setups. If I can reproduce the issue I can definitely debug through it to see what is wrong 👍

Brandon Stillwell · Answer 5 · Fri Jan 12 2018 06:14:20 GMT+0800 (China Standard Time)

I took some out that I didn't think I was using, I'll just post all of them:

"body-parser": "^1.17.2",
    "browserify": "^14.4.0",
    "client-sessions": "^0.8.0",
    "compression": "^1.0.3",
    "cors": "^2.5.2",
    "dexie": "^1.5.1",
    "form-data": "^2.2.0",
    "gasparesganga-jquery-loading-overlay": "^1.5.3",
    "geojson": "^0.4.1",
    "helmet": "^1.3.0",
    "kinto": "^9.0.2",
    "leaflet-control-topcenter": "git+https://github.com/FCOO/leaflet-control-topcenter.git",
    "leaflet.vectorgrid": "^1.3.0",
    "loopback": "^3.0.0",
    "loopback-boot": "^2.6.5",
    "loopback-cache": "^0.2.0",
    "loopback-component-explorer": "^4.0.0",
    "loopback-connector-postgresql": "^2.8.0",
    "pg-promise": "^7.3.3",
    "pug": "^2.0.0-rc.4",
    "serve-favicon": "^2.0.1",
    "shpjs": "^3.4.2",
    "strong-error-handler": "^2.0.0",
    "tilestrata-headers": "^0.3.0"

I used the curl -I command and I also used fiddler

Douglas Wilson · Answer 6 · Fri Jan 12 2018 06:14:26 GMT+0800 (China Standard Time)

Here is a new from Chrome for the app as well, which is also showing the Content-Encoding: gzip header for me:

Brandon Stillwell · Answer 7 · Fri Jan 12 2018 06:28:41 GMT+0800 (China Standard Time)

So I see the header when I run the curl -is --compress command but not when I just run curl -I. That's weird, when I try to set the header myself, then chrome and firefox will give me an error that it is not compressed properly.

Douglas Wilson · Answer 8 · Fri Jan 12 2018 06:29:50 GMT+0800 (China Standard Time)

This module will only compress if the client asks for it (which is what --compress does in cURL). Otherwise a client that doesn't understand gzip will just get garbage.

Brandon Stillwell · Answer 9 · Fri Jan 12 2018 06:39:50 GMT+0800 (China Standard Time)

Okay I get what you're saying. So shouldn't curl -I --compress return just the headers without sending my whole html? It's strange because I am able to see the content-encode header when I use the -is --compress but not -I --compress. Even when I used fiddler to check my page it still says 'No Compression'.

Brandon Stillwell · Answer 10 · Fri Jan 12 2018 06:40:26 GMT+0800 (China Standard Time)

By the way thank you for the fast replies.

Douglas Wilson · Answer 11 · Fri Jan 12 2018 06:43:28 GMT+0800 (China Standard Time)

The -I option makes a HEAD request, not a normal GET request. Since a HEAD has no body, there is nothing to compress.

Brandon Stillwell · Answer 12 · Fri Jan 12 2018 06:49:14 GMT+0800 (China Standard Time)

Okay that makes way more sense. I was so confused why I could see it happening but I couldn't find the header. I still don't know why fiddler still says there's no compression though. Thank you for the help. I will close the issue.