Why aren't dependencies explicit?

[Jesse Tane]

Looking around I noticed that many packages in the pex-gl organization do not appear to explicitly reference each other via the dependencies field in their package.json files.

I noticed this:
https://github.com/pex-gl/pex/blob/master/lib/pkg/pex-packages-core.json

Which seems to make sure that if you use pex init your project gets all the right things installed, but I'm curious to understand why this approach is used over the more traditional dependencies field?

[Marcin Ignac]

That's a very good question requiring a bit longer answer but I'll start here to keep the conversation going.

I wanted to avoid module duplication.

If every package require let's say pex-context then we might end up with the following folder structure

node_modules
   pex-sys
       node_modules
           pex-context@1.0.1
   pex-typo
       node_modules
           pex-context@1.0.2

While ideally this would be deduped as both versions are compatible in the previous versions of pex 0.3.x that wasn't the case (every minor version < 1.0.0 is treated like major by npm and potentially breaking)

That causes two problems

a) Couple of methods did and still do type checks. e.g.

pex-context@1.0.1/Texture2D != pex-context@1.0.2/Texture2D

I'm trying to avoid this in the newer code e.g. when comparing two params

if (param1.getTarget && param2.getTarget && (param1.getTarget() == param2.getTarget()))

b) In PEX <1.0 Context was global and you would call var gl = Context.getCurrent(). With module like above the singletons are not possible (you would end up with 2 Contexts)

This has changed in PEX@1.0.+ as now we pass ctx object to all dependencies

Peer references

We did use peer references in the past

"peerDependencies": {
    "pex-glu": "0.x",
    "pex-color": "0.x",
    "pex-gen": "0.x",
    "pex-geom": "0.x"
  },

But when npm@3 stopped installing them I didn't continue using them.

Solutions

I'm open for suggestions and best practices in this kind of plugin scenarios that pex and pex-* modules are at the moment.

[Jesse Tane]

That makes sense, thank you for the thorough explanation. Maybe you've looked at this:
mrdoob/three.js#4776

where some of the same problems are discussed. in particular the type checking issue comes up, with a feature detection approach being suggested over instanceof / constructor equality tests.

Generally it sounds like pex has fewer, more well defined problems than threejs though - maybe it's just a matter of spec'ing out all the deps, doing npm i and watching to see what breaks? I started to try that myself but got hung up on dealing with the require calls to plask. I see you can use browserify's ignore flag to work around the problem but that doesn't feel quite right. A "plask-wrap" module or similar with a browser field in its pkg json might do the trick, and the rest seems like it could fall into place naturally enough. Obviously still a bunch of work, I know :)

[Jesse Tane]

Also, personal taste I suppose, but I would risk code duplication to get familiar dependency management. Getting all the version mismatches fixed upstream to resolve duplication problems can be a pain, but at least no actual code needs to get changed - just version numbers in other package.json files.

[Marcin Ignac]

I like the idea about plask-wrap. I'll try to prototype it today and see what's possible.

I would risk code duplication to get familiar dependency management.

Can you explain how would that work? Do you expect the user to dedupe code after npm install?

[Marcin Ignac]

Ideally I would love to get rid of peer dependencies completely but i don't know if it's possible given the current architecture and WebGL state machine in general

This happened so many times for me and also the reason why modules like pex-typo are still not ported..

(Screen from https://www.youtube.com/watch?v=a97Mw2z1SAI)

[Marcin Ignac]

Ok, first step is behind us

plask-wrap@1.0.0

These modules no longer need -i plask flag
pex-sys v1.5.1 updated in pex-gl/pex-sys@e9b2407
pex-context v1.3.0 updated in pex-gl/pex-context@f4b2eea
pex-io v1.2.0 updated in pex-gl/pex-io@6b36dbc

Still need to update pex-gl/pex-gui#7

[Jesse Tane]

Awesome!

Ideally I would love to get rid of peer dependencies completely but i don't know if it's possible given the current architecture and WebGL state machine in general

I am not really familiar enough with the relevant packages, or even WebGL tbh, but it sounded like you were saying instances of pex-context represented the memory that needed to get shared, and that internally most of the required changes to allow consumers to pass their own reference had already been made.

Maybe that task is not as easy as I am assuming - I do see usage of an API this.getContext() in the example, which I am naively thinking might be changeable to something more like:

var Context      = require('pex-context');
var Window          = require('pex-sys/Window');

Window.create({
    context: new Context(),
    settings: {
        ...

[Jesse Tane]

Regarding code duplication, ideally npm3 improves the situation enough, but I think unless you pin exact versions or are able to be really meticulous about updating dependents in step, this is always a risk... I am just saying that as long as there is no technical requirement for there to be a single constructor for any given type, I think this is probably acceptable. People who care about bundle size will hopefully notice and if a reinstall or dedupe doesn't fix their problem, send patches to fix legitimately out of date dependencies.

[Marcin Ignac]

Window.create({
    context: new Context(),

This is currently not possible as Context creation is platform specific and that functionality is within Window. Peer dependencies problem in pex has two versions.

Explicit

In pex-sys/WindowImplBrowser:

var Context  = require('pex-context/Context');
...
windowPex._ctx = new Context(gl);

We should probably add pex-context to the package json of pex-sys as we require it.

Implicit

In pex-gui/GUI:

function GUI(ctx, windowWidth, windowHeight, pixelRatio) {
...
ctx.createMesh()

Where we don't do node require for pex-context but we assume that what we get as ctx is certain version of the pex-context/Context that has createMesh method. I don't know how to get rid of that kind of dependency.