Implement Halftone node

[Noah Rahm]

Implement a core node, the Halftone node using either glsl or numpy/opencv. It should take in an image and output an image.

It may be a good idea to implement this as a glsl shader node for speed.

Category: FILTER

[yonMaor]

Now that I finished the edge detection I'm looking into this issue (unless it's taken).
After a little bit of work, I'll admit that I'm a little stuck with the GLSL, but I'll see if I can find any solutions over the next few days.

[yonMaor]

I'm still working on this (still a little stuck on the shader, but making some progress).
I've run into a bug and I wondered whether you've seen this as well.
Sometimes, I get a keyerror where the program parameter does not contain the "input_img" key.

self._members[key].value = value KeyError: 'input_img'

This happens in the glsl_renderer, line 100.

Have you run into this in the past? I assume it has something to do with a failure to run the shader correctly, but I haven't been able to pinpoint why this happens exactly (this happens when I make trivial changes to the shader, that should not cause any run-time errors).

I'll open an issue about this later (unless one already exists, I'll check).
I think that it might be helpful to put a try except block over this part of the code.

[yonMaor]

Concerning the issues I'm having with the shader itself, my result from the shader is all NaNs. Have you run into an issue where the numpy.frombuffer function returns all NaN results?

Edit:
So the problem seems to be that we don't have access some variables that, as far as I understand, are supposed to be available in GLSL. These include things like u_resolution (resolution of our input texture). I wonder if this an issue with our setup (maybe something with our vertex shader?) or with the moderngl library.

[Noah Rahm]

Have you run into this in the past?

I don't think so.

I think that it might be helpful to put a try except block over this part of the code.

Good idea.

[Noah Rahm]

So the problem seems to be that we don't have access some variables that, as far as I understand, are supposed to be available in GLSL....I wonder if this an issue with our setup (maybe something with our vertex shader?) or with the moderngl library.

It is likely our setup. Could you share some example code that produces the NaaNs?

[yonMaor]

Have you run into this in the past?

I don't think so.

I think that it might be helpful to put a try except block over this part of the code.

Good idea.

What's the purpose of that line of code?

It is likely our setup. Could you share some example code that produces the NaaNs?

Currently this code doesn't output nans, just all zeros. Since the halftone example is a little long, I've added the pixelate code instead (which I've also started working on). It suffers from similar issues, where variables like tex_coords (and many others that I've tried to use and are supposed to be available) contain values of 0.

#version 330

uniform sampler2D input_img;
in vec2 tex_coord;
out vec4 output_img;

//uniform sampler2D input_img;

void main()
{
        float dx = 15.*(1./512.);
        float dy = 10.*(1./512.);
        vec2 coord = vec2(dx*floor(tex_coord.s/dx),
                   dy*floor(tex_coord.t/dy));
        output_img = texture2D(input_img, coord);
}

I've tried making tex_coord available through the vertex shader, but that didn't help either

[yonMaor]

By the way, with small adjustments to the platform, the shaders work on shadertoy, so I'm fairly convinced that either we have a problem with our setup, or modernGL reads GLSL differently in some kind of way that I'm not aware of (and haven't been able to find so far)

[Noah Rahm]

Ok, so indeed we just aren't passing in the attributes. Moderngl introspects uniforms, attributes etc

For example, passing in the text_coord attribute

            vertex_shader = """
                #version 330

                in vec2 in_position;

                out vec2 tex_coord;

                void main() {
                    gl_Position = vec4(in_position, 0.0, 1.0);
                }
                """

Let me know if that solves the issue.

[yonMaor]

Still seeing the same issue.
Do we maybe have to assign a value to tex_coord within the vertex shader's main function?

[Noah Rahm]

@yonMaor just wanted to let you know I am still looking into this. I haven't fully gotten to the bottom of why this happens...yet.

[yonMaor]

If there's anything I can do to help with this let me know.
In the meantime I'll look for other issues to solve

[Noah Rahm]

Ok. So I think I've figured out a few things:

1. For getting the resolution of the texture, use vec2 res = textureSize(input_img, 0);
2. For getting the texture coordinates, use gl_FragCoord.xy

I am still trying to wrap my head around a few of the other issues, but maybe this helps.

[yonMaor]

gl_FragCoord.xy seems to be working, but textureSize is returning some very large numbers: (6000,6000) as opposed to the actual size of the image

[Ilias Woithe]

but textureSize is returning some very large numbers: (6000,6000)

I am fairly sure that this is caused by these lines.

[yonMaor]

Yup, after a short test, that seems to be the case.
I think this might be worthy of its own issue at this point

[Noah Rahm]

textureSize is returning some very large numbers: (6000,6000)

Yes, that is intentional (for now). We do have an open issue to implement dynamically setting the texture size based on the input image (#80).

Though, I don't think the texture size is a cause of any issues necessarily. I've been working on a box blur shader and it works good as long as I use the texture size like I described above.

(Continue in #178)