Is it possible to test if an arbitrary pixel is modifiable by the shader?

Question

I am writing a spatial shader in godot to pixelate an object.

Previously, I tried to write outside of an object, however that is only possible in CanvasItem shaders, and now I am going back to 3D shaders due rendering annoyances (I am unable to selectively hide items without using the culling mask, which being limited to 20 layers is not an extensible solution.)

My naive approach:

1. Define a pixel "cell" resolution (ie. 3x3 real pixels)
2. For each fragment:

   • If the entire "cell" of real pixels is within the models draw bounds, color the current pixel as per the lower-left (where the pixel that has coordinates that are the multiple of the cell resolution).

   • If any pixel of the current "cell" is out of the draw bounds, set alpha to 1 to erase the entire cell.

psuedo-code for people asking for code of the likely non-existant functionality that I am seeking:

int cell_size = 3;

fragment {
  // check within a cell to see if all pixels are part of the object being drawn to

  for (int y = 0; y < cell_size; y++) {
    for (int x = 0; x < cell_size; x++) {
      int erase_pixel = 0;

      if ( uv_in_model(vec2(FRAGCOORD.x - (FRAGCOORD.x % x), FRAGCOORD.y - (FRAGCOORD.y % y))) == false) {
        int erase_pixel = 1;
      }

    }
  }

  albedo.a = erase_pixel
}

tl;dr, is it possible to know if any given point will be called by the fragment function?

Answer 1

On your object's material there should be a property called Next Pass. Add a new Spatial Material in this section, open up flags and check transparent and unshaded, and then right-click it to bring up the option to convert it to a Shader Material.

Now, open up the new Shader Material's Shader. The last process should have created a Shader formatted with a fragment() function containing the line vec4 albedo_tex = texture(texture_albedo, base_uv);

In this line, you can replace "texture_albedo" with "SCREEN_TEXTURE" and "base_uv" with "SCREEN_UV". This should make the new shader look like nothing has changed, because the next pass material is just sampling the screen from the last pass.

Above that, make a variable called something along the lines of "pixelated" and set it to the following expression: vec2 pixelated = floor(SCREEN_UV * scale) / scale; where scale is a float or vec2 containing the pixel size. Finally replace SCREEN_UV in the albedo_tex definition with pixelated.

After this, you can have a float depth which samples DEPTH_TEXTURE with pixelated like this: float depth = texture(DEPTH_TEXTURE, pixelated).r; This depth value will be very large for pixels that are just trying to render the background onto your object. So, add a conditional statement: if (depth > 100000.0f) { ALPHA = 0.0f; }

As long as the flags on this new next pass shader were set correctly (transparent and unshaded) you should have a quick-and-dirty pixelator. I say this because it has some minor artifacts around the edges, but you can make scale a uniform variable and set it from the editor and scripts, so I think it works nicely.

"Testing if a pixel is modifiable" in your case means testing if the object should be rendering it at all with that depth conditional.

Here's the full shader with my modifications from the comments

// NOTE: Shader automatically converted from Godot Engine 3.4.stable's SpatialMaterial.

shader_type spatial;
render_mode blend_mix,depth_draw_opaque,cull_back,unshaded;

//the size of pixelated blocks on the screen relative to pixels
uniform int scale;

void vertex() {
}

//vec2 representation of one used for calculation
const vec2 one = vec2(1.0f, 1.0f);

void fragment() {
    //scale SCREEN_UV up to the size of the viewport over the pixelation scale
    //assure scale is a multiple of 2 to avoid artefacts
    vec2 pixel_scale = VIEWPORT_SIZE / float(scale * 2);
    vec2 pixelated = SCREEN_UV * pixel_scale;
    //truncate the decimal place from the pixelated uvs and then shift them over by half a pixel
    pixelated = pixelated - mod(pixelated, one) + one / 2.0f;
    //scale the pixelated uvs back down to the screen
    pixelated /= pixel_scale;
    
    vec4 albedo_tex = texture(SCREEN_TEXTURE,pixelated);
    ALBEDO = albedo_tex.rgb;
    ALPHA = 1.0f;
    float depth = texture(DEPTH_TEXTURE, pixelated).r;
    if (depth > 10000.0f)
    {
        ALPHA = 0.0f;
    }
}