What are you working on?

[GiffE]

Made a glsl sobel edge detection filter effect for outlines.

Yes that's the Taov model, just using it temporarily.

The code in the post process, fragment shader.

Code: Select all

float result,temp;
     float col[9];
     int i;
     float x,y ;
     for(i=0;i<9;i++)
     {
        col[i] = texture2D(Texture_First, TexCoord1+PixelSize *offsets[i]).x * EdgeSize;
     }
   x= -col[6]-2*col[7]-col[8]+col[0]+2*col[1]+col[2];
   y= -col[0]-2*col[3]-col[6]+col[2]+2*col[5]+col[8];
     temp = x*x+y*y;
     result = sqrt(temp);
     if(result>255.0)result=255.0;
     gl_FragColor.r -= result;
     gl_FragColor.b -= result;
     gl_FragColor.g -= result;

Where:

Code: Select all

const float EdgeSize = 0.2;
vec2 offsets[9] =
{
         vec2(-1,-1),vec2(-1, 0 ),vec2(-1,1),
         vec2( 0,-1 ),vec2( 0,0),vec2( 0,1),
         vec2( 1,-1),vec2( 1, 0),vec2( 1,1)
};

[Urre]

Very cool! Can I steal this and release as GPL?

EDIT: Hm, I can't figure out where in default.glsl to put this?

[Irritant]

Very very cool shader, yes let us know if we can use it

[GiffE]

Thanks, you can GPL it all ya want
If you don't know where to put it make your entire MODE_POSTPROCESS part of glsl/default.glsl look like this:

Code: Select all

#ifdef MODE_POSTPROCESS
varying vec2 TexCoord1;
varying vec2 TexCoord2;

#ifdef VERTEX_SHADER
void main(void)
{
   gl_Position = ModelViewProjectionMatrix * gl_Vertex;
   TexCoord1 = gl_MultiTexCoord0.xy;
#ifdef USEBLOOM
   TexCoord2 = gl_MultiTexCoord1.xy;
#endif
}
#endif

#ifdef FRAGMENT_SHADER
uniform sampler2D Texture_First;
#ifdef USEBLOOM
uniform sampler2D Texture_Second;
#endif
#ifdef USEGAMMARAMPS
uniform sampler2D Texture_GammaRamps;
#endif
#ifdef USESATURATION
uniform float Saturation;
#endif
#ifdef USEVIEWTINT
uniform vec4 ViewTintColor;
#endif
//uncomment these if you want to use them:
uniform vec4 UserVec1;
// uniform vec4 UserVec2;
// uniform vec4 UserVec3;
// uniform vec4 UserVec4;
// uniform float ClientTime;
uniform vec2 PixelSize;
const float EdgeSize = 0.2;
vec2 offsets[9] =
{
         vec2(-1,-1),vec2(-1, 0 ),vec2(-1,1),
         vec2( 0,-1 ),vec2( 0,0),vec2( 0,1),
         vec2( 1,-1),vec2( 1, 0),vec2( 1,1)
};
void main(void)
{
   gl_FragColor = texture2D(Texture_First, TexCoord1);
#ifdef USEBLOOM
   gl_FragColor += texture2D(Texture_Second, TexCoord2);
#endif
#ifdef USEVIEWTINT
   gl_FragColor = mix(gl_FragColor, ViewTintColor, ViewTintColor.a);
#endif

#ifdef USEPOSTPROCESSING
// do r_glsl_dumpshader, edit glsl/default.glsl, and replace this by your own postprocessing if you want
// this code does a blur with the radius specified in the first component of r_glsl_postprocess_uservec1 and blends it using the second component

   float result,temp;
     float col[9];
     int i;
     float x,y ;
     for(i=0;i<9;i++)
     {
        col[i] = texture2D(Texture_First, TexCoord1+PixelSize *offsets[i]).x * EdgeSize;
     }
   x= -col[6]-2*col[7]-col[8]+col[0]+2*col[1]+col[2];
   y= -col[0]-2*col[3]-col[6]+col[2]+2*col[5]+col[8];
     temp = x*x+y*y;
     result = sqrt(temp);
     if(result>255.0)result=255.0;
     gl_FragColor.r -= result;
     gl_FragColor.b -= result;
     gl_FragColor.g -= result;
     
     gl_FragColor += texture2D(Texture_First, TexCoord1 + PixelSize*UserVec1.x*vec2(-0.987688, -0.156434)) * UserVec1.y;
   gl_FragColor += texture2D(Texture_First, TexCoord1 + PixelSize*UserVec1.x*vec2(-0.156434, -0.891007)) * UserVec1.y;
   gl_FragColor += texture2D(Texture_First, TexCoord1 + PixelSize*UserVec1.x*vec2( 0.891007, -0.453990)) * UserVec1.y;
   gl_FragColor += texture2D(Texture_First, TexCoord1 + PixelSize*UserVec1.x*vec2( 0.707107,  0.707107)) * UserVec1.y;
   gl_FragColor += texture2D(Texture_First, TexCoord1 + PixelSize*UserVec1.x*vec2(-0.453990,  0.891007)) * UserVec1.y;
   gl_FragColor /= (1 + 5 * UserVec1.y);
     
#endif

#ifdef USESATURATION
   //apply saturation BEFORE gamma ramps, so v_glslgamma value does not matter
   float y = dot(gl_FragColor.rgb, vec3(0.299, 0.587, 0.114));
   //gl_FragColor = vec3(y) + (gl_FragColor.rgb - vec3(y)) * Saturation;
   gl_FragColor.rgb = mix(vec3(y), gl_FragColor.rgb, Saturation);
#endif

#ifdef USEGAMMARAMPS
   gl_FragColor.r = texture2D(Texture_GammaRamps, vec2(gl_FragColor.r, 0)).r;
   gl_FragColor.g = texture2D(Texture_GammaRamps, vec2(gl_FragColor.g, 0)).g;
   gl_FragColor.b = texture2D(Texture_GammaRamps, vec2(gl_FragColor.b, 0)).b;
#endif
}
#endif
#else // !MODE_POSTPROCESS


Also playing around with the EdgeSize constant will change its intensity. I set it quite low for now.

[Urre]

[leileilol]

i'm not really a big fan of post processed celshading.


in DP i did a weirder way to do celshading, done by inclined normal edges. this screwed up map rendering though, but looks better on shapely models in rtworld and doesn't get in the way of the particles.

[Urre]

I like anything that fits the given game. I generally dig the look of post-process outlining, it's a yummy dirty look, reminds me of Okami.

[leileilol]

Here's what my shader looked like in Quake


It's not intended for Quake, but it does sort of make it look like Street Fighter IV. None of this is post-processed or edge detecting.

[Urre]

What is it if not post-process? Looks cool nonetheless! Might be cool combined with the edge detection post-process

[Spirit]

That looks even better than Tenebrae!

[frag.machine]

You can call it "skin seam edge magnifier"

[GiffE]

Fixed the warnings:
http://www.pasteall.org/13345

To fix them add:

Code: Select all

#version 120


to the very first line of glsl/default.glsl
Then change:

Code: Select all

vec2 offsets[9] =
{
         vec2(-1,-1),vec2(-1, 0 ),vec2(-1,1),
         vec2( 0,-1 ),vec2( 0,0),vec2( 0,1),
         vec2( 1,-1),vec2( 1, 0),vec2( 1,1)
};

to:

Code: Select all

vec2 offsets[9] = vec2[] (
         vec2(-1.0,-1.0),vec2(-1.0, 0 ),vec2(-1.0,1.0),
         vec2( 0,-1.0 ),vec2( 0,0),vec2( 0,1.0),
         vec2( 1.0,-1.0),vec2( 1.0, 0),vec2( 1.0,1.0)
);

And change:

Code: Select all

x= -col[6]-2*col[7]-col[8]+col[0]+2*col[1]+col[2];
   y= -col[0]-2*col[3]-col[6]+col[2]+2*col[5]+col[8];

to:

Code: Select all

x= -col[6]-2.0*col[7]-col[8]+col[0]+2.0*col[1]+col[2];
   y= -col[0]-2.0*col[3]-col[6]+col[2]+2.0*col[5]+col[8];

and lastly

Code: Select all

gl_FragColor /= (1 + 5 * UserVec1.y);

to:

Code: Select all

gl_FragColor /= (1.0 + 5.0 * UserVec1.y);

They were silly warnings for type casting and junk, so all that was needed was to change some of those int's to floats, also I didn't know how glsl array syntax worked, so I fixed that.

EDIT:
Oh and here's what start looks like:

Looks Borderlands-esque which was what I was shooting for

[Teiman]

you out-tenbrae tenebrae!

[r00k]

Is it also outlining the filtered textures not just the edges?
The texture looks paintbrushed though I like this effect!! Maybe skip the player texture so it stands out better.
Can I ask a stupid question: where do I put default.glsl?

[Sajt]

What is it if not post-process? Looks cool nonetheless! Might be cool combined with the edge detection post-process

What he did is probably like the "fresnel" type of shader, where you add an effect (such as darkening, or adding a colour) to a degree based on the dot product between its normal and the 3D vector between the point and your eye. So surfaces that are facing toward you are the same, but surfaces approaching perpendicular to your vision have the effect in full. Not a post process at all, it's like any other lighting shader. Here are two images of that type of shader being used to add a green glow:

http://quakery.quakedev.com/screenshots/screen114.jpg
http://quakery.quakedev.com/screenshots/screen115.jpg