HLSL pixel shaders 13 February, 2008 at 12:56 am
I’ve been trying to learn some HLSL (“HLSL is the High Level Shading Language for DirectX”), or at the very least use HLSL to create some new effects for use in Neon v2. Neon allows you to use real-time, hardware pixel shaders to manipulate each layer in the composition. Part of the problem has been that the example files I’ve been tinkering with give very little to no explanation of what exactly is going on, and when you make a mistake (or try to do something that isn’t allowed, isn’t supported, etc), all you get is… well, the effect doesn’t work (at all), and you get no further information.
When I had searched for information on other occasions, I might not have been looking particularly hard, but maybe it just wasn’t obvious where to find it. The MSDN (Microsoft Developers Network) documentation is ‘obviously’ where to find it, supposing that first of all you know that is obvious! :o)
What I’ve tried to do is combine more than one (existing) effect into a single, composite effect. One that I’ve managed to pull off that I quite like is being able to apply a sobel outline to just part of the image – that part defined by a separate texture (in this case loaded from a PNG image). Rather than just use a single ‘channel’ (red, green, blue, or the ‘alpha’ of an image) I allow three separate channels (R,G,B), or any combination of the three, and/or any combination of the inverse of each channel. That is, you can use the ‘shape’ or the negative of the shape. Yes, this of course means lots of inputs to the effect, but the more knobs the merrier, hey? Not necessarily, but it gives a lot of options!
My most recent attempt – and that is since having had a read of the docs! – has been to make a gradual fade-in type effect, based on an image, which goes from a totally ‘underexposed’ mask (for want of a better term, meaning that even the highest value is dropped to zero), through to 100% ‘exposed’ (using the mask just ‘as is’), and finally through to totally ‘overexposed’ (even the lowest value from the mask bumped up to 100%) – with the exception that a zero level from the mask always remains at zero.
Of course, I’ve employed the idea of using the red, green and blue channels of the mask image separately – mainly so that there can be three independent (or overlapping!) areas of audio reaction – which I managed to get working, so far as I could tell, but then went off on a mystery tour of attempting to colourize based on the three channels. What I have so far is a bit of a mess – not entirely uninteresting, mind you, but not what I envisaged… part of the problem is perhaps not having spent enough time on the envisaging part! More thought needs to go into what I actually mean for, say, the red and blue parts to be overlapping – what should the combined ‘alpha’ value be (since there is a single apha value for any given pixel), is one of the main problems – perhaps deciding the R,G,B values is less complex, but still not sorted out. When, for example, do you add in some blue? And how much blue? Use a value from the mask (percentage, or on/off?), on the source image/video, or both? Or what? Perhaps it is, or will be, a good thing that there are so many options. I just need to implement some and see how they look.
Another ‘problem’ I’m faced with is the mechanism within Neon for using effects – an effect is tied to a particular source, not a layer, which means if I switch to another item in the layer, the effect goes with the video/whatever. The newer version of Neon I’d downloaded ages ago proposes to allow effects layers for other than the last/top layer, but unfortunately I can’t get it to work on my machine… even if I could, it is distinctly lacking in usable functionality, which is not so good.
Perhaps I should invest some more time in selecting (indeed, creating!) some more clips, graphics, masks (for my existing effects), and getting on to creating some text animations… I might have a gig this week – at Q Bar on Thursday night – but haven’t heard back from the guy I emailed about it. And of course I lost his phone number when I put my SIM card in a new phone this evening… d’oh!
-G.