Page 1 of 2

Problem with coloring lines by hand

Posted: 30 Jun 2014, 13:50
by Animark
Hi there,

I have some black colored lineart where I want to color some parts of the lines in another color. I thought the best way should be to enable the layers stencil mode and draw over the lines with a bigger pen (or the filled stroke tool). That's also what I have found and read in the forum about line coloring.

The problem ist, that the smooth border of the black line (transparent pixels) is getting darker after coloring it. They are already transparent, but darker (and ugly). Here is a picture to demonstrate the problem. The black line was drawn with the pen tool (size = 2, Antialiasing and Drying is checked)
LineColorize.jpg
LineColorize.jpg (93.28 KiB) Viewed 29091 times
I remember that I had this problem before some years ago when using TVPaint 9 or 9.5. I think there was a different behaviour between using the layers stencil mode or the general stencil mode which had to be activated by pressing the icon on top of the screen. Today there is also a stencil mode there, but it only enables the layer stencil at all. - However, I rember that using the old "On Top Of Screen Stencil Mode" had much better results. Unfortunately I don't have an old TVPaint running to test it. Hope it is not only a memory ;-).

To come to an end: Is there another practical way for coloring lines without getting dark edges? (And sure, I don't want to clean my drawings using different colors from the beginning ;-))

Thanks from Animark

Re: Problem with coloring lines by hand

Posted: 30 Jun 2014, 14:24
by slowtiger
Left to the stencil button is the "preserve transparency" button, which does the trick.

Although I really don't know why we have both of these ...

Re: Problem with coloring lines by hand

Posted: 30 Jun 2014, 14:31
by Animark
Ha Ha Ha, thanks Slowtiger. I think I never touched that button, maybe because it looks like it has something to do with Gradiants and I am really not often interested in gradiants. Cool, you saved my day! Sometimes, solutions are just a few pixels away.

Re: Problem with coloring lines by hand

Posted: 30 Jun 2014, 15:05
by Svengali
You can also set a shortcut to TOOLS>Toggle Preserve Transparency
or
You can also set a button to SET KEYSTROKE>TOOLS>Toggle Preserve Transparency

Sven

Re: Problem with coloring lines by hand

Posted: 30 Jun 2014, 15:58
by D.T. Nethery
[quote="slowtiger"Left to the stencil button is the "preserve transparency" button, which does the trick.


Animark wrote:Ha Ha Ha, thanks Slowtiger. I think I never touched that button, maybe because it looks like it has something to do with Gradiants and I am really not often interested in gradiants. Cool, you saved my day! Sometimes, solutions are just a few pixels away.


Slowtiger beat me to the answer , but I do think you have raised a good point about the Preserve Transparency button -- it does look like it has something to do with setting a Gradient . Maybe that button design needs a change ?
Preserve Trans looks like Gradient.jpg
Preserve Trans looks like Gradient.jpg (21.38 KiB) Viewed 29061 times
Preserve Transparency button.jpg
Preserve Transparency button.jpg (53.95 KiB) Viewed 29061 times

.

Re: Problem with coloring lines by hand

Posted: 30 Jun 2014, 18:47
by Peter Wassink
slowtiger wrote:...
Although I really don't know why we have both of these ...
Slowtiger, we have both because they are different things...
The preserve transparency is an alpha lock for the current layer, whether it is on or off has no influence on any other layer.
The stencil switch, however, turns the layer on which it is activated into a global mask (meaning its setting affects all other layers)

so indeed for coloring the lines on a specific layer you'd choose the preserve transparency switch of that layer.
the stencil is typically used for all (animated)masking tasks

Re: Problem with coloring lines by hand

Posted: 30 Jun 2014, 19:35
by Animark
Thanks all for the nice explanation of the differences between the stencil and the preserve transparency switches and for the shortcuts.
D.T. Nethery wrote: -- it does look like it has something to do with setting a Gradient . Maybe that button design needs a change ?
I've taken a look at photoshop. It could be a good idea to take their picture as an inspiration for a better one in TVPaint. I know the one from photoshop since years and maybe it is established in the user community for its meaning.
PreserveTransparencyInPS.jpg
PreserveTransparencyInPS.jpg (67.29 KiB) Viewed 29036 times
Yes, even when I know the function now, a redesign would be helpful for new users :-).

Re: Problem with coloring lines by hand

Posted: 30 Jun 2014, 20:04
by Lukas
I agree the current icon design is very easily confused for some kind of gradient feature. Photoshop icon isn't super obvious either, but it's definitely about transparency.

Re: Problem with coloring lines by hand

Posted: 30 Jun 2014, 22:06
by Sewie
Though it is easier to use the 'preserve transparency' button, these artifacts shouldn't appear when using stencil mode, should they? Isn't this a bug?

Re: Problem with coloring lines by hand

Posted: 01 Jul 2014, 07:45
by Peter Wassink
Sewie wrote:Though it is easier to use the 'preserve transparency' button, these artifacts shouldn't appear when using stencil mode, should they? Isn't this a bug?
it is not a bug.
When recoloring on a layer that is itself in stencil mode, this will always happen.
The "artifacts" are caused by the anti aliased edges of the original drawing, the antialias pixels are partially transparent (thus the mask edge is partially transparent) and therefore when you apply color on the half transparent pixels the new color will only be applied with half opacity.
Therefore it results in a mix of both the new and old color. When the old color is darker, a dark halo will appear.

That is why if you want to perfectly recolor the layer itself you'll have to use the preserve transparency function.

Re: Problem with coloring lines by hand

Posted: 01 Jul 2014, 08:31
by slowtiger
OK, but why do we use the stencil mode then, or what's the reason stencil and transparency are implemented differently?

Re: Problem with coloring lines by hand

Posted: 01 Jul 2014, 08:43
by Sewie
Thanks for the explanation, Peter.

Re: Problem with coloring lines by hand

Posted: 01 Jul 2014, 08:44
by Peter Wassink
slowtiger wrote:OK, but why do we use the stencil mode then, or what's the reason stencil and transparency are implemented differently?
Peter Wassink wrote:...
The stencil switch, however, turns the layer on which it is activated into a global mask (meaning its setting affects all other layers)
Are you asking why you'd want a global mask?
Answer, in all cases where you need a mask that can't be part of the layer itself (which is the case when recoloring lines)

Re: Problem with coloring lines by hand

Posted: 01 Jul 2014, 09:03
by Eric Scholl
In Stencil Mode :
When you draw on a layer on which the stencil is activated, the mask itself is modified.
You can use this mode to draw on other layers using the layers on which the stencil is activated as a mask.

In Preserve transparency Mode :
The mask is set when you activate the option and will never change, but you can only draw on the layer on which the Preserve transparency is activated.
slowtiger wrote:OK, but why do we use the stencil mode then, or what's the reason stencil and transparency are implemented differently?
The problem is you can set multiple layers as stencil ( masks will be merged ). But the mask will not correspond to any layer when you do this.
So when you will draw on a layer on which the stencil is activated, it will modify the layer according to a mask which does not correspond to the layer itself and then the mask itself will be modified.

Re: Problem with coloring lines by hand

Posted: 01 Jul 2014, 09:10
by furushil
slowtiger wrote:OK, but why do we use the stencil mode then, or what's the reason stencil and transparency are implemented differently?
I am using the stencil mode all the time.
If you have colored the character on one layer and you want to have the shadow on another, you need the stencil mode.
Basically if you want to paint something on a separate layer but don't want to go over the boarder of the source layer, you need the stencil mode.

Ok now everybody is answering at the same time :mrgreen: