Problem with coloring lines by hand

Please use this part to report bugs & errors, ask questions & "How to..."
User avatar
slowtiger
Posts: 2949
Joined: 08 May 2008, 21:10
Location: berlin, germany
Contact:

Re: Problem with coloring lines by hand

Post by slowtiger »

Sorry, this doesn't explain anything. I try again.

The question was not about transparency = same layer affected vs. stencil = other/several layers affected.

The question is why the stencil is calculated in a different way than the transparency. Surely it must be possible to add up transparency masks the same way as stencil masks, and surely it should be possible to use one layer's transparency mask on another layer as well? I just want to know the technical decision behind this.
TVP 10.0.18 and 11.0 MacPro Quadcore 3GHz 16GB OS 10.6.8 Quicktime 7.6.6
TVP 11.0 and 11.7 MacPro 12core 3GHz 32GB OS 10.11 Quicktime 10.7.3
TVP 11.7 Mac Mini M2pro 32GB OS 13.5
User avatar
Peter Wassink
Posts: 4436
Joined: 17 Feb 2006, 15:38
Location: Amsterdam
Contact:

Re: Problem with coloring lines by hand

Post by Peter Wassink »

Markus, let ME try again... :wink:
The thing ist gerade that they are NOT calculated in a different way!
i tried explaining it here
Peter wrote: it is not a bug.
When recoloring on a layer that is itself in stencil mode, this will always happen.
The "artifacts" are caused by the anti aliased edges of the original drawing, the antialias pixels are partially transparent (thus the mask edge is partially transparent) and therefore when you apply color on the half transparent pixels the new color will only be applied with half opacity.
Therefore it results in a mix of both the new and old color. When the old color is darker, a dark halo will appear.

That is why if you want to perfectly recolor the layer itself you'll have to use the preserve transparency function.
What i didn't mention there is that they both already use the same alpha mask.

the difference is that they are displayed differently and this follows simply from the fact that in one case the result is ONE layer
and in the other case (Stencil) the result is one Two (or more) layers
see image:
diff-stencil-PT.jpg
Peter Wassink - 2D animator
• PC: Win11/64 Pro - AMD Ryzen 9 5900X 12-Core - 64Gb RAM
• laptop: Win10/64 Pro - i7-4600@2.1 GHz - 16Gb RAM
User avatar
slowtiger
Posts: 2949
Joined: 08 May 2008, 21:10
Location: berlin, germany
Contact:

Re: Problem with coloring lines by hand

Post by slowtiger »

Sorry, I don't follow that. There must be a technical difference since recoloring the same layer will yield different results when applying transparency preserve and stencil on itself.
TVP 10.0.18 and 11.0 MacPro Quadcore 3GHz 16GB OS 10.6.8 Quicktime 7.6.6
TVP 11.0 and 11.7 MacPro 12core 3GHz 32GB OS 10.11 Quicktime 10.7.3
TVP 11.7 Mac Mini M2pro 32GB OS 13.5
Elodie
Posts: 13912
Joined: 06 Jan 2009, 09:53
Location: Metz

Re: Problem with coloring lines by hand

Post by Elodie »

slowtiger wrote: Sorry, I don't follow that. There must be a technical difference since recoloring the same layer will yield different results when applying transparency preserve and stencil on itself.
Yes, because you are not supposed to paint on a layer using stencil mode, but on a different one.

You modify lines' colors on current layer = use preserve alpha
You want to change line's color, but having the result on different layer = turn the original layer into a stencil, create a new layer and paint on this new layer. I also advise you to turn off the original layer.
Black lines = stencil mode / layer turn off<br />New layer is used to paint a new very of the line
Black lines = stencil mode / layer turn off
New layer is used to paint a new very of the line
stencil or preserver 001.png (69.58 KiB) Viewed 22050 times
I use the original line and I change its color, using preserve alpha channel.
I use the original line and I change its color, using preserve alpha channel.
stencil or preserver 002.png (63.79 KiB) Viewed 22050 times
Result is the same : all depend on the way you work.

Actually, I think we should propose an alert window when you try do draw on a layer turn into a stencil.
Svengali
Posts: 1571
Joined: 28 Dec 2006, 10:08

Re: Problem with coloring lines by hand

Post by Svengali »

Other differences between Stencils and alpha masking worth mentioning
1. Stencils have an invert mode which let you toggle between the source and everything but the source (from one layer but applied to all layers).
2. if stencil is toggled after you move to another layer, that layer becomes the stencil source (the stencil derived from the other layer is lost).
3. If you move to a layer where alpha mask is still active, you receive a warning at the time you invoke a stencil on the same layer since both affect where you can and cannot paint.
4. Each alpha masking derives from (and affects) its own layer only... also it remains in effect when you move to another layer.
5. Every layer can simultaneously have its own, active alpha mask which is layer specific not frame specific.

Layer Finder
Layer Finder Button/script identifies and displays all layers which have either alpha masking or stencil applied.
Layer Finder also includes a one-click option to clear all layer settings including ALL active, layer specific masks AND which ever layer's stencil might be active (whether normal or inverted).

George Control and Persistence for Stencil Display Settings
Someday hopefully we'll have a George command which will let you turn the Stencil visibility on/off and set the checkerboard color and transparent level for a specific layer. Now, with every session startup you must use the Display Settings panel to set visibility, color and degree of transparency %, EVERY SESSION. Hopefully, in the future, those Stencil settings will be persistent and remain from session to session like Layer Attentuation is about to be, please!

Sven
TVP Pro 11.0.10-64bit Win10 - 64GB ram -2TB HHD - 256GB SSD - Wacom Cintiq 16, driver 6.3.41-1
Android Tablet: rel. 11, Samsung Galaxy Note10.1 - 32GB with microSD 32GB
Android Tablet: rel. 11.5, Samsung Galaxy Tab S7plus - 128GB with microSD 64GB
User avatar
Peter Wassink
Posts: 4436
Joined: 17 Feb 2006, 15:38
Location: Amsterdam
Contact:

Re: Problem with coloring lines by hand

Post by Peter Wassink »

slowtiger wrote:Sorry, I don't follow that. There must be a technical difference since recoloring the same layer will yield different results when applying transparency preserve and stencil on itself.
o.k. here is one last try to explain the difference:

Preserve transparency lets you replace the RGB value of ALL present pixels equally, no matter their transparency.
i think this is simple and clear.

Stencil gives a different result when you try to recolor on the layer itself.
This is not because the alpha mask of the stencil is different.
but its because you cannot replace the RGB value fully through a halftransparent mask.

o.k. lets try an example:
you want to paint a 50% transparent pixel from black to red
PreserveTransparency simply allows you to turn the RGB value of this pixel from black to red and because of the alpha channel, which is still at 50%, the pixel has become a 50% transparent red pixel (hurray!)
Stencil however is a different story, a bit more tricky to understand.
When you apply 100% red "paint" to the 50% black pixel, it won't turn into a 50% red pixel because the pixel is its own mask!
because of the stencil mask the red "paint" is blocked from reaching the 50% black pixel by its own 50% transparency, only 50% of the "red paint" comes through....

The half transparent pixels in the stencil mask prevent themselves to be recolored 100%
the more transparent a pixel is the less it can be recolored
that is why you see the effect mostly along the edges of lines, because of the anti alias these edges are transparent.
Peter Wassink - 2D animator
• PC: Win11/64 Pro - AMD Ryzen 9 5900X 12-Core - 64Gb RAM
• laptop: Win10/64 Pro - i7-4600@2.1 GHz - 16Gb RAM
Svengali
Posts: 1571
Joined: 28 Dec 2006, 10:08

Re: Problem with coloring lines by hand

Post by Svengali »

Peter Wassink wrote:o.k. here is one last try to explain the difference:
Well done Peter. :idea:

Sven
TVP Pro 11.0.10-64bit Win10 - 64GB ram -2TB HHD - 256GB SSD - Wacom Cintiq 16, driver 6.3.41-1
Android Tablet: rel. 11, Samsung Galaxy Note10.1 - 32GB with microSD 32GB
Android Tablet: rel. 11.5, Samsung Galaxy Tab S7plus - 128GB with microSD 64GB
Elodie
Posts: 13912
Joined: 06 Jan 2009, 09:53
Location: Metz

Re: Problem with coloring lines by hand

Post by Elodie »

Svengali wrote:
Peter Wassink wrote:o.k. here is one last try to explain the difference:
Well done Peter. :idea:

Sven
+1 !
User avatar
Sewie
Posts: 1310
Joined: 18 Jun 2008, 11:57
Location: The Netherlands, Europe
Contact:

Re: Problem with coloring lines by hand

Post by Sewie »

So if you would want to prevent these edges in stencil mode, is there a workaround for this?
Michael Sewnarain - Website
Windows 11/64b Pro - TVP11.7.0 & 11.7.1 - Pro/64b - Cintiq32 Pro - Intel i7-12700K - 64Gb RAM
User avatar
Peter Wassink
Posts: 4436
Joined: 17 Feb 2006, 15:38
Location: Amsterdam
Contact:

Re: Problem with coloring lines by hand

Post by Peter Wassink »

Sewie wrote:So if you would want to prevent these edges in stencil mode, is there a workaround for this?
No.

To prevent edges, either work in preserve transparency mode
or apply in stencil mode on another layer.

Two reasonable alternatives i think.
Peter Wassink - 2D animator
• PC: Win11/64 Pro - AMD Ryzen 9 5900X 12-Core - 64Gb RAM
• laptop: Win10/64 Pro - i7-4600@2.1 GHz - 16Gb RAM
User avatar
Sewie
Posts: 1310
Joined: 18 Jun 2008, 11:57
Location: The Netherlands, Europe
Contact:

Re: Problem with coloring lines by hand

Post by Sewie »

Thanks! :)
Michael Sewnarain - Website
Windows 11/64b Pro - TVP11.7.0 & 11.7.1 - Pro/64b - Cintiq32 Pro - Intel i7-12700K - 64Gb RAM
drude
Posts: 13
Joined: 14 Oct 2009, 23:30

Re: Problem with coloring lines by hand

Post by drude »

Hello!

I am very fond of the preserve transparency mode in tvp10! i use it all the time. I am now working on a project where the current type of TVP we use is 9.5 - and now my problem is, that i don't know where that button is! - is this feature not part of TVP 9.5? And if it is, does anyone know where to find it?

Thanks!:)
Svengali
Posts: 1571
Joined: 28 Dec 2006, 10:08

Re: Problem with coloring lines by hand

Post by Svengali »

Not sure about 9.5, but a shortcut key assigned to Tools:Toggle Preserve Transparency works in 10.

Sven
TVP Pro 11.0.10-64bit Win10 - 64GB ram -2TB HHD - 256GB SSD - Wacom Cintiq 16, driver 6.3.41-1
Android Tablet: rel. 11, Samsung Galaxy Note10.1 - 32GB with microSD 32GB
Android Tablet: rel. 11.5, Samsung Galaxy Tab S7plus - 128GB with microSD 64GB
User avatar
D.T. Nethery
Posts: 4225
Joined: 27 Sep 2006, 19:19

Re: Problem with coloring lines by hand

Post by D.T. Nethery »

drude wrote:Hello!

I am very fond of the preserve transparency mode in tvp10! i use it all the time. I am now working on a project where the current type of TVP we use is 9.5 - and now my problem is, that i don't know where that button is! - is this feature not part of TVP 9.5? And if it is, does anyone know where to find it?

Thanks!:)
The location of the Preserve Transparency button in TVP 9.5 is at the top. See attached screen shot.
Preserve Transparency TVP 9.5.jpg
Preserve Transparency TVP 9.5.jpg (36.31 KiB) Viewed 21868 times


.
drude
Posts: 13
Joined: 14 Oct 2009, 23:30

Re: Problem with coloring lines by hand

Post by drude »

THANK YOU! i completely forgot where it was, i was sure i had used it in 9.5 before - perfect! Now i remember! :)
Post Reply