drawing on layer when "Stencil" is "on"

[Joost]

Hi,

Why is it not possible to draw in a layer (in the area that already contains content) when the stencil is turned "on"?
Since TVPaint 11 it gives me a pop up "you cannot draw in this layer"

In TVPaint 10 (Which I used until last month) I always could draw on a layer when the stencil was turned "on". (of course in the area which already contained content/coloured pixels).

Why was this changed? The TVPaint 10 method was more logical in my opinion....

[Svengali]

This change in functionality of "masking" for the ACTIVE LAYER was a topic of discussion back when 11 first came out. Take a look here.

Bottom line, drawing on a layer where the existing pixels can be used as a mask or inverted mask was THROWN OUT WITHOUT DISCUSSION OR FEEDBACK FROM THOSE VETERAN USERS WHO DEPENDED UPON IT IN THEIR WORKFLOW.

Back then, it seems that the decision to disable masking of the active layer was summarily final... where simplifying critical functionality for new users won the day.

sorry.

sven

[Joost]

Thanks for your reaction!

I find it a really, really strange decision... It costs me a LOT more time now disable/enabling the layer stencil over and over again... And clicking away this annoying pop up.

Note: the link in your post didn't wok for me. It says: "you are not authorised to read this forum"

[D.T. Nethery]

Thanks for your reaction!

I find it a really, really strange decision... It costs me a LOT more time now disable/enabling the layer stencil over and over again... And clicking away this annoying pop up.

Note: the link in your post didn't wok for me. It says: "you are not authorised to read this forum"

The link is to a comprehensive post (with suggestions for improving the function ) by Svengali , but it's located in the Beta Tester's private forum which is why you can't access it.
I'll leave it to Svengali if he wants to copy and paste what he posted there. In short, he was advocating for a return to the TVPaint 10 behavior with stencils and/or making it an option that the user can enable in Preferences , so the default behavior would be as it is now in TVPaint 11 to avoid confusion for new users (apparently this was an issue) , but veteran TVPaint users would have the option in Preferences to have layers with stencil activated behave like in TVpaint 10.

I think it would be worth revisiting this change from the TVPaint 10 behavior with stencils to the TVPaint 11 behavior with stencils . You could make it a Feature Request .

[Svengali]

(Taking up on your suggestion David, I've moved a copy of the original beta posts over here - with hope that the current TVP interface and functionality design people might reconsider re-attaching the masking options for the CURRENT LAYER... )


It used to be possible to use the current layer image as a stencil (normal or inverted) and continue drawing with the stencil in effect. Since version 11, using the stencil option on the current active layer, was disabled because new users would turn it on accidentally and be confused and frustrated because they could no longer draw on certain parts of the frame - at times, they were prevented by the stencil to draw ANYWHERE because the screen was blank.

As things are NOW, if you even try to use a stencil on the current frame and layer, you get a firm warning which says:
YOU CANNOT DRAW ON THIS LAYER BECAUSE THE STENCIL FUNCTION IS ENABLED.
TO COLORIZE YOUR LINES, DISABLE THE STENCIL FUNCTION AND USE "PRESERVE TRANSPARENCY" FUNCTION ON YOUR LAYER.

No ifs, ands or buts…

So, because of those new users who become confused, all the rest of the veteran users are penalized and forbidden to deliberately invoke a stencil or inverted stencil on the current frame/layer. Even though there are useful things that can be done when drawing with active stencil or inverted stencil on the current frame/layer… it is forbidden!

Not sure about any other users, but before this revocation, I used stencils and inverted stencils on the current frame/layer, explicitly like that, VERY OFTEN. Not anymore.

In frustration I found several work-alike alternatives that get me close to that previous functionality.

Solution one: Using a combination of the "BEHIND" drawing mode for drawing behind objects on the current frame… OR alpha-masking for drawing "INSIDE" objects on the current frame and straight-forward drawing with "COLOR" mode to draw "OVER" background and objects already on screen. While clumsy to manage, I've made it work, accepting the fact that drawing in the "BEHIND" mode not only draws behind objects already on the current frame/layer, but EACH NEW STROKE will draw behind all previous strokes you've applied (in other words - by default, you are always drawing BACKWARDS into the screen's "Z" space, whether you want to or not).
"
Solution two: Another option I began using recently: To draw BEHIND or INSIDE existing objects on screen, I wrote a short script that creates a temporary copy of the current frame/layer, places it below the current frame/layer, and makes it inactive. Then, I select on that layer, a stencil or inverted stencil - which works perfectly even though the temporary layer is inactive. Now, I can again toggle between three different states:
1. NORMAL DRAWING where the temporary copy doesn't need to be made, or if it exists, it gets deleted and I can simply draw on the frame/layer in the normal way.
2. BEHIND DRAWING, where, after the temporary copy is created and made inactive, a normal stencil is applied. The added advantage here is that all NEW strokes still overlap one another moving forward in the space, but behind existing objects on the original frame/layer.
3. INSIDE DRAWING, where the temporary copy is created, made inactive and an inverted stencil is applied. This works the same as an alpha mask, drawing over/inside existing objects on the original frame/layer.

In addition, this version of the work-around maintains a transparent checkerboard mask that subtly shows where you can and can't draw: either outside (behind) the existing objects or inside of the existing objects.

The big drawback here is this: If I forget to toggle back to normal drawing, the temporary, inactive layer may become stranded (it never gets removed). And the next time I want to draw by creating a new temporary layer - a second, temporary layer gets created. Sometimes even a third or fourth temporary layer gets stranded during a more active drawing session.

Let me get to the REQUEST. Both of my two, improvised work-arounds have inherent problems.

Perhaps the BEST solution for NEW users and VETERAN users would be to provide an option in THE CONFIGURATION FILE that lets the more seasoned TVPaint users accept the risks of Stencil and Inverted Stencil on the current frame/layer (and provide a NEW popup warning that says "YOU HAVE AN ACTIVE STENCIL IN EFFECT WHICH PREVENTS YOU FROM DRAWING ON THE CURRENT SCREEN - PRESS ENTER TO TURN OFF THE STENCIL TO CONTINUE".

Solution two is shown in this short GIF demo…

MaskInvert-Mask4.gif (1.28 MiB) Viewed 12453 times

==================================================================================================================

OK, I'll make one last plea, NOT REALLY FOR A REVERSION, BUT INSTEAD, FOR A NEW IMPLEMENTATION which would permit advanced users to activate Stencil and Inverted Stencil in the form previously suggested. For those users who have a good grasp of the risks involved, a CONFIGURATION FILE OPTION should be provided that would TURN OFF the suggested popup warning.

In fact, a very similar popup warning already is implemented for new users who might unintentionally activate the ALPHA - Preserve Transparency option while the current frame is blank: the warning says "Image is empty and preserve transparency is active". From this terse message, the new user must first realize that the alpha button is, in fact, what turns Preserve Transparency off and on. Only then will they know that to correct the problem they must turning off the alpha button.

Which is why I argue that (in a similar fashion) perhaps the BEST solution for NEW users and VETERAN users involving the Stencil, would be to provide an option in THE CONFIGURATION FILE that lets the more seasoned TVPaint users accept the risks of Stencil and Inverted Stencil on the current frame/layer (and provide a NEW popup warning that says "YOU HAVE AN ACTIVE STENCIL IN EFFECT WHICH PREVENTS YOU FROM DRAWING ON THE CURRENT SCREEN - PRESS ENTER TO TURN OFF THE STENCIL TO CONTINUE". In fact the default popup warning message suggested here might be even more helpful because it gives the user an AUTOMATIC way to correct the active stencil by pressing ENTER.

My final thought on making the Stencil and Inverted Stencil option work on the current frame/layer is that it could provide users with an option that they don't even know they need... an option that could expand their approach to drawing in ways they would never otherwise learn about: Three alternate ways to apply new strokes to the page - OVER, UNDER (without using the "behind" mode) and INSIDE, (similar to existing ALPHA masking.)

sven

P.S. On May 29, 2021, I've currently posted a Script/Button .tvpx file that offers a work-alike and function-alike toggle that permits the artist to activate a temporary inverted or normal mask for the pixels on the current layer/images. It can be downloaded >> HERE <<.

[Joost]

Thanks Sven,

Your script works! Thanks!
But.... it's still not the same. It is clearly a workaround.

I think will make this a feature request. Because I'm starting to really miss TVP 10

[Xavier]

Guys, sorry I'm not very used to the stencil mode. Would you mind sharing a project, so that to show how it could be used ?
I'm going to test with TVPaint 10 too, so that I can see the difference.