Smooth Best leaves artefacts after rotation

[Soom]

I am doing some clean-up job now, and I have discovered a sad fact, that there is no good way to rotate a drawing without loosing too much of the line quality... Both Transform tool and Cut-Brush tool have Smooth settings (on Transform tool it's called AAliasing). Both have the same issue - If I set the smooth factor to None, the line gets crumbled after rotation. If I set Medium - it's Ok, but appears too much blurred. If I set Best (which I expect to do the job) - then the line creates artifacts all around it. I am attaching a very simple example of this issue. I don't know if it was addressed before, cause I didn't find anything specific in the forums and I don't know also if it was fixed in version 10 (I'm on 9.5).
I have developed a workaround, which consumes a bit more time, but it works: I set the FloodFill tool to Erase and then just erase all the artifacts with it.
Please advise me - is there a way to do it better?

Here you can see a simple example of the rotations of an original drawing on the left - it might seem Ok on this zoom level.

Picture 4.png (63.98 KiB) Viewed 27028 times

But here is a zoomed-in fragment of the 'Best' setting, and look at all the artifacts on the edges, and inside the line... It might appear not a big deal, but this is a cinema feature film I'm working on, and it will be on 35 mm film (!) and those artifacts can become visible on a big screen...

Picture 7.png (38.3 KiB) Viewed 27028 times

And here is the result after using the FloodFill to erase the artifacts - the problem here is that the artifacts INSIDE the line still remain...

Picture 6.png (24.24 KiB) Viewed 27028 times

[Elodie]

In fact, the problem in your example is the original object, which is already pixelized.
Smooth Medium and Best apply a blur (+ other complicated calculation), in order to be be as possibly true as the original object.

In your case, using Medium and Best are not necessary, as your original object is already pixelized.

If you need to rotate something more complicated, those options will be very handy :

[Soom]

In fact, the problem in your example is the original object, which is already pixelized.

Indeed, I know it - it's the Pencil tool with a bit changed profile - it's the standard cleanup tool for this project. Yet I think the rotation could be enhanced.
Look at the same line rotated in Photoshop:

Picture 10.png (45.18 KiB) Viewed 27021 times

[Fabrice]

I just can not see a big difference between TVPaint (smooth : medium) and Photoshop

[Soom]

I just can not see a big difference between TVPaint (smooth : medium) and Photoshop

Hmmm... well, maybe you are right... But then again - the 'best' setting becomes a bit confusing in this case...

[Fabrice]

"Best" is sharpening the images a little bit.
It explains the artifacts on your drawings (mostly line work), but it gives better results on photos.

[Soom]

"Best" is sharpening the images a little bit.
It explains the artifacts on your drawings (mostly line work), but it gives better results on photos.

Thanks. Probably that answers my questions.

[slowtiger]

The point is not "no difference between medium and PS", the point is "best is adding artefacts". The original line doesn't have any pixels of different colour, or holes, inside the line, so I wouldn't expect TVP to add any.

I did some tests on my own, and it seems to me that TVP is applying some kind of "sharpening" filter in "best" mode. Am I right? In that case, artifacts may be expected in all coloured lines which are not black or white, because in those two extremes the "sharpening" filter wouldn't be visible inside the line - you can't be blacker than black.

Here's some 800% enlargement of the result of turning a grey line on grey in "best" quality. Left is original: solid colour areas inside and outside the pencil stroke. Right: the sharpening along the edges is cearly visible, but also the pixel noise inside what should be uniform colour areas.

artifacts.png (6.86 KiB) Viewed 27005 times

I wonder if this could be avoided by changing that filter into a "sharpen edges only" mode? And maybe also rename the "best" option to "with sharpening" or similar, to indicate that there's an additional filter applied?


Soom: I was quite obsessed with a crisp pencil line when I started with TVP, and also noticed the blurring which occured after each operation with scale or rotate. But I learnt that nobody would notice in the final scene, when everything was coloured and moving. I happily have crisp and blurred lines in the same or consecutive frames, and it doesn't show, normally. Only in very fine detail it may be noticeable, and if it's really necessary, I just apply the Scan Cleaner. Another trick would be to colour your lines black first, do the rotation, then colour it back to what it was (or draw with plain black lines anyway).

[Fabrice]

I did some tests on my own, and it seems to me that TVP is applying some kind of "sharpening" filter in "best" mode. Am I right? In that case, artifacts may be expected in all coloured lines which are not black or white, because in those two extremes the "sharpening" filter wouldn't be visible inside the line - you can't be blacker than black.

yep, I just wrote it

[Soom]

Soom: I was quite obsessed with a crisp pencil line when I started with TVP, and also noticed the blurring which occured after each operation with scale or rotate. But I learnt that nobody would notice in the final scene, when everything was coloured and moving. I happily have crisp and blurred lines in the same or consecutive frames, and it doesn't show, normally. Only in very fine detail it may be noticeable, and if it's really necessary, I just apply the Scan Cleaner. Another trick would be to colour your lines black first, do the rotation, then colour it back to what it was (or draw with plain black lines anyway).

Ah, but I am actually drawing with black - the yellow was only used for clearer example in this post.
Even in a black line there appear artifacts inside the line after rotation with 'best' setting. Those artifacts are not actually a brighter color, but part of the alpha channel, because I draw on transparent layer. Therefore the artifacts will stay even if I paint over with black again, cause alpha always stays, unless I do some other complicated manipulations.

[Fabrice]

Soom: I was quite obsessed with a crisp pencil line when I started with TVP, and also noticed the blurring which occured after each operation with scale or rotate. But I learnt that nobody would notice in the final scene, when everything was coloured and moving. I happily have crisp and blurred lines in the same or consecutive frames, and it doesn't show, normally. Only in very fine detail it may be noticeable, and if it's really necessary, I just apply the Scan Cleaner. Another trick would be to colour your lines black first, do the rotation, then colour it back to what it was (or draw with plain black lines anyway).

Ah, but I am actually drawing with black - the yellow was only used for clearer example in this post.
Even in a black line there appear artifacts inside the line after rotation with 'best' setting. Those artifacts are not actually a brighter color, but part of the alpha channel, because I draw on transparent layer. Therefore the artifacts will stay even if I paint over with black again, cause alpha always stays, unless I do some other complicated manipulations.

Maybe can the "FX > Keying > Alpha Control" help you.

[furushil]

I have also experienced a similar "issue" when I cleaned up some drawing. The artifacts also occur when you just move the line without rotating it -with smooth set on "best". I attached two screenshots of the moved line. It appears that there is a 1 pixel gap between the black line and the artifact line (I hope you can see it).

The way I get rid of the artifacts is to use the "custom brush: magic wand" with range set to 25 and then just cut those artifacts away. Of course this only works when you have lines.
It is not a big issue I guess because you can hardly see the artifacts. I was just wondering, if this could somehow be resolved as there appears to be a 1 pixel gap between the line and the artifacts. They don't really stick to to the line.

[Soom]

I have also experienced a similar "issue" when I cleaned up some drawing. The artifacts also occur when you just move the line without rotating it -with smooth set on "best". I attached two screenshots of the moved line. It appears that there is a 1 pixel gap between the black line and the artifact line (I hope you can see it).

The way I get rid of the artifacts is to use the "custom brush: magic wand" with range set to 25 and then just cut those artifacts away. Of course this only works when you have lines.
It is not a big issue I guess because you can hardly see the artifacts. I was just wondering, if this could somehow be resolved as there appears to be a 1 pixel gap between the line and the artifacts. They don't really stick to to the line.

After I've started this post I was experimenting a bit more with the transformations and found out, that if you don't need to rotate the object, then the best thing is turn the smoothing off at all - it will give the most precise result. Another thing is moving the object by entering EXACT amount of pixels in the Transform panel. The gray outline appears when you move your object with the mouse and do not use exact pixels. It is true also to all other graphic software.
In any case never use the "best" setting when you need only to move the object - it will apply some blur/sharpen filter anyway.
Now if you need to rotate the object, then you might want to use the best setting...

[Mads Juul]

I will take the oppertunity to post a link to this feature request
viewtopic.php?f=21&t=5601
It is a new smooth option called "auto" for the tools using smooth. Transform. custombrush.
The idea is if theres is no rotaton or scale the smooth will be none else it will be best.

[Soom]

I will take the oppertunity to post a link to this feature request
viewtopic.php?f=21&t=5601
It is a new smooth option called "auto" for the tools using smooth. Transform. custombrush.
The idea is if theres is no rotaton or scale the smooth will be none else it will be best.

Thanks, this is a great request!