Jump to content



Sharky's Nightmare

by: mowub

Today, 03:33 PM


A Traumatic Moment

by: dew✿

Today, 03:09 PM


Woah No

by: Parappatherapper

Today, 02:03 PM


Casimir Greenfield...

by: kuhl

Today, 12:02 PM

Search Articles

Recent Comments

- - - - -

[Tutorial] The Art of Masking

One technique that has found a place in the repertoires of many YouTube Poopers is something most people refer to as Masking. Before there was even such a thing as film and video editing, "masking" referred to scenery used to block the audience's view of parts of the stage that should not be seen1. This concept carried over into early film editing, then over into today's video editing, where, essentially, parts of a video stream on one layer of a non-linear editor are "blocked" in order to composite it with video on another layer.

Or, to put it more simply, Masking is what you do when you cut out Mario's head in order to paste it on Link's ass.

Today's tutorial will cover how Masking is done in two non-linear editors: Sony's Vegas (v5) and Mediachance's Editstudio (v6). Vegas, chosen because it is a widely popular editor among YouTube Poopers (you might be using it yourself!), utilizes vector-based masks, while Editstudio, chosen because it's the other NLE that I have, utilizes bitmap-based masks. By the end of this tutorial, the advantages and differences between the two mask formats shall be clear, so as to give you the absolute best cutouts for your Poops to make you the next Walrusguy or Krobo2!

So, without further ado, let's dive right in:


Though this tutorial was done using Vegas version 5, other versions of Vegas, to my knowledge, keep this functionality in the same place and is done in the same way.

The first thing you want to do is import your clips and lay them onto different layers. You want the clip that is going to be Masked on top. Next, you right-click on the top clip and select "Video Event Pan/Crop", as this is where your Masking Workspace will be found.

Posted Image

for this and this article's other images, click for full size, if necessary

Once here, you want to select the checkbox by "Mask" near the bottom of the window. The selected tool shown in the screenshot above is the Anchor Creation Tool. This is your primary tool for making the rough mask (pictured above). It creates the mask as you click around. Above your ACT is the Normal Edit Tool, immediately below the ACT is the Anchor Deletion Tool, followed by the Split Tangent Tool, Zoom Edit Tool, Enable Snapping button, and the Move button.
  • The NET and ADT will see the most use after the rough mask has been drawn using the ACT. The NET is used for adjustment of the points and curve handles (when it comes to really knowing your way with curve handles, it pays to have practice with vector graphics!) while the ADT, true to its name, deletes any unneeded points.
  • The ZET is your standard-issue Zoom Tool. Experienced visual editing soldiers are very familiar with this baby.
  • ESB acts as a Shift button you don't have to hold if you want your point-moving to be at perfect angles. Holding Shift shuts it off, because if you're willing to use Shift to get the job done, then you don't even need this.
  • Move, when clicked, switches your Move Modes from Free to Horizontal Movement Only to Vertical Movement Only, then back again. You'll find Free Movement to be the most used, but the other options have their time and place.
After you've used your ACT to make a rough mask, unless your mask is already perfect (there's a 90% chance that you'd like to make some tweaks to it, 0% if you're really confident), next the NET, ADT, Move button, and other tools will be brought in to fix the mask. If the character is immobile, then voila! You're done! If you're not Masking an anime, though, then there are some extra steps to address the pesky movement of your plumber.

Posted Image

As your subject moves, the mask must move with him (or not; depends on which effect you're going for). This is where Vegas and its Vector-based masks come in handy, because all you have to do is set a new keyframe (move to the frame when the subject is in a different pose/position and press the blue diamond with the "+" marking) and use your editing tools to tweak the mask to its new position. Feel free to add more anchors if you have to! The general rule is that the more meticulous you are (more keyframes, more anchors and correct curve placement), the more professional your mask will look!

Then, it is a matter of repeating the process until your clip is complete! In our example, Mario is now composited onto the image! However, instead of a strictly clean mask, as a tutorial showcasing the differences between Vector and Bitmap based Masking, I could not resist the opportunity to create this "Sample Mask!"

As you can see, aside from the ease in which to tween between keyframes, Vector-based Masking allows for very easy creation of geometric shape masks and very crisp lines.

The final step, then, is to use the Position portion of Event Pan/Crop to move your cutout to the desired location. For extra fun, this positioning can be keyframed, too, so go all out and have Mario's upper half fly all over the screen if you'd like! (My example doesn't, sadly enough) Let's see how it turned out!

Posted Image

Ooh, not too shabby!


Editstudio isn't really a common program, but for what it is (an obscure, mid-priced NLE), it packs a nice amount of power in the editing capabilities. After all, popular plugin Wax supports it along with the more mainstream editors Vegas and Premiere. My point? ES is doing something right, and today, it is displaying what Bitmap-based masks3 look like and how to use them.

Like in Vegas, the first thing you want to do is import your clips and lay them onto different layers. You want the clip that is going to be Masked on top. Next, you right-click on the top clip and select "Quick Matte", as this is where your Masking Workspace will be found.

Posted Image

Editstudio's Matteing window is essentially a stripped-down paint program. The only tool here is a circular brush ( a preview of which is shown in the window so you know how big and soft your brush is before you begin making changes), which can be modified in the following ways:
  • It can be switched to paint away parts of the frame, or to put them back on (to undo mistakes or if you want to work in reverse, which is possible thanks to the half-black half-white square, which inverts the masking area, switching what parts are invisible and which are visible)
  • The Size adjuster adjusts the size of the brush (it's in the name, after all)
  • The Softness adjuster is similar, except it adjusts how soft, or blurred, your brush is
  • The Opacity adjuster adjusts your brush's opacity, which can be useful for foggy effect masks.
The Eraser icon is also there in case you want to start over completely (it undoes everything). The Load Brush Set imports brushes in .abr format (Photoshop and GIMP's format). With this, you can change your brush shape if you want something other than a circle.

After you've made your mask, ES will want it saved in as a bitmap, preferably, you should save it in the same folder as your project file.

Now, while bitmap-based masking can allow for transparency effects that would take forever to pull off fiddling with vectors and feathering, bitmap-based masks cannot be keyframed together, and so, each frame of animation on a non-static object has to be done by cutting up the video and drawing each new position a new mask.

However, because ES uses the standard Windows Bitmap for its mask, you could use an external editor to make some even crisper masks. So for the next new position, one could instead export the frame (ES exports frames in Bitmap), then
use whatever crisp cutout method you use (here, I am using Paint.NET and utilizing a process I typically use to cut out forumgoers' avatars to paste onto new images), and voila! You have a really clean mask! Just flatten and save the image and it's a monochrome bitmap ready to be a mask!

Now, you do back to ES, and this time, in the Matteing window, you click the button that looks like a folder and import the image. (In this example, the mofified image is "mario3.bmp")

So, let's see how that turned out, then!

Posted Image

Nearly indistinguishable... nice!

Now, to make this comprehensive (it's not everyday one gets to flaunt his preferred editor before an audience), I would like to share another method in which masking can be done in ES.

Posted Image

Using a third video track, the Matte effect can composite two layers by using the luminance information from the third. You can experiment greatly with this kind of masking. You could use a Taco Bell commercial or an image of Matt Damon as a matte if you like, but for this tutorial, I have instead created a pure black and white (whatever is in the Matte layer is not required to be pure BW, but it works the most obviously) version of the Vegas mask. After making adjustments to Mario's position, let's see the result!

Posted Image

Now THAT'S indistinguishable from the Vegas mask!4


Now that you know how to make a character drink liquor or touch a police officer, in bitmap or vector form, what do you do5? Take a look at the various possibilities each mask-producing method provides. Don't be afraid to think outside the box! Remember that masking, while primarily for cutting out characters, can also make holes in backgrounds, or make fogs. Be creative! Personally, while I've found Vegas's masking better because of the ability to keyframe it and not having to keep track of bitmap files everywhere, I've pretty much "grown up" on ES's masking and have found it to suit my editing needs most of the time. I'd say if you're a masking connoisseur, having both options would be great.

Here's a video showing how the outputs look in actual video form.


1 http://www.thefreedi...ary.com/masking (definition 3)
2 Using Masking doesn't actually guarantee that you'll become the next big thing.
3 ES refers to Masks as Mattes, which is another term used for the technique. Typically, I've seen the term come up in articles describing the processes used in old film editing.
4 Well, because it IS the Vegas mask
5 Joke stolen from Frustrated Sonic Insults the Kids
  • 0