• Announcements

    • tabull

      Browsing YouChew with an old OS/Browser   11/20/2016

      Some changes have been made and in the next few days, if you are using an older Operating System or web browser, you may have issues logging in.   Review the list below to make sure you are using a modern OS/Browser.  This should not impact a lot of people as "modern" is really most major OS/Browser from the last 10 years. However it cannot be guaranteed if a device you use is not listed here. Desktop Browsers installed on Windows Vista or OS X 10.6 or later: Internet Explorer 7 Firefox 2  Opera 8 (with TLS 1.1 enabled) Google Chrome v5.0.342.0 Safari 2.1 Mobile Browsers Mobile Safari for iOS 4.0 Android 3.0 (Honeycomb) and later Windows Phone 7 The error you may see will look like like this: ERR_SSL_VERSION_OR_CIPHER_MISMATCH
Sign in to follow this  
Followers 0

[Tutorial] The Art of Masking


One technique that has found a place in the repertoires of many YouTube Poopers is something most people refer to as Masking. Before there was even such a thing as film and video editing, "masking" referred to scenery used to block the audience's view of parts of the stage that should not be seen1. This concept carried over into early film editing, then over into today's video editing, where, essentially, parts of a video stream on one layer of a non-linear editor are "blocked" in order to composite it with video on another layer.

Or, to put it more simply, Masking is what you do when you cut out Mario's head in order to paste it on Link's ass.

Today's tutorial will cover how Masking is done in two non-linear editors: Sony's Vegas (v5) and Mediachance's Editstudio (v6). Vegas, chosen because it is a widely popular editor among YouTube Poopers (you might be using it yourself!), utilizes vector-based masks, while Editstudio, chosen because it's the other NLE that I have, utilizes bitmap-based masks. By the end of this tutorial, the advantages and differences between the two mask formats shall be clear, so as to give you the absolute best cutouts for your Poops to make you the next Walrusguy or Krobo2!

So, without further ado, let's dive right in:


Though this tutorial was done using Vegas version 5, other versions of Vegas, to my knowledge, keep this functionality in the same place and is done in the same way.

The first thing you want to do is import your clips and lay them onto different layers. You want the clip that is going to be Masked on top. Next, you right-click on the top clip and select "Video Event Pan/Crop", as this is where your Masking Workspace will be found.


for this and this article's other images, click for full size, if necessary

Once here, you want to select the checkbox by "Mask" near the bottom of the window. The selected tool shown in the screenshot above is the Anchor Creation Tool. This is your primary tool for making the rough mask (pictured above). It creates the mask as you click around. Above your ACT is the Normal Edit Tool, immediately below the ACT is the Anchor Deletion Tool, followed by the Split Tangent Tool, Zoom Edit Tool, Enable Snapping button, and the Move button.

  • The NET and ADT will see the most use after the rough mask has been drawn using the ACT. The NET is used for adjustment of the points and curve handles (when it comes to really knowing your way with curve handles, it pays to have practice with vector graphics!) while the ADT, true to its name, deletes any unneeded points.
  • The ZET is your standard-issue Zoom Tool. Experienced visual editing soldiers are very familiar with this baby.
  • ESB acts as a Shift button you don't have to hold if you want your point-moving to be at perfect angles. Holding Shift shuts it off, because if you're willing to use Shift to get the job done, then you don't even need this.
  • Move, when clicked, switches your Move Modes from Free to Horizontal Movement Only to Vertical Movement Only, then back again. You'll find Free Movement to be the most used, but the other options have their time and place.

    After you've used your ACT to make a rough mask, unless your mask is already perfect (there's a 90% chance that you'd like to make some tweaks to it, 0% if you're really confident), next the NET, ADT, Move button, and other tools will be brought in to fix the mask. If the character is immobile, then voila! You're done! If you're not Masking an anime, though, then there are some extra steps to address the pesky movement of your plumber.


    As your subject moves, the mask must move with him (or not; depends on which effect you're going for). This is where Vegas and its Vector-based masks come in handy, because all you have to do is set a new keyframe (move to the frame when the subject is in a different pose/position and press the blue diamond with the "+" marking) and use your editing tools to tweak the mask to its new position. Feel free to add more anchors if you have to! The general rule is that the more meticulous you are (more keyframes, more anchors and correct curve placement), the more professional your mask will look!

    Then, it is a matter of repeating the process until your clip is complete! In our example, Mario is now composited onto the image! However, instead of a strictly clean mask, as a tutorial showcasing the differences between Vector and Bitmap based Masking, I could not resist the opportunity to create this "Sample Mask!"

    As you can see, aside from the ease in which to tween between keyframes, Vector-based Masking allows for very easy creation of geometric shape masks and very crisp lines.

    The final step, then, is to use the Position portion of Event Pan/Crop to move your cutout to the desired location. For extra fun, this positioning can be keyframed, too, so go all out and have Mario's upper half fly all over the screen if you'd like! (My example doesn't, sadly enough) Let's see how it turned out!


    Ooh, not too shabby!


    Editstudio isn't really a common program, but for what it is (an obscure, mid-priced NLE), it packs a nice amount of power in the editing capabilities. After all, popular plugin Wax supports it along with the more mainstream editors Vegas and Premiere. My point? ES is doing something right, and today, it is displaying what Bitmap-based masks3 look like and how to use them.

    Like in Vegas, the first thing you want to do is import your clips and lay them onto different layers. You want the clip that is going to be Masked on top. Next, you right-click on the top clip and select "Quick Matte", as this is where your Masking Workspace will be found.


    Editstudio's Matteing window is essentially a stripped-down paint program. The only tool here is a circular brush ( a preview of which is shown in the window so you know how big and soft your brush is before you begin making changes), which can be modified in the following ways:

    • It can be switched to paint away parts of the frame, or to put them back on (to undo mistakes or if you want to work in reverse, which is possible thanks to the half-black half-white square, which inverts the masking area, switching what parts are invisible and which are visible)
    • The Size adjuster adjusts the size of the brush (it's in the name, after all)
    • The Softness adjuster is similar, except it adjusts how soft, or blurred, your brush is
    • The Opacity adjuster adjusts your brush's opacity, which can be useful for foggy effect masks.

    The Eraser icon is also there in case you want to start over completely (it undoes everything). The Load Brush Set imports brushes in .abr format (Photoshop and GIMP's format). With this, you can change your brush shape if you want something other than a circle.

    After you've made your mask, ES will want it saved in as a bitmap, preferably, you should save it in the same folder as your project file.

    Now, while bitmap-based masking can allow for transparency effects that would take forever to pull off fiddling with vectors and feathering, bitmap-based masks cannot be keyframed together, and so, each frame of animation on a non-static object has to be done by cutting up the video and drawing each new position a new mask.

    However, because ES uses the standard Windows Bitmap for its mask, you could use an external editor to make some even crisper masks. So for the next new position, one could instead export the frame (ES exports frames in Bitmap), then

    use whatever crisp cutout method you use (here, I am using Paint.NET and utilizing a process I typically use to cut out forumgoers' avatars to paste onto new images), and voila! You have a really clean mask! Just flatten and save the image and it's a monochrome bitmap ready to be a mask!

    Now, you do back to ES, and this time, in the Matteing window, you click the button that looks like a folder and import the image. (In this example, the mofified image is "mario3.bmp")

    So, let's see how that turned out, then!


    Nearly indistinguishable... nice!

    Now, to make this comprehensive (it's not everyday one gets to flaunt his preferred editor before an audience), I would like to share another method in which masking can be done in ES.


    Using a third video track, the Matte effect can composite two layers by using the luminance information from the third. You can experiment greatly with this kind of masking. You could use a Taco Bell commercial or an image of Matt Damon as a matte if you like, but for this tutorial, I have instead created a pure black and white (whatever is in the Matte layer is not required to be pure BW, but it works the most obviously) version of the Vegas mask. After making adjustments to Mario's position, let's see the result!


    Now THAT'S indistinguishable from the Vegas mask!4


    Now that you know how to make a character drink liquor or touch a police officer, in bitmap or vector form, what do you do5? Take a look at the various possibilities each mask-producing method provides. Don't be afraid to think outside the box! Remember that masking, while primarily for cutting out characters, can also make holes in backgrounds, or make fogs. Be creative! Personally, while I've found Vegas's masking better because of the ability to keyframe it and not having to keep track of bitmap files everywhere, I've pretty much "grown up" on ES's masking and have found it to suit my editing needs most of the time. I'd say if you're a masking connoisseur, having both options would be great.

    Here's a video showing how the outputs look in actual video form.


    1 http://www.thefreedi...ary.com/masking (definition 3)

    2 Using Masking doesn't actually guarantee that you'll become the next big thing.

    3 ES refers to Masks as Mattes, which is another term used for the technique. Typically, I've seen the term come up in articles describing the processes used in old film editing.

    4 Well, because it IS the Vegas mask

    5 Joke stolen from


Sign in to follow this  
Followers 0

User Feedback

Conway Twitty


well son I have to congratulate you on daring to tackle an explanation of how masking works in vegas. I'm sure theres plenty of people thats gonna ask how to do this in premiere WELL TOUGH COOKIES.

Great article. maybe this'll shut people up


Share this comment

Link to comment
Share on other sites


Good tutorial, though there are a couple of things that i can add to it on the vegas side of things.

Remember to feather masks. It looks so much better once you do, and it only takes about 2 seconds to set up.

Also, disable resampling on the clip (right click it, go to switches), it seems to stop the video in the masking window from becoming unsynced from the preview (and thus the result), and is a must when there is a lot of movement and you want the timing to be spot on.

Editstudio's bitmap masking looks interesting, i might have to give it a try sometime.


Share this comment

Link to comment
Share on other sites


Oh... It never even occurred to me to actually use the bendy thing.

This whole time I'd been doing this.



Share this comment

Link to comment
Share on other sites
Simon Belmont Painting


well son I have to congratulate you on daring to tackle an explanation of how masking works in vegas. I'm sure theres plenty of people thats gonna ask how to do this in premiere WELL TOUGH COOKIES.

Great article. maybe this'll shut people up

*4/8/16 pt. Garbage Matte+Keyframes

*Title shaped like character+Track Matte+Doing it all over again for every frame


Share this comment

Link to comment
Share on other sites

I remember my two biggest issues learning to mask was figuring out the proper layering (for the longest time I thought the topmost visible track was the lowest one on the timeline, I kid you not) and after that it was the keyframing for when I wanted fluid motion.

Anymore I just like to sloppy outline mask, feather and say, "NOBODY WILL EVER KNOW!"

Great tutorial, CIA.


Share this comment

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Winter Of Gamelon Theme by the Design Staff
Banner by Probo