2D Tooling & Development In Unreal Engine 5


Image courtesy of https://cainos.itch.io/

It's been a minute since I've made an entry on the blog, and the reason for that is simply due to work and burnout; I've just had my first break from work in 4 years and realized that burnout has been with me like Thor setting his hammer on someone's chest. The thought did occur to me that I could have worked on planets or any development related interest over my winter break, but said thought also made me want to run for the hills - I just really needed to step away for a bit before touching the computer again. That said, anyone who has been in the industry for some time or even just burns the midnight oil for years will tell you that 2 weeks is not enough time to fully recover from burnout, which is why I'm hoping that this post will be a sort of warm up for me before I dig back into planet work.

Now I know some of you may have expected the next post to be about procedural planets, but as the title of this posts suggests, we are going to talk about developing tools for 2D development in UE5. I can hear you, dear reader, asking "Why?" and here are a few reasons:

It's been stated in various places before, but I'd like to just showcase that you can of course develop for 2D games in UE. That said, there are very little resources online showing any tooling for 2D in UE.
I'd like to showcase some examples of how a technical artist can make artist lives easier by making simple editor tools.

This post will not be about best UE 2D practices, just to be clear; I simply want to demonstrate some tooling that will help level designers achieve things that may be difficult otherwise.

If you already follow me on the bird app or LinkedIn, you will have seen some of this already, but I'd like to talk about it a little more in depth.

Custom Importing and Pivot Mode Tool

This first showcase has features that are mainly meant to help assist in eliminating tedium in certain areas - it may not seem like much, but any time saved is a win.

The Importer

The custom importer was a sort of proof of concept born out of working with an artist new to UE5 who really hated repetitive actions that prohibited them from simply designing levels, and I cannot emphasize enough about how useful it can be to have forethought when designing tools for artists. Ideally, a good technical artist can anticipate artist needs even before giving the tool out for feedback during first passes, etc. If you're worried that this is not a skill you currently possess, fret not as you'll begin to develop a spidey sense for it as you work with artists over time. In this case, the irritations would have started as soon as the artist realized they would have had to change settings on every single texture/sprite asset they dragged in, so customizing the import process was where I began.

As for how the importer works: enter the Interchange Framework. This is a wonderful, if not a little confusing, feature for customizing how assets are processed when importing them into the engine. I won't be going over what I did here, but Epic staff have you covered if you find Interchange interesting and want to try it out: https://dev.epicgames.com/community/learning/tutorials/dp77/unreal-engine-import-customization-with-interchange
All I've done is make it so that any texture coming in with a specific suffix/prefix/name will have Paper2D texture settings applied, as well as creating a folder for them based on whatever criteria I desire.
So if you have assets labeled "tilemap", then interchange can create that folder automatically if it doesn't exist and then place your tilemap assets there. That can be extended, of course, and you can do anything you want with it.

The Pivot Mode Tool

The second feature shown here is setting a custom pivot point for sprite assets, and is considerably more advanced to implement, but only because of a self imposed challenge. To normally change this pivot point (which is useful for handling sorting in top down projects), you can edit the sprite asset and manually set it. Once again, to eliminate needless clicks and simplify the process, I decided I wanted to make this a bulk edit feature and have it as a right click action on sprite assets.

If you want to do this easily, you can either do it via the property matrix feature, which will allow for bulk editing of assets, or you can write an asset action script that only works on sprite classes. Both approaches are totally valid, but I always want to keep my knowledge of C++ tooling fresh, so I decided to implement this with a custom SWindow, details panel and menu entry. It's more complex to do it this way, but it allows for some nice features, such as hiding the custom pivot point uproperty when the pivot mode enum is set to a certain value, or adding the action directly to a class' right click menu. That may not seem worth it to you, and that's fair, but I wanted this to be a nice experience for the artist.

// Create the window with the details panel as content
	TSharedRef<SU2DWindow> Window = SU2DWindow::CreateU2DWindow(
		FText::FromString("Pivot Settings"),
		DetailsPanel,
		OnOkayClicked
	);

	// Add cleanup logic when the window is closed
	Window->SetOnWindowClosed(FOnWindowClosed::CreateLambda([PivotSettings](const TSharedRef<SWindow>&) {
		PivotSettings->RemoveFromRoot(); 
	}));
	
	// Add the window to the slate application
	FSlateApplication::Get().AddWindow(Window);

Now this snippet is only a very tiny part of what makes up the pivot mode window, but I wanted to include it just so there is a clear example.

That said, I'd like to explain why I don't plan on providing full source code when writing out these articles, as I suspect this approach may upset some readers: I never want to be a source of misinformation when it comes to game/tools development, as that behavior already runs rampant with awful YouTube/Twitter tutorials where the poster is simply parroting content they've seen elsewhere. What ends up happening is you'll see the same tutorials regurgitated numerous times with little to no changes, and no real explanation of why they are doing what you see onscreen. There are of course exceptions to this, and to those awesome tutors, I salute you.

As far as my articles go, I could do my best to explain every code snippet and structuring decision I made when making editor tools, but to that I'd retort that Unreal Engine's codebase is a monolith, and no single person knows it all. There are numerous ways to achieve a result, some better and some worse; I don't want to provide potentially harmful information to readers. Basically, I don't want to make mistakes for you, if that makes sense. So if I'm to ask one favor of you, don't blindly follow tutorials, and always search for the reason why something works the way it does.

Silhouette Masking

So this is a fun one. In my experiments with top down pixel art in Unreal, I found myself wanting to replicate effects you see in a lot of pixel art games where a character will move behind a tree, building, etc, and you will still be able to see the silhouette of your character. This is immensely useful for gameplay purposes, so I decided to see how this could be achieved.

To start, we need to think about sprite sorting and how it relates to this effect. Like most engines, Unreal can sort by axis and depth. For 2D purposes, sprite sorting is only going to work with translucent sprites, which is something to keep in mind. If you've spent time with materials/rendering, the immediate concept that will come to mind is custom depth, which is a great place to get started for this type of masking.

After a very brief chat with Christian Sparks (some of you may know him by his online handle, Hippowombat), a potential solution emerged. First, set Render Custom Depth Pass as true on the character actor, and then set its material to Translucent and Allow Custom Depth Writes to true. After that, in your object material, you subtract the Custom Depth from Scene Depth, and plug that result into an If statement to generate a silhouette mask. This gets passed into the alpha pin of a lerp node and you're done.

Moving on...just kidding, I'm not about to be a hypocrite and not explain why this works.

The first step is breaking down what we're actually wanting to do, and then figuring out how to get there. To start, we're wanting a binary mask that appears when one object goes behind another. We've already talked about depth sorting, so taking the leap to scene depth is the next logical choice.

This is an image of the scene being previewed with the scene depth buffer visualization - as you can see, pixels closest to the viewport camera are darker than pixels further away.

For those unfamiliar, scene depth is a node that can be utilized in translucent/post process materials to access the depth of a pixel at any location. It will return a value in cm unless you clamp the range - a saturate node or clamp node will get you there.

Now, scene depth alone will not solve the problem, as it's going to return values for almost all objects being rendered in your scene, so we need a way to isolate the depth mask. This brings us to custom depth, which is a feature that allows us to mark objects to be added to a buffer. Earlier, I mentioned enabling Allow Custom Depth Writes in the character material, and this is why. Great, so we have our two depth buffers in play, so what's next? Well, if we tried to simply use the custom depth buffer as a mask, as it's technically an isolated mask, then the character would be unable to partially render a silhouette if the character were to move a leg or arm behind another sprite.

Example of a character masked behind a tree

To allow for this type of masking, we need to subtract our character's depth mask from the scene depth. It is that simple. Depending on your needs, you can either clamp/saturate those results to isolate the values between 0-1 (might be left with a gradient around the edges) or you can run it through an If statement to generate a flat mask. In our material graph example, the if statement is used, but a saturate would probably have worked just fine without needing conditional logic. After that, we're left with a mask we can use to plug into a lerp with 2 colors - the sprite color and mask color. That's it.

This approach has some drawbacks, with a big one being that this custom depth due to a lack of fine control. If I were to make use of custom depth in a post process material, this could perhaps result in an unwanted interaction with the actors I've already tagged. The move here is to instead utilize the custom stencil buffer, where objects are marked with a value between 0-255, and then that stencil buffer/value is used in the material to basically do what we've done above.

Stencil buffer visualization where the character has a stencil value of 1.

The other drawback was that I wanted to sample the sprite's color information for the masked area so I could something other than a flat color (you can see it in the tree example above). This proved tricky, and sadly, I am not going to share the solution for this quite yet.

Future Plans with UE 2D Work

So before you get angry at me not tipping the method for the sprite color sampling, I'd like to explain something. I've wanted to make a substantial plugin that assists in UE 2D development for quite some time, and some of the things I've shown here will be included. The best part is, I will release this plugin for free when it's got more meat on its bones, and that content includes this silhouette masking.

Somethings I'd like to include are an autotiler for tilemaps, easier workflow with actors and the 2D layer system (bet most of you reading didn't know this feature even existed), good integration with Aseprite, and much more. Paper2D has not had active work done in literal years, so if I can pick up where it left off, without harming my mental well being, then I'd like to do so.

If you're curious about a timeline, it could take a while, as I work one full time job and sometimes a side gig. Not to mention, I want to have a very clean code structure, so I will take my time to be methodical in regards to how features are handled.

If you're working on 2D now and don't have time to wait on my plugin, please take a look at CobraCode's channel on YouTube. He is the best resource and advocate for 2D work in Unreal that I've seen.

With that, I'd like to leave you with a tease of a feature I am heavily debating on continuing or not. It would likely take more effort than the rest of the plugin itself, that is for certain.

A pixel art painter could be fun, but I'd be competing with Aseprite and other pixel art SDKs, which would probably be a full time job on its own. So as it stands currently, this was a fun experiment over the weekend.

With all that out of the way, keep an eye on future articles covering the plugins development if you are at all interested. Until next time!

Search This Blog

Unreal Engine Tech Art & Programming