Let's give everyone a little bit of time to filter in.
Yeah.
OK.
All right. Uh I think this is good enough to get started.
Uh Welcome everyone to uh my office hours.
Um This uh just so everyone is aware, this is
kind of like a sort of
primarily focused on rendering. Uh Alternatively, other areas of work that I do
on the team, um sort of stuff. Uh The other area that I focus on is uh the business team
uh along with a few other team members.
Uh I will say that for any sort of business related questions,
uh You may be better off waiting until the
uh monthly business office hours uh end up coming about
um which we will hopefully be getting scheduling finalized soon around that.
So, uh yeah, please feel free to go through and post questions in the um
in the uh form thread that I linked in the chat.
Uh These will be archived and this office
hours is also being recorded for posterity.
All right. So
uh we'll go ahead and uh get kicked off here.
Um Ozz asks uh boring question to start off with, but it came up the other day,
what is more efficient rendering wise,
a dual sided material on a mesh or a mesh doubling its verts via the make dual
sided mesh operation.
Uh That's easily just making just using the dual sided material.
Um So the problem with going through and using the uh make dual sided mesh operation
is you're just literally doubling the amount of vertex data for the mesh.
Uh This also has a pretty bad impact when it comes down to fill rate on uh that mesh.
Uh Because now you literally have to just printer it twice in a sense
um going through and using the material.
Uh the dual sided materials is always gonna be faster in this regard.
Uh We don't have double the vertex data to deal with.
We also don't have to deal with uh some of the
nasty bits around uh culling that on a per pixel level.
Uh We can just discard whenever we use the double sided material.
So just always go for the double sided material wherever you can.
Um I understand there are certain situations where you can't do that.
Uh Definitely our PBS variants
need to be kind of consolidated at this point,
which is one of the next big things I'll be working on.
Uh after the lib DLC stuff I think at this point.
Um
But yeah,
just try to avoid the make dual sided uh operation where you can,
I know that's painful for some people, especially like me where my
afar has like a lot of feathers where that doesn't always work.
But, yeah.
Ok. Uh,
dog bolt system says there, there's this odd hum in the default home.
How you, how do you turn that off?
Uh, that's not really
my
area of expertise. I don't really work on the cloud home.
Um, that's really more of a content team, uh, question, unfortunately.
Um,
Ozs, how is the VLC stuff going along?
Uh The VLC stuff is
going pretty ok. So far I ran into a couple of hiccups. Uh but we have uh VLC uh 4.0
bound and loaded.
Uh Right now for some reason,
there's some problems with it actually getting to the media.
It looks like we may be missing an access plug in somewhere.
Um So I, I may just end up pony up the 100 bucks to get the uh official unity version.
It could just be with a specific compile that I'm working with.
Um So I may just go through and just jump in on that.
Uh But yeah,
Z
Os, how are you doing today? Hopefully doing fantastic.
Yeah, I'm doing pretty good today. Thanks for asking
picks, asks, tell us about the sauce. Well,
uh let's go through and start with, um
and we'll go through and get uh Alex answered here as well. Um
So when it comes down to sauce, uh it's still in a pretty early state. Um
unfortunately,
just due to launch and everything leading up to launch that did kind of,
uh kick a lot of things, uh, down the road.
We do have other people working on it though. Um, people over at Vo
Anchor,
um, who are helping out with a lot of like the uh,
more boil boiler platy things here and there,
like build systems and all that fun stuff.
Um,
but, uh,
hopefully there'll be more news about that sooner rather than later for
those who aren't in the know as far as what sauce is
uh at present, we have a plan to drop unity support entirely.
Um
And uh you can also answer Cat Boy Slim's question here
as well since it kind of segues pretty well into that.
Um
So yeah, dropping unity is very much on the table.
Um It's not due to their weirdness around uh their pricing updates.
Um I know that a lot of people like, oh, you know, that seems like a great reason to,
there's actually a lot of better reasons too.
I've been working with unity for the better part of
15 years now back when it was Mac only,
they have a,
they have a fuck up like that about
once every five years and they immediately backtrack.
Uh But other reasons why we want to drop unity, um
It's using an older version of mono that just really isn't all that performance.
Uh sauce is using
dot net seven soon to be dot net eight.
So that will bring us a pretty hefty performance
uplift uh just from switching dot net versions.
Um So hopefully we can get uh more information about that sooner rather than later.
Um No timelines right now,
I'm not willing to commit to one until we have a
critical path that we're uh on top of.
Um And we're actually getting a lot of things uh knocked out.
Uh And we're getting to that uh last 3 to 5% of the of release for sauce.
Uh The reason why I say that last 3 to 5% and stuff like the halfway point
uh from experience that last 3 to 5% takes up about half your time.
So
anyways, uh that's kind of uh what the deal is with sauce, what it is.
Um And our plans for future facing uh unity support.
Uh Gareth 48 asks uh any plans uh for the future
to allow Alpha to support having shadows cast on it.
Alpha materials currently do not interface with shadows
in the same way opaque materials do and
a lack of support leads to a lot of simple texture bl effects not being possible.
Yes,
absolutely.
Um I do want to go through and take a pass at shadows sooner rather than
later before sauce really happens just because there's
a lot of weird outstanding bugs with that
and just some um
visual problems that I'm not a particular fan of uh
for example,
we have four shadow cascades at this point
and a pretty sizable shadow render distance.
Now, however,
um we do have problems uh with like uh
the different shadow cascades having these really harsh edges.
Um Additionally, I need to go through and take a good look at our shader.
This will probably end up coming with the PBS refactor
or, or consol or rather the PBS consolidation
um where this exact bug is something that I've noticed uh several times.
Uh So yeah, uh definitely something I wanna look at.
Definitely something I need to get on my own backlog.
Um So stay tuned.
Uh The basement nerd asks,
are we waiting for sep separation from unity rendering to be able to have proto
flux driven custom shader or is that a possibility for the separation of unity?
It is not a possibility at this time.
Um So custom shader
big ask always, uh I'm sure there's probably a few questions about it in here.
I just haven't scrolled down yet already.
Uh But custom shader are definitely something on our road map.
Um However,
the reason why we're not just jumping in and supporting those at this point in time,
a lot of it, it comes down to maintenance and long term support.
Uh This is always going to be a very big
problem whenever it comes down to shader as a rule.
Um
So if we were to just, just to jump in and do that today.
Um First,
we would need to go through and hack something
together so that you'd have to have the unity
install editor installed on your system.
In order to support this.
Second,
we'd have to use some hacking methods to go through
and use uh what's called the unity shader compiler.
Um People have figured out how to do this.
People have figured out how to uh sub IP CS uh streams
to these things to just feed it data and get shader out.
Um However, it's prone to breakage.
So, you know, there's that.
Um
and third. Ok, great. So we supported unity shader.
Uh How are those gonna break uh whenever we finally move over to sauce,
which uses an entirely different shading language.
Um So making sure that those shader are back,
uh like pretty much every shader you make today is forwards compatible
is going to be a massive pain.
Uh We'd rather go through and save that pain for whenever we do have sauce deployed
and we do have uh custom shares piping through sauce.
Um So yeah.
Um that's those are some of the biggest reasons
why we are not doing shader support today.
Um And before anyone gets any funny ideas to just
start hacking in their own shader and stuff like that,
uh We do do some,
some uh asset validation uh inside the client
to make sure that those shader are coming from
the correct place. Um, obviously we do have the asset system and that does drive,
uh, the shader, but we do a lot of validation if you mod that out, uh,
you bear the responsibility of breaking everyone else's game.
Uh, so
just a heads up on that one, don't do that.
You'll make a lot of people very unhappy if you do.
Um,
let's hear. Alpha asks. Uh, hey,
s was wondering if you're aware of the weird textures going
pale when porting from Neos and is there a fix?
Uh So this is one of the ones that kind of comes down to the asset variant system.
Um This also happens where if you import a
texture from your hard drive or something like that.
Uh Whenever the asset variant system is clogged, uh oftentimes
the um o oftentimes the uh metadata doesn't get calculated quite right.
Uh It ends up in entirely the wrong color space,
so it ends up looking pale and like super washed out.
Um So it's something we're aware of. Um One thing you can do is uh
in the texture.
If you in the texture, if you scroll all the way down uh on the static asset provider,
you'll notice that there's these buttons for like adjust GA
gamma,
uh
the 2.2 value, just click it once uh wait for it to process and do its thing
and then set that value to 0.0 0.4545 and then click it again.
Uh There will be a little bit of data loss.
So just a heads up, you may see banding on some textures for that.
Um But it should go through and get it looking about right. Um
Long term, this is not a great fix again, data loss being what it is. Um
But we should be able to uh hopefully get this problem fixed sooner than later.
Uh Let's see,
Trix Hart
asks, uh how are slash will draw calls uh handled, resonate uh
versus typical unity? Is there any notable differences?
Are they basically handled the same, basically handled the same right now?
Uh We do make use of dynamic batching.
So there's that static batching, not so much just because we,
that's a difficult thing to leverage from outside of the Unity editor.
We could probably add a little bit of logic to help out with that.
Um I'll definitely add it as an invest as an investigative task.
Uh Gareth 48 asked,
do you know exactly what unity weirdness is the
reason that the metallic map has to be?
Sr GB I've seen that talked about a fair amount, but I don't understand why. Exactly.
So
pretty much in a lot of unity's reference shader,
they specifically mark the metallic uh value as gamma.
Don't ask me why.
Uh This is like one of those weird things where it's like, well, why do they do that?
Um And that's just how it is additionally, they
sample that texture as SR GB.
Um
It could be that internally and one of their myriad of
shader files where they do a ton of uh conversions,
they're just uh
bumping it out of SR GB space and uh
just kind of assuming that it's linear somewhere along the way.
Um
But yeah, uh they, they do mark a few things as uh gamma being in gamma space,
metallic, uh metallic.
This included,
not sure why they do that.
Uh Excellent asking plans and SDFS uh or for people who don't know what that means,
assigned distance f uh functions probably signed distance fields is what you mean.
Um Some,
just some cool graphic stuff I've been reading up
on or if you have any interest in them.
Uh Just as a drone graphic question.
Uh SDFSR is an interesting one for people who don't know what they are or how,
how they are applied if you've seen
Unreal Engine
at all for the past,
I want to say almost decade.
Uh They've been leaning pretty more and more heavily into sign distance fields. Uh
Lumen their global Lumina system
uses them extremely heavily uh to uh function.
Uh They have their benefits.
Uh They also have a few drawbacks
particularly around uh the preprocessing that's required
to get STFS working. Um
That kind of makes it a bit of a weird one
for us in particular and for some of the effects,
there's just generally
more low end friendly ways to achieve some of these effects.
Uh For example, touching on dynamic global illumination,
um dynamic lumina
illumination is by far not a new problem.
It's been something that has been a problem in the industry since like
the mid two thousands.
Um
However,
we do have more efficient ways to do it
than creating a gigantic STF Atlas that we're just constantly
uh streaming texture data in and out of. Um
they're great for certain things like if you don't wanna have like geometry
and you just wanna have like an interesting uh surface like a um
uh metals is a great example of uh where SDFS really shine. Uh Because, you know,
rendering that many triangles is probably dynamically
is probably not a great idea but doing
it all inside of a shader and re marching it is probably a better idea.
Um But we'll probably end up
evaluating this one more case by case where it makes sense.
Uh And most importantly,
making sure that's something that can scale to a variety
of hardware uh made in like the last five years
ish.
So yeah,
we'll revisit it.
Uh Kittle
asks us a follow up to
Oz's question with regards to uh the
double sided question,
uh is there a point where manually doubling slash averting a
subset of the polygons in the mesh and blender worth it?
For example, the body and feathers of an volley are on the same mesh?
Is it worth it to double just the
feathers externally sub using a double sided material?
Uh
Well, that depends on what kind of material you want to use.
Uh So our PBS uh materials are
all over the place if I'm honest. Um So if you need like a specific effect, like
uh if you need like a specific effect, um
you probably want to use a double sided variant of the PBS uh materials.
Um You know, if you don't need like,
you know, anything particularly crazy if you have like say PBS R I,
where we don't have a double sided
uh variant,
you probably want to double up those uh th those faces.
Uh My avatar doesn't really use a lot of uh fancy materials like at all.
Um I do use some
LP materials, however, so that means I can't just go through and,
you know, use a PBS double sided on my feathers.
Um I do have to have the uh I do have
to have the vertices uh doubled up and flipped on that.
So that's one example where you'd want to go through and do that.
Uh She
asks if, and I'm gonna, and I'm gonna ignore the win because it's not a win.
Uh Well, it's actually a win. Sorry.
Um
When we get a custom rendering engine, what's the likelihood of node base shader,
editing and compute shader?
Uh So custom shader are obviously a yes.
Um That's probably gonna be handled through proto flux um or a subset thereof, more
specifically,
um
compute shader is also a yes.
Uh But we need to figure out the specific limits around that.
Uh One of the biggest things that we're trying to figure
out is how do we handle shader alongside all of this
in a more
safe way.
Uh Like for example, you build a shader and you create a crasher.
How do we go through and detect that uh compute shader are a great way to go through,
install the GP U.
Um And although we're gonna be doing a lot of mesh shader uh emulation,
uh Once we do have custom shader,
um
we're gonna end up uh you know, we,
we can go through and detect certain crasher
patterns and stuff like that uh on inside
the AVI by using a few special nodes with the GP U installed in them.
Um
We want to extend that to compute as well. Uh What those limits are gonna be.
I don't, I don't know. Right. I don't know right now.
Uh But I should be able to have more information
uh once we get to that point and once we've actually
done uh
more testing around the idea,
OK. Let's see here.
Yeah.
Uh Let's see here. Question about getting
optimized RBD animations and resite which of these would be more performant.
This is worst case scenario of the piece being deformed
uh having a high poly count.
Uh the force on the left
uh
has each rigid body separated into its own mesh that has its own trans
transform animation
force on the right. Has all rigid bodies merged
and is deforming one mesh with a skeleton.
Let me just watch the video real quick.
Yeah.
OK. So I'm gonna ask
some follow up on this one. Feel free to put it in the chat.
Uh This is directed at
Smiley Face here.
Um
When it comes down to the specific animation here, uh the one on the left
is that just, just, just to make sure I understand this
is that just straight up using um just transforms for each piece.
My understanding what I'm seeing here and what you're describing correctly
that's gonna be faster.
Uh The reason for that is uh skinning is kind of a pain and a lot of ways.
Uh Yeah, you're gonna have
a bigger draw call cost.
Um However,
it's generally gonna be faster than actually
applying skinning compared to the other one.
Yup. No problem.
OK. Um The base,
the basement nerd asks,
is there a difference to how LS work in resonate compared to other
platforms aside from linear coloration because I think a custom lut I made
uh I made broke uh was meant to provide winter coloration to a world. Uh
And now it causes drastic high contrast darkening effects
and I'm not sure if it's intentional or if that more
likely broke somewhere and I should submit a ticket about it.
This issue occur also occurs when creating a new light
in game from the external texture that created it.
So if it is a bug, uh it's not created just by the asset conversion process.
So yes, this is a known bug.
Um Lets right now are in a bit of a weird state uh as a result of color management.
Um
So
one problem we have with 3D textures is unity is Janky.
Um So with a two D texture or a Q map,
we can actually go through and upload our own textures like and just
manage them ourselves without unity really having anything to do with it.
But 3D textures
are different because unity demands that they are different.
So we do a direct uh upload pipeline
when it comes down to 3D textures and let unity handle it
as a result. This means that you can't have color management on 3D textures.
That's pretty much what the cause of this is.
Uh
There's a pretty significant uh merge request that we have internally
uh that completely refactors,
how 3D textures work from a texture upload perspective
and how they're processed by the A BS.
Uh I need fruits to go through and look at a
couple of problems that I've ran into on that one.
Mostly a couple of things broke in the process and we
don't want to break this even more as you might imagine.
But uh this is a bug we hope to get fixed soon
and also
have just generally a better uh texture management pipeline for that.
So three textures aren't kind of off to their,
off to the side, doing their own thing anymore.
OK. And uh that's really awesome to hear Vexus. Uh Good for you.
Snowy Ti Dragon. Um
Asks um
when resonate switches to sauce,
how will it affect things like file structure
uh in the modern community and et cetera?
Uh Well,
that's actually a really interesting question. So sauce is going to be open source.
So if the modern community wants to contribute to the rendering engine,
they will be uh permitted to do so
um probably gonna be like Mozilla public license or
something like that so others can use it.
But, you know,
we don't have to worry about the weird viral nature of the GP L or L GP L for that matter.
Um
But yeah, um that's,
that's one end of it in terms of how
that actually impacts uh the client and mod the client
honestly, probably not very much.
Uh There,
there are a couple of rendering mods out there that are
probably just going to straight up break and cease to function.
Uh But that kind of comes with the territory of
like
a paradigm shift in terms of how you're rendering of how you render things.
So that should be expected.
But most mods should generally be fine.
All right. Um
Let me just read through this post.
All right. Ras
Fire
asks, I'm looking to replicate an effect uh from the Poo
shader.
Uh
on the Tunney side of things,
I've seen some applications that allows the illusion of line art on models,
but only from a certain perspective as in
uh lines, only on the left side of the character
is this effect possible at all and resonate
not exactly due to how XYY
works. Um However, I have fun segue there, which we'll get to
uh
Xi
C uh only allows you to go through and um
apply the outline uh globally with a little bit of masking
to go through and affect the thickness of the outline.
But you can't really go through and do this based upon a specific perspective,
you can kind of get there by modulating the thickness of the outline.
Uh but it's not really a perfect solution.
Um That being said, Zig Cy
is definitely on the list of shader that's probably getting removed at some point.
And instead we're gonna be replacing it with our own tune shader
uh which will include uh some additional features that we just
don't really support or is well supported in the current version.
Ze that we're using.
Uh a lot of that is just to better prepare for sauce and custom shader.
Uh We do need to resolve some IP related issues before we move to sauce.
Um just things we don't own or would be difficult to pour over the sauce anyways.
Um But yeah, uh definitely something that I intend to add
in the future.
Uh but it's probably gonna come once we have our own tune shader um created.
Let's hear Xo
asks uh in relation to Snowe's uh question regarding uh the modding scene,
would that affect shader also? Absolutely.
If you have someone who has moded in shader
that is going to break,
am I going to help you fix it?
Absolutely not.
Am I going to feel bad that I did not help you fix it? Double? Absolutely not.
This is your warning.
You're gonna hear this warning again multiple times,
especially as questions like these get asked around custom shader
and hacking and modding. Those in today
heads up
Gareth Fortier asked,
will the consolidation you've mentioned also solve
the issue where ambient inclusion can only
be packed into green on specifically PBS
uh metallic and other PBS metallic variants
that will fix that problem.
Cobalt box asks,
would we have any eta on when collections would be implemented into proto
flex? Asking the wrong guy? That's really a freaks question.
So with the PB modern asks,
so with the PBS consolidation will allow certain effects to be masked and stuff like
that stuff like rem effects and LT being able to be used for transparency.
Yes,
actually, wait, hold on.
Not quite.
Um
So part of this
is material layering uh which is another,
uh which is another feature that I want to work on.
Um
largely just to kind of get
a little bit more flexibility around our shader uh almost in a poo
style way. Um
while we're still on unity. Um So the masking side of it is like secondary
um
the rim stuff, like if basically, if it's not a feature
that is already in the PBS variants that we have
in any of them,
you're not gonna see it with the PBS consolidation.
So let's just get that expectation set right now.
Um
So yeah,
um if it is something that's currently mask, you'll get that.
If it's something that is not currently mask,
sorry, you're gonna have to wait for material errors.
So specs asks follow up to Garrett's question as far as the sr GB metallic map stuff.
Does this metallic map being sr GB weirdness apply to any other masks uh slash maps
in particular? I'd like to know for Zizi
since I have a substance painter template
and want to ensure it's correct.
Uh
Yes,
yes.
And
um basically, uh
it just really depends on the specific shader that we're using. The Zi
one. I couldn't really talk to how accurate or correct your template is.
Uh Zixi is a third party shader uh that we li
well,
uh I don't think we license,
I think we actually used a older open source version of that one.
But yeah, it's it's just one of those things where
it's hard to say.
It depends and at least on the PBS ones,
the metallic map is the only particularly weird one
on that one
that I can think of off the top of my head.
So, yeah.
Uh which, oh yeah, by the way, just to note about the PBS stuff,
um I do want to get that into a GLTF compliant state
uh which does require uh more linear mappings uh particularly for metallic.
So
expect more stuff on that.
Eventually.
Uh Josh
asks what Chang
Lang language will
s use.
Well, that's a fun one.
in terms of what you can access proto flux in terms of what's using under the hood.
Um
WGSL,
so WGSL is a newer shading language. You could quite literally think of it as
Rust for shader.
Um
So that also gets translated into different platform dependent uh shader L
shading languages like on apple, we're gonna be using the metal shading language,
of course.
Uh Then on Vulcan,
we're gonna be using uh DLSL and if it makes
sense from a performance standpoint on direct X 12,
we'd be using HLSL.
Uh though I think maybe it compels directly to DXIL.
I'm not entirely sure I'd have to double check on that.
Um So like all the shader, uh we'll just go through that.
Uh And that's pretty much what we're going with there.
Uh The reason why we are not giving you direct access
to WGSL is largely just from an overall safety standpoint.
Um Obviously, once you can just go through it and throw your own shader at the wall,
um and like with full source code and everything, um that does kind of create
a few potential problems there.
Um Although WGSL is quite literally built for the web where you have to
take these sorts of considerations and to uh like in mind and the heart.
Um It does kind of create a few potential problems here and there. Um still
uh just because we have these things called
geometry shader because way back in the day,
someone thought it would be a good idea
to really mess up our geometry pipeline on the
GP U and make something that's very exploitable.
So
proto
flux all the way, that's what you guys need to worry about.
Uh Cat Boy Slim asks, are there any plans for UDIM slash UV tile discard?
Uh certain portions of meshes can be discarded in the vertex shader so
that toggles can be added without increasing the number of draw calls.
This is a fun one.
So
first off, let's talk about UDIM,
uh UTIM is not something that's really
super easy to support uh from the get go on GP US just because of how it works.
Uh You can do it.
Um However, for each tile that you add,
um your shader become a lot more bloated.
So that's a bit of a difficult one to really support uh straight up.
Um And the second part of this is the
uh going through and doing discards on the vertices.
Uh and the vertex shader,
that's not really how that works. Uh vertex shader. You, you, you have what you got,
you can't, you can modify its position, but you can't do discards on those vertices.
Uh Now with something like mesh shader,
you can do that sort of kind of uh by
going through and discarding entire mesh letts or just,
you know, going through and you know, not really
doing anything with them or moving them to a place where they're not visible.
Um
There's a lot more things you can do using mesh haters in this regard,
but that's done at the mesh
lit level.
Uh So if, for example, you have
like a jacket
and this jacket has multiple layers, multiple pieces to it,
but you only want to render one layer of it.
Um At any given point in time using a mesh lit, you can go through and do that.
Uh You can flag uh different parts of the mesh lit
uh different parts of the mesh just at the
mesh lit level as being visible or not visible.
Um And that's more something we would probably end up
doing around there is just letting you go through and uh
determine what uh parts of the mesh uh you
want to display or not through a mesh shader.
Uh But obviously, this requires custom shader.
Uh The short answer to this is no, we're not doing this right now.
It's due to the sheer complexity and bloat
that that will bring to our existing shader.
The long answer is custom shader.
Uh Let's see, just kind of going down the list
Gareth for
uh asks another more general question about authoring for
the engine which exported texture map should be SRB,
which should be linear from the following common texture maps.
Vido obviously should be Sr GB.
Uh You should always author albedo and Sr GB unless
you have a good reason to use floating point textures,
uh floating point textures just for the rest of the room.
Um Those do not have a color profile associated with them.
Uh The reason for that is you should be
applying whatever gamma curves and whatnot at author time
uh with floating point textures.
Um just because you have enough bit depth there to
handle whatever gamma curve that you want to throw out at
uh height. Uh rather well, let's back up normal.
Uh normal should always be linear
uh straight up.
Uh You will have problems if you try to make that SR GB or nonlinear uh
height
that should be linear as well.
Um Because you are dealing with like noncolor data
um
metallic honestly, it it should just straight up be linear but
you know, he does weird crap.
Um
smoothness should always be linear
uh ambient inclusion should be linear specular color should be SR GB.
And on ambient inclusion unity also does some weird
crap because that gets packed in with your metallic map
Stacy asked will will you guys focus on
performance going forward instead of new features?
OK. So let's talk about the performance problem.
So performance is always an incremental and iterative thing.
Um We did have a few performance boosts with proto flux. Um
And you can continue to see additional performance improvements as time goes on.
Uh You will never see us have a singular silver bullet to performance
even with the dot net upgrade.
Uh the dot net app create great although we expect
uh performance improvements across the board with that.
Um We don't really foresee that being like 90 Hertz
worth the worth of performance improvements for every single world.
Um
So there's that end of things.
Uh There are other parts of the engine that we know that needs uh some
performance love um
data model being amongst uh those um along
with various different other miscellaneous bits and pieces,
probably a few components as well.
Um
However, the other side of this question is
how do you make content that's performant?
So
just because we do all this work around performance, that's great. That's awesome.
Needs to happen one way or another.
Um
There's also gonna be the content side of this as well.
Uh Due to the very nature of resonate.
Um We can't really just go through and just make everyone's
stuff just automatically perform better uh in every single possible scenario.
Um So, for example, if you have a
proto flux graph that just doesn't really perform particularly well,
um you shouldn't really expect that thing to just
magically get better due to something that we do.
Uh You do need to have some engagement with other content creators when
it comes down to creating uh performant assets and worlds and so on.
So two parts to that one part. Absolutely. The other part
that really requires content creator engagement.
Uh Pliska.
Uh Well, this is multiple parts. Um
Luka says and also asks, um
as I understand you were in the process of updating libraries
like LVLC as well as fixing uh paths for them.
Does that also, does that include also a bunch of other libraries as well?
How much will it improve things on various platforms?
Windows Linux, especially since it's unusable.
Uh also are some tweaks to unity GC planned in the meantime before SS.
So
uh let's talk about the other libraries right now.
That's really strictly on a where it makes sense spaces, for example,
Ultra Leap is something that we are woefully behind on to the point that
it just straight up doesn't work on a lot of um on a lot of uh setups
anymore.
So we do need to update uh the SDK for that uh as one such example, LDLC, obviously,
we're using a multiple years old uh version of LIB VLC.
Um
And additionally, we're using uh universal media player uh presently,
which is deprecated doesn't really exist anymore.
Um So we couldn't update that if we wanted to.
Uh So we need to basically just rip out uh
universal media player in that old version of VLC.
Uh and just go for uh the official unity distribution of uh LVLC.
Uh So, yeah, it just really depends on the specific library you're asking about.
And if it's something that is currently particularly broken
or ha or poses a significant security risk,
uh we'll go through and look at those case by case.
Uh And also, Linux is something that I want to personally focus on just because like
it is extremely broken.
Um And I, I think uh I can afford to throw at least a couple of cycles at that.
All right, going down the list
prime geo asks and we have an option to switch to tri
linear slash
anisotropic and the value of switch as well as repeat slash
clamping.
Uh
You, you have that already
um
Like OK. So follow on question to that.
Are you talking about having this be automatic prime go?
So
we do the bilinear thing that's like an old carryover from the old quest builds.
Uh Just because quest um bilinear was faster.
Um
But like most people as far as repeat slash clamping, they're generally going to
want repeat. Um
So we're generally going to go for what we think is like a same default for that.
Uh We could probably bump it up to tri linear, maybe like
anisotropy level two.
Um at most.
Um But for the repeat slash clamping,
we're probably not going to go through and
uh switch from the current default on that.
Um For clamping from my experience,
that's usually a special case for a lot of people.
So that's probably how,
so we're probably gonna just stick with repeat for the time being
as far as auto conversion to BC one slash three.
That's also probably gonna stay in there.
You're getting texture compression, whether you like it or not.
Uh It's another one of those things where
um
not having texture compression or selecting a specific compression
method is often times going to be the exception,
not the norm we could potentially look into.
Uh once we have better uh hardware statistics,
uh switching to a STC
and just setting, uh what we think would be the most reasonable block size for a STC.
Um Just so you can get a little bit better quality out of your textures.
Uh But that's about the, the best I can offer you right now.
Oh It should automatically compress
because we don't really let you skip the texture compression stuff
whenever it hits the A vs unless you explicitly disable it.
So if that's not the behavior, you are observing then that is most likely a bug.
Granted, you know, A BS has been super backed up. So just heads up on that one
we do have.
So we do have the preferred
formats.
Um So that could be what's thrown you off there,
but we do, in fact compress textures a good bit.
So
uh preferred format just so everyone knows you're basically telling the A VS,
hey, I want this specific format.
I want BC one, I want BC three, I want a STC four by four. For example,
it's just uh it basically just allows you to set an explicit format. So,
but uh I can go through and take a look at it to see if it just does
a sanity check to make sure that it is actually doing what it's supposed to do here.
OK. VoIP Paw
asks, I thought shader just interfaced to the hardware API through some driver.
Why is there specifically unity shader?
Where is the compatibility disconnect?
So
how shader work at a fundamental level
has
little to do with a specific hardware API and everything to do
with how you go through and set up your engine around them.
Uh We have these things called uniforms or uh II,
I forget what um uh direct X calls them
basically properties that you can set on a shader.
Um You're going to have a lot of incredibly specific ones uh that are very,
incredibly specific to the specific engine that you are utilizing.
Um So that's one thing I'm not gonna go through
inside of sauce and Sprinkle unity underscore point light uh and
then have an array there and then try to handle everything
exactly as uh as unity handles them and so on.
That's not gonna be a thing.
I'm not gonna put that extra work in just to get you custom shader today. Boyd Paul,
sorry.
Modern asks, is there going to be something like baked lighting in the future?
Are we gonna have light probes?
Um, talking sauce? Well, actually this may not be a sauce thing specifically. Um
So yes,
there is interest in having baked lighting and I've been
doing some research around uh this cool thing called sphere co
gins.
Um The order 18 86 used these uh quite a bit for their uh static lighting uh method.
And let me tell you that game as far as being a close to launch title for uh the PS four.
It has no business looking as good as it does.
It looks great, basically.
Um And so we're probably gonna go in that direction for our own light probe solution.
Uh That's something we could go through and add to unity today uh without
really adding a lot of work for us down the road for sauce.
Uh The same goes for the baked lighting side of things. Um
It would be,
well, I mean, we could just go through and inject this,
inject the shader code into sauce shader later.
So it wouldn't be that big of a deal. Um So yes.
Uh baked lighting is an absolute, yeah, we're gonna do it.
Um The big,
the bigger question is how we're going to do it.
And I'm looking for a balance between quality and baking performance uh on that one
um baking performance in particular because since you don't really have since,
since the game is your editor.
Uh The obviously that means that we can't go through and like
do something like unreal light mass
where you have to have like a small swarm
of computers to compute uh your uh light maps.
Uh Same goes for unity nowadays or at least it's getting to that point.
Um So yeah, spherical gals
is what we're looking at. Feel free to Google that one.
M JP, a pretty known graphics programmer in the space.
Uh He has an outstanding series on his blog about that one
can always say nice things about it.
Uh Portal jumper 79
asks once everything currently is all sorted out.
Are there any plans to add more graphical features to play with
maybe better at uh A O and SSR on the post processing side, for example.
Uh Well, clearly, you know, I've, I've went through and said that yes, there,
there are more things coming but uh
like big lighting, for example,
uh but better A O is definitely on the list. Uh I'm not a fan of how we currently handle
Ambient Olu. Um There is
a version of color management
that actually makes ambient occlusion. Ambient.
Um I know that sounds weird to everyone
but uh right now,
um we currently go through a multiply ambient occlusion against the scene.
There's a few things in there to keep it from over darkening.
Uh But it creates a few problems in other places.
It's really just a temporary fix right now.
Um But eventually we do want ambient inclusion to be ambient only.
The only problem with doing it with the current uh
ambient inclusion solution that we're that we're using is
in order to do that, it only works on uh opaque surfaces
and surfaces that are not rendered using a
tomb shader or other forward rendered uh shader.
Uh And we want a O on those as well. So that creates a little bit of a problem for us.
Uh I do plan on going through and refactoring this entirely though.
Another one of those things where we're gonna
need uh our own solution for sauce anyways.
So it makes sense for us to go through
and just rewrite that uh in preparation for sauce.
Uh SSR is a
little bit of a weird one.
And that uh that's not typically very easy
or performant to do in VR. Uh We do support it in desktop,
but VR is a bit of a tricky one.
I'm not sure that that's one that makes sense to prioritize right now before sauce,
however,
uh, but other small things as they come up, uh, definitely,
I do want to take a look at better shadows is definitely on the list of things to
take a good look at since I'm pretty familiar with Shad, uh, unity's uh,
shadow rendering, uh, pipeline.
Um, but we'll see, we'll, we'll see what comes up.
Mhm. Ok.
Yeah.
Void Paul.
Mm.
Another great shader one.
Why not attempt to provide compatibility to sharers room for VR chat
to take advantage of the huge library of content already out there
shading languages.
That's really what it comes down to.
We're not going to support HLSL
and we're not going to create backwards compatibility just for unity shader.
That's not happening.
That's a large drain on compatibility. Uh That's a large,
that is a large productivity drain just to go through and have
compatibility with something that we don't intend to support going forward.
It will bit rot,
it will fall apart over time
and I'm not signing up for that amount of maintenance overhead for myself.
Yeah.
What a sauce. Your custom engine. Uh Well, sorry. Uh um
I'm not sure how to pronounce your name.
Apologize.
Uh I, I'll, I'll do my best and it's probably gonna sound funny.
Uh
You cover
for a look.
All right,
Clark.
Thank you for going through and putting that there. Uh Thank you.
Otherwise this is gonna
sound like I'm gargling Listerine or something like that.
Um What is sauce your custom engine custom ring pipeline? Yes.
Uh Stacia
S will po
tune and little tune shader.
Be able to be imported to resonate when importing 3D avatars. No,
you will have to go through and utilize the tune shader that we make available to you.
We can however,
add features to those Tomb shader and we can go
through and make certain accommodations for some level of compatibility,
but this is never going to be 1 to 1 just so you know,
uh Gareth asks in regards to the future custom tune shader,
will there be a place people can suggest things for it?
Uh
Yeah, github.
Uh
that should be going up soon.
The basement nerd asks, would it be possible to implement uh Oh, what,
sorry uh skipped over Gun Griffin.
Um
Gun Griffin asks, would a mod for the rendering engine be called a sauce
pack? No, no. Why, why, why would you do this?
Oh
my God,
please. No.
Uh
Of course you get, of course, we have a soy sauce
reaction. I hate it.
Um The basement nerd um ask,
would it be possible to implement a feature into PBS displays that
allows it to read an R GB texture to offset the vertices
in all three axis rather than just the tangent axis?
Yeah, that shouldn't be too big of a problem.
We do have uh we do have to be careful about our sampler account though.
Uh That's a great way to break a shader.
Um So, you know,
just a thing to keep in mind.
Also, given that it is 4 22 we've been at this for almost an hour.
Um We're going to cut off any questions uh from this point forward.
Um And I'm going to get through the remainder that we have here.
Uh Modern asks how is all going to be different in these aspects?
Transparency and render cue.
The reason why I ask, well, you've seen my avatars,
they use two meshes to really work and well, alpha ordering will be weird.
Is there anything that could really help uh
with that with a future rendering engine?
Or is that a big ask? It's not a big ask actually
uh owner order independent uh transparency is a thing and a
thing that I would like to go through and support.
Um So yeah, um that's the thing.
Render Q is also a thing that we're gonna need to support as well.
Um Just because there's a lot of content, even like, even if you weren't,
even if we weren't using unity today, there'd be a lot of content
that has a lot of hard dependencies on render cues.
So these are,
these are things that we're going, that we're going to work to support.
Uh However you shouldn't need to go through.
It's a goal,
we'll see how close we get.
Um
the hope is that you won't need to go through and
mess with render cues to have uh correct transparency sorting.
So
if anything you'd be using a render Q to override.
Um having correct transparency sorting.
Gareth asks uh any plans to implement rendering
layers earlier than the release of sauce?
To my knowledge,
this is the solution to one of the
common graphs you hear about regarding screen space,
screen space effects and resonate
uh specifically the refract, the refract and blur shader,
which is that they don't ignore the player and other specific objects.
That's a bit of an ask. We'll see. I'm not making any promises.
Uh Kittle
asks roughly how is material laying going to work?
Uh Is this going to be some kind of
encoding of the concept behind materials slash mesh stacking
where you render multiple materials uh using the same sub mesh or something else?
Uh If you, if you use filter materials
or
yeah, filter materials as part of your layering,
will this still involve a grab pass?
So
this is a really interesting one.
So material laying really, it's more about we have a set of outputs
uh that basically feed into our BRDF.
Uh By the way,
we're probably only going to make it available for PBS materials at first
with this eventually being able to incorporate entirely different
shading models like tune shading and whatnot later.
Um I don't know how many people in the chat have really been keeping
tabs on what Unreal Engine's been doing with their substrate uh material system.
Uh That's probably a better example of kind of where I'm thinking of going here.
Obviously, it won't be, may not be node based.
Uh
But you'll be able to just like go through and just have
um different material attributes that you can blend mask and mix together
in effect in different ways.
Um I will note that the more complex your material
layering gets the less performant it's going to be.
Uh So that's one thing to keep in mind once we have it,
uh one example would be
say you have a rim shader,
you know, you want to go through and highlight the edges of something.
Um But you want to go through and highlight the edges
uh in one color in some spots and a completely different color in other spots.
That's a good example of using material layering
or let's say you want to go through and have an albedo
that shifts over time.
Um And you want to go through and just have this masked off.
That's another good example of uh material layering.
Uh The specifics haven't quite been worked out yet,
uh particularly from a mechanical level
in terms of what you guys will be able to directly work with.
Um And a lot of it is going to come down
to what the specific performance impact and implications are going to be
uh for different usage patterns. Uh So that's one thing to consider here as well.
Uh More information is forthcoming,
Vo Paul. I'm going to start answering these questions around custom shader.
You're not going to have custom shader right now.
You can stop asking.
You're waiting until Sauce
Dr
Hart.
Uh asks, will the recording of this stage be stored when it's finished? Yes,
Kal
Stargazer asked, does reite
currently support Linux
and when the new rendering engine is released, will it still be compatible?
Yes,
ish, there's a lot of problems on with Lin support right now, but also yes,
it will be compatible.
Uh
CBO slim asks MP3 audio clips win. Um That's a fun, that's another fun one.
So MP3 is technically deprecated as a file format.
Um And uh a lot of that comes down to the um
uh the Fra
Hopper Institute does not want to uh issue a
license to the MP3 patents or encoders anymore.
Um That makes MP3 support pretty sparse.
Um Generally, my recommendation is to um convert things to OG wherever you can OG
VBS.
Ok.
Moving down uh Easton asks why do mirror lights hurt so much more now? Like I can't
feel like I can ever have the lights on anymore. Uh When previously it was no issue.
So mirrors actually rendered entirely with Ford rendering.
And additionally, we support up to 12 light sources uh at this point.
Um So that means if you have like a lot of meshes and stuff
like that and a lot of light sources that are maxing that out,
then it's going to hurt a little bit, it's gonna hurt a bit more.
And if you have pretty low end hardware, it's going to hurt a lot worse.
Um I have thought about reducing it down to 10 or eight light sources.
I know that's gonna make some content, not look as pretty.
Uh But this is largely just to mitigate the mirror cost uh here. So, yeah,
we'll see.
Um I'll take on Gigi Wentworth's question just because it is something
that we do want to do or at least some of this.
Um First, I don't
know what you mean by verbose mode on the menu options,
but search and proto flux components, et cetera.
That is something we do eventually want to do.
Um It's just a matter of figuring out where that needs to be on the priority stack.
Um Can you uh explain what verbose mode is, please?
So like a tool tip then am I understanding that? Right.
Yeah, that's something we do eventually want to support as well. Um
But again, it just, we're not,
we're just not sure uh where that's gonna be on the priority stack just yet.
Uh We only just recently launched.
So, you know,
we're still trying to work out where things need to be in terms of priorities.
Um So bear with us while we figure that out, please?
Ok.
Pien
asks, what about animated light maps?
Um
I mean, we could do something like what unreal does.
So what Unreal does
is that they have like their own system for setting
up light map variants for this particular use case.
Um Like having like a day night cycle utilizing nothing but static illumination.
Um We could do something like that.
Um That definitely would not be like a version one or MVP of a light mapping though.
Uh But in time, we'll see,
uh Prat asks,
a big ask apparently
could slash should we get tools for profiling assets,
at least at a high level or import time tools,
identifying common culprits of performance issues and notifying the user at
import time might help even non technical users would at least
be made aware that they should reach out and ask how
to fix their avatar if the U I warns them,
I'm thinking simple stuff like all like poly
count
a abb known expensive components, et cetera.
This is something that we've talked about a little bit internally here and there.
And I think it does make sense to go through and,
and offer some level profiling tools.
Um
The question is what level of profiling tools? Um
because you have, for example, on one end of the spectrum, something like Tracy,
which is like
a
microsecond level profiler um which you know, I I use at my 9 to 5
working on the second life viewer
uh to go through and find different performance pitfalls on the CPU
and to a more limited extent on the GP U as well.
For that,
you really need something like render doc or NVIDIA insight or
do you just want like a basic TLDR on your system?
This part of the hierarchy takes like, I don't know, 15 milliseconds to, to,
to work through
um on import.
That's always a bit of a fun one because it does require
that we make a bunch of assumptions as far as you know,
how your content is going to perform.
That may not necessarily be as relevant as just, you know,
understanding what the CPU time cost is,
uh especially given that we're very CPU bound today.
We could potentially look at that though.
Um
I'm definitely not think uh uh I'm definitely not thinking something
like VR Chat or second life's render cost uh stuff.
Uh Because those uh have a tendency to age like fine milk
uh which is a not at all.
Uh especially as hardware progresses forward,
you add support for different platforms or what not,
you end up finding that a lot of those estimations
are just all over the place in terms of the,
in terms of what actually ends up happening
versus like actual profiling to link. So,
um yeah,
uh I guess that the real question is
how in depth do you want your profiling tools to go?
That's really the, the, the, the, the, the counter question that I can think of here.
OK. So not like render
doc. All right. All right.
Um
Would just being able to measure how long it takes for a part of the hierarchy.
Do you think that would be useful? And having that broken down into
how specific components and parts of the hierarchy are performing?
Like get a little report, you just hit sample,
it, samples it poops out report
and you can figure out why, why something is slow from there.
Would that
be good, do you think?
Yeah.
OK, great.
Uh I can go through and look at reviving a little bit of work around that one.
Uh Sorry specs but uh doing it broken down by CP and GP U time
without sauce is gonna be a bit more difficult.
Uh unity only gives us so much information
and there's only so much we can really measure from dot net more generally.
But yeah.
OK.
Uh So next question,
just trying to wrap up the list here so we can wrap up this office hours.
Uh
uh
Zari uh
Tjin asks uh for those coming in with naive eyes and understanding how sla
slash graphics work do you have recommended resources to look
at in particular of the advent of custom shader and re
slash proto flex specifically?
Um
So that's always a really fun one.
Having worked on shader for pretty much my entire professional uh
career.
Um
There's definitely a lot of things that you learn along the way. Um
Like there's a lot of communal knowledge that
you just build up through experience there.
Um One of these days I'll probably just writing a guide
that's just gonna echo and compile a lot of the stuff.
For example, texture samplers
are slow, extremely slow.
Um You can oftentimes end up finding yourself in a
position where you're waiting on a texture sampler to finish
more for,
for a longer period of time than just going
through and calculating something on the GP U.
Um You may find that certain instructions are way slower
uh than just going through and using a texture sampler.
Uh For example, like Trigon functions and stuff like that.
Um
So like that's one of those things that I think it's
probably better if I if we provide some guidance on.
Uh So you don't have to like go
searching the internet, finding a bunch of highly technical
rendering posts and then trying to decipher what parts
you should actually be caring about from those.
Uh But
we'll get there.
Um Just generally getting started with shader programming though and just
playing around the side of unity or hell even unreal,
probably more so unreal is probably not a bad
call if you're looking to prepare yourself though.
Uh Prime G Os
is CTA a still a thing, it's still in there. Um It's not really used, there's a
bit of ghosting that occurs with it though. So,
you know, use at your own risk, I guess. Uh I think it's dash CTA A
something like that.
But yeah,
and Pix asks with sauce will shortcut slash binds be
similar to unity such as windows slash full screen toggle.
Uh So that's not really a sauce question. Uh That's really a resonate question.
Um If you are
seeking better
support for building in desktop,
uh we can talk about that.
Um However, I would advise you to once the github uh once the public github
issue trackers up to submit feature requests there
that we can then evaluate uh for example,
having more unity like uh keyboard bindings and shortcuts and stuff like that.
All right.
Uh So that's it for questions for today. Um We're doing this once every two weeks.
So our next one will be, let's see
on October 25th.
Um
Just kind of going up through any stragglers or anything like that.
Um The recording of this will hopefully be up pretty quick.
Uh We just need somewhere to drop it.
Um I'll probably coordinate with prime to figure out
like how to get that out to you guys.
Um But yeah,
thank you very much everyone for coming.
Uh see you again in two weeks and
always feel free to ask me questions and the rendering uh
in the rendering thread and the rendering discussion thread in the journal forum.
I'll copy and paste a link
into the questions thread.
There you go.
Um And if you ever see me in world, uh you know, feel free to, you know, chat me up.
I'm always uh down to have a conversation.
Um But yeah, thanks everyone.
See you around.
Yeah.