The DirectX Shader Compiler Summer 2023 release is live! The most significant change in this release is the enabling of the HLSL 2021 language version by default. HLSL 2021 includes templates, operator overloads, bitfields, short-circuiting, and modernizations to loop variable scope and implicit struct casting. You can read about the details of these features in Chris Bieneman’s dog-themed blog post.
This post documents the gotchas that you might encounter when compiling your existing shaders with HLSL 2021 to benefit from all this goodness. Specifically, it will focus on:
- Changes to implicit casting that affect assignments and how overloaded functions are matched to calls.
- Changes to binary logic operators disallowing vectors and introducing short-circuiting to scalars.
- Changes to the scope of loop variables that will prevent using them outside the loop.
To explain these changes and how they might create specific scenarios that impact your shaders, we have provided:
- Example code to illustrate what kind of pattern might have caused the problem.
- Background about the change to the language in HLSL 2021 that caused them.
- Quick fixes to get your shader compiling and running as it did before (for better or worse).
- Potential lurking bugs that these failures might indicate which might make you reconsider the quick fix.
- Ideal fixes if you want to spend the time.
Each scenario is meant to be self-contained so you can select the ones that apply to you and get all the information you need. However, you can feel free to read through the whole document as well and learn all there is to know about migrating to HLSL 2021.
My shaders compile successfully! Hooray!
Hang on! Double check your loop variable scoping and usage.
Congrats! Nevertheless, there is something you should try to ensure you don’t see any subtle behavior differences. You might have problems with using a loop variable that shadows another and uses it after the loop like this code does:
bool allPositive(float arr[3]) { uint i = 3; for (uint i = 0; i < 3; i++) { if (arr[i]< 0.0) break; } if (i >= 3) // Wait a minute. . . which i is that? return false; return true; }
The background
HLSL 2021 limits the scope of loop variables to the body of the loop itself. Previously, it conformed to an older language standard which extended the scope of such variables to the scope the loop was contained in. Because of the new limited scope, any usage of the variables outside the loop will refer only to variables in the outer scope, which is a change in behavior.
The ideal fix
In previous versions of HLSL, loop variables that shadowed outer scope variables produced a warning: “redefinition of ‘i’ shadows declaration in the outer scope; most recent declaration will be used”. It may be worth it to enable HLSL 2018 with the -HV 2018 flag and carefully examine such warnings to ensure that the change in scope won’t change the shader result. Alternately, you might enable all shadow warnings with -Wshadow and examine them all. There may be even more, but they may indicate other lurking bugs.
My struct assignment is failing!
error: cannot implicitly convert from <struct1> to <struct2>
You are trying to assign a variable of a different type to another variable with identical internal layout like this code does:
struct color { float r, g, b, a; }; struct vec4 {float x, y, z, w; }; color blue = { 0.0, 0.0, 1.0, 1.0 }; // my favorite color! vec4 vtxinfo[2]; // color and position info in one! vtxinfo[0] = blue; // error: cannot implicitly convert from 'color' to 'vec4' doh!
The background
You are running into a change in implicit casting. Previously, structs were considered to match for the purposes of implicit casts if their members had the same types in the same places.
In HLSL 2018, this assignment would have copied the internal values of the blue color value into the generic vec4 struct. In HLSL 2021, the names of the struct are also required to match similar to C++ and other languages.
The quick fix
You can still cast one compatible struct to another. Any structs that run into this error are definitely compatible, so you can just add a cast to the failing assignment:
vtxinfo[0] = (vec4)blue; // No errors!
The potential lurking bug
The situation above showed where this was clearly intended. There may be cases where this shows a mistaken assignment of two values that aren’t meant to have the same types. Examining the code carefully with the knowledge of its context might be necessary to ensure that this assignment is really what you want. Different types with the same internal representations are meant to prevent mistakes. HLSL 2021 gives the ability to identify these mistakes.
The (possibly) ideal fix
In the situation above, you might choose to replace the array of vec4s with a struct containing both color and position information. If you were to actually write this shader, you would probably want to use float4s. The structs containing only a list of floats were used to make a convenient example.
My function call can’t find its implementation!
No matching function for call to <myfunction>
You have a function call that passes a different, but internally identical struct to the defined function like this code does:
struct LinearRGB {float3 RGB;}; struct LinearYCoCg {float3 YCoCg;}; void InitColor(inout LinearRGB V) { V.RGB= 0.0; } ... LinearYCoCg V; InitColor(V); // error: no matching function for call to 'InitColor'
The background
It might not be initially obvious, but you are running into a change in implicit casting. Previously, structs were considered to match for the purposes of implicit casts if their members had the same types in the same places, so functions that took structs as parameters could be called with structs that had identical internals.
In HLSL 2018, the call to InitColor would have identified V as having the same internal representation as the parameter to the defined function and included an implicit cast and then used that implementation for the call. However, HLSL 2021 requires the name of the struct casts to match as well much as C++ and other languages do.
The quick fix
You can still cast one compatible struct to another. Any structs that run into this error are definitely compatible, so you can just add a cast to the parameter of the failing function call:
InitColor((LinearRGB)V); // no errors!
The potential lurking bug
The code above represents a scenario where the call does an adequate job of initializing both RGB and YCoCg values. However, this quirk of HLSL allowed for some erroneous code to be accepted and run fine. If instead of initializing to 0.0, the code tried to initialize it to blue, the correct YCoCg initialization would have to be very different from the RGB initialization. What’s worse, these overloads weren’t possible for structs with identical internals because the compiler didn’t see any difference between them. It’s worth taking a minute to consider if these calls are intended.
The ideal fix
While it’s not unusual that the same operation would be needed for variables of different types, there is a preferred way of doing that is more useful than relying on structs that happen to have the same internals and it’s templates. Fortunately, templates are included in HLSL 2021! What’s more, operator overloads expand the utility of templates even more.
struct LinearRGB { float3 RGB; void assign(float3 f) { RGB = f; } }; struct LinearYCoCg { float3 YCoCg; void assign(float3 f) { YCoCg = f; } }; template<class T> void InitColor(inout T V) { V.assign(0.0); } ... LinearYCoCg V; InitColor(V); // no errors!
My logic operator fails!
error: operands for short-circuiting logical binary operator must be scalar
error: condition for short-circuiting logical ternary operator must be scalar
You have a binary or ternary logic operator that has vectors as operands like this code does
float4 main(int3 i3: I3, int3 j3: J3): SV_Target { bool3 b3 = i3 || j3; // error: operands for short-circuiting logical binary operator must be scalar int4 res = b3?i3:j3; // error: condition for short-circuiting ternary operator must be scalar
The background
In order to enable short-circuiting for logical operations involving scalars, vectors had to be disallowed from using those same operations. Uniform short-circuiting for a group of values contained in a vector or matrix isn’t possible as different values within might have different results. This is why, prior to HLSL 2021, short-circuiting was not supported at all. In order to enable it, non-scalars had to be excluded.
The quick fix
HLSL 2021 adds new built-in functions that maintain the non-short-circuiting behavior of previous language versions. These are and(c1,c2), or(c1,c2) and select(c,v1,v2) which replace &&, || and ?: respectively. Substituting these built-in functions for the operators for non-scalars will silence the errors and maintain previous behavior:
float4 main(int3 i3: I3, int3 j3: J3): SV_Target { bool3 b3 = or(i3, j3); // no errors! int i4 = select(b3,i3,j3); // no errors!
Note that although they are intended for use with non-scalars, and(), or() and select() can be used with scalar values to perform non-sort-circuiting logical evaluations as well.
The potentially lurking bug
With the exception of the conditional argument for the ?: ternary operator, boolean vectors can’t be directly used by the usual conditional control flow mechanisms in HLSL. As such, it may be the case that the vectors were being used as operands to these logic ops in error. If applied directly to conditional statements, they would be rejected, but if assigned to a scalar boolean value, the boolean vector would be implicitly truncated with a warning that might have been ignored. It’s unlikely that calculation that throws out the rest of the values was intended. It may be worth examining these errors to ensure that the vectors were intended to be applied to these logic operators.
The ideal fix
In cases where non-scalars were intended to be evaluated for logic calculations, it may be that the intent was to:
- evaluate something about a specific aggregate element
- evaluate something about any of the aggregate’s elements
- evaluate something about all of the aggregate’s elements
In these cases, rather than preserve the possibly unintended behavior, it would be better to respectively:
- Reference the specific element: i3.x || j3.x
- Use the any() built-in function: any(i3) || any(j3)
- Use the all() built-in function: all(i3) || all(j3)
My loop variable is undeclared!
error: use of undeclared identifier ‘i’
You have a variable declared in the loop construct that is used after the end of the loop scope like this code does
for (uint i = 0; i < ARR_SIZE; i++) { if (arr[i] > 0.0) break; } if (i >= ARR_SIZE) return false;
The background
HLSL 2021 limits the scope of loop variables to the body of the loop itself. Previously, it conformed to an older language standard which extended the scope of such variables to the scope the loop was contained in. Because of the new limited scope, any usage of the variables outside the loop will fail.
The quick fix
By declaring the variable just before the loop, it will be in the same scope as it was previously and accessible after end of the loop scope:
uint i; for (i = 0; i < ARR_SIZE; i++) { if (arr[i] > 0.0) break; } if (i >= ARR_SIZE) return false;
The potentially lurking bug
In the event that the loop variable shadows a variable in the outer scope and is used after the loop scope, the results of the computation might change. In previous versions of HLSL, such variables produced a warning: “redefinition of ‘i’ shadows declaration in the outer scope; most recent declaration will be used”. It may be worth it to enable HLSL 2018 with the -HV 2018 flag and carefully examine such warnings to ensure that the change in scope won’t change the shader result.
The ideal fix
Barring shader-specific details, in this case, the quick fix is also the ideal fix!
I have a problem that isn’t listed here!
Sorry about that! Please file an issue in our GitHub repo using the appropriate template. If you include the prefix [HLSL2021] it will help us identify it faster.
You can also reach out to us at askhlsl@microsoft.com
Conclusion
Thanks for reading! We hope your migration to HLSL 2021 is smooth and it enables you to build great things!
The post HLSL 2021 Migration Guide appeared first on DirectX Developer Blog.