The HLSL team is excited to announce DXC 1.8.2405 which is a packed release! In addition to a healthy assortment of bug fixes and quality of life improvements, this release features two things we’re really excited about. First this is the first DXC release to contain Windows binaries built with Clang alongside the binaries built with MSVC, and it includes the first feature planned for the next HLSL language version, which is ready for user testing. We will dig into a lot more details later in this post, but two quick TL;DRs are:
- Using the Clang to build DXC binaries showed a modest performance improvement for compiling shaders. These binaries should work as drop-in replacements instead of the MSVC-built ones and might give you a cheap win to speed up compile times.
- To prepare your HLSL codebase for the new language features you can add these arguments to your shader compiler options -HV 202x -Wconversion -Wdouble-promotion -Whlsl-legacy-literal. These options will enable the new language behavior and emit warnings for type conversions which could indicate important changes in behavior or precision.
As with all HLSL language versions, once our new language version is feature complete, stabilized, and has a proper name, we will make this the default language version in DXC (it already is for Clang). We don’t have a timeline for that yet, but likely sometime in 2025 or 2026.
Note for our Xbox developers: the new language feature will be coming soon to a GDK near you, so keep reading.
Clang-built DXC
The DXC 1.8.2405 release is the first release that we’re shipping Windows binaries built with Clang alongside the existing binaries built with MSVC. We’ve included these binaries in addition to the MSVC binaries so that users can try them with minimal risk and revert back to the MSVC binaries if they encounter problems specific to the Clang-built binaries (although we don’t anticipate any).
You might be asking yourself: why should you care what compiler we built DXC with? Or what even is this Clang thing?
Clang is the LLVM compiler for C-like languages. DXC is a fork of an older version of LLVM & Clang.
This matters because different compilers optimize different patterns in code more effectively. LLVM’s optimizer has always been pretty good at optimizing LLVM’s codebase, which seems both silly to say and obvious once you think about it. For this reason, LLVM’s optimizer is pretty good at optimizing DXC’s code, which inherits most of its code and design from LLVM & Clang.
We spent some time this year looking for easy wins to speed up compile time for users of DXC and building our binaries with Clang seemed like a good candidate.
In practice we tested compile time against a proprietary suite of real-world shaders and saw promising results. Comparing compile times with Clang-built DXC against MSVC-built DXC we saw on average around 10% reduction in compile time with the Clang-built DXC, and as much as 20-30% in some cases.
This probably makes you wonder why we’re still shipping the MSVC-built binaries. In 88% of our tests Clang-built DXC compiled shaders as quickly or faster than MSVC-built DXC (with some small margin of test error). In about 4.5% of cases the Clang-built binaries were significantly slower than the DXC-built ones (10-25%). In the remaining 7.5% the slowdown was less than 10%, but still measurable.
As we continue to look for ways to speed up compile time for shaders we’re really interested for your feedback. Particularly we’d like to know if the Clang-built DXC helps or hurts your shader compile time, and if you encounter any unexpected difficulties in using the Clang-built binaries. Please try the Clang-built DXC binaries in your project and let us know how it goes!
Evolving HLSL
As many of you are probably aware, DXC is based on LLVM & Clang from version 3.7, which is getting older by the second. Because of how HLSL & DXIL support were introduced to the forked LLVM code we don’t have a reasonable path to update to a newer version of LLVM, and we’re stuck with an aging and degrading codebase. To address this, we’ve been working on a new initiative to implement HLSL support in Clang.
One of the challenges we’re facing with this effort is that HLSL has no formal specification (yet!). For programming languages, a formal specification (like the ISO C and C++ specifications) details the language grammar, and semantic rules that describe program behavior. Not having a specification for HLSL makes it extremely challenging to implement a new HLSL compiler with the same behavior as the existing one. Any developer who has been writing HLSL for a while may recognize the problems porting code between FXC and DXC. Because of the continued lack of formal specification, a similar set of problems may occur with the transition from DXC to Clang.
To help limit this impact we’re doing two things:
- We’re writing a language specification for HLSL.
- We’re defining HLSL 202x as a common language mode between DXC and Clang
HLSL 202x is the working name for the next iteration of the HLSL programming language. HLSL 202x is planned as a narrowly scoped set of language changes to focus on making HLSL more consistent and regular. Our goal with HLSL 202x is to bridge the gap between DXC and Clang’s HLSL support.
We still don’t expect that DXC and Clang will match 100% even under HLSL 202x, but we’re trying to catalog differences and keep them to a minimum. Even beyond the expected differences, compilers are software made by mere mortals, so they have bugs too. Clang is no exception (even if we wish it would be). By allowing users to phase the changes into their codebases we hope to minimize the disruptions this transition will cause.
Clang will support only HLSL 202x initially and will have full support for 202x and later versions of HLSL. Some earlier versions of HLSL may be implemented in the future if there is user demand and resourcing allow, however some HLSL features from DXC and FXC we never plan to implement or match in Clang, so earlier language versions will always behave differently in Clang from DXC.
We’re planning and developing HLSL 202x and its successor HLSL 202y in parallel. We don’t yet have anything to share about HLSL 202y, but we’re planning it as a feature packed release to bring a bunch of powerful new language features to HLSL only in Clang. This may include features like constant expressions (C++ constexpr), variable declaration type inference (C++ auto), variadic templates, lambdas, and more!
Conforming Literals
The DXC 1.8.2405 release includes the new HLSL 202x Conforming Literals feature. You can enable HLSL 202x in your codebase by adding the -HV 202x flag to your compiler arguments.
The technical proposal for the feature is available in the hlsl-specs repository on GitHub. That document contains a lot of information including links to the PRs with the precise wording for the draft language specification.
Literal Types
If it ain’t broke…
All this sounds cool right? So why are we changing it?
There are a couple reasons why we are changing this behavior. First is that it isn’t strictly compatible with C++ templates (which we added in HLSL 2021), nor is it compatible with other C++ features that infer types (like auto). For example, if you have the expression auto v = 1.0;, what is the type of v? With literal types we actually don’t know.
This incompatibility with type deduction has been causing subtle bugs in shaders forever, but they’re exacerbated by the introduction of templates to HLSL 2021 (see related issues #3973, #5493, & #6147).
The second big motivation for this change is that the existing behavior is really complicated and not well understood, specified, or documented. That means that we’re unlikely to be able to make Clang match DXC’s behavior even if we tried. Standardizing on a simpler behavior meets our goal of reducing hurdles for users transitioning code between DXC and Clang. If you’re still not fully convinced, the literal behavior has been tangentially related to a host of other issues in DXC. Here are a few more related issues: #4683, #5961, #6410, #6565, and #6566.
Because of all these issues and more, we decided to do something different, simpler, and more like C/C++ and other shading languages.
HLSL 202x Literals
In HLSL 202x, un-suffixed integer literals behave exactly like C/C++. Base-10 un-suffixed literals are int unless the value is too large to be represented in which case they are int64_t. Un-suffixed hexadecimal and octal literals are signed unless their most significant bit is set in which case they are unsigned, and they will be 32-bit if the specified value fits, otherwise they will extend to 64-bits. HLSL 202x continues to support the suffixes u, l, ll for integer literals. The suffix u marks a type as uint. The suffix l or ll mark it as int64_t, and when paired with the u suffix the type becomes uint64_t.
There is no suffix to specify integer types smaller than int in C or HLSL. When explicit sizing is desired for smaller types a cast is the recommended approach (e.g., (int16_t)4).
Un-suffixed floating-point literals are of type float. This is different from C/C++ where they would be double. This is the same behavior as other common shading languages and more closely aligns to HLSL as implemented by FXC. This divergence was chosen in part because it closely aligns with FXC and other shading languages, but also because it seemed like the right approach since double values often have significant performance impact on GPU code and require an optional feature bit in DirectX.
HLSL 202x continues to support the suffixes h, f and l for floating-point literals. The h suffix marks the type as half (which may be 16 or 32-bit based on -enable-16bit-types), the f suffix marks the type as float, and the l suffix marks the type as double. There is no suffix to specify min16float. When using a literal in an expression with min16float values you need to be mindful to use explicit casts where appropriate.
Estimating Impact: Integer Signedness
The changes in this language feature impact the fundamental types of HLSL, and how types are implicitly resolved. As a result, this change can cause behavior changes.
To estimate the impact of this change we created a modified build of DXC that enabled this feature in all HLSL language modes. We then ran that build through our proprietary internal test suite and had our partners at Google who maintain DXC’s SPIR-V support do the same. Google reported no issues in their SPIR-V test suite. We encountered a few, which I’ll discuss. Our internal test suite is comprised of captured frames from shipping games and demo content. Our test suite is large enough to be useful, but not extensive enough to be fully representative of all use cases.
Most of the shaders in our test suite had no visible difference in the final frame rendering. Many of the shaders compiled to bit-for-bit identical byte code with this change.
All the cases where significant rendering differences appeared were caused by changes in integer literal handling. Specifically, DXC always treats integer literals as signed integers, and the C/C++ rules treat hexadecimal literals as unsigned if the most significant bit is set. This produces significantly different results for bit shifts. To illustrate this difference, consider the following code:
export int Fn(int inInt) {
return (inInt & 0xffff0000) >> 16);
}Following HLSL 2021 rules, 0xffff0000 is a signed integer. This means that all operations in this code are arithmetic. The result of the & operation produces a signed integer. If the most significant bit of that integer is 1 the shift will fill the new high order bits with 1.
Following HLSL 202x rules, consistent with C/C+, 0xffff0000 is an unsigned integer. This results in inInt being promoted to an unsigned integer, the & operation produces an unsigned result, and the shift is a logical shift filling the top bits with 0 regardless of the value. This also matches the effective behavior in FXC because hexadecimal literals are unsigned. To help users identify cases in their code where this may cause issues we’ve added a new disabled-by-default warning to warn on literal values that will change signed-ness under the new language rules. You can enable this warning on your shaders by passing the -Whlsl-legacy-literal flag. This new warning works the same under HLSL 202x and earlier versions, and it notifies the user of all cases where a literal integer changes signed-ness based on the language version.
Estimating Impact: Floating-point Precision
Our team has also speculated but not observed two other situations related to floating point literals that may cause problems in your codebases.
One issue is that legacy literals in DXC compute constant evaluated math at a higher precision than under the new rules. This may cause rounding errors due to intermediate values being computed at lower precisions in HLSL 202x. To identify these cases, the -Wconversion warnings are your best tool. They report any implicit conversion that results in a loss in precision.
The other issue we believe is possible is where math intended to be 16-bit math is implicitly promoted to 32-bit math. This likely won’t cause correctness problems but could cause runtime performance regressions for shaders that rely on min16float or half data types. To identify these cases, we’ve back-ported the (poorly named) Clang warning -Wdouble-promotion which reports any implicit floating-point promotions (conversions that don’t lose precision).
These warnings will behave differently under HLSL 202x and earlier versions. In HLSL 2021 and earlier literal types are excluded from all conversion warnings (promotion or truncation) because of their unsized nature. For this reason, in HLSL 2021 and earlier these warnings will only trigger on conversions of non-literal types. For HLSL 202x, the warnings will trigger for all conversions. It may be useful to run code through DXC with the warnings enabled for HLSL 2021 then run it again with 202x and compare the output. Regardless of which language version you are building your shaders with, being warning-clean for -Wconversion and -Wdouble-promotion is recommended to avoid unintended behavior and performance pitfalls.
Call to Action
This is the point in the post where I ask (or beg) you to try this feature and our new Clang-built DXC binaries in your shader codebases and give us feedback. We’ve done a lot of work to estimate the impact that these changes will have on existing HLSL, but we aren’t omniscient so the only way we’ll know how this impacts your codebase is if you try it and tell us how it goes.
If you encounter a bug in the DXC implementation, please file an issue against DXC on GitHub.
If you have other feedback about the language change positive or negative, we’d love to hear it. We collected public feedback on this GitHub issue, and we’ll continue to track and respond to feedback even though the issue is closed.
The post DXC 1.8.2405 Available Now, Including HLSL 202x appeared first on DirectX Developer Blog.