![]() |
Welcome to ftp.nluug.nl Current directory: /NetBSD/NetBSD-release-10/xsrc/external/mit/MesaLib/dist/src/compiler/glsl/ |
|
Contents of README:Welcome to Mesa's GLSL compiler. A brief overview of how things flow: 1) lex and yacc-based preprocessor takes the incoming shader string and produces a new string containing the preprocessed shader. This takes care of things like #if, #ifdef, #define, and preprocessor macro invocations. Note that #version, #extension, and some others are passed straight through. See glcpp/* 2) lex and yacc-based parser takes the preprocessed string and generates the AST (abstract syntax tree). Almost no checking is performed in this stage. See glsl_lexer.ll and glsl_parser.yy. 3) The AST is converted to "HIR". This is the intermediate representation of the compiler. Constructors are generated, function calls are resolved to particular function signatures, and all the semantic checking is performed. See ast_*.cpp for the conversion, and ir.h for the IR structures. 4) The driver (Mesa, or main.cpp for the standalone binary) performs optimizations. These include copy propagation, dead code elimination, constant folding, and others. Generally the driver will call optimizations in a loop, as each may open up opportunities for other optimizations to do additional work. See most files called ir_*.cpp 5) linking is performed. This does checking to ensure that the outputs of the vertex shader match the inputs of the fragment shader, and assigns locations to uniforms, attributes, and varyings. See linker.cpp. 6) The driver may perform additional optimization at this point, as for example dead code elimination previously couldn't remove functions or global variable usage when we didn't know what other code would be linked in. 7) The driver performs code generation out of the IR, taking a linked shader program and producing a compiled program for each stage. See ../mesa/program/ir_to_mesa.cpp for Mesa IR code generation. FAQ: Q: What is HIR versus IR versus LIR? A: The idea behind the naming was that ast_to_hir would produce a high-level IR ("HIR"), with things like matrix operations, structure assignments, etc., present. A series of lowering passes would occur that do things like break matrix multiplication into a series of dot products/MADs, make structure assignment be a series of assignment of components, flatten if statements into conditional moves, and such, producing a low level IR ("LIR"). However, it now appears that each driver will have different requirements from a LIR. A 915-generation chipset wants all functions inlined, all loops unrolled, all ifs flattened, no variable array accesses, and matrix multiplication broken down. The Mesa IR backend for swrast would like matrices and structure assignment broken down, but it can support function calls and dynamic branching. A 965 vertex shader IR backend could potentially even handle some matrix operations without breaking them down, but the 965 fragment shader IR backend would want to break to have (almost) all operations down channel-wise and perform optimization on that. As a result, there's no single low-level IR that will make everyone happy. So that usage has fallen out of favor, and each driver will perform a series of lowering passes to take the HIR down to whatever restrictions it wants to impose before doing codegen. Q: How is the IR structured? A: The best way to get started seeing it would be to run the standalone compiler against a shader: ./glsl_compiler --dump-lir \ ~/src/piglit/tests/shaders/glsl-orangebook-ch06-bump.frag So for example one of the ir_instructions in main() contains: (assign (constant bool (1)) (var_ref litColor) (expression vec3 * (var_ref Surf aceColor) (var_ref __retval) ) ) Or more visually: (assign) / | \ (var_ref) (expression *) (constant bool 1) / / \ (litColor) (var_ref) (var_ref) / \ (SurfaceColor) (__retval) which came from: litColor = SurfaceColor * max(dot(normDelta, LightDir), 0.0); (the max call is not represented in this expression tree, as it was a function call that got inlined but not brought into this expression tree) Each of those nodes is a subclass of ir_instruction. A particular ir_instruction instance may only appear once in the whole IR tree with the exception of ir_variables, which appear once as variable declarations: (declare () vec3 normDelta) and multiple times as the targets of variable dereferences: ... (assign (constant bool (1)) (var_ref __retval) (expression float dot (var_ref normDelta) (var_ref LightDir) ) ) ... (assign (constant bool (1)) (var_ref __retval) (expression vec3 - (var_ref LightDir) (expression vec3 * (constant float (2.000000)) (expression vec3 * (expression float dot (var_ref normDelta) (var_ref LightDir) ) (var_ref normDelta) ) ) ) ) ... Each node has a type. Expressions may involve several different types: (declare (uniform ) mat4 gl_ModelViewMatrix) ((assign (constant bool (1)) (var_ref constructor_tmp) (expression vec4 * (var_ref gl_ModelViewMatrix) (var_ref gl_Vertex) ) ) An expression tree can be arbitrarily deep, and the compiler tries to keep them structured like that so that things like algebraic optimizations ((color * 1.0 == color) and ((mat1 * mat2) * vec == mat1 * (mat2 * vec))) or recognizing operation patterns for code generation (vec1 * vec2 + vec3 == mad(vec1, vec2, vec3)) are easier. This comes at the expense of additional trickery in implementing some optimizations like CSE where one must navigate an expression tree. Q: Why no SSA representation? A: Converting an IR tree to SSA form makes dead code elimination, common subexpression elimination, and many other optimizations much easier. However, in our primarily vector-based language, there's some major questions as to how it would work. Do we do SSA on the scalar or vector level? If we do it at the vector level, we're going to end up with many different versions of the variable when encountering code like: (assign (constant bool (1)) (swiz x (var_ref __retval) ) (var_ref a) ) (assign (constant bool (1)) (swiz y (var_ref __retval) ) (var_ref b) ) (assign (constant bool (1)) (swiz z (var_ref __retval) ) (var_ref c) ) If every masked update of a component relies on the previous value of the variable, then we're probably going to be quite limited in our dead code elimination wins, and recognizing common expressions may just not happen. On the other hand, if we operate channel-wise, then we'll be prone to optimizing the operation on one of the channels at the expense of making its instruction flow different from the other channels, and a vector-based GPU would end up with worse code than if we didn't optimize operations on that channel! Once again, it appears that our optimization requirements are driven significantly by the target architecture. For now, targeting the Mesa IR backend, SSA does not appear to be that important to producing excellent code, but we do expect to do some SSA-based optimizations for the 965 fragment shader backend when that is developed. Q: How should I expand instructions that take multiple backend instructions? Sometimes you'll have to do the expansion in your code generation -- see, for example, ir_to_mesa.cpp's handling of ir_unop_sqrt. However, in many cases you'll want to do a pass over the IR to convert non-native instructions to a series of native instructions. For example, for the Mesa backend we have ir_div_to_mul_rcp.cpp because Mesa IR (and many hardware backends) only have a reciprocal instruction, not a divide. Implementing non-native instructions this way gives the chance for constant folding to occur, so (a / 2.0) becomes (a * 0.5) after codegen instead of (a * (1.0 / 2.0)) Q: How shoud I handle my special hardware instructions with respect to IR? Our current theory is that if multiple targets have an instruction for some operation, then we should probably be able to represent that in the IR. Generally this is in the form of an ir_{bin,un}op expression type. For example, we initially implemented fract() using (a - floor(a)), but both 945 and 965 have instructions to give that result, and it would also simplify the implementation of mod(), so ir_unop_fract was added. The following areas need updating to add a new expression type: ir.h (new enum) ir.cpp:operator_strs (used for ir_reader) ir_constant_expression.cpp (you probably want to be able to constant fold) ir_validate.cpp (check users have the right types) You may also need to update the backends if they will see the new expr type: ../mesa/program/ir_to_mesa.cpp You can then use the new expression from builtins (if all backends would rather see it), or scan the IR and convert to use your new expression type (see ir_mod_to_floor, for example). Q: How is memory management handled in the compiler? The hierarchical memory allocator "talloc" developed for the Samba project is used, so that things like optimization passes don't have to worry about their garbage collection so much. It has a few nice features, including low performance overhead and good debugging support that's trivially available. Generally, each stage of the compile creates a talloc context and allocates its memory out of that or children of it. At the end of the stage, the pieces still live are stolen to a new context and the old one freed, or the whole context is kept for use by the next stage. For IR transformations, a temporary context is used, then at the end of all transformations, reparent_ir reparents all live nodes under the shader's IR list, and the old context full of dead nodes is freed. When developing a single IR transformation pass, this means that you want to allocate instruction nodes out of the temporary context, so if it becomes dead it doesn't live on as the child of a live node. At the moment, optimization passes aren't passed that temporary context, so they find it by calling talloc_parent() on a nearby IR node. The talloc_parent() call is expensive, so many passes will cache the result of the first talloc_parent(). Cleaning up all the optimization passes to take a context argument and not call talloc_parent() is left as an exercise. Q: What is the file naming convention in this directory? Initially, there really wasn't one. We have since adopted one: - Files that implement code lowering passes should be named lower_* (e.g., lower_builtins.cpp). - Files that implement optimization passes should be named opt_*. - Files that implement a class that is used throught the code should take the name of that class (e.g., ir_hierarchical_visitor.cpp). - Files that contain code not fitting in one of the previous categories should have a sensible name (e.g., glsl_parser.yy). |
Name Last modified Size
Parent Directory -
CVS/ 17-Dec-2022 21:58 -
glcpp/ 17-Dec-2022 21:57 -
tests/ 17-Aug-2024 05:24 -
README 09-May-2022 03:23 11K
TODO 10-Mar-2019 04:42 689
ast.h 09-May-2022 03:23 37K
ast_array_index.cpp 09-May-2022 03:23 15K
ast_expr.cpp 10-Mar-2019 04:42 2.2K
ast_function.cpp 09-May-2022 03:23 95K
ast_to_hir.cpp 09-May-2022 03:23 343K
ast_type.cpp 09-May-2022 03:23 32K
builtin_functions.cpp 09-May-2022 03:23 375K
builtin_functions.h 09-May-2022 03:23 2.5K
builtin_int64.h 10-Mar-2019 04:42 43K
builtin_types.cpp 09-May-2022 03:23 19K
builtin_variables.cpp 09-May-2022 03:23 64K
float64.glsl 09-May-2022 03:23 55K
generate_ir.cpp 10-Mar-2019 04:42 1.3K
gl_nir.h 09-May-2022 03:23 1.9K
gl_nir_link_atomics.c 09-May-2022 03:23 13K
gl_nir_link_uniform_blocks.c 09-May-2022 03:23 23K
gl_nir_link_uniform_initializers.c 09-May-2022 03:23 11K
gl_nir_link_uniforms.c 09-May-2022 03:23 69K
gl_nir_link_xfb.c 09-May-2022 03:23 7.3K
gl_nir_linker.c 09-May-2022 03:23 26K
gl_nir_linker.h 09-May-2022 03:23 2.9K
gl_nir_lower_atomics.c 24-Sep-2019 18:39 5.7K
gl_nir_lower_buffers.c 09-May-2022 03:23 14K
gl_nir_lower_images.c 09-May-2022 03:23 4.3K
gl_nir_lower_samplers.c 09-May-2022 03:23 1.7K
gl_nir_lower_samplers_as_deref.c 09-May-2022 03:23 13K
glsl_lexer.ll 09-May-2022 03:23 42K
glsl_parser.yy 09-May-2022 03:23 95K
glsl_parser_extras.cpp 09-May-2022 03:27 80K
glsl_parser_extras.h 09-May-2022 03:23 34K
glsl_symbol_table.cpp 10-Mar-2019 04:42 9.0K
glsl_symbol_table.h 10-Mar-2019 04:42 3.6K
glsl_to_nir.cpp 09-May-2022 03:23 89K
glsl_to_nir.h 24-Sep-2019 18:40 1.8K
hir_field_selection.cpp 24-Sep-2019 18:40 3.1K
int64.glsl 24-Sep-2019 18:40 2.6K
ir.cpp 09-May-2022 03:23 67K
ir.h 09-May-2022 03:23 73K
ir_array_refcount.cpp 09-May-2022 03:23 6.0K
ir_array_refcount.h 09-May-2022 03:23 3.6K
ir_basic_block.cpp 10-Mar-2019 04:42 3.3K
ir_basic_block.h 10-Mar-2019 04:42 1.4K
ir_builder.cpp 10-Mar-2019 04:42 11K
ir_builder.h 10-Mar-2019 04:42 7.1K
ir_builder_print_visitor.cpp 09-May-2022 03:23 23K
ir_builder_print_visitor.h 10-Mar-2019 04:42 1.3K
ir_clone.cpp 09-May-2022 03:23 13K
ir_constant_expression.cpp 09-May-2022 03:23 35K
ir_equals.cpp 10-Mar-2019 04:42 5.3K
ir_expression_flattening.cpp 10-Mar-2019 04:42 2.6K
ir_expression_flattening.h 10-Mar-2019 04:42 1.8K
ir_expression_operation.py 09-May-2022 03:23 43K
ir_function.cpp 09-May-2022 03:23 14K
ir_function_can_inline.cpp 10-Mar-2019 04:42 2.4K
ir_function_detect_recursion.cpp 24-Sep-2019 18:41 11K
ir_function_inlining.h 10-Mar-2019 04:42 1.4K
ir_hierarchical_visitor.cpp 09-May-2022 03:23 9.0K
ir_hierarchical_visitor.h 09-May-2022 03:23 9.1K
ir_hv_accept.cpp 09-May-2022 03:23 12K
ir_optimization.h 09-May-2022 03:23 8.6K
ir_print_visitor.cpp 09-May-2022 03:23 16K
ir_print_visitor.h 09-May-2022 03:23 3.1K
ir_reader.cpp 01-Jun-2019 09:41 34K
ir_reader.h 10-Mar-2019 04:42 1.4K
ir_rvalue_visitor.cpp 10-Mar-2019 04:42 6.8K
ir_rvalue_visitor.h 10-Mar-2019 04:42 3.8K
ir_set_program_inouts.cpp 09-May-2022 03:23 15K
ir_uniform.h 10-Mar-2019 04:42 6.2K
ir_validate.cpp 09-May-2022 03:23 38K
ir_variable_refcount.cpp 24-Sep-2019 18:41 4.5K
ir_variable_refcount.h 10-Mar-2019 04:42 2.9K
ir_visitor.h 09-May-2022 03:23 3.6K
link_atomics.cpp 10-Mar-2019 04:42 12K
link_functions.cpp 24-Sep-2019 18:41 12K
link_interface_blocks.cpp 09-May-2022 03:23 20K
link_uniform_block_active_visitor.cpp 09-May-2022 03:23 10K
link_uniform_block_active_visitor.h 09-May-2022 03:23 2.7K
link_uniform_blocks.cpp 09-May-2022 03:23 21K
link_uniform_initializers.cpp 09-May-2022 03:23 11K
link_uniforms.cpp 09-May-2022 03:23 63K
link_varyings.cpp 09-May-2022 03:23 126K
link_varyings.h 09-May-2022 03:23 8.3K
linker.cpp 09-May-2022 03:23 181K
linker.h 09-May-2022 03:23 8.6K
linker_util.cpp 09-May-2022 03:23 14K
linker_util.h 09-May-2022 03:23 3.7K
list.h 09-May-2022 03:23 22K
loop_analysis.cpp 09-May-2022 03:23 24K
loop_analysis.h 09-May-2022 03:23 6.4K
loop_unroll.cpp 09-May-2022 03:23 19K
lower_blend_equation_advanced.cpp 09-May-2022 03:23 18K
lower_buffer_access.cpp 09-May-2022 03:23 17K
lower_buffer_access.h 10-Mar-2019 04:42 2.7K
lower_builtins.cpp 09-May-2022 03:23 1.9K
lower_const_arrays_to_uniforms.cpp 09-May-2022 03:23 4.7K
lower_cs_derived.cpp 09-May-2022 03:23 7.5K
lower_discard.cpp 10-Mar-2019 04:42 4.7K
lower_discard_flow.cpp 10-Mar-2019 04:42 4.6K
lower_distance.cpp 10-Mar-2019 04:42 24K
lower_if_to_cond_assign.cpp 09-May-2022 03:23 11K
lower_instructions.cpp 09-May-2022 03:23 67K
lower_int64.cpp 09-May-2022 03:23 12K
lower_jumps.cpp 09-May-2022 03:23 39K
lower_mat_op_to_vec.cpp 09-May-2022 03:23 12K
lower_named_interface_blocks.cpp 09-May-2022 03:23 11K
lower_offset_array.cpp 10-Mar-2019 04:42 2.7K
lower_output_reads.cpp 09-May-2022 03:23 6.0K
lower_packed_varyings.cpp 09-May-2022 03:23 37K
lower_packing_builtins.cpp 10-Mar-2019 04:42 46K
lower_precision.cpp 09-May-2022 03:23 44K
lower_shared_reference.cpp 09-May-2022 03:23 17K
lower_subroutine.cpp 10-Mar-2019 04:42 3.7K
lower_tess_level.cpp 10-Mar-2019 04:42 16K
lower_ubo_reference.cpp 09-May-2022 03:23 38K
lower_variable_index_to_cond_assign.cpp 09-May-2022 03:23 18K
lower_vec_index_to_cond_assign.cpp 10-Mar-2019 04:42 8.0K
lower_vec_index_to_swizzle.cpp 10-Mar-2019 04:42 3.3K
lower_vector.cpp 10-Mar-2019 04:42 6.1K
lower_vector_derefs.cpp 09-May-2022 03:23 7.7K
lower_vector_insert.cpp 09-May-2022 03:23 5.7K
lower_vertex_id.cpp 10-Mar-2019 04:42 4.7K
lower_xfb_varying.cpp 09-May-2022 03:23 7.2K
main.cpp 09-May-2022 03:23 3.4K
meson.build 09-May-2022 03:23 8.5K
opt_add_neg_to_sub.h 10-Mar-2019 04:42 2.0K
opt_algebraic.cpp 09-May-2022 03:23 32K
opt_array_splitting.cpp 09-May-2022 03:23 15K
opt_conditional_discard.cpp 10-Mar-2019 04:42 2.7K
opt_constant_folding.cpp 10-Mar-2019 04:42 6.1K
opt_constant_propagation.cpp 09-May-2022 03:23 15K
opt_constant_variable.cpp 09-May-2022 03:23 6.9K
opt_copy_propagation_elements.cpp 24-Sep-2019 18:43 21K
opt_dead_builtin_variables.cpp 10-Mar-2019 04:42 3.3K
opt_dead_builtin_varyings.cpp 09-May-2022 03:23 21K
opt_dead_code.cpp 09-May-2022 03:23 7.1K
opt_dead_code_local.cpp 09-May-2022 03:23 9.5K
opt_dead_functions.cpp 10-Mar-2019 04:42 3.9K
opt_flatten_nested_if_blocks.cpp 10-Mar-2019 04:42 2.7K
opt_flip_matrices.cpp 10-Mar-2019 04:42 3.9K
opt_function_inlining.cpp 24-Sep-2019 18:43 13K
opt_if_simplification.cpp 10-Mar-2019 04:42 3.7K
opt_minmax.cpp 09-May-2022 03:23 16K
opt_rebalance_tree.cpp 10-Mar-2019 04:42 9.4K
opt_redundant_jumps.cpp 10-Mar-2019 04:42 3.6K
opt_structure_splitting.cpp 24-Sep-2019 18:43 11K
opt_swizzle.cpp 10-Mar-2019 04:42 3.3K
opt_tree_grafting.cpp 10-Mar-2019 04:42 11K
opt_vectorize.cpp 10-Mar-2019 04:42 12K
program.h 09-May-2022 03:23 2.0K
propagate_invariance.cpp 09-May-2022 03:23 3.7K
s_expression.cpp 10-Mar-2019 04:42 6.0K
s_expression.h 10-Mar-2019 04:42 4.6K
serialize.cpp 09-May-2022 03:23 48K
serialize.h 10-Mar-2019 04:42 1.6K
shader_cache.cpp 09-May-2022 03:23 9.4K
shader_cache.h 10-Mar-2019 04:42 1.5K
standalone.cpp 09-May-2022 03:23 22K
standalone.h 09-May-2022 03:23 1.7K
standalone_scaffolding.cpp 09-May-2022 03:23 8.7K
standalone_scaffolding.h 24-Sep-2019 18:43 3.8K
string_to_uint_map.cpp 10-Mar-2019 04:42 1.5K
string_to_uint_map.h 09-May-2022 03:23 5.1K
test.cpp 10-Mar-2019 04:42 2.4K
test_optpass.cpp 09-May-2022 03:23 9.7K
test_optpass.h 10-Mar-2019 04:42 1.2K
NLUUG - Open Systems. Open Standards
Become a member
and get discounts on conferences and more, see the NLUUG website!