Skip to content
Snippets Groups Projects
Closed In compile_and_load C-code is generated twice
  • View options
  • In compile_and_load C-code is generated twice

  • View options
  • Closed Issue created by Stephan Seitz

    When I was reviewing my old code I saw that I placed a TODO here. generate_c is just called to generate the hash and then again for the real code (if not loaded from shared object on disk). Maybe the ast can be hashed directly instead (danger point: generate_c may have changed since last generation).

    I remember that I directly hashed the assignments for generation of torch/tensorflow code. This had the disadvantage that I had to deactivate caching when developing/changing code affecting generate_c.

    def compile_and_load(ast):
        cache_config = get_cache_config()
        # TODO: inefficient to generate_c just for hash? could reuse it
        code_hash_str = "mod_" + hashlib.sha256(generate_c(ast).encode()).hexdigest()
        code = ExtensionModuleCode(module_name=code_hash_str)
        code.add_function(ast, ast.function_name)
    
        if cache_config['object_cache'] is False:
            with TemporaryDirectory() as base_dir:
                lib_file = compile_module(code, code_hash_str, base_dir)
                result = load_kernel_from_file(code_hash_str, ast.function_name, lib_file)
        else:
            lib_file = compile_module(code, code_hash_str, base_dir=cache_config['object_cache'])
            result = load_kernel_from_file(code_hash_str, ast.function_name, lib_file)
    
        rtn = KernelWrapper(result, ast.get_parameters(), ast)
        rtn.code = code.code
        return rtn

    Linked items 0

  • Link items together to show that they're related.

    Activity

    • All activity
    • Comments only
    • History only
    • Newest first
    • Oldest first