-
Frederik Hennig authored
Composer and language Frontend: - Minimialize exposition of `SfgVar`, use `AugExpr` in all composer interfaces instead - Move `SfgVar` into `lang` module - Introduce `VarLike` and `ExprLike` protocols to the `lang` module - Treat `TypedSymbol` equivalently to SfgVar in interfaces - Disallow `sp.Symbol` in expressions - Deprecate `map_param` in favor of newly introduced `set_param`, which uses `AugExpr` - Deprecate `sfg.define` in favor of `sfg.code` - Introduce `Ref` type IR Postprocessing: - Check for type conflicts during live-variable collection Documentation: - Add docstrings to `lang.expressions` module - Add various doctest examples to docstrings - Link to pystencils via intersphinx Tests: - Fix bugs in generator script testing - Add new testscripts - Add unit tests for parts of `lang.expressions` and `ir.postprocessing` Squashed commit of the following: commit c75b939d Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 13:17:52 2024 +0200 fix badges in readme and doc homepage commit 425efea7 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 13:00:35 2024 +0200 Update CONTRIBUTING & test documentation commit 3d41d1de Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 11:51:47 2024 +0200 add coverage badge to readme. commit 3023541e Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 11:45:54 2024 +0200 Update .gitlab-ci.yml file commit fdfaf307 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 11:40:14 2024 +0200 Update .gitlab-ci.yml file commit ac133b75 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 11:38:54 2024 +0200 add `coverage` key to CI task commit 72ed2ab8 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 11:28:43 2024 +0200 keep coverage.xml artifact commit 31c65ed8 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 11:23:42 2024 +0200 change testsuite in CI to run coverage.py directly commit f6893443 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 11:10:34 2024 +0200 try to fix coverage, pt. 2 commit a7fc061b Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 11:05:17 2024 +0200 try to fix coverage commit 3cb194c5 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 11:01:11 2024 +0200 Deprecate map_param. Fix test suite commit f0d11ee2 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 10:52:39 2024 +0200 Disable non-const SymPy expressions in AugExpr formatting. Add tests. commit b2ebfb02 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 09:42:20 2024 +0200 deprecate `define` in favor of `code` commit d97191f5 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 09:31:08 2024 +0200 fix mypy; reformat all commit 3182652c Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 09:29:38 2024 +0200 Update documentation - use pystencils 2.0 intersphinx - add doc for composer builders, SfgException, config subobjects commit fbd9d9b4 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 09:29:03 2024 +0200 changes to lang and ir modules - Move `SfgVar` to `lang` - Remove builder for `init`, use nested function instead - Add various docstrings commit 6a5c6936 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 08:50:53 2024 +0200 move VarLike, ExprLike, asvar, depends to lang module and add docstrings. commit fce5897c Author: Frederik Hennig <frederik.hennig@fau.de> Date: Thu Oct 17 17:38:16 2024 +0200 remove duplicate standard import from test script commit 5c595075 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Thu Oct 17 17:29:47 2024 +0200 More frontend updates - Add `Ref` type - Allow multi-arg `init` in constructor builder - Change `CustomGenerator` to take a composer instead of a context. - Allow a class to have multiple methods with the same name. commit 7a4ff746 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Thu Oct 17 16:00:51 2024 +0200 Add CustomGenerator to docs. Fix bug in postprocessing. commit 2edd363e Author: Frederik Hennig <frederik.hennig@fau.de> Date: Thu Oct 17 15:47:13 2024 +0200 More examples for composer. Fix generator script tests ground-truth comparison. commit a662f194 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Thu Oct 17 14:56:58 2024 +0200 add doctests to testsuite commit ab113916 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Thu Oct 17 14:56:27 2024 +0200 Extend doctests commit 8f04e828 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Thu Oct 17 13:56:37 2024 +0200 move generator script tests commit af32c802 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Thu Oct 17 13:56:23 2024 +0200 Extend conflict resolution in postprocessing + add tests commit 7ff8e893 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Thu Oct 17 12:27:24 2024 +0200 fix testsuite CI task commit 3534ed16 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Thu Oct 17 10:22:15 2024 +0200 Toward cleaning up variables and expressions in the composer commit 91889646 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Wed Oct 16 22:44:22 2024 +0200 bugfixes + more AugExpr in interfaces
Frederik Hennig authoredComposer and language Frontend: - Minimialize exposition of `SfgVar`, use `AugExpr` in all composer interfaces instead - Move `SfgVar` into `lang` module - Introduce `VarLike` and `ExprLike` protocols to the `lang` module - Treat `TypedSymbol` equivalently to SfgVar in interfaces - Disallow `sp.Symbol` in expressions - Deprecate `map_param` in favor of newly introduced `set_param`, which uses `AugExpr` - Deprecate `sfg.define` in favor of `sfg.code` - Introduce `Ref` type IR Postprocessing: - Check for type conflicts during live-variable collection Documentation: - Add docstrings to `lang.expressions` module - Add various doctest examples to docstrings - Link to pystencils via intersphinx Tests: - Fix bugs in generator script testing - Add new testscripts - Add unit tests for parts of `lang.expressions` and `ir.postprocessing` Squashed commit of the following: commit c75b939d Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 13:17:52 2024 +0200 fix badges in readme and doc homepage commit 425efea7 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 13:00:35 2024 +0200 Update CONTRIBUTING & test documentation commit 3d41d1de Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 11:51:47 2024 +0200 add coverage badge to readme. commit 3023541e Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 11:45:54 2024 +0200 Update .gitlab-ci.yml file commit fdfaf307 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 11:40:14 2024 +0200 Update .gitlab-ci.yml file commit ac133b75 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 11:38:54 2024 +0200 add `coverage` key to CI task commit 72ed2ab8 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 11:28:43 2024 +0200 keep coverage.xml artifact commit 31c65ed8 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 11:23:42 2024 +0200 change testsuite in CI to run coverage.py directly commit f6893443 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 11:10:34 2024 +0200 try to fix coverage, pt. 2 commit a7fc061b Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 11:05:17 2024 +0200 try to fix coverage commit 3cb194c5 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 11:01:11 2024 +0200 Deprecate map_param. Fix test suite commit f0d11ee2 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 10:52:39 2024 +0200 Disable non-const SymPy expressions in AugExpr formatting. Add tests. commit b2ebfb02 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 09:42:20 2024 +0200 deprecate `define` in favor of `code` commit d97191f5 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 09:31:08 2024 +0200 fix mypy; reformat all commit 3182652c Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 09:29:38 2024 +0200 Update documentation - use pystencils 2.0 intersphinx - add doc for composer builders, SfgException, config subobjects commit fbd9d9b4 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 09:29:03 2024 +0200 changes to lang and ir modules - Move `SfgVar` to `lang` - Remove builder for `init`, use nested function instead - Add various docstrings commit 6a5c6936 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Fri Oct 18 08:50:53 2024 +0200 move VarLike, ExprLike, asvar, depends to lang module and add docstrings. commit fce5897c Author: Frederik Hennig <frederik.hennig@fau.de> Date: Thu Oct 17 17:38:16 2024 +0200 remove duplicate standard import from test script commit 5c595075 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Thu Oct 17 17:29:47 2024 +0200 More frontend updates - Add `Ref` type - Allow multi-arg `init` in constructor builder - Change `CustomGenerator` to take a composer instead of a context. - Allow a class to have multiple methods with the same name. commit 7a4ff746 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Thu Oct 17 16:00:51 2024 +0200 Add CustomGenerator to docs. Fix bug in postprocessing. commit 2edd363e Author: Frederik Hennig <frederik.hennig@fau.de> Date: Thu Oct 17 15:47:13 2024 +0200 More examples for composer. Fix generator script tests ground-truth comparison. commit a662f194 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Thu Oct 17 14:56:58 2024 +0200 add doctests to testsuite commit ab113916 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Thu Oct 17 14:56:27 2024 +0200 Extend doctests commit 8f04e828 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Thu Oct 17 13:56:37 2024 +0200 move generator script tests commit af32c802 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Thu Oct 17 13:56:23 2024 +0200 Extend conflict resolution in postprocessing + add tests commit 7ff8e893 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Thu Oct 17 12:27:24 2024 +0200 fix testsuite CI task commit 3534ed16 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Thu Oct 17 10:22:15 2024 +0200 Toward cleaning up variables and expressions in the composer commit 91889646 Author: Frederik Hennig <frederik.hennig@fau.de> Date: Wed Oct 16 22:44:22 2024 +0200 bugfixes + more AugExpr in interfaces
configuration.py 12.14 KiB
# mypy: strict_optional=False
from __future__ import annotations
from typing import Sequence, Any
from os import path
from enum import Enum, auto
from dataclasses import dataclass, replace, fields, InitVar
from argparse import ArgumentParser
from textwrap import indent
from importlib import util as iutil
from .exceptions import SfgException
class SfgConfigSource(Enum):
DEFAULT = auto()
PROJECT = auto()
COMMANDLINE = auto()
SCRIPT = auto()
class SfgConfigException(Exception):
def __init__(self, cfg_src: SfgConfigSource | None, message: str):
super().__init__(cfg_src, message)
self.message = message
self.config_source = cfg_src
@dataclass
class SfgCodeStyle:
indent_width: int = 2
code_style: str = "file"
"""Code style to be used by clang-format. Passed verbatim to `--style` argument of the clang-format CLI.
Similar to clang-format itself, the default value is `file`, such that a `.clang-format` file found in the build
tree will automatically be used.
"""
force_clang_format: bool = False
"""If set to True, abort code generation if ``clang-format`` binary cannot be found."""
skip_clang_format: bool = False
"""If set to True, skip formatting using ``clang-format``."""
clang_format_binary: str = "clang-format"
"""Path to the clang-format executable"""
def indent(self, s: str):
prefix = " " * self.indent_width
return indent(s, prefix)
class SfgOutputMode(Enum):
STANDALONE = auto()
"""Generate a header/implementation file pair (e.g. ``.hpp/.cpp``) where the implementation file will
be compiled to a standalone object."""
INLINE = auto()
"""Generate a header/inline implementation file pair (e.g. ``.hpp/.ipp``) where all implementations
are inlined by including the implementation file at the end of the header file."""
HEADER_ONLY = auto()
"""Generate only a header file.
At the moment, header-only mode does not support generation of kernels and requires that all functions
and methods are marked `inline`.
"""
HEADER_FILE_EXTENSIONS = {"h", "hpp", "cuh"}
IMPL_FILE_EXTENSIONS: dict[SfgOutputMode, set[str]] = {
SfgOutputMode.STANDALONE: {"c", "cpp", "cu"},
SfgOutputMode.INLINE: {".impl.h", "ipp"},
SfgOutputMode.HEADER_ONLY: set(),
}
@dataclass
class SfgOutputSpec:
"""Name and path specification for files output by the code generator.
Filenames are constructed as `<output_directory>/<basename>.<extension>`."""
output_directory: str
"""Directory to which the generated files should be written."""
basename: str
"""Base name for output files."""
header_extension: str
"""File extension for generated header file."""
impl_extension: str
"""File extension for generated implementation file."""
def get_header_filename(self):
return f"{self.basename}.{self.header_extension}"
def get_impl_filename(self):
return f"{self.basename}.{self.impl_extension}"
def get_header_filepath(self):
return path.join(self.output_directory, self.get_header_filename())
def get_impl_filepath(self):
return path.join(self.output_directory, self.get_impl_filename())
@dataclass
class SfgConfiguration:
"""
Configuration for the `SfgSourceFileGenerator`.
The source file generator draws configuration from a total of four sources:
- The default configuration (`pystencilssfg.configuration.DEFAULT_CONFIG`);
- The project configuration;
- Command-line arguments;
- The user configuration passed to the constructor of `SourceFileGenerator`.
They take precedence in the following way:
- Project configuration overrides the default configuration
- Command line arguments override the project configuration
- User configuration overrides default and project configuration,
and must not conflict with command-line arguments; otherwise, an error is thrown.
**Project Configuration via Configurator Script**
Currently, the only way to define the project configuration is via a configuration module.
A configurator module is a Python file defining the following function at the top-level:
.. code-block:: Python
from pystencilssfg import SfgConfiguration
def sfg_config() -> SfgConfiguration:
# ...
return SfgConfiguration(
# ...
)
The configuration module is passed to the code generation script via the command-line argument
`--sfg-config-module`.
"""
config_source: InitVar[SfgConfigSource | None] = None
header_extension: str | None = None
"""File extension for generated header file."""
impl_extension: str | None = None
"""File extension for generated implementation file."""
output_mode: SfgOutputMode | None = None
"""The generator's output mode; defines which files to generate, and the set of legal file extensions."""
outer_namespace: str | None = None
"""The outermost namespace in the generated file. May be a valid C++ nested namespace qualifier
(like ``a::b::c``) or `None` if no outer namespace should be generated."""
codestyle: SfgCodeStyle | None = None
"""Code style that should be used by the code generator."""
output_directory: str | None = None
"""Directory to which the generated files should be written."""
project_info: Any = None
"""Object for managing project-specific information. To be set by the configurator script."""
def __post_init__(self, cfg_src: SfgConfigSource | None = None):
if self.header_extension and self.header_extension[0] == ".":
self.header_extension = self.header_extension[1:]
if self.impl_extension and self.impl_extension[0] == ".":
self.impl_extension = self.impl_extension[1:]
def override(self, other: SfgConfiguration):
other_dict: dict[str, Any] = {
k: v for k, v in _shallow_dict(other).items() if v is not None
}
return replace(self, **other_dict)
def get_output_spec(self, basename: str) -> SfgOutputSpec:
assert self.header_extension is not None
assert self.impl_extension is not None
assert self.output_directory is not None
return SfgOutputSpec(
self.output_directory, basename, self.header_extension, self.impl_extension
)
DEFAULT_CONFIG = SfgConfiguration(
config_source=SfgConfigSource.DEFAULT,
header_extension="h",
impl_extension="cpp",
output_mode=SfgOutputMode.STANDALONE,
outer_namespace=None,
codestyle=SfgCodeStyle(),
output_directory=".",
)
"""Default configuration for the `SourceFileGenerator`."""
def run_configurator(configurator_script: str):
cfg_modulename = path.splitext(path.split(configurator_script)[1])[0]
cfg_spec = iutil.spec_from_file_location(cfg_modulename, configurator_script)
if cfg_spec is None:
raise SfgConfigException(
SfgConfigSource.PROJECT,
f"Unable to load configurator script {configurator_script}",
)
configurator = iutil.module_from_spec(cfg_spec)
cfg_spec.loader.exec_module(configurator)
if not hasattr(configurator, "sfg_config"):
raise SfgConfigException(
SfgConfigSource.PROJECT,
"Project configurator does not define function `sfg_config`.",
)
project_config = configurator.sfg_config()
if not isinstance(project_config, SfgConfiguration):
raise SfgConfigException(
SfgConfigSource.PROJECT,
"sfg_config did not return a SfgConfiguration object.",
)
return project_config
def add_config_args_to_parser(parser: ArgumentParser):
config_group = parser.add_argument_group("Configuration")
config_group.add_argument(
"--sfg-output-dir", type=str, default=None, dest="output_directory"
)
config_group.add_argument(
"--sfg-file-extensions",
type=str,
default=None,
dest="file_extensions",
help="Comma-separated list of file extensions",
)
config_group.add_argument(
"--sfg-output-mode",
type=str,
default=None,
choices=("standalone", "inline", "header-only"),
dest="output_mode",
)
config_group.add_argument(
"--sfg-config-module", type=str, default=None, dest="configurator_script"
)
return parser
def config_from_parser_args(args):
if args.configurator_script is not None:
project_config = run_configurator(args.configurator_script)
else:
project_config = None
if args.output_mode is not None:
match args.output_mode.lower():
case "standalone":
output_mode = SfgOutputMode.STANDALONE
case "inline":
output_mode = SfgOutputMode.INLINE
case "header-only":
output_mode = SfgOutputMode.HEADER_ONLY
case _:
assert False, "invalid output mode"
else:
output_mode = None
if args.file_extensions is not None:
file_extentions = list(args.file_extensions.split(","))
h_ext, src_ext = _get_file_extensions(
SfgConfigSource.COMMANDLINE, file_extentions, output_mode
)
else:
h_ext, src_ext = None, None
cmdline_config = SfgConfiguration(
config_source=SfgConfigSource.COMMANDLINE,
header_extension=h_ext,
impl_extension=src_ext,
output_mode=output_mode,
output_directory=args.output_directory,
)
return project_config, cmdline_config
def config_from_commandline(argv: list[str]):
parser = ArgumentParser(
"pystencilssfg",
description="pystencils Source File Generator",
allow_abbrev=False,
)
add_config_args_to_parser(parser)
args, script_args = parser.parse_known_args(argv)
project_config, cmdline_config = config_from_parser_args(args)
return project_config, cmdline_config, script_args
def merge_configurations(
project_config: SfgConfiguration | None,
cmdline_config: SfgConfiguration | None,
script_config: SfgConfiguration | None,
):
# Project config completely overrides default config
config = DEFAULT_CONFIG
if project_config is not None:
config = config.override(project_config)
if cmdline_config is not None:
cmdline_dict = _shallow_dict(cmdline_config)
# Commandline config completely overrides project and default config
config = config.override(cmdline_config)
else:
cmdline_dict = {}
if script_config is not None:
# User config may only set values not specified on the command line
script_dict = _shallow_dict(script_config)
for key, cmdline_value in cmdline_dict.items():
if cmdline_value is not None and script_dict[key] is not None:
raise SfgException(
"Conflicting configuration:"
+ f" Parameter {key} was specified both in the script and on the command line."
)
config = config.override(script_config)
return config
def _get_file_extensions(
cfgsrc: SfgConfigSource, extensions: Sequence[str], output_mode: SfgOutputMode
):
h_ext = None
src_ext = None
extensions = tuple((ext[1:] if ext[0] == "." else ext) for ext in extensions)
for ext in extensions:
if ext in HEADER_FILE_EXTENSIONS:
if h_ext is not None:
raise SfgConfigException(
cfgsrc, "Multiple header file extensions specified."
)
h_ext = ext
elif ext in IMPL_FILE_EXTENSIONS[output_mode]:
if src_ext is not None:
raise SfgConfigException(
cfgsrc, "Multiple source file extensions specified."
)
src_ext = ext
else:
raise SfgConfigException(
cfgsrc, f"Invalid file extension '.{ext}' for output mode {output_mode}"
)
return h_ext, src_ext
def _shallow_dict(obj):
"""Workaround to create a shallow dict of a dataclass object, see
https://docs.python.org/3/library/dataclasses.html#dataclasses.asdict."""
return dict((field.name, getattr(obj, field.name)) for field in fields(obj))