Skip to content
Snippets Groups Projects
Commit 78e27f1c authored by Frederik Hennig's avatar Frederik Hennig
Browse files

Extended documentation: Usage Guides 1

parent 841e40eb
Branches
Tags
No related merge requests found
Pipeline #57739 passed
# Source File Composition
# Source File Generator
::: pystencilssfg.generator.SourceFileGenerator
# Code Generator Configuration
::: pystencilssfg.configuration.SfgConfiguration
# Generation Context
::: pystencilssfg.context.SfgContext
# Source File Composition and Components
::: pystencilssfg.composer.SfgComposer
::: pystencilssfg.source_components.SfgKernelNamespace
::: pystencilssfg.source_components.SfgKernelHandle
::: pystencilssfg.source_components.SfgFunction
Generator scripts are the primary way *pystencils-sfg* is meant to be used.
A generator script is a single Python script, say `kernels.py`, which contains *pystencils-sfg*
code at the top level such that, when executed, it emits source code to a pair of files `kernels.h`
and `kernels.cpp`. This guide describes how to write such a generator script, its structure, and how
it can be used to generate code.
This page gives a general overview over the code generation process, but introduces only the
convenient high-level interface provided by the [SourceFileGenerator][pystencilssfg.SourceFileGenerator]
and [SfgComposer][pystencilssfg.SfgComposer] classes.
For a more in-depth look into building source files, and about using *pystencils-sfg* outside
of a generator script, please take a look at the [In-Depth Guide](building.md).
## Anatomy
The code generation process in a generator script is controlled by the
[SourceFileGenerator][pystencilssfg.SourceFileGenerator] context manager.
It configures the code generator by combining configuration options from the
environment (e.g. a CMake build system) with options specified in the script,
and infers the names of the output files from the script's name.
It then prepares a code generation [context][pystencilssfg.SfgContext] and a
[composer][pystencilssfg.SfgComposer].
The latter is returned when entering the context manager, and provides a convenient
interface to constructing the source files.
To start, place the following code in a Python script, e.g. `kernels.py`:
```Python
from pystencilssfg import SourceFileGenerator, SfgConfiguration
sfg_config = SfgConfiguration()
with SourceFileGenerator(sfg_config) as sfg:
pass
```
The source file is constructed within the context manager's managed region.
During execution of the script, when the region ends, a header/source file pair
`kernels.h` and `kernels.cpp` will be written to the file system next to your script.
Execute the script as-is and inspect the generated files, which will of course
still be empty.
A few notes on configuration:
- The [SourceFileGenerator][pystencilssfg.SourceFileGenerator] parses the script's command line arguments
for configuration options (refer to [CLI and Build System Integration](cli.md)).
If you intend to use command-line parameters in your
generation script, use [`sfg.context.argv`][pystencilssfg.SfgContext.argv] instead of `sys.argv`.
There, all arguments meant for the code generator are already removed.
- The code generator's configuration is consolidated from a global project configuration which may
be provided by the build system; a number of command line arguments; and the
[SfgConfiguration][pystencilssfg.SfgConfiguration] provided in the script.
The project configuration may safely be overridden by the latter two; however, conflicts
between command-line arguments and the configuration defined in the script will cause
an exception to be thrown.
## Using the Composer
The object `sfg` returned by the context manager is an [SfgComposer](pystencilssfg.SfgComposer).
It is a stateless builder object which provides a convenient interface to construct the source
file's contents. Here's an overview:
### Includes and Definitions
With [`SfgComposer.include`][pystencilssfg.SfgComposer.include], the code generator can be instructed
to include header files.
```Python
with SourceFileGenerator(sfg_config) as sfg:
# ...
sfg.include("<vector>")
sfg.incldue("custom_header.h")
```
### Adding Kernels
`pystencils`-generated kernels are managed in
[kernel namespaces][pystencilssfg.source_components.SfgKernelNamespace].
The default kernel namespace is called `kernels` and is available via
[`SfgComposer.kernels`][pystencilssfg.SfgComposer.kernels].
Adding an existing `pystencils` AST, or creating one from a list of assignments, is possible
through [`add`][pystencilssfg.source_components.SfgKernelNamespace.add]
and [`create`][pystencilssfg.source_components.SfgKernelNamespace.create].
The latter is a wrapper around
[`pystencils.create_kernel`](
https://pycodegen.pages.i10git.cs.fau.de/pystencils/sphinx/kernel_compile_and_call.html#pystencils.create_kernel
).
Both functions return a [kernel handle][pystencilssfg.source_components.SfgKernelHandle]
through which the kernel can be accessed, e.g. for calling it in a function.
If required, use [`SfgComposer.kernel_namespace`][pystencilssfg.SfgComposer.kernel_namespace]
to access other kernel namespaces than the default one.
```Python
with SourceFileGenerator(sfg_config) as sfg:
# ...
ast = ps.create_kernel(assignments, config)
khandle = sfg.kernels.add(ast, "kernel_a")
# is equivalent to
khandle = sfg.kernels.create(assignments, "kernel_a", config)
# You may use a different namespace
nspace = sfg.kernel_namespace("group_of_kernels")
nspace.create(assignments, "kernel_a", config)
```
### Building Functions
[Functions][pystencilssfg.source_components.SfgFunction] form the link between your `pystencils` kernels
and your C++ framework. A function in *pystencils-sfg* translates to a simple C++ function, and should
fulfill just the following tasks:
- Extract kernel parameters (pointers, sizes, strides, numerical coefficients)
from C++ objects (like fields, vectors, other data containers)
- Call one or more kernels in sequence or in conditional branches
It is the philosophy of this project that anything more complicated than this should happen in handwritten
code; these generated functions are merely meant to close the remaining gap.
The composer provides an interface for constructing functions that tries to mimic the look of the generated C++
code.
Use [`SfgComposer.function`][pystencilssfg.SfgComposer.function] to create a function,
and [`SfgComposer.call`][pystencilssfg.SfgComposer.call] to call a kernel by its handle:
```Python
with SourceFileGenerator(sfg_config) as sfg:
# ...
sfg.function("MyFunction")(
sfg.call(khandle)
)
```
Note the special syntax: To mimic the look of a C++ function, the composer uses a sequence of two calls
to construct the function.
The function body may further be populated with the following things:
#### Parameter Mappings
Extract kernel parameters from C++ objects:
- [`map_param`][pystencilssfg.SfgComposer.map_param]: Add a single line of code to define one parameter
depending on one other.
- [`map_field`][pystencilssfg.SfgComposer.map_field] maps a pystencils
[`Field`](https://pycodegen.pages.i10git.cs.fau.de/pystencils/sphinx/field.html)
to a field data structure providing the necessary pointers, sizes and stride information.
The field data structure must be provided as an instance of a subclass of
[`SrcField`][pystencilssfg.source_concepts.SrcField].
Currently, *pystencils-sfg* provides mappings to
[`std::vector`](https://en.cppreference.com/w/cpp/container/vector)
(via [`std_vector_ref`][pystencilssfg.source_components.cpp.std_vector_ref])
and
[`std::mdspan`](https://en.cppreference.com/w/cpp/container/mdspan)
(via [`mdspan_ref`][pystencilssfg.source_components.cpp.mdspan_ref])
from the C++ standard library.
- [`map_vector`][pystencilssfg.SfgComposer.map_vector] maps a sequence of scalar numerical values
(given as `pystencils.TypedSymbol`s) to a vector data type. Currently, only `std::vector` is provided.
#### Conditional Branches
A conditonal branch may be added with [`SfgComposer.branch`][pystencilssfg.SfgComposer.branch]
using a special syntax:
```Python
with SourceFileGenerator(sfg_config) as sfg:
# ...
sfg.function("myFunction")(
# ...
sfg.branch("condition")(
# then-body
)(
# else-body (may be omitted)
)
)
```
\ No newline at end of file
These pages provide an overview of how to use the pystencils Source File Generator.
A basic understanding of [pystencils](https://pycodegen.pages.i10git.cs.fau.de/pystencils/index.html)
is required.
## Guides
- [Writing Generator Scripts](generator_scripts.md) explains about the primary interface of *pystencils-sfg*:
Generator scripts, which are Python scripts that, when executed, emit *pystencils*-generated code to a header/source
file pair with the same name as the script.
- [In-Depth: Building Source Files](building.md)
- [CLI and Build System Integration](cli.md)
\ No newline at end of file
......@@ -36,6 +36,11 @@ markdown_extensions:
nav:
- Home: index.md
- 'Usage Guide':
- 'Overview': usage/index.md
- 'Writing Generator Scripts': usage/generator_scripts.md
- 'In-Depth: Building Source Files': usage/building.md
- 'CLI and Build System Integration': usage/cli.md
- 'API Documentation':
- 'Overview': api/index.md
- 'Source File Generator Front-End': api/frontend.md
......
from __future__ import annotations
from typing import TYPE_CHECKING, Optional, Sequence
from typing import TYPE_CHECKING, Sequence
from abc import ABC, abstractmethod
from pystencils import Field
......@@ -27,7 +27,11 @@ class SfgComposer:
@property
def kernels(self) -> SfgKernelNamespace:
"""The default kernel namespace."""
"""The default kernel namespace. Add kernels like:
```Python
sfg.kernels.add(ast, "kernel_name")
sfg.kernels.create(assignments, "kernel_name", config)
```"""
return self._ctx._default_kernel_namespace
def kernel_namespace(self, name: str) -> SfgKernelNamespace:
......@@ -63,6 +67,16 @@ class SfgComposer:
self._ctx.add_function(func)
def function(self, name: str):
"""Add a function.
The syntax of this function adder uses a chain of two calls to mimic C++ syntax:
```Python
sfg.function("FunctionName")(
# Function Body
)
```
"""
if self._ctx.get_function(name) is not None:
raise ValueError(f"Function {name} already exists.")
......@@ -74,6 +88,11 @@ class SfgComposer:
return sequencer
def call(self, kernel_handle: SfgKernelHandle) -> SfgKernelCallNode:
"""Use inside a function body to generate a kernel call.
Args:
kernel_handle: Handle to a kernel previously added to some kernel namespace.
"""
return SfgKernelCallNode(kernel_handle)
def seq(self, *args: SfgCallTreeNode) -> SfgSequence:
......@@ -81,18 +100,36 @@ class SfgComposer:
@property
def branch(self) -> SfgBranchBuilder:
"""Use inside a function body to create an if/else conditonal branch.
The syntax is:
```Python
sfg.branch("condition")(
# then-body
)(
# else-body (may be omitted)
)
```
"""
return SfgBranchBuilder()
def map_field(self, field: Field, src_object: Optional[SrcField] = None) -> SfgDeferredFieldMapping:
if src_object is None:
raise NotImplementedError("Automatic field extraction is not implemented yet.")
else:
return SfgDeferredFieldMapping(field, src_object)
def map_field(self, field: Field, src_object: SrcField) -> SfgDeferredFieldMapping:
"""Map a pystencils field to a field data structure, from which pointers, sizes
and strides should be extracted.
Args:
field: The pystencils field to be mapped
src_object: A `SrcField` object representing a field data structure.
"""
return SfgDeferredFieldMapping(field, src_object)
def map_param(self, lhs: TypedSymbolOrObject, rhs: TypedSymbolOrObject, mapping: str):
"""Arbitrary parameter mapping: Add a single line of code to define a left-hand
side object from a right-hand side."""
return SfgStatements(mapping, (lhs,), (rhs,))
def map_vector(self, lhs_components: Sequence[TypedSymbolOrObject], rhs: SrcVector):
"""Extracts scalar numerical values from a vector data type."""
return make_sequence(*(
rhs.extract_component(dest, coord) for coord, dest in enumerate(lhs_components)
))
......
......@@ -3,10 +3,11 @@ from typing import Generator, Sequence
from .configuration import SfgConfiguration, SfgCodeStyle
from .tree.visitors import CollectIncludes
from .source_components import SfgHeaderInclude, SfgKernelNamespace, SfgFunction
from .exceptions import SfgException
class SfgContext:
def __init__(self, argv, config: SfgConfiguration):
def __init__(self, config: SfgConfiguration, argv: Sequence[str] | None = None):
self._argv = argv
self._config = config
self._default_kernel_namespace = SfgKernelNamespace(self, "kernels")
......@@ -20,6 +21,13 @@ class SfgContext:
@property
def argv(self) -> Sequence[str]:
"""If this context was created by a `pystencilssfg.SourceFileGenerator`, provides the command
line arguments given to the generator script, with configuration arguments for the code generator
stripped away.
Otherwise, throws an exception."""
if self._argv is None:
raise SfgException("This context provides no command-line arguments.")
return self._argv
@property
......
......@@ -21,7 +21,7 @@ class SourceFileGenerator:
config = merge_configurations(project_config, cmdline_config, sfg_config)
self._context = SfgContext(script_args, config)
self._context = SfgContext(config, argv=script_args)
from .emitters.cpu.basic_cpu import BasicCpuEmitter
self._emitter = BasicCpuEmitter(basename, config)
......
......@@ -56,7 +56,8 @@ class SfgKernelNamespace:
yield from self._asts.values()
def add(self, ast: KernelFunction, name: str | None = None):
"""Adds an existing pystencils AST to this namespace."""
"""Adds an existing pystencils AST to this namespace.
If a name is specified, the AST's function name is changed."""
if name is not None:
astname = name
else:
......@@ -73,6 +74,14 @@ class SfgKernelNamespace:
return SfgKernelHandle(self._ctx, astname, self, ast.get_parameters())
def create(self, assignments, name: str | None = None, config: CreateKernelConfig | None = None):
"""Creates a new pystencils kernel from a list of assignments and a configuration.
This is a wrapper around
[`pystencils.create_kernel`](
https://pycodegen.pages.i10git.cs.fau.de/pystencils/
sphinx/kernel_compile_and_call.html#pystencils.create_kernel
)
with a subsequent call to [`add`][pystencilssfg.source_components.SfgKernelNamespace.add].
"""
if config is None:
config = CreateKernelConfig()
......
......@@ -59,7 +59,7 @@ class std_vector(SrcVector, SrcField):
else:
return SfgStatements(f"assert( 1 == {stride} );", (), ())
def extract_component(self, destination: TypedSymbolOrObject, coordinate: int):
def extract_component(self, destination: TypedSymbolOrObject, coordinate: int) -> SfgStatements:
if self._unsafe:
mapping = f"{destination.dtype} {destination.name} = {self._identifier}[{coordinate}];"
else:
......
......@@ -82,5 +82,5 @@ class SrcField(SrcObject, ABC):
class SrcVector(SrcObject, ABC):
@abstractmethod
def extract_component(self, destination: TypedSymbolOrObject, coordinate: int):
def extract_component(self, destination: TypedSymbolOrObject, coordinate: int) -> SfgStatements:
pass
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment