Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found
Select Git revision

Target

Select target project
  • ob28imeq/pystencils-sfg
  • brendan-waters/pystencils-sfg
  • pycodegen/pystencils-sfg
3 results
Select Git revision
Show changes
Showing
with 2690 additions and 0 deletions
**/_sfg_out
\ No newline at end of file
This diff is collapsed.
This diff is collapsed.
{{ fullname | escape | underline}}
.. currentmodule:: {{ module }}
.. autoclass:: {{ objname }}
:members:
{% extends "!autosummary/class.rst" %}
{% block methods %}
{% if methods %}
.. rubric:: {{ _('Methods') }}
.. autosummary::
:toctree:
{% for item in methods %}
~{{ name }}.{{ item }}
{%- endfor %}
{% endif %}
{% endblock %}
{% block attributes %}
{% if attributes %}
.. rubric:: {{ _('Attributes') }}
.. autosummary::
:toctree:
{% for item in attributes %}
~{{ name }}.{{ item }}
{%- endfor %}
{% endif %}
{% endblock %}
import pystencilssfg
from pystencilssfg.config import SfgConfig
from os.path import splitext
class DocsPatchedGenerator(pystencilssfg.SourceFileGenerator):
"""Mockup wrapper around SourceFileGenerator for use in documentation
notebooks to print the generated code directly to the HTML
instead of writing it to file."""
scriptname: str = "demo"
glue: bool = False
display: bool = True
@classmethod
def setup(cls, scriptname: str, glue: bool = False, display: bool = True):
cls.scriptname = scriptname
cls.glue = glue
cls.display = display
def _scriptname(self) -> str:
return f"{DocsPatchedGenerator.scriptname}.py"
def __init__(
self, sfg_config: SfgConfig | None = None, keep_unknown_argv: bool = False
):
super().__init__(sfg_config, keep_unknown_argv=True)
def __exit__(self, exc_type, exc_value, traceback):
if exc_type is None:
self._finish_files()
emitter = self._get_emitter()
header_code = emitter.dumps(self._header_file)
header_ext = splitext(self._header_file.name)[1]
mdcode = ":::::{tab-set}\n"
mdcode += f"::::{{tab-item}} Generated Header ({header_ext})\n"
mdcode += ":::{code-block} C++\n\n"
mdcode += header_code
mdcode += "\n:::\n::::\n"
if self._impl_file is not None:
impl_code = emitter.dumps(self._impl_file)
impl_ext = splitext(self._impl_file.name)[1]
mdcode += f"::::{{tab-item}} Generated Implementation ({impl_ext})\n"
mdcode += ":::{code-block} C++\n\n"
mdcode += impl_code
mdcode += "\n:::\n::::\n"
mdcode += ":::::"
from IPython.display import Markdown
mdobj = Markdown(mdcode)
if self.glue:
from myst_nb import glue
glue(f"sfg_out_{self.scriptname}", mdobj, display=False)
if self.display:
from IPython.display import display
display(mdobj)
pystencilssfg.SourceFileGenerator = DocsPatchedGenerator
*****************************************
Composer API (``pystencilssfg.composer``)
*****************************************
.. module:: pystencilssfg.composer
.. autoclass:: SfgComposer
:members:
.. autoclass:: SfgIComposer
:members:
.. autoclass:: SfgBasicComposer
:members:
.. autoclass:: SfgClassComposer
:members:
.. autoclass:: SfgGpuComposer
:members:
Custom Generators
=================
.. module:: pystencilssfg.composer.custom
.. autoclass:: CustomGenerator
:members:
Helper Methods and Builders
===========================
.. module:: pystencilssfg.composer.basic_composer
.. autofunction:: make_sequence
.. autoclass:: KernelsAdder
:members:
.. autoclass:: SfgFunctionSequencer
:members:
:inherited-members:
.. autoclass:: SfgNodeBuilder
:members:
.. autoclass:: SfgBranchBuilder
:members:
.. autoclass:: SfgSwitchBuilder
:members:
.. module:: pystencilssfg.composer.class_composer
.. autoclass:: SfgMethodSequencer
:members:
:inherited-members:
Context and Cursor
==================
.. module:: pystencilssfg.context
.. autoclass:: SfgContext
:members:
.. autoclass:: SfgCursor
:members:
*********************
Errors and Exceptions
*********************
.. automodule:: pystencilssfg.exceptions
:members:
**************************
Generator Script Interface
**************************
.. autoclass:: pystencilssfg.SourceFileGenerator
:members:
Configuration
=============
.. module:: pystencilssfg.config
.. autoclass:: SfgConfig
:members:
:exclude-members: __init__
Categories, Parameter Types, and Special Values
-----------------------------------------------
.. autoclass:: _GlobalNamespace
.. autodata:: GLOBAL_NAMESPACE
.. autoclass:: FileExtensions
:members:
.. autoclass:: CodeStyle
:members:
.. autoclass:: ClangFormatOptions
:members:
Internal Code Representation (`pystencilssfg.ir`)
=================================================
.. automodule:: pystencilssfg.ir
:members:
Postprocessing
--------------
.. automodule:: pystencilssfg.ir.postprocessing
:members:
Language Modelling (`pystencilssfg.lang`)
=========================================
.. automodule:: pystencilssfg.lang
Expressions
-----------
.. automodule:: pystencilssfg.lang.expressions
:members:
Header Files
------------
.. automodule:: pystencilssfg.lang.headers
:members:
Data Types
----------
.. automodule:: pystencilssfg.lang.types
:members:
Extraction Protocols
--------------------
.. automodule:: pystencilssfg.lang.extractions
:members:
C++ Standard Library (``pystencilssfg.lang.cpp``)
-------------------------------------------------
Quick Access
^^^^^^^^^^^^
.. automodule:: pystencilssfg.lang.cpp.std
:members:
Implementation
^^^^^^^^^^^^^^
.. automodule:: pystencilssfg.lang.cpp
:members:
GPU Runtime APIs
----------------
.. automodule:: pystencilssfg.lang.gpu
:members:
from pystencilssfg import __version__ as sfg_version
from packaging.version import Version
# Configuration file for the Sphinx documentation builder.
#
# For the full list of built-in configuration values, see the documentation:
# https://www.sphinx-doc.org/en/master/usage/configuration.html
# -- Project information -----------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
project = "pystencils-sfg"
copyright = "2024, Frederik Hennig"
author = "Frederik Hennig"
parsed_version = Version(sfg_version)
version = ".".join([parsed_version.public])
release = sfg_version
html_title = f"pystencils-sfg v{version} Documentation"
# -- General configuration ---------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
extensions = [
"myst_nb",
"sphinx.ext.autodoc",
"sphinx.ext.napoleon",
"sphinx.ext.autosummary",
"sphinx.ext.doctest",
"sphinx.ext.intersphinx",
"sphinx_autodoc_typehints",
"sphinx_design",
"sphinx_copybutton"
]
templates_path = ["_templates"]
exclude_patterns = []
master_doc = "index"
# -- Options for HTML output -------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
html_theme = "sphinx_book_theme"
html_static_path = ['_static']
html_theme_options = {
"logo": {
"image_light": "_static/sfg-logo-light.svg",
"image_dark": "_static/sfg-logo-dark.svg",
}
}
# Intersphinx
intersphinx_mapping = {
"python": ("https://docs.python.org/3.8", None),
"numpy": ("https://numpy.org/doc/stable/", None),
"sympy": ("https://docs.sympy.org/latest/", None),
"pystencils": ("https://pycodegen.pages.i10git.cs.fau.de/docs/pystencils/2.0dev/", None),
}
# References
# Treat `single-quoted` code blocks as references to any
default_role = "any"
# Autodoc options
autodoc_member_order = "bysource"
autodoc_typehints = "description"
# autodoc_type_aliases = {
# "VarLike": "pystencilssfg.lang.expressions.VarLike",
# "ExprLike": "pystencilssfg.lang.expressions.ExprLike"
# }
# Doctest Setup
doctest_global_setup = '''
from pystencilssfg import SfgContext, SfgComposer
sfg = SfgComposer(SfgContext())
'''
# -- Options for MyST / MyST-NB ----------------------------------------------
nb_execution_mode = "cache" # do not execute notebooks by default
myst_enable_extensions = [
"dollarmath",
"colon_fence",
]
nb_render_markdown_format = "myst"
---
file_format: mystnb
kernelspec:
name: python3
---
(getting_started_guide)=
# Getting Started
```{code-cell} ipython3
:tags: [remove-cell]
import sys
from pathlib import Path
mockup_path = Path("_util").resolve()
sys.path.append(str(mockup_path))
from sfg_monkeypatch import DocsPatchedGenerator # monkeypatch SFG for docs
```
This guide will explain the basics of using pystencils-sfg through generator scripts.
Generator scripts are the primary way to run code generation with pystencils-sfg.
A generator script is a Python script that, when executed, produces one or more
C++ source files with the same base name, but different file extensions.
## Writing a Basic Generator Script
To start using pystencils-sfg, create a new empty Python file and populate it with the
following minimal skeleton:
```{code-block} python
from pystencilssfg import SourceFileGenerator
with SourceFileGenerator() as sfg:
...
```
The above snippet defines the basic structure of a *generator script*.
When executed, the above will produce two (nearly) empty C++ files
in the current folder, both with the same name as your Python script
but with `.hpp` and `.cpp` file extensions instead.
In the generator script, code generation is orchestrated by the `SourceFileGenerator` context manager.
When entering into the region controlled by the `SourceFileGenerator`,
it supplies us with a *composer object*, customarily called `sfg`.
Through the composer, we can declaratively populate the generated files with code.
## Adding a pystencils Kernel
One of the core applications of pystencils-sfg is to generate and wrap pystencils-kernels
for usage within C++ applications.
To register a kernel, pass its assignments to `sfg.kernels.create`, which returns a *kernel handle* object:
```{code-block} python
src, dst = ps.fields("src, dst: double[1D]")
c = sp.Symbol("c")
@ps.kernel
def scale():
dst.center @= c * src.center()
# Register the kernel for code generation
scale_kernel = sfg.kernels.create(scale, "scale_kernel")
```
In order to call the kernel, and expose it to the outside world,
we have to create a wrapper function for it, using `sfg.function`.
In its body, we use `sfg.call` to invoke the kernel:
```{code-block} python
sfg.function("scale")(
sfg.call(scale_kernel)
)
```
The `function` composer has a special syntax that mimics the generated C++ code.
We call it twice in sequence,
first providing the name of the function, and then populating its body.
Here's our full first generator script:
```{code-cell} ipython3
:tags: [remove-cell]
DocsPatchedGenerator.scriptname = "add_kernel_demo"
DocsPatchedGenerator.glue = True
DocsPatchedGenerator.display = False
```
```{code-cell} ipython3
from pystencilssfg import SourceFileGenerator
import pystencils as ps
import sympy as sp
with SourceFileGenerator() as sfg:
# Define a copy kernel
src, dst = ps.fields("src, dst: double[1D]")
c = sp.Symbol("c")
@ps.kernel
def scale():
dst.center @= c * src.center()
# Register the kernel for code generation
scale_kernel = sfg.kernels.create(scale, "scale_kernel")
# Wrap it in a function
sfg.function("scale")(
sfg.call(scale_kernel)
)
```
When executing the above script, two files will be generated: a C++ header and implementation file containing
the `scale_kernel` and its wrapper function:
:::{glue:md} sfg_out_add_kernel_demo
:format: myst
:::
As you can see, the header file contains a declaration `void scale(...)` of a function
which is defined in the associated implementation file,
and there calls our generated numerical kernel.
As of now, it forwards the entire set of low-level kernel arguments -- array pointers and indexing information --
to the outside.
In numerical applications, this information is most of the time hidden from the user by encapsulating
it in high-level C++ data structures.
Pystencils-sfg offers means of representing such data structures in the code generator, and supports the
automatic extraction of the low-level indexing information from them.
## Mapping Fields to Data Structures
Since C++23 there exists the archetypical [std::mdspan][mdspan], which represents a non-owning n-dimensional view
on a contiguous data array.
Pystencils-sfg offers native support for mapping pystencils fields onto `mdspan` instances in order to
hide their memory layout details.
Import `std` from `pystencilssfg.lang.cpp` and use `std.mdspan.from_field` to create representations
of your pystencils fields as `std::mdspan` objects:
```{code-block} python
from pystencilssfg.lang.cpp import std
...
src_mdspan = std.mdspan.from_field(src)
dst_mdspan = std.mdspan.from_field(dst)
```
Then, inside the wrapper function, instruct the SFG to map the fields onto their corresponding mdspans:
```{code-block} python
sfg.function("scale")(
sfg.map_field(src, src_mdspan),
sfg.map_field(dst, dst_mdspan),
sfg.call(scale_kernel)
)
```
Here's the full script and its output:
```{code-cell} ipython3
:tags: [remove-cell]
DocsPatchedGenerator.setup("mdspan_demo", False, True)
```
```{code-cell} ipython3
from pystencilssfg import SourceFileGenerator
import pystencils as ps
import sympy as sp
from pystencilssfg.lang.cpp import std
with SourceFileGenerator() as sfg:
# Define a copy kernel
src, dst = ps.fields("src, dst: double[1D]")
c = sp.Symbol("c")
@ps.kernel
def scale():
dst.center @= c * src.center()
# Register the kernel for code generation
scale_kernel = sfg.kernels.create(scale, "scale_kernel")
# Create mdspan objects
src_mdspan = std.mdspan.from_field(src)
dst_mdspan = std.mdspan.from_field(dst)
# Wrap it in a function
sfg.function("scale")(
sfg.map_field(src, src_mdspan),
sfg.map_field(dst, dst_mdspan),
sfg.call(scale_kernel)
)
```
:::{note}
As of early 2025, `std::mdspan` is still not fully adopted by standard library implementors
(see [cppreference.com][cppreference_compiler_support]);
most importantly, the GNU libstdc++ does not yet ship an implementation of it.
However, a reference implementation is available at https://github.com/kokkos/mdspan.
If you are using the reference implementation, refer to the documentation of {any}`StdMdspan`
for advice on how to configure the header file and namespace where the class is defined.
:::
[mdspan]: https://en.cppreference.com/w/cpp/container/mdspan
[cppreference_compiler_support]: https://en.cppreference.com/w/cpp/compiler_support
# The pystencils Source File Generator
[![pipeline](https://i10git.cs.fau.de/pycodegen/pystencils-sfg/badges/master/pipeline.svg)](https://i10git.cs.fau.de/pycodegen-/pystencils-sfg/commits/master)
[![coverage](https://i10git.cs.fau.de/pycodegen/pystencils-sfg/badges/master/coverage.svg)](https://i10git.cs.fau.de/pycodegen-/pystencils-sfg/commits/master)
[![licence](https://img.shields.io/gitlab/license/pycodegen%2Fpystencils-sfg?gitlab_url=https%3A%2F%2Fi10git.cs.fau.de)](https://i10git.cs.fau.de/pycodegen/pystencils-sfg/-/blob/master/LICENSE)
*A bridge over the semantic gap between [pystencils](https://pypi.org/project/pystencils/) and C++ HPC frameworks.*
The pystencils Source File Generator is a code generation tool that allows you to
declaratively describe and automatically generate C++ code using its Python API.
It is part of the wider [pycodegen][pycodegen] family of packages for scientific code generation.
The primary purpose of pystencils-sfg is to embed the [pystencils][pystencils] code generator for
high-performance stencil computations into C++ HPC applications and frameworks of all scales.
Its features include:
- Exporting pystencils kernels to C++ source files for use in larger projects
- Mapping of symbolic pystencils fields onto a wide variety of n-dimensional array data structures
- Orchestration of code generation as part of a Makefile or CMake project
- Declarative description of C++ code structure including functions and classes using the versatile composer API
- Reflection of C++ APIs in the code generator, including automatic tracking of variables and `#include`s
## Table of Contents
```{toctree}
:maxdepth: 1
installation
getting_started
```
```{toctree}
:maxdepth: 1
:caption: User Guide
usage/how_to_composer
usage/api_modelling
usage/config_and_cli
usage/project_integration
usage/tips_n_tricks
```
```{toctree}
:maxdepth: 1
:caption: API Reference
api/generation
api/composer
api/lang
api/ir
api/errors
```
[pycodegen]: https://pycodegen.pages.i10git.cs.fau.de
[pystencils]: https://pycodegen.pages.i10git.cs.fau.de/docs/pystencils/2.0dev
# Installation and Setup
## Prequesites
To use pystencils-sfg, you will need at least Python 3.10.
You will also need the appropriate compilers for building the generated code,
such as
- a modern C++ compiler (e.g. GCC, clang)
- `nvcc` for CUDA or `hipcc` for HIP
- Intel OneAPI or AdaptiveCpp for SYCL
Furthermore, an installation of clang-format for automatic code formatting is strongly recommended.
## Install the Latest Development Revision
As pystencils-sfg is still unreleased, it can at this time only be obtained directly
from its Git repository.
Create a fresh [virtual environment](https://docs.python.org/3/library/venv.html) or activate
an existing one. Install both the pystencils 2.0 and pystencils-sfg development revisions from Git:
```{code-block} bash
pip install "git+https://i10git.cs.fau.de/pycodegen/pystencils.git@v2.0-dev"
pip install "git+https://i10git.cs.fau.de/pycodegen/pystencils-sfg.git"
```
````{caution}
*pystencils-sfg* is not compatible with the *pystencils 1.3.x* releases available from PyPI;
at the moment, you will still have to manually install the latest version of pystencils 2.0.
````
## Check your Installation
To verify that the SFG was successfully installed, execute the following command:
```{code-block} bash
sfg-cli version
```
You should see an output like `0.1a4+...`.
## Next Steps
Move on to [](#getting_started_guide) for a guide on how to author simple generator scripts.
---
file_format: mystnb
kernelspec:
name: python3
---
(how_to_cpp_api_modelling)=
# How To Reflect C++ APIs
```{code-cell} ipython3
:tags: [remove-cell]
from __future__ import annotations
import sys
from pathlib import Path
mockup_path = Path("../_util").resolve()
sys.path.append(str(mockup_path))
from sfg_monkeypatch import DocsPatchedGenerator # monkeypatch SFG for docs
from pystencilssfg import SourceFileGenerator
```
Pystencils-SFG is designed to help you generate C++ code that interfaces with pystencils on the one side,
and with your handwritten code on the other side.
This requires that the C++ classes and APIs of your framework or application be represented within the SFG system.
This guide shows how you can use the facilities of the {any}`pystencilssfg.lang` module to model your C++ interfaces
for use with the code generator.
To begin, import the `lang` module:
```{code-cell} ipython3
from pystencilssfg import lang
```
## Defining C++ Types and Type Templates
The first C++ entities that need to be mirrored for the SFGs are the types and type templates a library
or application uses or exposes.
### Non-Templated Types
To define a C++ type, we use {any}`pystencilssfg.lang.cpptype <pystencilssfg.lang.types.cpptype>`:
```{code-cell} ipython3
MyClassTypeFactory = lang.cpptype("my_namespace::MyClass", "MyClass.hpp")
MyClassTypeFactory
```
This defines two properties of the type: its fully qualified name, and the set of headers
that need to be included when working with the type.
Now, whenever this type occurs as the type of a variable given to pystencils-sfg,
the code generator will make sure that `MyClass.hpp` is included into the respective
generated code file.
The object returned by `cpptype` is not the type itself, but a factory for instances of the type.
Even as `MyClass` does not have any template parameters, we can create different instances of it:
`const` and non-`const`, as well as references and non-references.
We do this by calling the factory:
```{code-cell} ipython3
MyClass = MyClassTypeFactory()
str(MyClass)
```
To produce a `const`-qualified version of the type:
```{code-cell} ipython3
MyClassConst = MyClassTypeFactory(const=True)
str(MyClassConst)
```
And finally, to produce a reference instead:
```{code-cell} ipython3
MyClassRef = MyClassTypeFactory(ref=True)
str(MyClassRef)
```
Of course, `const` and `ref` can also be combined to create a reference-to-const.
### Types with Template Parameters
We can add template parameters to our type by the use of
[Python format strings](https://docs.python.org/3/library/string.html#formatstrings):
```{code-cell} ipython3
MyClassTemplate = lang.cpptype("my_namespace::MyClass< {T1}, {T2} >", "MyClass.hpp")
MyClassTemplate
```
Here, the type parameters `T1` and `T2` are specified in braces.
For them, values must be provided when calling the factory to instantiate the type:
```{code-cell} ipython3
MyClassIntDouble = MyClassTemplate(T1="int", T2="double")
str(MyClassIntDouble)
```
The way type parameters are passed to the factory is identical to the behavior of {any}`str.format`,
except that it does not support attribute or element accesses.
In particular, this means that we can also use unnamed, implicit positional parameters:
```{code-cell} ipython3
MyClassTemplate = lang.cpptype("my_namespace::MyClass< {}, {} >", "MyClass.hpp")
MyClassIntDouble = MyClassTemplate("int", "double")
str(MyClassIntDouble)
```
## Creating Variables and Expressions
Type templates and types will not get us far on their own.
To use them in APIs, as function or constructor parameters,
or as class members and local objects,
we need to create *variables* with certain types.
To do so, we need to inject our defined types into the expression framework of pystencils-sfg.
We wrap the type in an interface that allows us to create variables and, later, more complex expressions,
using {any}`lang.CppClass <pystencilssfg.lang.expressions.CppClass>`:
```{code-cell} ipython3
class MyClass(lang.CppClass):
template = lang.cpptype("my_namespace::MyClass< {T1}, {T2} >", "MyClass.hpp")
```
Instances of `MyClass` can now be created via constructor call, in the same way as above.
This gives us an unbound `MyClass` object, which we can bind to a variable name by calling `var` on it:
```{code-cell} ipython3
my_obj = MyClass(T1="int", T2="void").var("my_obj")
my_obj, str(my_obj.dtype)
```
## Reflecting C++ Class APIs
In the previous section, we showed how to reflect a C++ class in pystencils-sfg in order to create
a variable representing an object of that class.
We can now extend this to reflect the public API of the class, in order to create complex expressions
involving objects of `MyClass` during code generation.
### Public Methods
Assume `MyClass` has the following public interface:
```C++
template< typename T1, typename T2 >
class MyClass {
public:
T1 & getA();
std::tuple< T1, T2 > getBoth();
void replace(T1 a_new, T2 b_new);
}
```
We mirror this in our Python reflection of `CppClass` using methods that create `AugExpr` objects,
which represent C++ expressions annotated with variables they depend on.
A possible implementation might look like this:
```{code-cell} ipython3
---
tags: [remove-cell]
---
class MyClass(lang.CppClass):
template = lang.cpptype("my_namespace::MyClass< {T1}, {T2} >", "MyClass.hpp")
def ctor(self, a: lang.AugExpr, b: lang.AugExpr) -> MyClass:
return self.ctor_bind(a, b)
def getA(self) -> lang.AugExpr:
return lang.AugExpr.format("{}.getA()", self)
def getBoth(self) -> lang.AugExpr:
return lang.AugExpr.format("{}.getBoth()", self)
def replace(self, a_new: lang.AugExpr, b_new: lang.AugExpr) -> lang.AugExpr:
return lang.AugExpr.format("{}.replace({}, {})", self, a_new, b_new)
```
```{code-block} python
class MyClass(lang.CppClass):
template = lang.cpptype("my_namespace::MyClass< {T1}, {T2} >", "MyClass.hpp")
def getA(self) -> lang.AugExpr:
return lang.AugExpr.format("{}.getA()", self)
def getBoth(self) -> lang.AugExpr:
return lang.AugExpr.format("{}.getBoth()", self)
def replace(self, a_new: lang.AugExpr, b_new: lang.AugExpr) -> lang.AugExpr:
return lang.AugExpr.format("{}.replace({}, {})", self, a_new, b_new)
```
Each method of `MyClass` reflects a method of the same name in its public C++ API.
These methods do not return values, but *expressions*;
here, we use the generic `AugExpr` class to model expressions that we don't know anything
about except how they should be constructed.
We create these expressions using `AugExpr.format`, which takes a format string
and interpolation arguments in the same way as `cpptype`.
Internally, it will analyze the format arguments (e.g. `self`, `a_new` and `b_new` in `replace`),
and combine information from any `AugExpr`s found among them.
These are:
- **Variables**: If any of the input expression depend on variables, the resulting expression will
depend on the union of all these variable sets
- **Headers**: If any of the input expression requires certain header files to be evaluated,
the resulting expression will require the same header files.
We can see this in action by calling one of the methods on a variable of type `MyClass`:
```{code-cell} ipython3
my_obj = MyClass(T1="int", T2="void").var("my_obj")
expr = my_obj.getBoth()
expr, lang.depends(expr), lang.includes(expr)
```
We can see: the newly created expression `my_obj.getBoth()` depends on the variable `my_obj` and
requires the header `MyClass.hpp` to be included; this header it has inherited from `my_obj`.
### Constructors
Using the `AugExpr` system, we can also model constructors of `MyClass`.
Assume `MyClass` has the constructor `MyClass(T1 a, T2 b)`.
We implement this by adding a `ctor` method to our Python interface:
```{code-block} python
class MyClass(lang.CppClass):
...
def ctor(self, a: lang.AugExpr, b: lang.AugExpr) -> MyClass:
return self.ctor_bind(a, b)
```
Here, we don't use `AugExpr.format`; instead, we use `ctor_bind`, which is exposed by `CppClass`.
This will generate the correct constructor invocation from the type of our `MyClass` object
and also ensure the headers required by `MyClass` are correctly attached to the resulting
expression:
```{code-cell} ipython3
a = lang.AugExpr("int").var("a")
b = lang.AugExpr("double").var("b")
expr = MyClass(T1="int", T2="double").ctor(a, b)
expr, lang.depends(expr), lang.includes(expr)
```
(field_data_structure_reflection)=
## Reflecting Field Data Structures
One key feature of pystencils-sfg is its ability to map symbolic fields
onto arbitrary array data structures
using the composer's {any}`map_field <SfgBasicComposer.map_field>` method.
The APIs of a custom field data structure can naturally be injected into pystencils-sfg
using the modelling framework described above.
However, for them to be recognized by `map_field`,
the reflection class also needs to implement the {any}`SupportsFieldExtraction` protocol.
This requires that the following three methods are implemented:
```{code-block} python
def _extract_ptr(self) -> AugExpr: ...
def _extract_size(self, coordinate: int) -> AugExpr | None: ...
def _extract_stride(self, coordinate: int) -> AugExpr | None: ...
```
The first, `_extract_ptr`, must return an expression that evaluates
to the base pointer of the field's memory buffer.
This pointer has to point at the field entry which pystencils accesses
at all-zero index and offsets (see [](#note-on-ghost-layers)).
The other two, when called with a coordinate $c \ge 0$, shall return
the size and linearization stride of the field in that direction.
If the coordinate is equal or larger than the field's dimensionality,
return `None` instead.
### Sample Field API Reflection
Consider the following class template for a field, which takes its element type
and dimensionality as template parameters
and exposes its data pointer, shape, and strides through public methods:
```{code-block} C++
template< std::floating_point ElemType, size_t DIM >
class MyField {
public:
size_t get_shape(size_t coord);
size_t get_stride(size_t coord);
ElemType * data_ptr();
}
```
It could be reflected by the following class.
Note that in this case we define a custom `__init__` method in order to
intercept the template arguments `elem_type` and `dim`
and store them as instance members.
Our `__init__` then forwards all its arguments up to `CppClass.__init__`.
We then define reflection methods for `shape`, `stride` and `data` -
the implementation of the field extraction protocol then simply calls these methods.
```{code-cell} ipython3
from pystencilssfg.lang import SupportsFieldExtraction
from pystencils.types import UserTypeSpec
class MyField(lang.CppClass, SupportsFieldExtraction):
template = lang.cpptype(
"MyField< {ElemType}, {DIM} >",
"MyField.hpp"
)
def __init__(
self,
elem_type: UserTypeSpec,
dim: int,
**kwargs,
) -> None:
self._elem_type = elem_type
self._dim = dim
super().__init__(ElemType=elem_type, DIM=dim, **kwargs)
# Reflection of Public Methods
def get_shape(self, coord: int | lang.AugExpr) -> lang.AugExpr:
return lang.AugExpr.format("{}.get_shape({})", self, coord)
def get_stride(self, coord: int | lang.AugExpr) -> lang.AugExpr:
return lang.AugExpr.format("{}.get_stride({})", self, coord)
def data_ptr(self) -> lang.AugExpr:
return lang.AugExpr.format("{}.data_ptr()", self)
# Field Extraction Protocol that uses the above interface
def _extract_ptr(self) -> lang.AugExpr:
return self.data_ptr()
def _extract_size(self, coordinate: int) -> lang.AugExpr | None:
if coordinate > self._dim:
return None
else:
return self.get_shape(coordinate)
def _extract_stride(self, coordinate: int) -> lang.AugExpr | None:
if coordinate > self._dim:
return None
else:
return self.get_stride(coordinate)
```
Our custom field reflection is now ready to be used.
The following generator script demonstrates what code is generated when an instance of `MyField`
is passed to `sfg.map_field`:
```{code-cell} ipython3
import pystencils as ps
from pystencilssfg.lang.cpp import std
with SourceFileGenerator() as sfg:
# Create symbolic fields
f = ps.fields("f: double[3D]")
f_myfield = MyField(f.dtype, f.ndim, ref=True).var(f.name)
# Create the kernel
asm = ps.Assignment(f(0), 2 * f(0))
khandle = sfg.kernels.create(asm)
# Create the wrapper function
sfg.function("invoke")(
sfg.map_field(f, f_myfield),
sfg.call(khandle)
)
```
### Add a Factory Function
In the above example, an instance of `MyField` representing the field `f` is created by the
slightly verbose expression `MyField(f.dtype, f.ndim, ref=True).var(f.name)`.
Having to write this sequence every time, for every field, introduces unnecessary
cognitive load and lots of potential sources of error.
Whenever it is possible to create a field reflection using just information contained in a
pystencils {any}`Field <pystencils.field.Field>` object,
the API reflection should therefore implement a factory method `from_field`:
```{code-cell} ipython3
class MyField(lang.CppClass, SupportsFieldExtraction):
...
@classmethod
def from_field(cls, field: ps.Field, const: bool = False, ref: bool = False) -> MyField:
return cls(f.dtype, f.ndim, const=const, ref=ref).var(f.name)
```
The above signature is idiomatic for `from_field`, and you should stick to it as far as possible.
We can now use it inside the generator script:
```{code-block} python
f = ps.fields("f: double[3D]")
f_myfield = MyField.from_field(f)
```
(note-on-ghost-layers)=
### A Note on Ghost Layers
Some care has to be taken when reflecting data structures that model the notion
of ghost layers.
Consider an array with the index space $[0, N_x) \times [0, N_y)$,
its base pointer identifying the entry $(0, 0)$.
When a pystencils kernel is generated with a shell of $k$ ghost layers
(see {any}`CreateKernelConfig.ghost_layers <pystencils.codegen.config.CreateKernelConfig.ghost_layers>`),
it will process only the subspace $[k, N_x - k) \times [k, N_x - k)$.
If your data structure is implemented such that ghost layer nodes have coordinates
$< 0$ and $\ge N_{x, y}$,
you must hence take care that
- either, `_extract_ptr` returns a pointer identifying the array entry at `(-k, -k)`;
- or, ensure that kernels operating on your data structure are always generated
with `ghost_layers = 0`.
In either case, you must make sure that the number of ghost layers in your data structure
matches the expected number of ghost layers of the kernel.
(how_to_generator_scripts_config)=
# Generator Script Configuration and Command-Line Interface
There are several ways to affect the behavior and output of a generator script.
For one, the `SourceFileGenerator` itself may be configured from the combination of three
different configuration sources:
- **Inline Configuration:** The generator script may set up an {any}`SfgConfig` object,
which is passed to the `SourceFileGenerator` at its creation; see [Inline Configuration](#inline_config)
- **Command-Line Options:** The `SourceFileGenerator` parses the command line arguments of
the generator script to set some of its configuration options; see [Command-Line Options](#cmdline_options)
- **Project Configuration:** When embedded into a larger project, using a build system such as CMake, generator scripts
may be configured globally within that project by the use of a *configuration module*.
Settings specified inside that configuration module are always overridden by the two other configuration sources listed above.
For details on configuration modules, refer to the guide on [Project and Build System Integration](#guide_project_integration).
(inline_config)=
## Inline Configuration
To configure the source file generator within your generator script, import the {any}`SfgConfig` from `pystencilssfg`.
You may then set up the configuration object before passing it to the `SourceFileGenerator` constructor.
To illustrate, the following snippet alters the code indentation width and changes the output directory
of the generator script to `gen_src`:
```{literalinclude} examples/guide_generator_scripts/inline_config/kernels.py
```
For a selection of common configuration options, see [below](#config_options).
The inline configuration will override any values set by the [project configuration](#config_module)
and must not conflict with any [command line arguments](#custom_cli_args).
(config_options)=
## Configuration Options
Here is a selection of common configuration options to be set in the [inline configuration](#inline_config) or
[project configuration](#config_module).
### Output Options
The file extensions of the generated files can be modified through
{any}`cfg.extensions.header <FileExtensions.header>`
and {any}`cfg.extensions.impl <FileExtensions.impl>`;
and the output directory of the code generator can be set through {any}`cfg.output_directory <SfgConfig.output_directory>`.
The [header-only mode](#header_only_mode) can be enabled using {any}`cfg.header_only <SfgConfig.header_only>`.
:::{danger}
When running generator scripts through [CMake](#cmake_integration), the file extensions,
output directory, and header-only mode settings will be managed fully by the pystencils-sfg
CMake module and the (optional) project configuration module.
They should therefore not be set in the inline configuration,
as this will likely lead to errors being raised during code generation.
:::
### Outer Namespace
To specify the outer namespace to which all generated code should be emitted,
set {any}`cfg.outer_namespace <SfgConfig.outer_namespace>`.
### Code Style and Formatting
Pystencils-sfg gives you some options to affect its output code style.
These are controlled by the options in the {any}`cfg.code_style <CodeStyle>` category.
Furthermore, pystencils-sfg uses `clang-format` to beautify generated code.
The behaviour of the clang-format integration is managed by the
the {any}`cfg.clang_format <ClangFormatOptions>` category,
where you can set options to skip or enforce formatting,
or change the formatter binary.
To set the code style used by `clang-format` either create a `.clang-format` file
in any of the parent folders of your generator script,
or modify the {any}`cfg.clang_format.code_style <ClangFormatOptions.code_style>` option.
:::{seealso}
[Clang-Format Style Options](https://clang.llvm.org/docs/ClangFormatStyleOptions.html)
:::
Clang-format will, by default, sort `#include` statements alphabetically and separate
local and system header includes.
To override this, you can set a custom sorting key for `#include` sorting via
{any}`cfg.code_style.includes_sorting_key <CodeStyle.includes_sorting_key>`.
(cmdline_options)=
## Command-Line Options
The `SourceFileGenerator` consumes a number of command-line parameters that may be passed to the script
on invocation. These include:
- `--sfg-output-dir <path>`: Set the output directory of the generator script. This corresponds to {any}`SfgConfig.output_directory`.
- `--sfg-file-extensions <exts>`: Set the file extensions used for the generated files;
`exts` must be a comma-separated list not containing any spaces. Corresponds to {any}`SfgConfig.extensions`.
- `[--no]--sfg-header-only`: Enable or disable header-only code generation. Corresponds to {any}`SfgConfig.header_only`.
If any configuration option is set to conflicting values on the command line and in the inline configuration,
the generator script will terminate with an error.
You may examine the full set of possible command line parameters by invoking a generator script
with the `--help` flag:
```bash
$ python kernels.py --help
```
(header_only_mode)=
## Header-Only Mode
When the header-only output mode is enabled,
the code generator will emit only a header file and no separate implementation file.
In this case, the composer will automatically place all function, method,
and kernel definitions in the header file.
Header-only code generation can be enabled by setting the `--header-only` command-line flag
or the {any}`SfgConfig.header_only` configuration option.
(custom_cli_args)=
## Adding Custom Command-Line Options
Sometimes, you might want to add your own command-line options to a generator script
in order to affect its behavior from the shell,
for instance by using {any}`argparse` to set up an argument parser.
If you parse your options directly from {any}`sys.argv`,
as {any}`parse_args <argparse.ArgumentParser.parse_args>` does by default,
your parser will also receive any options meant for the `SourceFileGenerator`.
To filter these out of the argument list,
pass the additional option `keep_unknown_argv=True` to your `SourceFileGenerator`.
This will instruct it to store any unknown command line arguments into `sfg.context.argv`,
where you can then retrieve them from and pass on to your custom parser:
```{literalinclude} examples/guide_generator_scripts/custom_cmdline_args/kernels.py
```
Any SFG-specific arguments will already have been filtered out of this argument list.
As a consequence of the above, if the generator script is invoked with a typo in some SFG-specific argument,
which the `SourceFileGenerator` therefore does not recognize,
that argument will be passed on to your downstream parser instead.
:::{important}
If you do *not* pass on `sfg.context.argv` to a downstream parser, make sure that `keep_unknown_argv` is set to
`False` (which is the default), such that typos or illegal arguments will not be ignored.
:::
*.cpp
*.h
*.hpp
\ No newline at end of file
import os
import glob
import subprocess
GROUPS = ["guide_generator_scripts"]
THIS_DIR = os.path.split(__file__)[0]
def main():
for group in GROUPS:
group_folder = os.path.join(THIS_DIR, group)
for group_entry in os.listdir(group_folder):
scripts_dir = os.path.join(group_folder, group_entry)
if os.path.isdir(scripts_dir):
query = os.path.join(scripts_dir, "*.py")
for script in glob.glob(query):
subprocess.run(["python", script], cwd=scripts_dir).check_returncode()
if __name__ == "__main__":
main()
from pystencilssfg import SourceFileGenerator
with SourceFileGenerator() as sfg:
pass