Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found
Select Git revision

Target

Select target project
  • anirudh.jonnalagadda/pystencils
  • hyteg/pystencils
  • jbadwaik/pystencils
  • jngrad/pystencils
  • itischler/pystencils
  • ob28imeq/pystencils
  • hoenig/pystencils
  • Bindgen/pystencils
  • hammer/pystencils
  • da15siwa/pystencils
  • holzer/pystencils
  • alexander.reinauer/pystencils
  • ec93ujoh/pystencils
  • Harke/pystencils
  • seitz/pystencils
  • pycodegen/pystencils
16 results
Select Git revision
Show changes
Commits on Source (4)
Showing
with 193 additions and 968 deletions
__pycache__ __pycache__
.ipynb_checkpoints .ipynb_checkpoints
.coverage .coverage*
*.pyc *.pyc
*.vti *.vti
/build /build
...@@ -18,4 +18,4 @@ pystencils/boundaries/createindexlistcython.c ...@@ -18,4 +18,4 @@ pystencils/boundaries/createindexlistcython.c
pystencils/boundaries/createindexlistcython.*.so pystencils/boundaries/createindexlistcython.*.so
pystencils_tests/tmp pystencils_tests/tmp
pystencils_tests/kerncraft_inputs/.2d-5pt.c_kerncraft/ pystencils_tests/kerncraft_inputs/.2d-5pt.c_kerncraft/
pystencils_tests/kerncraft_inputs/.3d-7pt.c_kerncraft/ pystencils_tests/kerncraft_inputs/.3d-7pt.c_kerncraft/
\ No newline at end of file
...@@ -14,7 +14,7 @@ ...@@ -14,7 +14,7 @@
name: pystencils name: pystencils
dependencies: dependencies:
# Basic dependencies: # Basic dependencies:
- python >= 3.6 - python >= 3.8
- numpy - numpy
- sympy >= 1.1 - sympy >= 1.1
- appdirs # to find default cache directory on each platform - appdirs # to find default cache directory on each platform
......
...@@ -51,7 +51,7 @@ nbsphinx_execute = 'never' ...@@ -51,7 +51,7 @@ nbsphinx_execute = 'never'
nbsphinx_codecell_lexer = 'python3' nbsphinx_codecell_lexer = 'python3'
# Example configuration for intersphinx: refer to the Python standard library. # Example configuration for intersphinx: refer to the Python standard library.
intersphinx_mapping = {'python': ('https://docs.python.org/3.6', None), intersphinx_mapping = {'python': ('https://docs.python.org/3.8', None),
'numpy': ('https://docs.scipy.org/doc/numpy/', None), 'numpy': ('https://docs.scipy.org/doc/numpy/', None),
'matplotlib': ('https://matplotlib.org/', None), 'matplotlib': ('https://matplotlib.org/', None),
'sympy': ('https://docs.sympy.org/latest/', None), 'sympy': ('https://docs.sympy.org/latest/', None),
......
Source diff could not be displayed: it is too large. Options to address this: view the blob.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
import psutil
from mpl_toolkits.mplot3d import Axes3D from mpl_toolkits.mplot3d import Axes3D
from matplotlib import pyplot, cm from matplotlib import pyplot, cm
from pystencils.session import * from pystencils.session import *
from pystencils.boundaries import add_neumann_boundary, Neumann, Dirichlet, BoundaryHandling from pystencils.boundaries import add_neumann_boundary, Neumann, Dirichlet, BoundaryHandling
from pystencils.slicing import slice_from_direction from pystencils.slicing import slice_from_direction
import math import math
import time import time
%matplotlib inline %matplotlib inline
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Test to see if pycuda is installed which is needed to run calculations on the GPU Test to see if pycuda is installed which is needed to run calculations on the GPU
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
try: try:
import pycuda import pycuda
except ImportError: except ImportError:
pycuda = None pycuda = None
print('No pycuda installed') print('No pycuda installed')
if pycuda: if pycuda:
import pycuda.gpuarray as gpuarray import pycuda.gpuarray as gpuarray
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# Tutorial 03: Datahandling # Tutorial 03: Datahandling
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
This is a tutorial about the `DataHandling` class of pystencils. This class is an abstraction layer to This is a tutorial about the `DataHandling` class of pystencils. This class is an abstraction layer to
- link numpy arrays to pystencils fields - link numpy arrays to pystencils fields
- handle CPU-GPU array transfer, such that one can write code that works on CPU and GPU - handle CPU-GPU array transfer, such that one can write code that works on CPU and GPU
- makes it possible to write MPI parallel simulations to run on distributed-memory clusters using the waLBerla library - makes it possible to write MPI parallel simulations to run on distributed-memory clusters using the waLBerla library
We will look at a small and easy example to demonstrate the usage of `DataHandling` objects. We will define an averaging kernel to every cell of an array, that writes the average of the neighbor cell values to the center. We will look at a small and easy example to demonstrate the usage of `DataHandling` objects. We will define an averaging kernel to every cell of an array, that writes the average of the neighbor cell values to the center.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## 1. Manual ## 1. Manual
### 1.1. CPU kernels ### 1.1. CPU kernels
In this first part, we set up a scenario manually without a `DataHandling`. In the next sections we then repeat the same setup with the help of the data handling. In this first part, we set up a scenario manually without a `DataHandling`. In the next sections we then repeat the same setup with the help of the data handling.
One concept of *pystencils* that may be confusing at first, is the differences between pystencils fields and numpy arrays. Fields are used to describe the computation *symbolically* with sympy, while numpy arrays hold the actual values where the computation is executed on. One concept of *pystencils* that may be confusing at first, is the differences between pystencils fields and numpy arrays. Fields are used to describe the computation *symbolically* with sympy, while numpy arrays hold the actual values where the computation is executed on.
One option to create and execute a *pystencils* kernel is listed below. For reasons that become clear later we call this the **variable-field-size workflow**: One option to create and execute a *pystencils* kernel is listed below. For reasons that become clear later we call this the **variable-field-size workflow**:
1. define pystencils fields 1. define pystencils fields
2. use sympy and the pystencils fields to define an update rule, that describes what should be done on *every cell* 2. use sympy and the pystencils fields to define an update rule, that describes what should be done on *every cell*
3. compile the update rule to a real function, that can be called from Python. For each field that was referenced in the symbolic description the function expects a numpy array, passed as named parameter 3. compile the update rule to a real function, that can be called from Python. For each field that was referenced in the symbolic description the function expects a numpy array, passed as named parameter
4. create some numpy arrays with actual data 4. create some numpy arrays with actual data
5. call the kernel - usually many times 5. call the kernel - usually many times
Now, lets see how this actually looks in Python code: Now, lets see how this actually looks in Python code:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
# 1. field definitions # 1. field definitions
src_field, dst_field = ps.fields("src, dst:[2D]") src_field, dst_field = ps.fields("src, dst:[2D]")
# 2. define update rule # 2. define update rule
update_rule = [ps.Assignment(lhs=dst_field[0, 0], update_rule = [ps.Assignment(lhs=dst_field[0, 0],
rhs=(src_field[1, 0] + src_field[-1, 0] + rhs=(src_field[1, 0] + src_field[-1, 0] +
src_field[0, 1] + src_field[0, -1]) / 4)] src_field[0, 1] + src_field[0, -1]) / 4)]
# 3. compile update rule to function # 3. compile update rule to function
kernel_function = ps.create_kernel(update_rule).compile() kernel_function = ps.create_kernel(update_rule).compile()
# 4. create numpy arrays and call kernel # 4. create numpy arrays and call kernel
src_arr, dst_arr = np.random.rand(30, 30), np.zeros([30, 30]) src_arr, dst_arr = np.random.rand(30, 30), np.zeros([30, 30])
# 5. call kernel # 5. call kernel
kernel_function(src=src_arr, dst=dst_arr) # names of arguments have to match names passed to ps.fields() kernel_function(src=src_arr, dst=dst_arr) # names of arguments have to match names passed to ps.fields()
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
This workflow separates the symbolic and the numeric stages very cleanly. The separation also makes it possible to stop after step 3, write the C-code to a file and call the kernel from a C program. Speaking of the C-Code - lets have a look at the generated sources: This workflow separates the symbolic and the numeric stages very cleanly. The separation also makes it possible to stop after step 3, write the C-code to a file and call the kernel from a C program. Speaking of the C-Code - lets have a look at the generated sources:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
ps.show_code(kernel_function.ast) ps.show_code(kernel_function.ast)
``` ```
%% Output %% Output
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Even if it looks very ugly and low-level :) lets look at this code in a bit more detail. The code is generated in a way that it works for different array sizes. The size of the array is passed in the `_size_dst_` variables that specifiy the shape of the array for each dimension. Also, the memory layout (linearization) of the array can be different. That means the array could be stored in row-major or column-major order - if we pass in the array strides correctly the kernel does the right thing. If you're not familiar with the concept of strides check out [this stackoverflow post](https://stackoverflow.com/questions/53097952/how-to-understand-numpy-strides-for-layman) or search in the numpy documentation for strides - C vs Fortran order. Even if it looks very ugly and low-level :) lets look at this code in a bit more detail. The code is generated in a way that it works for different array sizes. The size of the array is passed in the `_size_dst_` variables that specifiy the shape of the array for each dimension. Also, the memory layout (linearization) of the array can be different. That means the array could be stored in row-major or column-major order - if we pass in the array strides correctly the kernel does the right thing. If you're not familiar with the concept of strides check out [this stackoverflow post](https://stackoverflow.com/questions/53097952/how-to-understand-numpy-strides-for-layman) or search in the numpy documentation for strides - C vs Fortran order.
The goal of *pystencils* is to produce the fastest possible code. One technique to do this is to use all available information already on compile time and generate code that is highly adapted to the specific problem. In our case we already know the shape and strides of the arrays we want to apply the kernel to, so we can make use of this information. This idea leads to the **fixed-field-size workflow**. The main difference there is that we define the arrays first and therefore let *pystencils* know about the array shapes and strides, so that it can generate more specific code: The goal of *pystencils* is to produce the fastest possible code. One technique to do this is to use all available information already on compile time and generate code that is highly adapted to the specific problem. In our case we already know the shape and strides of the arrays we want to apply the kernel to, so we can make use of this information. This idea leads to the **fixed-field-size workflow**. The main difference there is that we define the arrays first and therefore let *pystencils* know about the array shapes and strides, so that it can generate more specific code:
1. create numpy arrays that hold your data 1. create numpy arrays that hold your data
2. define pystencils fields, this time telling pystencils already which arrays they correspond to, so that it knows about the size and strides 2. define pystencils fields, this time telling pystencils already which arrays they correspond to, so that it knows about the size and strides
in the other steps nothing has changed in the other steps nothing has changed
3. define the update rule 3. define the update rule
4. compile update rule to kernel 4. compile update rule to kernel
5. run the kernel 5. run the kernel
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
# 1. create arrays first # 1. create arrays first
src_arr, dst_arr = np.random.rand(30, 30), np.zeros([30, 30]) src_arr, dst_arr = np.random.rand(30, 30), np.zeros([30, 30])
# 2. define symbolic fields - note the additional parameter that link an array to each field # 2. define symbolic fields - note the additional parameter that link an array to each field
src_field, dst_field = ps.fields("src, dst:[2D]", src=src_arr, dst=dst_arr) src_field, dst_field = ps.fields("src, dst:[2D]", src=src_arr, dst=dst_arr)
# 3. define update rule # 3. define update rule
update_rule = [ps.Assignment(lhs=dst_field[0, 0], update_rule = [ps.Assignment(lhs=dst_field[0, 0],
rhs=(src_field[1, 0] + src_field[-1, 0] + rhs=(src_field[1, 0] + src_field[-1, 0] +
src_field[0, 1] + src_field[0, -1]) / 4)] src_field[0, 1] + src_field[0, -1]) / 4)]
# 4. compile it # 4. compile it
kernel_function = ps.create_kernel(update_rule).compile() kernel_function = ps.create_kernel(update_rule).compile()
# 5. call kernel # 5. call kernel
kernel_function(src=src_arr, dst=dst_arr) # names of arguments have to match names passed to ps.fields() kernel_function(src=src_arr, dst=dst_arr) # names of arguments have to match names passed to ps.fields()
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Functionally, both variants are equivalent. We see the difference only when we look at the generated code Functionally, both variants are equivalent. We see the difference only when we look at the generated code
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
ps.show_code(kernel_function.ast) ps.show_code(kernel_function.ast)
``` ```
%% Output %% Output
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Compare this to the code above! It looks much simpler. The reason is that all index computations are already simplified since the exact field sizes and strides are known. This kernel now only works on arrays of the previously specified size. Compare this to the code above! It looks much simpler. The reason is that all index computations are already simplified since the exact field sizes and strides are known. This kernel now only works on arrays of the previously specified size.
Lets try what happens if we use a different array: Lets try what happens if we use a different array:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
src_arr2, dst_arr2 = np.random.rand(40, 40), np.zeros([40, 40]) src_arr2, dst_arr2 = np.random.rand(40, 40), np.zeros([40, 40])
try: try:
kernel_function(src=src_arr2, dst=dst_arr2) kernel_function(src=src_arr2, dst=dst_arr2)
except ValueError as e: except ValueError as e:
print(e) print(e)
``` ```
%% Output %% Output
Wrong shape of array dst. Expected (30, 30) Wrong shape of array dst. Expected (30, 30)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### 1.2. GPU simulations ### 1.2. GPU simulations
Let's now jump to a seemingly unrelated topic: running kernels on the GPU. Let's now jump to a seemingly unrelated topic: running kernels on the GPU.
When creating the kernel, an additional parameter `target='gpu'` has to be passed. Also, the compiled kernel cannot be called with numpy arrays directly, but has to be called with `pycuda.gpuarray`s instead. That means, we have to transfer our numpy array to GPU first. From this step we obtain a gpuarray, then we can run the kernel, hopefully multiple times so that the data transfer was worth the time. Finally we transfer the finished result back to CPU: When creating the kernel, an additional parameter `target=ps.Target.GPU` has to be passed. Also, the compiled kernel cannot be called with numpy arrays directly, but has to be called with `pycuda.gpuarray`s instead. That means, we have to transfer our numpy array to GPU first. From this step we obtain a gpuarray, then we can run the kernel, hopefully multiple times so that the data transfer was worth the time. Finally we transfer the finished result back to CPU:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
if pycuda: if pycuda:
kernel_function_gpu = ps.create_kernel(update_rule, target='gpu').compile() config = ps.CreateKernelConfig(target=ps.Target.GPU)
kernel_function_gpu = ps.create_kernel(update_rule, config=config).compile()
# transfer to GPU # transfer to GPU
src_arr_gpu = pycuda.gpuarray.to_gpu(src_arr) src_arr_gpu = pycuda.gpuarray.to_gpu(src_arr)
dst_arr_gpu = pycuda.gpuarray.to_gpu(dst_arr) dst_arr_gpu = pycuda.gpuarray.to_gpu(dst_arr)
# run kernel on GPU, this is done many times in real setups # run kernel on GPU, this is done many times in real setups
kernel_function_gpu(src=src_arr_gpu, dst=dst_arr_gpu) kernel_function_gpu(src=src_arr_gpu, dst=dst_arr_gpu)
# transfer result back to CPU # transfer result back to CPU
dst_arr_gpu.get(dst_arr) dst_arr_gpu.get(dst_arr)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### 1.3. Summary: manual way ### 1.3. Summary: manual way
- Don't confuse *pystencils* fields and *numpy* arrays - Don't confuse *pystencils* fields and *numpy* arrays
- fields are symbolic - fields are symbolic
- arrays are numeric - arrays are numeric
- Use the fixed-field-size workflow whenever possible, since code might be faster. Create arrays first, then create fields from arrays - Use the fixed-field-size workflow whenever possible, since code might be faster. Create arrays first, then create fields from arrays
- if we run GPU kernels, arrays have to transferred to the GPU first - if we run GPU kernels, arrays have to transferred to the GPU first
As demonstrated in the examples above we have to define 2 or 3 corresponding objects for each grid: As demonstrated in the examples above we have to define 2 or 3 corresponding objects for each grid:
- symbolic pystencils field - symbolic pystencils field
- numpy array on CPU - numpy array on CPU
- for GPU run also a pycuda.gpuarray to mirror the data on the GPU - for GPU run also a pycuda.gpuarray to mirror the data on the GPU
Managing these three objects manually is tedious and error-prone. We'll see in the next section how the data handling object takes care of this problem. Managing these three objects manually is tedious and error-prone. We'll see in the next section how the data handling object takes care of this problem.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## 2. Introducing the data handling - serial version ## 2. Introducing the data handling - serial version
### 2.1. Example for CPU simulations ### 2.1. Example for CPU simulations
The data handling internally keeps a mapping between symbolic fields and numpy arrays. When we create a field, automatically a corresponding array is allocated as well. Optionally we can also allocate memory on the GPU for the array as well. Lets dive right in and see how our example looks like, when implemented with a data handling. The data handling internally keeps a mapping between symbolic fields and numpy arrays. When we create a field, automatically a corresponding array is allocated as well. Optionally we can also allocate memory on the GPU for the array as well. Lets dive right in and see how our example looks like, when implemented with a data handling.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
dh = ps.create_data_handling(domain_size=(30, 30)) dh = ps.create_data_handling(domain_size=(30, 30))
# fields are now created using the data handling # fields are now created using the data handling
src_field = dh.add_array('src', values_per_cell=1) src_field = dh.add_array('src', values_per_cell=1)
dst_field = dh.add_array('dst', values_per_cell=1) dst_field = dh.add_array('dst', values_per_cell=1)
# kernel is created just like before # kernel is created just like before
update_rule = [ps.Assignment(lhs=dst_field[0, 0], update_rule = [ps.Assignment(lhs=dst_field[0, 0],
rhs=(src_field[1, 0] + src_field[-1, 0] + src_field[0, 1] + src_field[0, -1]) / 4)] rhs=(src_field[1, 0] + src_field[-1, 0] + src_field[0, 1] + src_field[0, -1]) / 4)]
kernel_function = ps.create_kernel(update_rule).compile() kernel_function = ps.create_kernel(update_rule).compile()
# have a look at the generated code - it uses # have a look at the generated code - it uses
# the fast version where array sizes are compiled-in # the fast version where array sizes are compiled-in
# ps.show_code(kernel_function.ast) # ps.show_code(kernel_function.ast)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
The data handling has methods to create fields - but where are the corresponding arrays? The data handling has methods to create fields - but where are the corresponding arrays?
In the serial case you can access them as a member of the data handling, for example to initialize our 'src' array we can write In the serial case you can access them as a member of the data handling, for example to initialize our 'src' array we can write
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
src_arr = dh.cpu_arrays['src'] src_arr = dh.cpu_arrays['src']
src_arr.fill(0.0) src_arr.fill(0.0)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
This method is nice and easy, but you should not use it if you want your simulation to run on distributed-memory clusters. We'll see why in the last section. So it is good habit to not access the arrays directly but use the data handling to do so. We can, for example, initialize the array also with the following code: This method is nice and easy, but you should not use it if you want your simulation to run on distributed-memory clusters. We'll see why in the last section. So it is good habit to not access the arrays directly but use the data handling to do so. We can, for example, initialize the array also with the following code:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
dh.fill('src', 0.0) dh.fill('src', 0.0)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
To run the kernels with the same code as before, we would also need the arrays. We could do that accessing the `cpu_arrays`: To run the kernels with the same code as before, we would also need the arrays. We could do that accessing the `cpu_arrays`:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
kernel_function(src=dh.cpu_arrays['src'], kernel_function(src=dh.cpu_arrays['src'],
dst=dh.cpu_arrays['dst']) dst=dh.cpu_arrays['dst'])
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
but to be prepared for MPI parallel simulations, again a method of the data handling should be used for this. but to be prepared for MPI parallel simulations, again a method of the data handling should be used for this.
Besides, this method is also simpler to use - since it automatically detects which arrays a kernel uses and passes them in. Besides, this method is also simpler to use - since it automatically detects which arrays a kernel uses and passes them in.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
dh.run_kernel(kernel_function) dh.run_kernel(kernel_function)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
To access the data read-only instead of using `cpu_arrays` the gather function should be used. To access the data read-only instead of using `cpu_arrays` the gather function should be used.
This function gives you a read-only copy of the domain or part of the domain. This function gives you a read-only copy of the domain or part of the domain.
We will discuss this function later in more detail when we look at MPI parallel simulations. We will discuss this function later in more detail when we look at MPI parallel simulations.
For serial simulations keep in mind that modifying the resulting array does not change your original data! For serial simulations keep in mind that modifying the resulting array does not change your original data!
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
read_only_copy = dh.gather_array('src', ps.make_slice[:, :], ghost_layers=False) read_only_copy = dh.gather_array('src', ps.make_slice[:, :], ghost_layers=False)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### 2.2. Example for GPU simulations ### 2.2. Example for GPU simulations
In this section we have a look at GPU simulations using the data handling. Only minimal changes are required. In this section we have a look at GPU simulations using the data handling. Only minimal changes are required.
When creating the data handling we can pass a 'default_target'. This means for every added field an array on the CPU and the GPU is allocated. This is a useful default, for more fine-grained control the `add_array` method also takes additional parameters controlling where the array should be allocated. When creating the data handling we can pass a 'default_target'. This means for every added field an array on the CPU and the GPU is allocated. This is a useful default, for more fine-grained control the `add_array` method also takes additional parameters controlling where the array should be allocated.
Additionally we also need to compile a GPU version of the kernel. Additionally we also need to compile a GPU version of the kernel.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
dh = ps.create_data_handling(domain_size=(30, 30), default_target='gpu') dh = ps.create_data_handling(domain_size=(30, 30), default_target=ps.Target.GPU)
# fields are now created using the data handling # fields are now created using the data handling
src_field = dh.add_array('src', values_per_cell=1) src_field = dh.add_array('src', values_per_cell=1)
dst_field = dh.add_array('dst', values_per_cell=1) dst_field = dh.add_array('dst', values_per_cell=1)
# kernel is created just like before # kernel is created just like before
update_rule = [ps.Assignment(lhs=dst_field[0, 0], update_rule = [ps.Assignment(lhs=dst_field[0, 0],
rhs=(src_field[1, 0] + src_field[-1, 0] + src_field[0, 1] + src_field[0, -1]) / 4)] rhs=(src_field[1, 0] + src_field[-1, 0] + src_field[0, 1] + src_field[0, -1]) / 4)]
kernel_function = ps.create_kernel(update_rule, target=dh.default_target).compile() config = ps.CreateKernelConfig(target=dh.default_target)
kernel_function = ps.create_kernel(update_rule, config=config).compile()
dh.fill('src', 0.0) dh.fill('src', 0.0)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
The data handling provides function to transfer data between CPU and GPU The data handling provides function to transfer data between CPU and GPU
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
dh.to_gpu('src') dh.to_gpu('src')
dh.run_kernel(kernel_function) dh.run_kernel(kernel_function)
dh.to_cpu('dst') dh.to_cpu('dst')
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
usually one wants to transfer all fields that have been allocated on CPU and GPU at the same time: usually one wants to transfer all fields that have been allocated on CPU and GPU at the same time:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
dh.all_to_gpu() dh.all_to_gpu()
dh.run_kernel(kernel_function) dh.run_kernel(kernel_function)
dh.all_to_cpu() dh.all_to_cpu()
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
We can always include the `all_to_*` functions in our code, since they do nothing if there are no arrays allocated on the GPU. Thus there is only a single point in the code where we can switch between CPU and GPU version: the `default_target` of the data handling. We can always include the `all_to_*` functions in our code, since they do nothing if there are no arrays allocated on the GPU. Thus there is only a single point in the code where we can switch between CPU and GPU version: the `default_target` of the data handling.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### 2.2 Ghost Layers and periodicity ### 2.2 Ghost Layers and periodicity
The data handling can also provide periodic boundary conditions. Therefor the domain is extended by one layer of cells, the so-called ghost layer or halo layer. The data handling can also provide periodic boundary conditions. Therefor the domain is extended by one layer of cells, the so-called ghost layer or halo layer.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
print("Shape of domain ", dh.shape) print("Shape of domain ", dh.shape)
print("Direct access to arrays ", dh.cpu_arrays['src'].shape) print("Direct access to arrays ", dh.cpu_arrays['src'].shape)
print("Gather ", dh.gather_array('src', ghost_layers=True).shape) print("Gather ", dh.gather_array('src', ghost_layers=True).shape)
``` ```
%% Output %% Output
Shape of domain (30, 30) Shape of domain (30, 30)
Direct access to arrays (32, 32) Direct access to arrays (32, 32)
Gather (32, 32) Gather (32, 32)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
So the actual arrays are 2 cells larger than what you asked for. This additional layer is used to copy over the data from the other end of the array, such that for the stencil algorithm effectively the domain is periodic. This copying operation has to be started manually though: So the actual arrays are 2 cells larger than what you asked for. This additional layer is used to copy over the data from the other end of the array, such that for the stencil algorithm effectively the domain is periodic. This copying operation has to be started manually though:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
dh = ps.create_data_handling((4, 4), periodicity=(True, False)) dh = ps.create_data_handling((4, 4), periodicity=(True, False))
dh.add_array("src") dh.add_array("src")
# get copy function # get copy function
copy_fct = dh.synchronization_function(['src']) copy_fct = dh.synchronization_function(['src'])
dh.fill('src', 0.0, ghost_layers=True) dh.fill('src', 0.0, ghost_layers=True)
dh.fill('src', 3.0, ghost_layers=False) dh.fill('src', 3.0, ghost_layers=False)
before_sync = dh.gather_array('src', ghost_layers=True).copy() before_sync = dh.gather_array('src', ghost_layers=True).copy()
copy_fct() # copy over to get periodicity in x direction copy_fct() # copy over to get periodicity in x direction
after_sync = dh.gather_array('src', ghost_layers=True).copy() after_sync = dh.gather_array('src', ghost_layers=True).copy()
plt.subplot(1,2,1) plt.subplot(1,2,1)
plt.scalar_field(before_sync); plt.scalar_field(before_sync);
plt.title("Before") plt.title("Before")
plt.subplot(1,2,2) plt.subplot(1,2,2)
plt.scalar_field(after_sync); plt.scalar_field(after_sync);
plt.title("After"); plt.title("After");
``` ```
%% Output %% Output
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## 3. Going (MPI) parallel - the parallel data handling ## 3. Going (MPI) parallel - the parallel data handling
### 3.1. Conceptual overview ### 3.1. Conceptual overview
To run MPI parallel simulations the waLBerla framework is used. waLBerla has to be compiled against your local MPI library and thus is a bit hard to install. We suggest to use the version shipped with conda for testing. For production, when you want to run on a cluster the best option is to build waLBerla yourself against the MPI library that is installed on your cluster. To run MPI parallel simulations the waLBerla framework is used. waLBerla has to be compiled against your local MPI library and thus is a bit hard to install. We suggest to use the version shipped with conda for testing. For production, when you want to run on a cluster the best option is to build waLBerla yourself against the MPI library that is installed on your cluster.
Now lets have a look on how to write code that runs MPI parallel. Now lets have a look on how to write code that runs MPI parallel.
Since the data is distributed, we don't have access to the full array any more but only to the part that is stored locally. The domain is split into so called blocks, where one process might get one (or sometimes multiple) blocks. To do anything with the local part of the data we iterate over the **local** blocks to get the contents as numpy arrays. The blocks returned in the loop differ from process to process. Since the data is distributed, we don't have access to the full array any more but only to the part that is stored locally. The domain is split into so called blocks, where one process might get one (or sometimes multiple) blocks. To do anything with the local part of the data we iterate over the **local** blocks to get the contents as numpy arrays. The blocks returned in the loop differ from process to process.
Copy the following snippet to a python file and run with multiple processes e.g.: Copy the following snippet to a python file and run with multiple processes e.g.:
``mpirun -np 4 myscript.py`` you will see that there are as many blocks as processes. ``mpirun -np 4 myscript.py`` you will see that there are as many blocks as processes.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
dh = ps.create_data_handling(domain_size=(30, 30), parallel=True) dh = ps.create_data_handling(domain_size=(30, 30), parallel=True)
field = dh.add_array('field') field = dh.add_array('field')
for block in dh.iterate(): for block in dh.iterate():
# offset is in global coordinates, where first inner cell has coordiante (0,0) # offset is in global coordinates, where first inner cell has coordiante (0,0)
# and ghost layers have negative coordinates # and ghost layers have negative coordinates
print(block.offset, block['field'].shape) print(block.offset, block['field'].shape)
# use offset to create a local array 'my_data' for the part of the domain # use offset to create a local array 'my_data' for the part of the domain
#np.copyto(block[field.name], my_data) #np.copyto(block[field.name], my_data)
``` ```
%% Output %% Output
(-1, -1) (32, 32) ---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-20-779cf1a58f81> in <module>
----> 1 dh = ps.create_data_handling(domain_size=(30, 30), parallel=True)
2 field = dh.add_array('field')
3 for block in dh.iterate():
4 # offset is in global coordinates, where first inner cell has coordiante (0,0)
5 # and ghost layers have negative coordinates
~/git/pystencils/pystencils/datahandling/__init__.py in create_data_handling(domain_size, periodicity, default_layout, default_target, parallel, default_ghost_layers, opencl_queue)
38 assert not opencl_queue, "OpenCL is only supported for SerialDataHandling"
39 if wlb is None:
---> 40 raise ValueError("Cannot create parallel data handling because walberla module is not available")
41
42 if periodicity is False or periodicity is None:
ValueError: Cannot create parallel data handling because walberla module is not available
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
To get some more interesting results here in the notebook we put multiple blocks onto our single notebook process. This makes not much sense for real simulations, but for testing and demonstration purposes this is useful. To get some more interesting results here in the notebook we put multiple blocks onto our single notebook process. This makes not much sense for real simulations, but for testing and demonstration purposes this is useful.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
from waLBerla import createUniformBlockGrid from waLBerla import createUniformBlockGrid
from pystencils.datahandling import ParallelDataHandling from pystencils.datahandling import ParallelDataHandling
blocks = createUniformBlockGrid(blocks=(2,1,2), cellsPerBlock=(20, 10, 20), blocks = createUniformBlockGrid(blocks=(2,1,2), cellsPerBlock=(20, 10, 20),
oneBlockPerProcess=False, periodic=(1, 0, 0)) oneBlockPerProcess=False, periodic=(1, 0, 0))
dh = ParallelDataHandling(blocks) dh = ParallelDataHandling(blocks)
field = dh.add_array('field') field = dh.add_array('field')
for block in dh.iterate(): for block in dh.iterate():
print(block.offset, block['field'].shape) print(block.offset, block['field'].shape)
``` ```
%% Output
(-1, -1, -1) (22, 12, 22)
(19, -1, -1) (22, 12, 22)
(-1, -1, 19) (22, 12, 22)
(19, -1, 19) (22, 12, 22)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Now we see that we have four blocks with (20, 10, 20) block each, and the global domain is (40, 10, 40) big. Now we see that we have four blocks with (20, 10, 20) block each, and the global domain is (40, 10, 40) big.
All subblock also have a ghost layer around them, which is used to synchronize with their neighboring blocks (over the network). For ghost layer synchronization the same `synchronization_function` is used that we used above for periodic boundaries, because copying between blocks and copying the ghost layer for periodicity uses the same mechanism. All subblock also have a ghost layer around them, which is used to synchronize with their neighboring blocks (over the network). For ghost layer synchronization the same `synchronization_function` is used that we used above for periodic boundaries, because copying between blocks and copying the ghost layer for periodicity uses the same mechanism.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
dh.gather_array('field').shape dh.gather_array('field').shape
``` ```
%% Output
$\displaystyle \left( 40, \ 10, \ 40\right)$
(40, 10, 40)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### 3.2. Parallel example ### 3.2. Parallel example
To illustrate this in more detail, lets run a simple kernel on a parallel domain. waLBerla can handle 3D domains only, so we choose a z-size of 1. To illustrate this in more detail, lets run a simple kernel on a parallel domain. waLBerla can handle 3D domains only, so we choose a z-size of 1.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
blocks = createUniformBlockGrid(blocks=(2,4,1), cellsPerBlock=(20, 10, 1), blocks = createUniformBlockGrid(blocks=(2,4,1), cellsPerBlock=(20, 10, 1),
oneBlockPerProcess=False, periodic=(1, 1, 0)) oneBlockPerProcess=False, periodic=(1, 1, 0))
dh = ParallelDataHandling(blocks, dim=2) dh = ParallelDataHandling(blocks, dim=2)
src_field = dh.add_array('src') src_field = dh.add_array('src')
dst_field = dh.add_array('dst') dst_field = dh.add_array('dst')
update_rule = [ps.Assignment(lhs=dst_field[0, 0 ], update_rule = [ps.Assignment(lhs=dst_field[0, 0 ],
rhs=(src_field[1, 0] + src_field[-1, 0] + rhs=(src_field[1, 0] + src_field[-1, 0] +
src_field[0, 1] + src_field[0, -1]) / 4)] src_field[0, 1] + src_field[0, -1]) / 4)]
# 3. compile update rule to function # 3. compile update rule to function
kernel_function = ps.create_kernel(update_rule).compile() kernel_function = ps.create_kernel(update_rule).compile()
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Now lets initialize the arrays. To do this we can get arrays (meshgrids) from the block with the coordinates of the local cells. We use this to initialize a circular shape. Now lets initialize the arrays. To do this we can get arrays (meshgrids) from the block with the coordinates of the local cells. We use this to initialize a circular shape.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
dh.fill('src', 0.0) dh.fill('src', 0.0)
for block in dh.iterate(ghost_layers=False, inner_ghost_layers=False): for block in dh.iterate(ghost_layers=False, inner_ghost_layers=False):
x, y = block.midpoint_arrays x, y = block.midpoint_arrays
inside_sphere = (x -20)**2 + (y-25)**2 < 8 ** 2 inside_sphere = (x -20)**2 + (y-25)**2 < 8 ** 2
block['src'][inside_sphere] = 1.0 block['src'][inside_sphere] = 1.0
plt.scalar_field( dh.gather_array('src') ); plt.scalar_field( dh.gather_array('src') );
``` ```
%% Output
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Now we can run our compute kernel on this data as usual. We just have to make sure that the field is synchronized after every step, i.e. that the ghost layers are correctly updated. Now we can run our compute kernel on this data as usual. We just have to make sure that the field is synchronized after every step, i.e. that the ghost layers are correctly updated.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
sync = dh.synchronization_function(['src']) sync = dh.synchronization_function(['src'])
for i in range(40): for i in range(40):
sync() sync()
dh.run_kernel(kernel_function) dh.run_kernel(kernel_function)
dh.swap('src', 'dst') dh.swap('src', 'dst')
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
plt.scalar_field( dh.gather_array('src') ); plt.scalar_field( dh.gather_array('src') );
``` ```
%% Output
......
%% Cell type:code id: tags: %% Cell type:code id: tags:
   
``` python ``` python
from pystencils.session import * from pystencils.session import *
import pystencils as ps
sp.init_printing() sp.init_printing()
frac = sp.Rational frac = sp.Rational
``` ```
   
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
   
# Tutorial 06: Phase-field simulation of dentritic solidification # Tutorial 06: Phase-field simulation of dentritic solidification
   
This is the second tutorial on phase field methods with pystencils. Make sure to read the previous tutorial first. This is the second tutorial on phase field methods with pystencils. Make sure to read the previous tutorial first.
   
In this tutorial we again implement a model described in **Programming Phase-Field Modelling** by S. Bulent Biner. In this tutorial we again implement a model described in **Programming Phase-Field Modelling** by S. Bulent Biner.
This time we implement the model from chapter 4.7 that describes dentritic growth. So get ready for some beautiful snowflake pictures. This time we implement the model from chapter 4.7 that describes dentritic growth. So get ready for some beautiful snowflake pictures.
   
We start again by adding all required arrays fields. This time we explicitly store the change of the phase variable φ in time, since the dynamics is calculated using an Allen-Cahn formulation where a term $\partial_t \phi$ occurs. We start again by adding all required arrays fields. This time we explicitly store the change of the phase variable φ in time, since the dynamics is calculated using an Allen-Cahn formulation where a term $\partial_t \phi$ occurs.
   
%% Cell type:code id: tags: %% Cell type:code id: tags:
   
``` python ``` python
dh = ps.create_data_handling(domain_size=(300, 300), periodicity=True, dh = ps.create_data_handling(domain_size=(300, 300), periodicity=True,
default_target='cpu') default_target=ps.Target.CPU)
φ_field = dh.add_array('phi', latex_name='φ') φ_field = dh.add_array('phi', latex_name='φ')
φ_field_tmp = dh.add_array('phi_temp', latex_name='φ_temp') φ_field_tmp = dh.add_array('phi_temp', latex_name='φ_temp')
φ_delta_field = dh.add_array('phidelta', latex_name='φ_D') φ_delta_field = dh.add_array('phidelta', latex_name='φ_D')
t_field = dh.add_array('T') t_field = dh.add_array('T')
``` ```
   
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
   
This model has a lot of parameters that are created here in a symbolic fashion. This model has a lot of parameters that are created here in a symbolic fashion.
   
%% Cell type:code id: tags: %% Cell type:code id: tags:
   
``` python ``` python
ε, m, δ, j, θzero, α, γ, Teq, κ, τ = sp.symbols("ε m δ j θ_0 α γ T_eq κ τ") ε, m, δ, j, θzero, α, γ, Teq, κ, τ = sp.symbols("ε m δ j θ_0 α γ T_eq κ τ")
εb = sp.Symbol("\\bar{\\epsilon}") εb = sp.Symbol("\\bar{\\epsilon}")
   
φ = φ_field.center φ = φ_field.center
φ_tmp = φ_field_tmp.center φ_tmp = φ_field_tmp.center
T = t_field.center T = t_field.center
   
def f(φ, m): def f(φ, m):
return φ**4 / 4 - (frac(1, 2) - m/3) * φ**3 + (frac(1,4)-m/2)*φ**2 return φ**4 / 4 - (frac(1, 2) - m/3) * φ**3 + (frac(1,4)-m/2)*φ**2
   
free_energy_density = ε**2 / 2 * (ps.fd.Diff(φ,0)**2 + ps.fd.Diff(φ,1)**2 ) + f(φ, m) free_energy_density = ε**2 / 2 * (ps.fd.Diff(φ,0)**2 + ps.fd.Diff(φ,1)**2 ) + f(φ, m)
free_energy_density free_energy_density
``` ```
   
%% Output %% Output
   
$\displaystyle \frac{{{φ}_{(0,0)}}^{4}}{4} - {{φ}_{(0,0)}}^{3} \left(\frac{1}{2} - \frac{m}{3}\right) + {{φ}_{(0,0)}}^{2} \left(\frac{1}{4} - \frac{m}{2}\right) + \frac{ε^{2} \left({\partial_{0} {{φ}_{(0,0)}}}^{2} + {\partial_{1} {{φ}_{(0,0)}}}^{2}\right)}{2}$ $\displaystyle \frac{{{φ}_{(0,0)}}^{4}}{4} - {{φ}_{(0,0)}}^{3} \left(\frac{1}{2} - \frac{m}{3}\right) + {{φ}_{(0,0)}}^{2} \left(\frac{1}{4} - \frac{m}{2}\right) + \frac{ε^{2} \left({\partial_{0} {{φ}_{(0,0)}}}^{2} + {\partial_{1} {{φ}_{(0,0)}}}^{2}\right)}{2}$
4 2 ⎛ 2 2⎞ 4 2 ⎛ 2 2⎞
φ_C 3 ⎛1 m⎞ 2 ⎛1 m⎞ ε ⋅⎝D(φ[0,0]) + D(φ[0,0]) ⎠ φ_C 3 ⎛1 m⎞ 2 ⎛1 m⎞ ε ⋅⎝D(φ[0,0]) + D(φ[0,0]) ⎠
──── - φ_C ⋅⎜─ - ─⎟ + φ_C ⋅⎜─ - ─⎟ + ──────────────────────────── ──── - φ_C ⋅⎜─ - ─⎟ + φ_C ⋅⎜─ - ─⎟ + ────────────────────────────
4 ⎝2 3⎠ ⎝4 2⎠ 2 4 ⎝2 3⎠ ⎝4 2⎠ 2
   
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
   
The free energy is again composed of a bulk and interface part. The free energy is again composed of a bulk and interface part.
   
%% Cell type:code id: tags: %% Cell type:code id: tags:
   
``` python ``` python
plt.figure(figsize=(7,4)) plt.figure(figsize=(7,4))
plt.sympy_function(f(φ, m=1), x_values=(-1.05, 1.5) ) plt.sympy_function(f(φ, m=1), x_values=(-1.05, 1.5) )
plt.xlabel("φ") plt.xlabel("φ")
plt.title("Bulk free energy"); plt.title("Bulk free energy");
``` ```
   
%% Output %% Output
   
   
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
   
Compared to last tutorial, this bulk free energy has also two minima, but at different values. Compared to last tutorial, this bulk free energy has also two minima, but at different values.
   
Another complexity of the model is its anisotropy. The gradient parameter $\epsilon$ depends on the interface normal. Another complexity of the model is its anisotropy. The gradient parameter $\epsilon$ depends on the interface normal.
   
%% Cell type:code id: tags: %% Cell type:code id: tags:
   
``` python ``` python
def σ(θ): def σ(θ):
return 1 + δ * sp.cos(j * (θ - θzero)) return 1 + δ * sp.cos(j * (θ - θzero))
   
θ = sp.atan2(ps.fd.Diff(φ, 1), ps.fd.Diff(φ, 0)) θ = sp.atan2(ps.fd.Diff(φ, 1), ps.fd.Diff(φ, 0))
   
ε_val = εb * σ(θ) ε_val = εb * σ(θ)
ε_val ε_val
``` ```
   
%% Output %% Output
   
$\displaystyle \bar{\epsilon} \left(δ \cos{\left(j \left(- θ_{0} + \operatorname{atan_{2}}{\left({\partial_{1} {{φ}_{(0,0)}}},{\partial_{0} {{φ}_{(0,0)}}} \right)}\right) \right)} + 1\right)$ $\displaystyle \bar{\epsilon} \left(δ \cos{\left(j \left(- θ_{0} + \operatorname{atan_{2}}{\left({\partial_{1} {{φ}_{(0,0)}}},{\partial_{0} {{φ}_{(0,0)}}} \right)}\right) \right)} + 1\right)$
\bar{\epsilon}⋅(δ⋅cos(j⋅(-θ₀ + atan2(D(φ[0,0]), D(φ[0,0])))) + 1) \bar{\epsilon}⋅(δ⋅cos(j⋅(-θ₀ + atan2(D(φ[0,0]), D(φ[0,0])))) + 1)
   
%% Cell type:code id: tags: %% Cell type:code id: tags:
   
``` python ``` python
def m_func(T): def m_func(T):
return (α / sp.pi) * sp.atan(γ * (Teq - T)) return (α / sp.pi) * sp.atan(γ * (Teq - T))
``` ```
   
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
   
However, we just insert these parameters into the free energy formulation before doing the functional derivative, to make the dependence of $\epsilon$ on $\nabla \phi$ explicit. However, we just insert these parameters into the free energy formulation before doing the functional derivative, to make the dependence of $\epsilon$ on $\nabla \phi$ explicit.
   
%% Cell type:code id: tags: %% Cell type:code id: tags:
   
``` python ``` python
fe = free_energy_density.subs({ fe = free_energy_density.subs({
m: m_func(T), m: m_func(T),
ε: εb * σ(θ), ε: εb * σ(θ),
}) })
   
dF_dφ = ps.fd.functional_derivative(fe, φ) dF_dφ = ps.fd.functional_derivative(fe, φ)
dF_dφ = ps.fd.expand_diff_full(dF_dφ, functions=[φ]) dF_dφ = ps.fd.expand_diff_full(dF_dφ, functions=[φ])
dF_dφ dF_dφ
``` ```
   
%% Output %% Output
   
$\displaystyle {{φ}_{(0,0)}}^{3} - \frac{{{φ}_{(0,0)}}^{2} α \operatorname{atan}{\left({{T}_{(0,0)}} γ - T_{eq} γ \right)}}{\pi} - \frac{3 {{φ}_{(0,0)}}^{2}}{2} + \frac{{{φ}_{(0,0)}} α \operatorname{atan}{\left({{T}_{(0,0)}} γ - T_{eq} γ \right)}}{\pi} + \frac{{{φ}_{(0,0)}}}{2} - \bar{\epsilon}^{2} δ^{2} \cos^{2}{\left(j θ_{0} - j \operatorname{atan_{2}}{\left({\partial_{1} {{φ}_{(0,0)}}},{\partial_{0} {{φ}_{(0,0)}}} \right)} \right)} {\partial_{0} {\partial_{0} {{φ}_{(0,0)}}}} - \bar{\epsilon}^{2} δ^{2} \cos^{2}{\left(j θ_{0} - j \operatorname{atan_{2}}{\left({\partial_{1} {{φ}_{(0,0)}}},{\partial_{0} {{φ}_{(0,0)}}} \right)} \right)} {\partial_{1} {\partial_{1} {{φ}_{(0,0)}}}} - 2 \bar{\epsilon}^{2} δ \cos{\left(j θ_{0} - j \operatorname{atan_{2}}{\left({\partial_{1} {{φ}_{(0,0)}}},{\partial_{0} {{φ}_{(0,0)}}} \right)} \right)} {\partial_{0} {\partial_{0} {{φ}_{(0,0)}}}} - 2 \bar{\epsilon}^{2} δ \cos{\left(j θ_{0} - j \operatorname{atan_{2}}{\left({\partial_{1} {{φ}_{(0,0)}}},{\partial_{0} {{φ}_{(0,0)}}} \right)} \right)} {\partial_{1} {\partial_{1} {{φ}_{(0,0)}}}} - \bar{\epsilon}^{2} {\partial_{0} {\partial_{0} {{φ}_{(0,0)}}}} - \bar{\epsilon}^{2} {\partial_{1} {\partial_{1} {{φ}_{(0,0)}}}}$ $\displaystyle {{φ}_{(0,0)}}^{3} - \frac{{{φ}_{(0,0)}}^{2} α \operatorname{atan}{\left({{T}_{(0,0)}} γ - T_{eq} γ \right)}}{\pi} - \frac{3 {{φ}_{(0,0)}}^{2}}{2} + \frac{{{φ}_{(0,0)}} α \operatorname{atan}{\left({{T}_{(0,0)}} γ - T_{eq} γ \right)}}{\pi} + \frac{{{φ}_{(0,0)}}}{2} - \bar{\epsilon}^{2} δ^{2} \cos^{2}{\left(j θ_{0} - j \operatorname{atan_{2}}{\left({\partial_{1} {{φ}_{(0,0)}}},{\partial_{0} {{φ}_{(0,0)}}} \right)} \right)} {\partial_{0} {\partial_{0} {{φ}_{(0,0)}}}} - \bar{\epsilon}^{2} δ^{2} \cos^{2}{\left(j θ_{0} - j \operatorname{atan_{2}}{\left({\partial_{1} {{φ}_{(0,0)}}},{\partial_{0} {{φ}_{(0,0)}}} \right)} \right)} {\partial_{1} {\partial_{1} {{φ}_{(0,0)}}}} - 2 \bar{\epsilon}^{2} δ \cos{\left(j θ_{0} - j \operatorname{atan_{2}}{\left({\partial_{1} {{φ}_{(0,0)}}},{\partial_{0} {{φ}_{(0,0)}}} \right)} \right)} {\partial_{0} {\partial_{0} {{φ}_{(0,0)}}}} - 2 \bar{\epsilon}^{2} δ \cos{\left(j θ_{0} - j \operatorname{atan_{2}}{\left({\partial_{1} {{φ}_{(0,0)}}},{\partial_{0} {{φ}_{(0,0)}}} \right)} \right)} {\partial_{1} {\partial_{1} {{φ}_{(0,0)}}}} - \bar{\epsilon}^{2} {\partial_{0} {\partial_{0} {{φ}_{(0,0)}}}} - \bar{\epsilon}^{2} {\partial_{1} {\partial_{1} {{φ}_{(0,0)}}}}$
2 2 2 2
3 φ_C ⋅α⋅atan(T_C⋅γ - T_eq⋅γ) 3⋅φ_C φ_C⋅α⋅atan(T_C⋅γ - T_eq⋅γ) φ_C 3 φ_C ⋅α⋅atan(T_C⋅γ - T_eq⋅γ) 3⋅φ_C φ_C⋅α⋅atan(T_C⋅γ - T_eq⋅γ) φ_C
φ_C - ─────────────────────────── - ────── + ────────────────────────── + ─── φ_C - ─────────────────────────── - ────── + ────────────────────────── + ───
π 2 π 2 π 2 π 2
2 2 2 2 2 2
- \bar{\epsilon} ⋅δ ⋅cos (j⋅θ₀ - j⋅atan2(D(φ[0,0]), D(φ[0,0])))⋅D(D(φ[0,0])) - \bar{\epsilon} ⋅δ ⋅cos (j⋅θ₀ - j⋅atan2(D(φ[0,0]), D(φ[0,0])))⋅D(D(φ[0,0]))
2 2 2 2 2 2
- \bar{\epsilon} ⋅δ ⋅cos (j⋅θ₀ - j⋅atan2(D(φ[0,0]), D(φ[0,0])))⋅D(D(φ[0,0])) - - \bar{\epsilon} ⋅δ ⋅cos (j⋅θ₀ - j⋅atan2(D(φ[0,0]), D(φ[0,0])))⋅D(D(φ[0,0])) -
2 2
2⋅\bar{\epsilon} ⋅δ⋅cos(j⋅θ₀ - j⋅atan2(D(φ[0,0]), D(φ[0,0])))⋅D(D(φ[0,0])) - 2⋅\bar{\epsilon} ⋅δ⋅cos(j⋅θ₀ - j⋅atan2(D(φ[0,0]), D(φ[0,0])))⋅D(D(φ[0,0])) -
2 2
2⋅\bar{\epsilon} ⋅δ⋅cos(j⋅θ₀ - j⋅atan2(D(φ[0,0]), D(φ[0,0])))⋅D(D(φ[0,0])) - \ 2⋅\bar{\epsilon} ⋅δ⋅cos(j⋅θ₀ - j⋅atan2(D(φ[0,0]), D(φ[0,0])))⋅D(D(φ[0,0])) - \
2 2 2 2
bar{\epsilon} ⋅D(D(φ[0,0])) - \bar{\epsilon} ⋅D(D(φ[0,0])) bar{\epsilon} ⋅D(D(φ[0,0])) - \bar{\epsilon} ⋅D(D(φ[0,0]))
   
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
   
Then we insert all the numeric parameters and discretize: Then we insert all the numeric parameters and discretize:
   
%% Cell type:code id: tags: %% Cell type:code id: tags:
   
``` python ``` python
discretize = ps.fd.Discretization2ndOrder(dx=0.03, dt=1e-5) discretize = ps.fd.Discretization2ndOrder(dx=0.03, dt=1e-5)
parameters = { parameters = {
τ: 0.0003, τ: 0.0003,
κ: 1.8, κ: 1.8,
εb: 0.01, εb: 0.01,
δ: 0.02, δ: 0.02,
γ: 10, γ: 10,
j: 6, j: 6,
α: 0.9, α: 0.9,
Teq: 1.0, Teq: 1.0,
θzero: 0.2, θzero: 0.2,
sp.pi: sp.pi.evalf() sp.pi: sp.pi.evalf()
} }
parameters parameters
``` ```
   
%% Output %% Output
   
$\displaystyle \left\{ \pi : 3.14159265358979, \ T_{eq} : 1.0, \ \bar{\epsilon} : 0.01, \ j : 6, \ α : 0.9, \ γ : 10, \ δ : 0.02, \ θ_{0} : 0.2, \ κ : 1.8, \ τ : 0.0003\right\}$ $\displaystyle \left\{ \pi : 3.14159265358979, \ T_{eq} : 1.0, \ \bar{\epsilon} : 0.01, \ j : 6, \ α : 0.9, \ γ : 10, \ δ : 0.02, \ θ_{0} : 0.2, \ κ : 1.8, \ τ : 0.0003\right\}$
{π: 3.14159265358979, T_eq: 1.0, \bar{\epsilon}: 0.01, j: 6, α: 0.9, γ: 10, δ: {π: 3.14159265358979, T_eq: 1.0, \bar{\epsilon}: 0.01, j: 6, α: 0.9, γ: 10, δ:
0.02, θ₀: 0.2, κ: 1.8, τ: 0.0003} 0.02, θ₀: 0.2, κ: 1.8, τ: 0.0003}
   
%% Cell type:code id: tags: %% Cell type:code id: tags:
   
``` python ``` python
dφ_dt = - dF_dφ / τ dφ_dt = - dF_dφ / τ
φ_eqs = ps.simp.sympy_cse_on_assignment_list([ps.Assignment(φ_delta_field.center, φ_eqs = ps.simp.sympy_cse_on_assignment_list([ps.Assignment(φ_delta_field.center,
discretize(dφ_dt.subs(parameters)))]) discretize(dφ_dt.subs(parameters)))])
φ_eqs.append(ps.Assignment(φ_tmp, discretize(ps.fd.transient(φ) - φ_delta_field.center))) φ_eqs.append(ps.Assignment(φ_tmp, discretize(ps.fd.transient(φ) - φ_delta_field.center)))
   
temperature_evolution = -ps.fd.transient(T) + ps.fd.diffusion(T, 1) + κ * φ_delta_field.center temperature_evolution = -ps.fd.transient(T) + ps.fd.diffusion(T, 1) + κ * φ_delta_field.center
temperature_eqs = [ temperature_eqs = [
ps.Assignment(T, discretize(temperature_evolution.subs(parameters))) ps.Assignment(T, discretize(temperature_evolution.subs(parameters)))
] ]
``` ```
   
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
   
When creating the kernels we pass as target (which may be 'cpu' or 'gpu') the default target of the target handling. This enables to switch to a GPU simulation just by changing the parameter of the data handling. When creating the kernels we pass as target (which may be 'Target.CPU' or 'Target.GPU') the default target of the target handling. This enables to switch to a GPU simulation just by changing the parameter of the data handling.
   
The rest is similar to the previous tutorial. The rest is similar to the previous tutorial.
   
%% Cell type:code id: tags: %% Cell type:code id: tags:
   
``` python ``` python
φ_kernel = ps.create_kernel(φ_eqs, cpu_openmp=4, target=dh.default_target).compile() φ_kernel = ps.create_kernel(φ_eqs, cpu_openmp=4, target=dh.default_target).compile()
temperature_kernel = ps.create_kernel(temperature_eqs, target=dh.default_target).compile() temperature_kernel = ps.create_kernel(temperature_eqs, target=dh.default_target).compile()
``` ```
   
%% Cell type:code id: tags: %% Cell type:code id: tags:
   
``` python ``` python
def timeloop(steps=200): def timeloop(steps=200):
φ_sync = dh.synchronization_function(['phi']) φ_sync = dh.synchronization_function(['phi'])
temperature_sync = dh.synchronization_function(['T']) temperature_sync = dh.synchronization_function(['T'])
dh.all_to_gpu() # this does nothing when running on CPU dh.all_to_gpu() # this does nothing when running on CPU
for t in range(steps): for t in range(steps):
φ_sync() φ_sync()
dh.run_kernel(φ_kernel) dh.run_kernel(φ_kernel)
dh.swap('phi', 'phi_temp') dh.swap('phi', 'phi_temp')
temperature_sync() temperature_sync()
dh.run_kernel(temperature_kernel) dh.run_kernel(temperature_kernel)
dh.all_to_cpu() dh.all_to_cpu()
return dh.gather_array('phi') return dh.gather_array('phi')
   
def init(nucleus_size=np.sqrt(5)): def init(nucleus_size=np.sqrt(5)):
for b in dh.iterate(): for b in dh.iterate():
x, y = b.cell_index_arrays x, y = b.cell_index_arrays
x, y = x-b.shape[0]//2, y-b.shape[0]//2 x, y = x-b.shape[0]//2, y-b.shape[0]//2
bArr = (x**2 + y**2) < nucleus_size**2 bArr = (x**2 + y**2) < nucleus_size**2
b['phi'].fill(0) b['phi'].fill(0)
b['phi'][(x**2 + y**2) < nucleus_size**2] = 1.0 b['phi'][(x**2 + y**2) < nucleus_size**2] = 1.0
b['T'].fill(0.0) b['T'].fill(0.0)
   
def plot(): def plot():
plt.subplot(1,3,1) plt.subplot(1,3,1)
plt.scalar_field(dh.gather_array('phi')) plt.scalar_field(dh.gather_array('phi'))
plt.title("φ") plt.title("φ")
plt.colorbar() plt.colorbar()
plt.subplot(1,3,2) plt.subplot(1,3,2)
plt.title("T") plt.title("T")
plt.scalar_field(dh.gather_array('T')) plt.scalar_field(dh.gather_array('T'))
plt.colorbar() plt.colorbar()
plt.subplot(1,3,3) plt.subplot(1,3,3)
plt.title("∂φ") plt.title("∂φ")
plt.scalar_field(dh.gather_array('phidelta')) plt.scalar_field(dh.gather_array('phidelta'))
plt.colorbar() plt.colorbar()
``` ```
   
%% Cell type:code id: tags: %% Cell type:code id: tags:
   
``` python ``` python
timeloop(10) timeloop(10)
init() init()
plot() plot()
print(dh) print(dh)
``` ```
   
%% Output %% Output
   
Name| Inner (min/max)| WithGl (min/max) Name| Inner (min/max)| WithGl (min/max)
---------------------------------------------------- ----------------------------------------------------
T| ( 0, 0)| ( 0, 0) T| ( 0, 0)| ( 0, 0)
phi| ( 0, 1)| ( 0, 1) phi| ( 0, 1)| ( 0, 1)
phi_temp| ( 0, 0)| ( 0, 0) phi_temp| ( 0, 0)| ( 0, 0)
phidelta| ( 0, 0)| ( 0, 0) phidelta| ( 0, 0)| ( 0, 0)
   
   
%% Cell type:code id: tags: %% Cell type:code id: tags:
   
``` python ``` python
result = None result = None
if 'is_test_run' not in globals(): if 'is_test_run' not in globals():
ani = ps.plot.scalar_field_animation(timeloop, rescale=True, frames=600) ani = ps.plot.scalar_field_animation(timeloop, rescale=True, frames=600)
result = ps.jupyter.display_as_html_video(ani) result = ps.jupyter.display_as_html_video(ani)
result result
``` ```
   
%% Output %% Output
   
<IPython.core.display.HTML object> <IPython.core.display.HTML object>
   
%% Cell type:code id: tags: %% Cell type:code id: tags:
   
``` python ``` python
assert np.isfinite(dh.max('phi')) assert np.isfinite(dh.max('phi'))
assert np.isfinite(dh.max('T')) assert np.isfinite(dh.max('T'))
``` ```
......
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
import psutil
from pystencils.session import * from pystencils.session import *
import shutil import shutil
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# Demo: Finite differences - 2D wave equation # Demo: Finite differences - 2D wave equation
In this tutorial we show how to use the finite difference module of *pystencils* to solve a 2D wave equations. The time derivative is discretized by a simple forward Euler method. In this tutorial we show how to use the finite difference module of *pystencils* to solve a 2D wave equations. The time derivative is discretized by a simple forward Euler method.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
$$ \frac{\partial^2 u}{\partial t^2} = \mbox{div} \left( q(x,y) \nabla u \right)$$ $$ \frac{\partial^2 u}{\partial t^2} = \mbox{div} \left( q(x,y) \nabla u \right)$$
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
We begin by creating three *numpy* arrays for the current, the previous and the next timestep. Actually we will see later that two fields are enough, but let's keep it simple. From these *numpy* arrays we create *pystencils* fields to formulate our update rule. We begin by creating three *numpy* arrays for the current, the previous and the next timestep. Actually we will see later that two fields are enough, but let's keep it simple. From these *numpy* arrays we create *pystencils* fields to formulate our update rule.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
size = (60, 70) # domain size size = (60, 70) # domain size
u_arrays = [np.zeros(size), np.zeros(size), np.zeros(size)] u_arrays = [np.zeros(size), np.zeros(size), np.zeros(size)]
u_fields = [ps.Field.create_from_numpy_array("u%s" % (name,), arr) u_fields = [ps.Field.create_from_numpy_array("u%s" % (name,), arr)
for name, arr in zip(["0", "1", "2"], u_arrays)] for name, arr in zip(["0", "1", "2"], u_arrays)]
# Nicer display for fields # Nicer display for fields
for i, field in enumerate(u_fields): for i, field in enumerate(u_fields):
field.latex_name = "u^{(%d)}" % (i,) field.latex_name = "u^{(%d)}" % (i,)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
*pystencils* contains already simple rules to discretize the a diffusion term. The time discretization is done manually. *pystencils* contains already simple rules to discretize the a diffusion term. The time discretization is done manually.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
discretize = ps.fd.Discretization2ndOrder() discretize = ps.fd.Discretization2ndOrder()
def central2nd_time_derivative(fields): def central2nd_time_derivative(fields):
f_next, f_current, f_last = fields f_next, f_current, f_last = fields
return (f_next[0, 0] - 2 * f_current[0, 0] + f_last[0, 0]) / discretize.dt**2 return (f_next[0, 0] - 2 * f_current[0, 0] + f_last[0, 0]) / discretize.dt**2
rhs = ps.fd.diffusion(u_fields[1], 1) rhs = ps.fd.diffusion(u_fields[1], 1)
wave_eq = sp.Eq(central2nd_time_derivative(u_fields), discretize(rhs)) wave_eq = sp.Eq(central2nd_time_derivative(u_fields), discretize(rhs))
wave_eq = sp.simplify(wave_eq) wave_eq = sp.simplify(wave_eq)
wave_eq wave_eq
``` ```
%% Output %% Output
$$\frac{{{u^{(0)}}_{(0,0)}} - 2 {{u^{(1)}}_{(0,0)}} + {{u^{(2)}}_{(0,0)}}}{dt^{2}} = \frac{{{u^{(1)}}_{(-1,0)}} + {{u^{(1)}}_{(0,-1)}} - 4 {{u^{(1)}}_{(0,0)}} + {{u^{(1)}}_{(0,1)}} + {{u^{(1)}}_{(1,0)}}}{dx^{2}}$$ $$\frac{{{u^{(0)}}_{(0,0)}} - 2 {{u^{(1)}}_{(0,0)}} + {{u^{(2)}}_{(0,0)}}}{dt^{2}} = \frac{{{u^{(1)}}_{(-1,0)}} + {{u^{(1)}}_{(0,-1)}} - 4 {{u^{(1)}}_{(0,0)}} + {{u^{(1)}}_{(0,1)}} + {{u^{(1)}}_{(1,0)}}}{dx^{2}}$$
u_0_C - 2⋅u_1_C + u_2_C u_1_W + u_1_S - 4⋅u_1_C + u_1_N + u_1_E u_0_C - 2⋅u_1_C + u_2_C u_1_W + u_1_S - 4⋅u_1_C + u_1_N + u_1_E
─────────────────────── = ─────────────────────────────────────── ─────────────────────── = ───────────────────────────────────────
2 2 2 2
dt dx dt dx
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
The explicit Euler scheme is now obtained by solving above equation with respect to $u_C^{next}$. The explicit Euler scheme is now obtained by solving above equation with respect to $u_C^{next}$.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
u_next_C = u_fields[-1][0, 0] u_next_C = u_fields[-1][0, 0]
update_rule = ps.Assignment(u_next_C, sp.solve(wave_eq, u_next_C)[0]) update_rule = ps.Assignment(u_next_C, sp.solve(wave_eq, u_next_C)[0])
update_rule update_rule
``` ```
%% Output %% Output
$${{u^{(2)}}_{(0,0)}} \leftarrow \frac{{{u^{(1)}}_{(-1,0)}} dt^{2} + {{u^{(1)}}_{(0,-1)}} dt^{2} - 4 {{u^{(1)}}_{(0,0)}} dt^{2} + {{u^{(1)}}_{(0,1)}} dt^{2} + {{u^{(1)}}_{(1,0)}} dt^{2} + dx^{2} \left(- {{u^{(0)}}_{(0,0)}} + 2 {{u^{(1)}}_{(0,0)}}\right)}{dx^{2}}$$ $${{u^{(2)}}_{(0,0)}} \leftarrow \frac{{{u^{(1)}}_{(-1,0)}} dt^{2} + {{u^{(1)}}_{(0,-1)}} dt^{2} - 4 {{u^{(1)}}_{(0,0)}} dt^{2} + {{u^{(1)}}_{(0,1)}} dt^{2} + {{u^{(1)}}_{(1,0)}} dt^{2} + dx^{2} \left(- {{u^{(0)}}_{(0,0)}} + 2 {{u^{(1)}}_{(0,0)}}\right)}{dx^{2}}$$
2 2 2 2 2 2 2 2 2 2 2 2
u_1_W⋅dt + u_1_S⋅dt - 4⋅u_1_C⋅dt + u_1_N⋅dt + u_1_E⋅dt + dx ⋅(-u u_1_W⋅dt + u_1_S⋅dt - 4⋅u_1_C⋅dt + u_1_N⋅dt + u_1_E⋅dt + dx ⋅(-u
u_2_C := ───────────────────────────────────────────────────────────────────── u_2_C := ─────────────────────────────────────────────────────────────────────
2 2
dx dx
_0_C + 2⋅u_1_C) _0_C + 2⋅u_1_C)
─────────────── ───────────────
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Before creating the kernel, we substitute numeric values for $dx$ and $dt$. Before creating the kernel, we substitute numeric values for $dx$ and $dt$.
Then a kernel is created just like in the last tutorial. Then a kernel is created just like in the last tutorial.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
update_rule = update_rule.subs({discretize.dx: 0.1, discretize.dt: 0.05}) update_rule = update_rule.subs({discretize.dx: 0.1, discretize.dt: 0.05})
ast = ps.create_kernel(update_rule) ast = ps.create_kernel(update_rule)
kernel = ast.compile() kernel = ast.compile()
ps.show_code(ast) ps.show_code(ast)
``` ```
%% Output %% Output
FUNC_PREFIX void kernel(double * RESTRICT const _data_u0, double * RESTRICT const _data_u1, double * RESTRICT _data_u2) FUNC_PREFIX void kernel(double * RESTRICT const _data_u0, double * RESTRICT const _data_u1, double * RESTRICT _data_u2)
{ {
for (int ctr_0 = 1; ctr_0 < 59; ctr_0 += 1) for (int ctr_0 = 1; ctr_0 < 59; ctr_0 += 1)
{ {
double * RESTRICT _data_u2_00 = _data_u2 + 70*ctr_0; double * RESTRICT _data_u2_00 = _data_u2 + 70*ctr_0;
double * RESTRICT const _data_u1_00 = _data_u1 + 70*ctr_0; double * RESTRICT const _data_u1_00 = _data_u1 + 70*ctr_0;
double * RESTRICT const _data_u0_00 = _data_u0 + 70*ctr_0; double * RESTRICT const _data_u0_00 = _data_u0 + 70*ctr_0;
double * RESTRICT const _data_u1_01 = _data_u1 + 70*ctr_0 + 70; double * RESTRICT const _data_u1_01 = _data_u1 + 70*ctr_0 + 70;
double * RESTRICT const _data_u1_0m1 = _data_u1 + 70*ctr_0 - 70; double * RESTRICT const _data_u1_0m1 = _data_u1 + 70*ctr_0 - 70;
for (int ctr_1 = 1; ctr_1 < 69; ctr_1 += 1) for (int ctr_1 = 1; ctr_1 < 69; ctr_1 += 1)
{ {
_data_u2_00[ctr_1] = 0.25*_data_u1_00[ctr_1 + 1] + 0.25*_data_u1_00[ctr_1 - 1] + 0.25*_data_u1_01[ctr_1] + 0.25*_data_u1_0m1[ctr_1] - 1.0*_data_u0_00[ctr_1] + 1.0*_data_u1_00[ctr_1]; _data_u2_00[ctr_1] = 0.25*_data_u1_00[ctr_1 + 1] + 0.25*_data_u1_00[ctr_1 - 1] + 0.25*_data_u1_01[ctr_1] + 0.25*_data_u1_0m1[ctr_1] - 1.0*_data_u0_00[ctr_1] + 1.0*_data_u1_00[ctr_1];
} }
} }
} }
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
ast.get_parameters() ast.get_parameters()
``` ```
%% Output %% Output
[_data_u0, _data_u1, _data_u2] [_data_u0, _data_u1, _data_u2]
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
ast.fields_accessed ast.fields_accessed
``` ```
%% Output %% Output
{u0, u1, u2} {u0, u1, u2}
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
To run simulation a suitable initial condition and boundary treatment is required. We chose an initial condition which is zero at the borders of the domain. The outermost layer is not changed by the update kernel, so we have an implicit homogenous Dirichlet boundary condition. To run simulation a suitable initial condition and boundary treatment is required. We chose an initial condition which is zero at the borders of the domain. The outermost layer is not changed by the update kernel, so we have an implicit homogenous Dirichlet boundary condition.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
X,Y = np.meshgrid( np.linspace(0, 1, size[1]), np.linspace(0,1, size[0])) X,Y = np.meshgrid( np.linspace(0, 1, size[1]), np.linspace(0,1, size[0]))
Z = np.sin(2*X*np.pi) * np.sin(2*Y*np.pi) Z = np.sin(2*X*np.pi) * np.sin(2*Y*np.pi)
# Initialize the previous and current values with the initial function # Initialize the previous and current values with the initial function
np.copyto(u_arrays[0], Z) np.copyto(u_arrays[0], Z)
np.copyto(u_arrays[1], Z) np.copyto(u_arrays[1], Z)
# The values for the next timesteps do not matter, since they are overwritten # The values for the next timesteps do not matter, since they are overwritten
u_arrays[2][:, :] = 0 u_arrays[2][:, :] = 0
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
One timestep now consists of applying the kernel once, then shifting the arrays. One timestep now consists of applying the kernel once, then shifting the arrays.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
def run(timesteps=1): def run(timesteps=1):
for t in range(timesteps): for t in range(timesteps):
kernel(u0=u_arrays[0], u1=u_arrays[1], u2=u_arrays[2]) kernel(u0=u_arrays[0], u1=u_arrays[1], u2=u_arrays[2])
u_arrays[0], u_arrays[1], u_arrays[2] = u_arrays[1], u_arrays[2], u_arrays[0] u_arrays[0], u_arrays[1], u_arrays[2] = u_arrays[1], u_arrays[2], u_arrays[0]
return u_arrays[2] return u_arrays[2]
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Lets create an animation of the solution: Lets create an animation of the solution:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
if shutil.which("ffmpeg") is not None: if shutil.which("ffmpeg") is not None:
ani = plt.surface_plot_animation(run, zlim=(-1, 1)) ani = plt.surface_plot_animation(run, zlim=(-1, 1))
ps.jupyter.display_as_html_video(ani) ps.jupyter.display_as_html_video(ani)
else: else:
print("No ffmpeg installed") print("No ffmpeg installed")
``` ```
%% Output %% Output
<IPython.core.display.HTML object> <IPython.core.display.HTML object>
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
assert np.isfinite(np.max(u_arrays[2])) assert np.isfinite(np.max(u_arrays[2]))
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
__LLVM backend__ __LLVM backend__
The next cell demonstrates how to run the same simulation with the LLVM backend. The only difference is that in the `create_kernel` function the `target` is set to llvm. The next cell demonstrates how to run the same simulation with the LLVM backend. The only difference is that in the `create_kernel` function the `target` is set to llvm.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
try: try:
import llvmlite import llvmlite
except ImportError: except ImportError:
llvmlite=None llvmlite=None
print('No llvmlite installed') print('No llvmlite installed')
if llvmlite: if llvmlite:
kernel = ps.create_kernel(update_rule, target='llvm').compile() kernel = ps.create_kernel(update_rule, backend=ps.Backend.LLVM).compile()
X,Y = np.meshgrid( np.linspace(0, 1, size[1]), np.linspace(0,1, size[0])) X,Y = np.meshgrid( np.linspace(0, 1, size[1]), np.linspace(0,1, size[0]))
Z = np.sin(2*X*np.pi) * np.sin(2*Y*np.pi) Z = np.sin(2*X*np.pi) * np.sin(2*Y*np.pi)
# Initialize the previous and current values with the initial function # Initialize the previous and current values with the initial function
np.copyto(u_arrays[0], Z) np.copyto(u_arrays[0], Z)
np.copyto(u_arrays[1], Z) np.copyto(u_arrays[1], Z)
# The values for the next timesteps do not matter, since they are overwritten # The values for the next timesteps do not matter, since they are overwritten
u_arrays[2][:, :] = 0 u_arrays[2][:, :] = 0
def run_LLVM(timesteps=1): def run_LLVM(timesteps=1):
for t in range(timesteps): for t in range(timesteps):
kernel(u0=u_arrays[0], u1=u_arrays[1], u2=u_arrays[2]) kernel(u0=u_arrays[0], u1=u_arrays[1], u2=u_arrays[2])
u_arrays[0], u_arrays[1], u_arrays[2] = u_arrays[1], u_arrays[2], u_arrays[0] u_arrays[0], u_arrays[1], u_arrays[2] = u_arrays[1], u_arrays[2], u_arrays[0]
return u_arrays[2] return u_arrays[2]
assert np.isfinite(np.max(u_arrays[2])) assert np.isfinite(np.max(u_arrays[2]))
if shutil.which("ffmpeg") is not None: if shutil.which("ffmpeg") is not None:
ani = plt.surface_plot_animation(run_LLVM, zlim=(-1, 1)) ani = plt.surface_plot_animation(run_LLVM, zlim=(-1, 1))
ps.jupyter.display_as_html_video(ani) ps.jupyter.display_as_html_video(ani)
else: else:
print("No ffmpeg installed") print("No ffmpeg installed")
``` ```
%% Output %% Output
<IPython.core.display.HTML object> <IPython.core.display.HTML object>
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
__Runing on GPU__ __Runing on GPU__
We can also run the same kernel on the GPU, by using the ``pycuda`` package. We can also run the same kernel on the GPU, by using the ``pycuda`` package.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
try: try:
import pycuda import pycuda
except ImportError: except ImportError:
pycuda=None pycuda=None
print('No pycuda installed') print('No pycuda installed')
res = None res = None
if pycuda: if pycuda:
gpu_ast = ps.create_kernel(update_rule, target='gpu') gpu_ast = ps.create_kernel(update_rule, target=ps.Target.GPU)
gpu_kernel = gpu_ast.compile() gpu_kernel = gpu_ast.compile()
res = ps.show_code(gpu_ast) res = ps.show_code(gpu_ast)
res res
``` ```
%% Output %% Output
FUNC_PREFIX __launch_bounds__(256) void kernel(double * RESTRICT const _data_u0, double * RESTRICT const _data_u1, double * RESTRICT _data_u2) FUNC_PREFIX __launch_bounds__(256) void kernel(double * RESTRICT const _data_u0, double * RESTRICT const _data_u1, double * RESTRICT _data_u2)
{ {
if (blockDim.x*blockIdx.x + threadIdx.x + 1 < 59 && blockDim.y*blockIdx.y + threadIdx.y + 1 < 69) if (blockDim.x*blockIdx.x + threadIdx.x + 1 < 59 && blockDim.y*blockIdx.y + threadIdx.y + 1 < 69)
{ {
const int64_t ctr_0 = blockDim.x*blockIdx.x + threadIdx.x + 1; const int64_t ctr_0 = blockDim.x*blockIdx.x + threadIdx.x + 1;
const int64_t ctr_1 = blockDim.y*blockIdx.y + threadIdx.y + 1; const int64_t ctr_1 = blockDim.y*blockIdx.y + threadIdx.y + 1;
double * RESTRICT _data_u2_10 = _data_u2 + ctr_1; double * RESTRICT _data_u2_10 = _data_u2 + ctr_1;
double * RESTRICT const _data_u1_10 = _data_u1 + ctr_1; double * RESTRICT const _data_u1_10 = _data_u1 + ctr_1;
double * RESTRICT const _data_u0_10 = _data_u0 + ctr_1; double * RESTRICT const _data_u0_10 = _data_u0 + ctr_1;
double * RESTRICT const _data_u1_11 = _data_u1 + ctr_1 + 1; double * RESTRICT const _data_u1_11 = _data_u1 + ctr_1 + 1;
double * RESTRICT const _data_u1_1m1 = _data_u1 + ctr_1 - 1; double * RESTRICT const _data_u1_1m1 = _data_u1 + ctr_1 - 1;
_data_u2_10[70*ctr_0] = 0.25*_data_u1_10[70*ctr_0 + 70] + 0.25*_data_u1_10[70*ctr_0 - 70] + 0.25*_data_u1_11[70*ctr_0] + 0.25*_data_u1_1m1[70*ctr_0] - 1.0*_data_u0_10[70*ctr_0] + 1.0*_data_u1_10[70*ctr_0]; _data_u2_10[70*ctr_0] = 0.25*_data_u1_10[70*ctr_0 + 70] + 0.25*_data_u1_10[70*ctr_0 - 70] + 0.25*_data_u1_11[70*ctr_0] + 0.25*_data_u1_1m1[70*ctr_0] - 1.0*_data_u0_10[70*ctr_0] + 1.0*_data_u1_10[70*ctr_0];
} }
} }
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
The run function has to be changed now slightly, since the data has to be transfered to the GPU first, then the kernel can be executed, and in the end the data has to be transfered back The run function has to be changed now slightly, since the data has to be transfered to the GPU first, then the kernel can be executed, and in the end the data has to be transfered back
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
if pycuda: if pycuda:
import pycuda.gpuarray as gpuarray import pycuda.gpuarray as gpuarray
def run_on_gpu(timesteps=1): def run_on_gpu(timesteps=1):
# Transfer arrays to GPU # Transfer arrays to GPU
gpuArrs = [gpuarray.to_gpu(a) for a in u_arrays] gpuArrs = [gpuarray.to_gpu(a) for a in u_arrays]
for t in range(timesteps): for t in range(timesteps):
gpu_kernel(u0=gpuArrs[0], u1=gpuArrs[1], u2=gpuArrs[2]) gpu_kernel(u0=gpuArrs[0], u1=gpuArrs[1], u2=gpuArrs[2])
gpuArrs[0], gpuArrs[1], gpuArrs[2] = gpuArrs[1], gpuArrs[2], gpuArrs[0] gpuArrs[0], gpuArrs[1], gpuArrs[2] = gpuArrs[1], gpuArrs[2], gpuArrs[0]
# Transfer arrays to CPU # Transfer arrays to CPU
for gpuArr, cpuArr in zip(gpuArrs, u_arrays): for gpuArr, cpuArr in zip(gpuArrs, u_arrays):
gpuArr.get(cpuArr) gpuArr.get(cpuArr)
assert np.isfinite(np.max(u_arrays[2])) assert np.isfinite(np.max(u_arrays[2]))
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
if pycuda: if pycuda:
run_on_gpu(400) run_on_gpu(400)
``` ```
......
...@@ -8,6 +8,10 @@ Creating kernels ...@@ -8,6 +8,10 @@ Creating kernels
.. autofunction:: pystencils.create_kernel .. autofunction:: pystencils.create_kernel
.. autofunction:: pystencils.CreateKernelConfig
.. autofunction:: pystencils.create_domain_kernel
.. autofunction:: pystencils.create_indexed_kernel .. autofunction:: pystencils.create_indexed_kernel
.. autofunction:: pystencils.create_staggered_kernel .. autofunction:: pystencils.create_staggered_kernel
......
"""Module to generate stencil kernels in C or CUDA using sympy expressions and call them as Python functions""" """Module to generate stencil kernels in C or CUDA using sympy expressions and call them as Python functions"""
from .enums import Backend, Target
from . import fd from . import fd
from . import stencil as stencil from . import stencil as stencil
from .assignment import Assignment, assignment_from_stencil from .assignment import Assignment, assignment_from_stencil
from .data_types import TypedSymbol from .data_types import TypedSymbol
from .datahandling import create_data_handling from .datahandling import create_data_handling
from .display_utils import show_code, get_code_obj, get_code_str, to_dot from .display_utils import get_code_obj, get_code_str, show_code, to_dot
from .field import Field, FieldType, fields from .field import Field, FieldType, fields
from .kernel_decorator import kernel from .kernel_decorator import kernel
from .kernelcreation import create_indexed_kernel, create_kernel, create_staggered_kernel from .kernelcreation import (
CreateKernelConfig, create_domain_kernel, create_indexed_kernel, create_kernel, create_staggered_kernel)
from .simp import AssignmentCollection from .simp import AssignmentCollection
from .slicing import make_slice from .slicing import make_slice
from .spatial_coordinates import x_, x_staggered, x_staggered_vector, x_vector, y_, y_staggered, z_, z_staggered
from .sympyextensions import SymbolCreator from .sympyextensions import SymbolCreator
from .spatial_coordinates import (x_, x_staggered, x_staggered_vector, x_vector,
y_, y_staggered, z_, z_staggered)
try: try:
import pystencils_autodiff import pystencils_autodiff
autodiff = pystencils_autodiff autodiff = pystencils_autodiff
except ImportError: except ImportError:
pass pass
__all__ = ['Field', 'FieldType', 'fields', __all__ = ['Field', 'FieldType', 'fields',
'TypedSymbol', 'TypedSymbol',
'make_slice', 'make_slice',
'create_kernel', 'create_indexed_kernel', 'create_staggered_kernel', 'create_kernel', 'create_domain_kernel', 'create_indexed_kernel', 'create_staggered_kernel',
'CreateKernelConfig',
'Target', 'Backend',
'show_code', 'to_dot', 'get_code_obj', 'get_code_str', 'show_code', 'to_dot', 'get_code_obj', 'get_code_str',
'AssignmentCollection', 'AssignmentCollection',
'Assignment', 'Assignment',
...@@ -39,5 +42,6 @@ __all__ = ['Field', 'FieldType', 'fields', ...@@ -39,5 +42,6 @@ __all__ = ['Field', 'FieldType', 'fields',
'stencil'] 'stencil']
from ._version import get_versions from ._version import get_versions
__version__ = get_versions()['version'] __version__ = get_versions()['version']
del get_versions del get_versions
...@@ -7,6 +7,7 @@ import sympy as sp ...@@ -7,6 +7,7 @@ import sympy as sp
import pystencils import pystencils
from pystencils.data_types import TypedImaginaryUnit, TypedSymbol, cast_func, create_type from pystencils.data_types import TypedImaginaryUnit, TypedSymbol, cast_func, create_type
from pystencils.enums import Target, Backend
from pystencils.field import Field from pystencils.field import Field
from pystencils.kernelparameters import FieldPointerSymbol, FieldShapeSymbol, FieldStrideSymbol from pystencils.kernelparameters import FieldPointerSymbol, FieldShapeSymbol, FieldStrideSymbol
from pystencils.sympyextensions import fast_subs from pystencils.sympyextensions import fast_subs
...@@ -136,7 +137,6 @@ class Conditional(Node): ...@@ -136,7 +137,6 @@ class Conditional(Node):
class KernelFunction(Node): class KernelFunction(Node):
class Parameter: class Parameter:
"""Function parameter. """Function parameter.
...@@ -176,7 +176,9 @@ class KernelFunction(Node): ...@@ -176,7 +176,9 @@ class KernelFunction(Node):
def field_name(self): def field_name(self):
return self.fields[0].name return self.fields[0].name
def __init__(self, body, target, backend, compile_function, ghost_layers, function_name="kernel", assignments=None): def __init__(self, body, target: Target, backend: Backend, compile_function, ghost_layers,
function_name: str = "kernel",
assignments=None):
super(KernelFunction, self).__init__() super(KernelFunction, self).__init__()
self._body = body self._body = body
body.parent = self body.parent = self
...@@ -194,12 +196,12 @@ class KernelFunction(Node): ...@@ -194,12 +196,12 @@ class KernelFunction(Node):
@property @property
def target(self): def target(self):
"""Currently either 'cpu' or 'gpu' """ """See pystencils.Target"""
return self._target return self._target
@property @property
def backend(self): def backend(self):
"""Backend for generating the code e.g. 'llvm', 'c', 'cuda' """ """Backend for generating the code: `Backend`"""
return self._backend return self._backend
@property @property
......
...@@ -14,6 +14,7 @@ from pystencils.cpu.vectorization import vec_all, vec_any, CachelineSize ...@@ -14,6 +14,7 @@ from pystencils.cpu.vectorization import vec_all, vec_any, CachelineSize
from pystencils.data_types import ( from pystencils.data_types import (
PointerType, VectorType, address_of, cast_func, create_type, get_type_of_expression, PointerType, VectorType, address_of, cast_func, create_type, get_type_of_expression,
reinterpret_cast_func, vector_memory_access, BasicType, TypedSymbol) reinterpret_cast_func, vector_memory_access, BasicType, TypedSymbol)
from pystencils.enums import Backend
from pystencils.fast_approximation import fast_division, fast_inv_sqrt, fast_sqrt from pystencils.fast_approximation import fast_division, fast_inv_sqrt, fast_sqrt
from pystencils.integer_functions import ( from pystencils.integer_functions import (
bit_shift_left, bit_shift_right, bitwise_and, bitwise_or, bitwise_xor, bit_shift_left, bit_shift_right, bitwise_and, bitwise_or, bitwise_xor,
...@@ -34,7 +35,7 @@ KERNCRAFT_NO_TERNARY_MODE = False ...@@ -34,7 +35,7 @@ KERNCRAFT_NO_TERNARY_MODE = False
def generate_c(ast_node: Node, def generate_c(ast_node: Node,
signature_only: bool = False, signature_only: bool = False,
dialect='c', dialect: Backend = Backend.C,
custom_backend=None, custom_backend=None,
with_globals=True) -> str: with_globals=True) -> str:
"""Prints an abstract syntax tree node as C or CUDA code. """Prints an abstract syntax tree node as C or CUDA code.
...@@ -46,7 +47,7 @@ def generate_c(ast_node: Node, ...@@ -46,7 +47,7 @@ def generate_c(ast_node: Node,
Args: Args:
ast_node: ast representation of kernel ast_node: ast representation of kernel
signature_only: generate signature without function body signature_only: generate signature without function body
dialect: 'c', 'cuda' or opencl dialect: `Backend`: 'C', 'CUDA' or 'OPENCL'
custom_backend: use own custom printer for code generation custom_backend: use own custom printer for code generation
with_globals: enable usage of global variables with_globals: enable usage of global variables
Returns: Returns:
...@@ -60,21 +61,21 @@ def generate_c(ast_node: Node, ...@@ -60,21 +61,21 @@ def generate_c(ast_node: Node,
ast_node.global_variables = d.symbols_defined ast_node.global_variables = d.symbols_defined
if custom_backend: if custom_backend:
printer = custom_backend printer = custom_backend
elif dialect == 'c': elif dialect == Backend.C:
try: try:
instruction_set = ast_node.instruction_set instruction_set = ast_node.instruction_set
except Exception: except Exception:
instruction_set = None instruction_set = None
printer = CBackend(signature_only=signature_only, printer = CBackend(signature_only=signature_only,
vector_instruction_set=instruction_set) vector_instruction_set=instruction_set)
elif dialect == 'cuda': elif dialect == Backend.CUDA:
from pystencils.backends.cuda_backend import CudaBackend from pystencils.backends.cuda_backend import CudaBackend
printer = CudaBackend(signature_only=signature_only) printer = CudaBackend(signature_only=signature_only)
elif dialect == 'opencl': elif dialect == Backend.OPENCL:
from pystencils.backends.opencl_backend import OpenClBackend from pystencils.backends.opencl_backend import OpenClBackend
printer = OpenClBackend(signature_only=signature_only) printer = OpenClBackend(signature_only=signature_only)
else: else:
raise ValueError("Unknown dialect: " + str(dialect)) raise ValueError(f'Unknown {dialect=}')
code = printer(ast_node) code = printer(ast_node)
if not signature_only and isinstance(ast_node, KernelFunction): if not signature_only and isinstance(ast_node, KernelFunction):
if with_globals and global_declarations: if with_globals and global_declarations:
...@@ -189,7 +190,7 @@ class CFunction(TypedSymbol): ...@@ -189,7 +190,7 @@ class CFunction(TypedSymbol):
# noinspection PyPep8Naming # noinspection PyPep8Naming
class CBackend: class CBackend:
def __init__(self, sympy_printer=None, signature_only=False, vector_instruction_set=None, dialect='c'): def __init__(self, sympy_printer=None, signature_only=False, vector_instruction_set=None, dialect=Backend.C):
if sympy_printer is None: if sympy_printer is None:
if vector_instruction_set is not None: if vector_instruction_set is not None:
self.sympy_printer = VectorizedCustomSympyPrinter(vector_instruction_set) self.sympy_printer = VectorizedCustomSympyPrinter(vector_instruction_set)
...@@ -228,7 +229,7 @@ class CBackend: ...@@ -228,7 +229,7 @@ class CBackend:
function_arguments = [f"{self._print(s.symbol.dtype)} {s.symbol.name}" for s in node.get_parameters() function_arguments = [f"{self._print(s.symbol.dtype)} {s.symbol.name}" for s in node.get_parameters()
if not type(s.symbol) is CFunction] if not type(s.symbol) is CFunction]
launch_bounds = "" launch_bounds = ""
if self._dialect == 'cuda': if self._dialect == Backend.CUDA:
max_threads = node.indexing.max_threads_per_block() max_threads = node.indexing.max_threads_per_block()
if max_threads: if max_threads:
launch_bounds = f"__launch_bounds__({max_threads}) " launch_bounds = f"__launch_bounds__({max_threads}) "
......
...@@ -2,6 +2,7 @@ from os.path import dirname, join ...@@ -2,6 +2,7 @@ from os.path import dirname, join
from pystencils.astnodes import Node from pystencils.astnodes import Node
from pystencils.backends.cbackend import CBackend, CustomSympyPrinter, generate_c from pystencils.backends.cbackend import CBackend, CustomSympyPrinter, generate_c
from pystencils.enums import Backend
from pystencils.fast_approximation import fast_division, fast_inv_sqrt, fast_sqrt from pystencils.fast_approximation import fast_division, fast_inv_sqrt, fast_sqrt
from pystencils.interpolation_astnodes import DiffInterpolatorAccess, InterpolationMode from pystencils.interpolation_astnodes import DiffInterpolatorAccess, InterpolationMode
...@@ -22,7 +23,7 @@ def generate_cuda(ast_node: Node, signature_only: bool = False, custom_backend=N ...@@ -22,7 +23,7 @@ def generate_cuda(ast_node: Node, signature_only: bool = False, custom_backend=N
Returns: Returns:
CUDA code for the ast node and its descendants CUDA code for the ast node and its descendants
""" """
return generate_c(ast_node, signature_only, dialect='cuda', return generate_c(ast_node, signature_only, dialect=Backend.CUDA,
custom_backend=custom_backend, with_globals=with_globals) custom_backend=custom_backend, with_globals=with_globals)
...@@ -33,7 +34,7 @@ class CudaBackend(CBackend): ...@@ -33,7 +34,7 @@ class CudaBackend(CBackend):
if not sympy_printer: if not sympy_printer:
sympy_printer = CudaSympyPrinter() sympy_printer = CudaSympyPrinter()
super().__init__(sympy_printer, signature_only, dialect='cuda') super().__init__(sympy_printer, signature_only, dialect=Backend.CUDA)
def _print_SharedMemoryAllocation(self, node): def _print_SharedMemoryAllocation(self, node):
dtype = node.symbol.dtype dtype = node.symbol.dtype
......
...@@ -4,6 +4,7 @@ import pystencils.data_types ...@@ -4,6 +4,7 @@ import pystencils.data_types
from pystencils.astnodes import Node from pystencils.astnodes import Node
from pystencils.backends.cbackend import CustomSympyPrinter, generate_c from pystencils.backends.cbackend import CustomSympyPrinter, generate_c
from pystencils.backends.cuda_backend import CudaBackend, CudaSympyPrinter from pystencils.backends.cuda_backend import CudaBackend, CudaSympyPrinter
from pystencils.enums import Backend
from pystencils.fast_approximation import fast_division, fast_inv_sqrt, fast_sqrt from pystencils.fast_approximation import fast_division, fast_inv_sqrt, fast_sqrt
with open(join(dirname(__file__), 'opencl1.1_known_functions.txt')) as f: with open(join(dirname(__file__), 'opencl1.1_known_functions.txt')) as f:
...@@ -12,7 +13,7 @@ with open(join(dirname(__file__), 'opencl1.1_known_functions.txt')) as f: ...@@ -12,7 +13,7 @@ with open(join(dirname(__file__), 'opencl1.1_known_functions.txt')) as f:
def generate_opencl(ast_node: Node, signature_only: bool = False, custom_backend=None, with_globals=True) -> str: def generate_opencl(ast_node: Node, signature_only: bool = False, custom_backend=None, with_globals=True) -> str:
"""Prints an abstract syntax tree node (made for target 'gpu') as OpenCL code. """Prints an abstract syntax tree node (made for `Target` 'GPU') as OpenCL code. # TODO Backend instead of Target?
Args: Args:
ast_node: ast representation of kernel ast_node: ast representation of kernel
...@@ -23,7 +24,7 @@ def generate_opencl(ast_node: Node, signature_only: bool = False, custom_backend ...@@ -23,7 +24,7 @@ def generate_opencl(ast_node: Node, signature_only: bool = False, custom_backend
Returns: Returns:
OpenCL code for the ast node and its descendants OpenCL code for the ast node and its descendants
""" """
return generate_c(ast_node, signature_only, dialect='opencl', return generate_c(ast_node, signature_only, dialect=Backend.OPENCL,
custom_backend=custom_backend, with_globals=with_globals) custom_backend=custom_backend, with_globals=with_globals)
...@@ -36,7 +37,7 @@ class OpenClBackend(CudaBackend): ...@@ -36,7 +37,7 @@ class OpenClBackend(CudaBackend):
sympy_printer = OpenClSympyPrinter() sympy_printer = OpenClSympyPrinter()
super().__init__(sympy_printer, signature_only) super().__init__(sympy_printer, signature_only)
self._dialect = 'opencl' self._dialect = Backend.OPENCL
def _print_Type(self, node): def _print_Type(self, node):
code = super()._print_Type(node) code = super()._print_Type(node)
......
import numpy as np import numpy as np
import sympy as sp import sympy as sp
from pystencils import create_indexed_kernel from pystencils import create_kernel, CreateKernelConfig, Target
from pystencils.assignment import Assignment from pystencils.assignment import Assignment
from pystencils.backends.cbackend import CustomCodeNode from pystencils.backends.cbackend import CustomCodeNode
from pystencils.boundaries.createindexlist import ( from pystencils.boundaries.createindexlist import (
...@@ -84,7 +84,7 @@ class FlagInterface: ...@@ -84,7 +84,7 @@ class FlagInterface:
class BoundaryHandling: class BoundaryHandling:
def __init__(self, data_handling, field_name, stencil, name="boundary_handling", flag_interface=None, def __init__(self, data_handling, field_name, stencil, name="boundary_handling", flag_interface=None,
target='cpu', openmp=True): target: Target = Target.CPU, openmp=True):
assert data_handling.has_data(field_name) assert data_handling.has_data(field_name)
assert data_handling.dim == len(stencil[0]), "Dimension of stencil and data handling do not match" assert data_handling.dim == len(stencil[0]), "Dimension of stencil and data handling do not match"
self._data_handling = data_handling self._data_handling = data_handling
...@@ -442,9 +442,10 @@ class BoundaryOffsetInfo(CustomCodeNode): ...@@ -442,9 +442,10 @@ class BoundaryOffsetInfo(CustomCodeNode):
INV_DIR_SYMBOL = TypedSymbol("invdir", np.int64) INV_DIR_SYMBOL = TypedSymbol("invdir", np.int64)
def create_boundary_kernel(field, index_field, stencil, boundary_functor, target='cpu', **kernel_creation_args): def create_boundary_kernel(field, index_field, stencil, boundary_functor, target=Target.CPU, **kernel_creation_args):
elements = [BoundaryOffsetInfo(stencil)] elements = [BoundaryOffsetInfo(stencil)]
dir_symbol = TypedSymbol("dir", np.int64) dir_symbol = TypedSymbol("dir", np.int64)
elements += [Assignment(dir_symbol, index_field[0]('dir'))] elements += [Assignment(dir_symbol, index_field[0]('dir'))]
elements += boundary_functor(field, direction_symbol=dir_symbol, index_field=index_field) elements += boundary_functor(field, direction_symbol=dir_symbol, index_field=index_field)
return create_indexed_kernel(elements, [index_field], target=target, **kernel_creation_args) config = CreateKernelConfig(index_fields=[index_field], target=target, **kernel_creation_args)
return create_kernel(elements, config=config)
...@@ -5,6 +5,7 @@ import numpy as np ...@@ -5,6 +5,7 @@ import numpy as np
import pystencils.astnodes as ast import pystencils.astnodes as ast
from pystencils.assignment import Assignment from pystencils.assignment import Assignment
from pystencils.enums import Target, Backend
from pystencils.astnodes import Block, KernelFunction, LoopOverCoordinate, SympyAssignment from pystencils.astnodes import Block, KernelFunction, LoopOverCoordinate, SympyAssignment
from pystencils.cpu.cpujit import make_python_function from pystencils.cpu.cpujit import make_python_function
from pystencils.data_types import StructType, TypedSymbol, create_type from pystencils.data_types import StructType, TypedSymbol, create_type
...@@ -70,7 +71,7 @@ def create_kernel(assignments: AssignmentOrAstNodeList, function_name: str = "ke ...@@ -70,7 +71,7 @@ def create_kernel(assignments: AssignmentOrAstNodeList, function_name: str = "ke
loop_order = get_optimal_loop_ordering(fields_without_buffers) loop_order = get_optimal_loop_ordering(fields_without_buffers)
loop_node, ghost_layer_info = make_loop_over_domain(body, iteration_slice=iteration_slice, loop_node, ghost_layer_info = make_loop_over_domain(body, iteration_slice=iteration_slice,
ghost_layers=ghost_layers, loop_order=loop_order) ghost_layers=ghost_layers, loop_order=loop_order)
ast_node = KernelFunction(loop_node, 'cpu', 'c', compile_function=make_python_function, ast_node = KernelFunction(loop_node, Target.CPU, Backend.C, compile_function=make_python_function,
ghost_layers=ghost_layer_info, function_name=function_name, assignments=assignments) ghost_layers=ghost_layer_info, function_name=function_name, assignments=assignments)
implement_interpolations(body) implement_interpolations(body)
...@@ -151,7 +152,7 @@ def create_indexed_kernel(assignments: AssignmentOrAstNodeList, index_fields, fu ...@@ -151,7 +152,7 @@ def create_indexed_kernel(assignments: AssignmentOrAstNodeList, index_fields, fu
loop_body.append(assignment) loop_body.append(assignment)
function_body = Block([loop_node]) function_body = Block([loop_node])
ast_node = KernelFunction(function_body, "cpu", "c", make_python_function, ast_node = KernelFunction(function_body, Target.CPU, Backend.C, make_python_function,
ghost_layers=None, function_name=function_name, assignments=assignments) ghost_layers=None, function_name=function_name, assignments=assignments)
fixed_coordinate_mapping = {f.name: coordinate_typed_symbols for f in non_index_fields} fixed_coordinate_mapping = {f.name: coordinate_typed_symbols for f in non_index_fields}
......
import warnings
from typing import Tuple, Union from typing import Tuple, Union
from .datahandling_interface import DataHandling from .datahandling_interface import DataHandling
from ..enums import Target
from .serial_datahandling import SerialDataHandling from .serial_datahandling import SerialDataHandling
try: try:
...@@ -18,7 +21,7 @@ except ImportError: ...@@ -18,7 +21,7 @@ except ImportError:
def create_data_handling(domain_size: Tuple[int, ...], def create_data_handling(domain_size: Tuple[int, ...],
periodicity: Union[bool, Tuple[bool, ...]] = False, periodicity: Union[bool, Tuple[bool, ...]] = False,
default_layout: str = 'SoA', default_layout: str = 'SoA',
default_target: str = 'cpu', default_target: Target = Target.CPU,
parallel: bool = False, parallel: bool = False,
default_ghost_layers: int = 1, default_ghost_layers: int = 1,
opencl_queue=None) -> DataHandling: opencl_queue=None) -> DataHandling:
...@@ -29,10 +32,16 @@ def create_data_handling(domain_size: Tuple[int, ...], ...@@ -29,10 +32,16 @@ def create_data_handling(domain_size: Tuple[int, ...],
periodicity: either True, False for full or no periodicity or a tuple of booleans indicating periodicity periodicity: either True, False for full or no periodicity or a tuple of booleans indicating periodicity
for each coordinate for each coordinate
default_layout: default array layout, that is used if not explicitly specified in 'add_array' default_layout: default array layout, that is used if not explicitly specified in 'add_array'
default_target: either 'cpu' or 'gpu' default_target: `Target`
parallel: if True a parallel domain is created using walberla - each MPI process gets a part of the domain parallel: if True a parallel domain is created using walberla - each MPI process gets a part of the domain
default_ghost_layers: default number of ghost layers if not overwritten in 'add_array' default_ghost_layers: default number of ghost layers if not overwritten in 'add_array'
""" """
if isinstance(default_target, str):
new_target = Target[default_target.upper()]
warnings.warn(f'Target "{default_target}" as str is deprecated. Use {new_target} instead',
category=DeprecationWarning)
default_target = new_target
if parallel: if parallel:
assert not opencl_queue, "OpenCL is only supported for SerialDataHandling" assert not opencl_queue, "OpenCL is only supported for SerialDataHandling"
if wlb is None: if wlb is None:
......
...@@ -3,6 +3,7 @@ from typing import Callable, Dict, Iterable, Optional, Sequence, Tuple, Union ...@@ -3,6 +3,7 @@ from typing import Callable, Dict, Iterable, Optional, Sequence, Tuple, Union
import numpy as np import numpy as np
from pystencils.enums import Target, Backend
from pystencils.field import Field, FieldType from pystencils.field import Field, FieldType
...@@ -16,8 +17,8 @@ class DataHandling(ABC): ...@@ -16,8 +17,8 @@ class DataHandling(ABC):
'gather' function that has collects (parts of the) distributed data on a single process. 'gather' function that has collects (parts of the) distributed data on a single process.
""" """
_GPU_LIKE_TARGETS = ['gpu', 'opencl'] _GPU_LIKE_TARGETS = [Target.GPU, Target.OPENCL]
_GPU_LIKE_BACKENDS = ['gpucuda', 'opencl'] _GPU_LIKE_BACKENDS = [Backend.CUDA, Backend.OPENCL]
# ---------------------------- Adding and accessing data ----------------------------------------------------------- # ---------------------------- Adding and accessing data -----------------------------------------------------------
...@@ -56,7 +57,7 @@ class DataHandling(ABC): ...@@ -56,7 +57,7 @@ class DataHandling(ABC):
layout: memory layout of array, either structure of arrays 'SoA' or array of structures 'AoS'. layout: memory layout of array, either structure of arrays 'SoA' or array of structures 'AoS'.
this is only important if values_per_cell > 1 this is only important if values_per_cell > 1
cpu: allocate field on the CPU cpu: allocate field on the CPU
gpu: allocate field on the GPU, if None, a GPU field is allocated if default_target is 'gpu' gpu: allocate field on the GPU, if None, a GPU field is allocated if default_target is 'GPU'
alignment: either False for no alignment, or the number of bytes to align to alignment: either False for no alignment, or the number of bytes to align to
Returns: Returns:
pystencils field, that can be used to formulate symbolic kernels pystencils field, that can be used to formulate symbolic kernels
...@@ -91,7 +92,7 @@ class DataHandling(ABC): ...@@ -91,7 +92,7 @@ class DataHandling(ABC):
layout: memory layout of array, either structure of arrays 'SoA' or array of structures 'AoS'. layout: memory layout of array, either structure of arrays 'SoA' or array of structures 'AoS'.
this is only important if values_per_cell > 1 this is only important if values_per_cell > 1
cpu: allocate field on the CPU cpu: allocate field on the CPU
gpu: allocate field on the GPU, if None, a GPU field is allocated if default_target is 'gpu' gpu: allocate field on the GPU, if None, a GPU field is allocated if default_target is 'GPU'
alignment: either False for no alignment, or the number of bytes to align to alignment: either False for no alignment, or the number of bytes to align to
Returns: Returns:
Fields representing the just created arrays Fields representing the just created arrays
...@@ -280,7 +281,7 @@ class DataHandling(ABC): ...@@ -280,7 +281,7 @@ class DataHandling(ABC):
names: what data to synchronize: name of array or sequence of names names: what data to synchronize: name of array or sequence of names
stencil: stencil as string defining which neighbors are synchronized e.g. 'D2Q9', 'D3Q19' stencil: stencil as string defining which neighbors are synchronized e.g. 'D2Q9', 'D3Q19'
if None, a full synchronization (i.e. D2Q9 or D3Q27) is done if None, a full synchronization (i.e. D2Q9 or D3Q27) is done
target: either 'cpu' or 'gpu target: `Target` either 'CPU' or 'GPU'
kwargs: implementation specific, optional optimization parameters for communication kwargs: implementation specific, optional optimization parameters for communication
Returns: Returns:
......
...@@ -7,16 +7,18 @@ import waLBerla as wlb ...@@ -7,16 +7,18 @@ import waLBerla as wlb
from pystencils.datahandling.blockiteration import block_iteration, sliced_block_iteration from pystencils.datahandling.blockiteration import block_iteration, sliced_block_iteration
from pystencils.datahandling.datahandling_interface import DataHandling from pystencils.datahandling.datahandling_interface import DataHandling
from pystencils.enums import Backend
from pystencils.field import Field, FieldType from pystencils.field import Field, FieldType
from pystencils.kernelparameters import FieldPointerSymbol from pystencils.kernelparameters import FieldPointerSymbol
from pystencils.utils import DotDict from pystencils.utils import DotDict
from pystencils import Target
class ParallelDataHandling(DataHandling): class ParallelDataHandling(DataHandling):
GPU_DATA_PREFIX = "gpu_" GPU_DATA_PREFIX = "gpu_"
VTK_COUNTER = 0 VTK_COUNTER = 0
def __init__(self, blocks, default_ghost_layers=1, default_layout='SoA', dim=3, default_target='cpu'): def __init__(self, blocks, default_ghost_layers=1, default_layout='SoA', dim=3, default_target=Target.CPU):
""" """
Creates data handling based on walberla block storage Creates data handling based on walberla block storage
...@@ -27,8 +29,9 @@ class ParallelDataHandling(DataHandling): ...@@ -27,8 +29,9 @@ class ParallelDataHandling(DataHandling):
dim: dimension of scenario, dim: dimension of scenario,
walberla always uses three dimensions, so if dim=2 the extend of the walberla always uses three dimensions, so if dim=2 the extend of the
z coordinate of blocks has to be 1 z coordinate of blocks has to be 1
default_target: either 'cpu' or 'gpu' . If set to 'gpu' for each array also a GPU version is allocated default_target: `Target`, either 'CPU' or 'GPU' . If set to 'GPU' for each array also a GPU version is
if not overwritten in add_array, and synchronization functions are for the GPU by default allocated if not overwritten in add_array, and synchronization functions are for the GPU by
default
""" """
super(ParallelDataHandling, self).__init__() super(ParallelDataHandling, self).__init__()
assert dim in (2, 3) assert dim in (2, 3)
...@@ -94,7 +97,7 @@ class ParallelDataHandling(DataHandling): ...@@ -94,7 +97,7 @@ class ParallelDataHandling(DataHandling):
if ghost_layers is None: if ghost_layers is None:
ghost_layers = self.default_ghost_layers ghost_layers = self.default_ghost_layers
if gpu is None: if gpu is None:
gpu = self.default_target == 'gpu' gpu = self.default_target == Target.GPU
if layout is None: if layout is None:
layout = self.default_layout layout = self.default_layout
if len(self.blocks) == 0: if len(self.blocks) == 0:
...@@ -230,7 +233,7 @@ class ParallelDataHandling(DataHandling): ...@@ -230,7 +233,7 @@ class ParallelDataHandling(DataHandling):
kernel_function(**arg_dict) kernel_function(**arg_dict)
def get_kernel_kwargs(self, kernel_function, **kwargs): def get_kernel_kwargs(self, kernel_function, **kwargs):
if kernel_function.ast.backend == 'gpucuda': if kernel_function.ast.backend == Backend.CUDA:
name_map = self._field_name_to_gpu_data_name name_map = self._field_name_to_gpu_data_name
to_array = wlb.cuda.toGpuArray to_array = wlb.cuda.toGpuArray
else: else:
...@@ -283,10 +286,10 @@ class ParallelDataHandling(DataHandling): ...@@ -283,10 +286,10 @@ class ParallelDataHandling(DataHandling):
self.to_gpu(name) self.to_gpu(name)
def synchronization_function_cpu(self, names, stencil=None, buffered=True, stencil_restricted=False, **_): def synchronization_function_cpu(self, names, stencil=None, buffered=True, stencil_restricted=False, **_):
return self.synchronization_function(names, stencil, 'cpu', buffered, stencil_restricted) return self.synchronization_function(names, stencil, Target.CPU, buffered, stencil_restricted)
def synchronization_function_gpu(self, names, stencil=None, buffered=True, stencil_restricted=False, **_): def synchronization_function_gpu(self, names, stencil=None, buffered=True, stencil_restricted=False, **_):
return self.synchronization_function(names, stencil, 'gpu', buffered, stencil_restricted) return self.synchronization_function(names, stencil, Target.GPU, buffered, stencil_restricted)
def synchronization_function(self, names, stencil=None, target=None, buffered=True, stencil_restricted=False): def synchronization_function(self, names, stencil=None, target=None, buffered=True, stencil_restricted=False):
if target is None: if target is None:
...@@ -299,12 +302,12 @@ class ParallelDataHandling(DataHandling): ...@@ -299,12 +302,12 @@ class ParallelDataHandling(DataHandling):
names = [names] names = [names]
create_scheme = wlb.createUniformBufferedScheme if buffered else wlb.createUniformDirectScheme create_scheme = wlb.createUniformBufferedScheme if buffered else wlb.createUniformDirectScheme
if target == 'cpu': if target == Target.CPU:
create_packing = wlb.field.createPackInfo if buffered else wlb.field.createMPIDatatypeInfo create_packing = wlb.field.createPackInfo if buffered else wlb.field.createMPIDatatypeInfo
if buffered and stencil_restricted: if buffered and stencil_restricted:
create_packing = wlb.field.createStencilRestrictedPackInfo create_packing = wlb.field.createStencilRestrictedPackInfo
else: else:
assert target == 'gpu' assert target == Target.GPU
create_packing = wlb.cuda.createPackInfo if buffered else wlb.cuda.createMPIDatatypeInfo create_packing = wlb.cuda.createPackInfo if buffered else wlb.cuda.createMPIDatatypeInfo
names = [self.GPU_DATA_PREFIX + name for name in names] names = [self.GPU_DATA_PREFIX + name for name in names]
......
...@@ -8,6 +8,7 @@ from pystencils.datahandling.blockiteration import SerialBlock ...@@ -8,6 +8,7 @@ from pystencils.datahandling.blockiteration import SerialBlock
from pystencils.datahandling.datahandling_interface import DataHandling from pystencils.datahandling.datahandling_interface import DataHandling
from pystencils.datahandling.pycuda import PyCudaArrayHandler, PyCudaNotAvailableHandler from pystencils.datahandling.pycuda import PyCudaArrayHandler, PyCudaNotAvailableHandler
from pystencils.datahandling.pyopencl import PyOpenClArrayHandler from pystencils.datahandling.pyopencl import PyOpenClArrayHandler
from pystencils.enums import Target
from pystencils.field import ( from pystencils.field import (
Field, FieldType, create_numpy_array_with_layout, layout_string_to_tuple, Field, FieldType, create_numpy_array_with_layout, layout_string_to_tuple,
spatial_layout_string_to_tuple) spatial_layout_string_to_tuple)
...@@ -22,7 +23,7 @@ class SerialDataHandling(DataHandling): ...@@ -22,7 +23,7 @@ class SerialDataHandling(DataHandling):
default_ghost_layers: int = 1, default_ghost_layers: int = 1,
default_layout: str = 'SoA', default_layout: str = 'SoA',
periodicity: Union[bool, Sequence[bool]] = False, periodicity: Union[bool, Sequence[bool]] = False,
default_target: str = 'cpu', default_target: Target = Target.CPU,
opencl_queue=None, opencl_queue=None,
opencl_ctx=None, opencl_ctx=None,
array_handler=None) -> None: array_handler=None) -> None:
...@@ -33,8 +34,9 @@ class SerialDataHandling(DataHandling): ...@@ -33,8 +34,9 @@ class SerialDataHandling(DataHandling):
domain_size: size of the spatial domain as tuple domain_size: size of the spatial domain as tuple
default_ghost_layers: default number of ghost layers used, if not overridden in add_array() method default_ghost_layers: default number of ghost layers used, if not overridden in add_array() method
default_layout: default layout used, if not overridden in add_array() method default_layout: default layout used, if not overridden in add_array() method
default_target: either 'cpu' or 'gpu' . If set to 'gpu' for each array also a GPU version is allocated default_target: `Target` either 'CPU' or 'GPU'. If set to 'GPU' for each array also a GPU version is
if not overwritten in add_array, and synchronization functions are for the GPU by default allocated if not overwritten in add_array, and synchronization functions are for the GPU by
default
""" """
super(SerialDataHandling, self).__init__() super(SerialDataHandling, self).__init__()
self._domainSize = tuple(domain_size) self._domainSize = tuple(domain_size)
...@@ -55,7 +57,7 @@ class SerialDataHandling(DataHandling): ...@@ -55,7 +57,7 @@ class SerialDataHandling(DataHandling):
except Exception: except Exception:
self.array_handler = PyCudaNotAvailableHandler() self.array_handler = PyCudaNotAvailableHandler()
if default_target == 'opencl' or opencl_queue: if default_target == Target.OPENCL or opencl_queue:
self.array_handler = PyOpenClArrayHandler(opencl_queue) self.array_handler = PyOpenClArrayHandler(opencl_queue)
else: else:
self.array_handler = array_handler self.array_handler = array_handler
...@@ -107,7 +109,7 @@ class SerialDataHandling(DataHandling): ...@@ -107,7 +109,7 @@ class SerialDataHandling(DataHandling):
} }
if not hasattr(values_per_cell, '__len__'): if not hasattr(values_per_cell, '__len__'):
values_per_cell = (values_per_cell, ) values_per_cell = (values_per_cell,)
if len(values_per_cell) == 1 and values_per_cell[0] == 1: if len(values_per_cell) == 1 and values_per_cell[0] == 1:
values_per_cell = () values_per_cell = ()
...@@ -266,17 +268,17 @@ class SerialDataHandling(DataHandling): ...@@ -266,17 +268,17 @@ class SerialDataHandling(DataHandling):
return name in self.gpu_arrays return name in self.gpu_arrays
def synchronization_function_cpu(self, names, stencil_name=None, **_): def synchronization_function_cpu(self, names, stencil_name=None, **_):
return self.synchronization_function(names, stencil_name, target='cpu') return self.synchronization_function(names, stencil_name, target=Target.CPU)
def synchronization_function_gpu(self, names, stencil_name=None, **_): def synchronization_function_gpu(self, names, stencil_name=None, **_):
return self.synchronization_function(names, stencil_name, target='gpu') return self.synchronization_function(names, stencil_name, target=Target.GPU)
def synchronization_function(self, names, stencil=None, target=None, functor=None, **_): def synchronization_function(self, names, stencil=None, target=None, functor=None, **_):
if target is None: if target is None:
target = self.default_target target = self.default_target
if target == 'opencl': if target == Target.OPENCL: # TODO potential misuse between Target and Backend
target = 'gpu' target = Target.GPU
assert target in ('cpu', 'gpu') assert target in (Target.CPU, Target.GPU)
if not hasattr(names, '__len__') or type(names) is str: if not hasattr(names, '__len__') or type(names) is str:
names = [names] names = [names]
...@@ -305,12 +307,12 @@ class SerialDataHandling(DataHandling): ...@@ -305,12 +307,12 @@ class SerialDataHandling(DataHandling):
gls = self._field_information[name]['ghost_layers'] gls = self._field_information[name]['ghost_layers']
values_per_cell = self._field_information[name]['values_per_cell'] values_per_cell = self._field_information[name]['values_per_cell']
if values_per_cell == (): if values_per_cell == ():
values_per_cell = (1, ) values_per_cell = (1,)
if len(values_per_cell) == 1: if len(values_per_cell) == 1:
values_per_cell = values_per_cell[0] values_per_cell = values_per_cell[0]
if len(filtered_stencil) > 0: if len(filtered_stencil) > 0:
if target == 'cpu': if target == Target.CPU:
if functor is None: if functor is None:
from pystencils.slicing import get_periodic_boundary_functor from pystencils.slicing import get_periodic_boundary_functor
functor = get_periodic_boundary_functor functor = get_periodic_boundary_functor
...@@ -318,7 +320,8 @@ class SerialDataHandling(DataHandling): ...@@ -318,7 +320,8 @@ class SerialDataHandling(DataHandling):
else: else:
if functor is None: if functor is None:
from pystencils.gpucuda.periodicity import get_periodic_boundary_functor as functor from pystencils.gpucuda.periodicity import get_periodic_boundary_functor as functor
target = 'gpu' if not isinstance(self.array_handler, PyOpenClArrayHandler) else 'opencl' target = Target.GPU if not isinstance(self.array_handler,
PyOpenClArrayHandler) else Target.OPENCL
result.append(functor(filtered_stencil, self._domainSize, result.append(functor(filtered_stencil, self._domainSize,
index_dimensions=self.fields[name].index_dimensions, index_dimensions=self.fields[name].index_dimensions,
index_dim_shape=values_per_cell, index_dim_shape=values_per_cell,
...@@ -328,7 +331,7 @@ class SerialDataHandling(DataHandling): ...@@ -328,7 +331,7 @@ class SerialDataHandling(DataHandling):
opencl_queue=self._opencl_queue, opencl_queue=self._opencl_queue,
opencl_ctx=self._opencl_ctx)) opencl_ctx=self._opencl_ctx))
if target == 'cpu': if target == Target.CPU:
def result_functor(): def result_functor():
for arr_name, func in zip(names, result): for arr_name, func in zip(names, result):
func(pdfs=self.cpu_arrays[arr_name]) func(pdfs=self.cpu_arrays[arr_name])
...@@ -379,6 +382,7 @@ class SerialDataHandling(DataHandling): ...@@ -379,6 +382,7 @@ class SerialDataHandling(DataHandling):
raise NotImplementedError("VTK export for fields with more than one index " raise NotImplementedError("VTK export for fields with more than one index "
"coordinate not implemented") "coordinate not implemented")
image_to_vtk(full_file_name, cell_data=cell_data) image_to_vtk(full_file_name, cell_data=cell_data)
return writer return writer
def create_vtk_writer_for_flag_array(self, file_name, data_name, masks_to_name, ghost_layers=False): def create_vtk_writer_for_flag_array(self, file_name, data_name, masks_to_name, ghost_layers=False):
......
...@@ -3,6 +3,7 @@ from typing import Any, Dict, Optional, Union ...@@ -3,6 +3,7 @@ from typing import Any, Dict, Optional, Union
import sympy as sp import sympy as sp
from pystencils.astnodes import KernelFunction from pystencils.astnodes import KernelFunction
from pystencils.enums import Backend
from pystencils.kernel_wrapper import KernelWrapper from pystencils.kernel_wrapper import KernelWrapper
...@@ -45,12 +46,9 @@ def get_code_obj(ast: Union[KernelFunction, KernelWrapper], custom_backend=None) ...@@ -45,12 +46,9 @@ def get_code_obj(ast: Union[KernelFunction, KernelWrapper], custom_backend=None)
if isinstance(ast, KernelWrapper): if isinstance(ast, KernelWrapper):
ast = ast.ast ast = ast.ast
if ast.backend == 'gpucuda': if ast.backend not in {Backend.C, Backend.CUDA, Backend.OPENCL}:
dialect = 'cuda' raise NotImplementedError(f'get_code_obj is not implemented for backend {ast.backend}')
elif ast.backend == 'opencl': dialect = ast.backend
dialect = 'opencl'
else:
dialect = 'c'
class CodeDisplay: class CodeDisplay:
def __init__(self, ast_input): def __init__(self, ast_input):
......