Improve non-temporal stores

ARM doesn't have real non-temporal stores like x86 does. It does, however, have a special instruction (dc zva) that sets a cacheline to zero without reading it from memory, thus saving memory bandwidth. This pull request makes use of it. The other important part of non-temporal stores is that they don't pollute the cache. Some architectures like PowerPC can emulate that with a special store instruction that flushes a cache line from cache. It's not available on ARM, but this pull requests adds support anyway.

Furthermore, this pull request adds fence instructions for non-temporal stores on x86. This ensures that everything has actually arrived in memory before the kernel returns, thus ensuring that subsequent memory accesses will not get stale data. This was not an issue in practice as the overhead of exiting and calling a kernel would have usually taken enough time for the data to arrive in memory, but it wasn't guaranteed.

Note that non-temporal stores are not guaranteed to be faster than regular stores on all processors. For example, on my Apple M1, the non-temporal stores are actually slightly slower (so I did not gain anything by implementing this). This is actually not due to the overhead of checking whether a vector is the first one in a cache line (which can easily be confirmed by replacing dc zva with nop), but likely an artifact of Apple implementing dc zva differently than the ARM documentation suggests. So on ARM, one should always compare regular and non-temporal stores and use whichever is faster.

Fixes #25 (closed). Supersedes !225 (closed).

pystencils/backends/arm_instruction_sets.py

@@ -28,7 +28,6 @@ def get_vector_instruction_set_arm(data_type='double', instruction_set='neon'):
         'loadA': 'ld1[0]',
         'storeU': 'st1[0, 1]',
         'storeA': 'st1[0, 1]',
-        'stream': 'st1[0, 1]',
 
         'abs': 'abs[0]',
         '==': 'ceq[0, 1]',
@@ -47,6 +46,7 @@ def get_vector_instruction_set_arm(data_type='double', instruction_set='neon'):
     suffix = f'q_f{bits[data_type]}'
 
     result = dict()
+    result['bytes'] = 16
 
     for intrinsic_id, function_shortcut in base_names.items():
         function_shortcut = function_shortcut.strip()
@@ -81,4 +81,7 @@ def get_vector_instruction_set_arm(data_type='double', instruction_set='neon'):
     result['any'] = f'vaddlvq_u8(vreinterpretq_u8_u{bits[data_type]}({{0}})) > 0'
     result['all'] = f'vaddlvq_u8(vreinterpretq_u8_u{bits[data_type]}({{0}})) == 16*0xff'
 
+    result['cachelineSize'] = 'cachelineSize()'
+    result['cachelineZero'] = 'cachelineZero((void*) {0})'
+
     return result