Transformation pipelines

Important

Loki is still under active development and has not yet seen a stable release. Interfaces can change at any time, objects may be renamed, or concepts may be re-thought. Make sure to sync your work to the current release frequently by rebasing feature branches and upstreaming more general applicable work in the form of pull requests.

Transformations

Transformations are the building blocks of a transformation pipeline in Loki. They encode the workflow of converting a Sourcefile or an individual program unit (such as Module or Subroutine) to the desired output format.

A transformation can encode a single modification, combine multiple steps, or call other transformations to create complex changes. If a transformation depends on another transformation, inheritance can be used to combine them.

Every transformation in a pipeline should implement the interface defined by Transformation. It provides generic entry points for transforming different objects and thus allows for batch processing. To implement a new transformation, only one or all of the relevant methods Transformation.transform_subroutine, Transformation.transform_module, or Transformation.transform_file need to be implemented.

Example: A transformation that renames every module and subroutine by appending a given suffix:

class RenameTransformation(Transformation):

    def __init__(self, suffix):
        super().__init__()
        self._suffix = suffix

    def _rename(self, item):
        item.name += self._suffix

    def transform_subroutine(self, routine, **kwargs):
        self._rename(routine)

    def transform_module(self, module, **kwargs):
        self._rename(module)

The transformation can be applied by calling apply() with the relevant object.

source = Sourcefile(...)  # may contain modules and subroutines
transformation = RenameTransformation(suffix='_example')
transformation.apply(source)

Note that, despite only implementing logic for transforming modules and subroutines, it works also for sourcefiles. While the Sourcefile object itself is not modified (because we did not implement transform_file()), the dispatch mechanism in the Transformation base class takes care of calling the relevant method for each member of the given object.

Typically, transformations should be implemented by users to encode the transformation pipeline for their individual use-case. However, Loki comes with a few built-in transformations for common tasks and we expect this list to grow in the future:

loki.transform.transformation.Transformation()

Base class for source code transformations that manipulate source items like Subroutine or Module in place via the item.apply(transform) method.

loki.transform.build_system_transform.CMakePlanner(...)

Generates a list of files to add, remove or replace in a CMake target's list of sources

loki.transform.build_system_transform.FileWriteTransformation([...])

Write out modified source files to a select build directory

loki.transform.dependency_transform.DependencyTransformation(suffix)

Basic Transformation class that facilitates dependency injection for transformed Module and Subroutine into complex source trees.

loki.transform.fortran_c_transform.FortranCTransformation([...])

Fortran-to-C transformation that translates the given routine into C and generates the corresponding ISO-C wrappers.

loki.transform.fortran_max_transform.FortranMaxTransformation()

Fortran-to-Maxeler transformation that translates the given routine into .maxj and generates a matching manager and host code with corresponding ISO-C wrappers.

loki.transform.fortran_python_transform.FortranPythonTransformation(...)

A transformer class to convert Fortran to Python or DaCe.

loki.transform.transform_hoist_variables.HoistVariablesAnalysis([key])

Base class for the Analysis part of the hoist variables functionality/transformation.

loki.transform.transform_hoist_variables.HoistVariablesTransformation([key])

Base class for the Synthesis part of the hoist variables functionality/transformation.

loki.transform.transform_hoist_variables.HoistTemporaryArraysAnalysis([...])

Specialisation for the Analysis part of the hoist variables functionality/transformation, to hoist only temporary arrays and if provided only temporary arrays with specific variables/variable names within the array dimensions.

loki.transform.transform_hoist_variables.HoistTemporaryArraysTransformationAllocatable([key])

Specialisation for the Synthesis part of the hoist variables functionality/transformation, to hoist temporary arrays and make them allocatable, including the actual allocation and de-allocation.

loki.transform.transform_parametrise.ParametriseTransformation(dic2p)

Parametrise variables with provided values.

Further transformations are defined for specific use-cases but may prove useful in a wider context. These are defined in transformations:

Additionally, a number of tools for common transformation tasks are provided as functions that can be readily used in a step of the transformation pipeline:

loki.transform.transform_array_indexing.shift_to_zero_indexing(routine)

Shift all array indices to adjust to 0-based indexing conventions (eg.

loki.transform.transform_array_indexing.invert_array_indices(routine)

Invert data/loop accesses from column to row-major

loki.transform.transform_array_indexing.resolve_vector_notation(routine)

Resolve implicit vector notation by inserting explicit loops

loki.transform.transform_array_indexing.normalize_range_indexing(routine)

Replace the (1:size) indexing in array sizes that OMNI introduces.

loki.transform.transform_array_indexing.promote_variables(...)

Promote a list of variables by inserting new array dimensions of given size and updating all uses of these variables with a given index expression.

loki.transform.transform_array_indexing.promote_nonmatching_variables(...)

Promote multiple variables with potentially non-matching promotion dimensions or index expressions.

loki.transform.transform_array_indexing.promotion_dimensions_from_loop_nest(...)

Determine promotion dimensions corresponding to the iteration space of a loop nest.

loki.transform.transform_associates.resolve_associates(routine)

Resolve Associate mappings in the body of a given routine.

loki.transform.transform_inline.inline_constant_parameters(routine)

Replace instances of variables with known constant values by Literals.

loki.transform.transform_inline.inline_elemental_functions(routine)

Replaces InlineCall expression to elemental functions with the called functions body.

loki.transform.transform_loop.loop_interchange(routine)

Search for loops annotated with the loki loop-interchange pragma and attempt to reorder them.

loki.transform.transform_loop.loop_fusion(routine)

Search for loops annotated with the loki loop-fusion pragma and attempt to fuse them into a single loop.

loki.transform.transform_loop.loop_fission(routine)

Search for !$loki loop-fission pragmas in loops and split them.

loki.transform.transform_region.region_hoist(routine)

Hoist one or multiple code regions annotated by pragma ranges and insert them at a specified target location.

loki.transform.transform_region.region_to_call(routine)

Convert regions annotated with !$loki region-to-call pragmas to subroutine calls.

loki.transform.transform_utilities.convert_to_lower_case(routine)

Converts all variables and symbols in a subroutine to lower-case.

loki.transform.transform_utilities.replace_intrinsics(routine)

Replace known intrinsic functions and symbols.

loki.transform.transform_utilities.sanitise_imports(...)

Sanitise imports by removing unused symbols and eliminating imports with empty symbol lists.

loki.transform.transform_utilities.replace_selected_kind(routine)

Find all uses of selected_real_kind or selected_int_kind and replace them by their iso_fortran_env counterparts.

loki.transform.transform_utilities.single_variable_declaration(routine)

Modify/extend variable declarations to

Bulk processing large source trees

Transformations can be applied over source trees using the Scheduler. It is a work queue manager that automatically discovers source files in a list of paths. Given the name of an entry routine, it allows to build a call graph and thus derive the dependencies within this source tree.

Calling Scheduler.process on a source tree and providing it with a Transformation applies this transformation to all modules and routines, making sure that routines with the relevant CallStatement are always processed before their target Subroutine.

When applying the transformation to an item in the source tree, the scheduler provides certain information about the item to the transformation:

  • the transformation mode (provided in the scheduler’s config),

  • the item’s role (e.g., ‘driver’ or ‘kernel’, configurable via the scheduler’s config), and

  • targets (routines that are called from the item and are included in the scheduler’s tree, i.e., will be processed afterwards).

loki.bulk.scheduler.Scheduler(paths[, ...])

Work queue manager to enqueue and process individual Item routines/modules with a given kernel.

loki.bulk.scheduler.SchedulerConfig(default, ...)

Configuration object for the transformation Scheduler that encapsulates default behaviour and item-specific behaviour.

loki.bulk.item.Item(name, source[, config])

Base class of a work item that represents a single source routine to be processed

loki.bulk.item.SubroutineItem(name, source)

Implementation of Item to represent a Fortran subroutine work item

loki.bulk.item.ProcedureBindingItem(name, source)

Implementation of Item to represent a Fortran procedure binding