print · source · login   

Theme: HPC

With the increased demand for artificial intelligence and big data applications, High-Performance Computing needs to make the transition from supercomputing centres to the wider audience. The increasing popularity of frameworks such as TensorFLow, PyTorch, Matlab, or NumPy are witness of this development.

This research theme tries to look at application areas, domain specific languages (DSLs), and language designs that bridge the help making HPC accessible to the masses. Some concrete topics of interest are:

a) Functional approaches to efficient graph processing; declarative graph processing.

Graph-processing typically is based on the idea of pointers, making it ideally suited for an imperative setting. While non-cyclic graphs can be easily manipulated in a declarative style, cyclic graphs pose a bigger challenge. Some non-trivial approaches exist, however, they lack an efficient implementation. Here, we look into ways to find simpler solutions as well as a code generator for generating efficient parallel code.

b) Advanced whole-world optimisations vs highly tuned libraries; how to beat highly-tuned libraries such as MKL or BLAS?

Libraries such as MKL or BLAS are often considered the non-plus-ultra. However, in several cases, it could be shown that compilers that take the calling context into consideration can outperform such libraries. Here, we look at applications areas such as vision or AI and investigate how whole-world compilers compete against tuned libraries.

c) New number formats for reduced memory bandwidth; control your number representations :-)

Many modern applications are memory bound, i.e., their performance is not dominated by the compute power that is thrown at them but by the throughput the memory subsystem can provide. Here, we look at new number formats such as POSITS instead of IEEE floating point, n-bit integers, or n-bit reals.


Theme: Compiler Technology

a) Out of core computations for GPUs; how to run GPU kernels on data that does not fit the memory of the GPU.

Here, we envision some compiler transformation that re-structures code for manipulating large arrays into semantically equivalent versions that stream computations over smaller arrays.

b) Advanced Reference Counting; on combining reference counting with uniqueness attributes.

Reference Counting sofar has not made it into the mainstream. With its advantages in a parallel setting there is a rise in interest in this area. Combining techniques such as uniqueness-inference and reference counting offers opportunities for further improvements. Another area of investigation in the context of reference counting is efficient support for nested data structures.

c) A new form of Distributed Shared Memory; investigating ideas for revolutionising shared memory abstractions for clusters.

The functional paradigm allows for better control over data placement which enables a new, by far more efficient implementations of Distributed Shared Memory on clusters than what pre-exists. Based on a few ideas, we have a prototypical implementation that relies on the MMU. This can serve as a starting point for a challenging master thesis involving OS level programming.


Theme: Array Programming

Array programming languages such as SaC offer high-levels of programmer productivity. This theme looks at extending such languages in ways that allow to further improve expressiveness of these languages and, with it, the programmer productivity. Some topics of interest are:

a) A new semantics for array programming languages; how to introduce some laziness without being lazy :-)

For some programs, it is possible to statically infer that not all data is needed for computing the overall result. Languages based on normal-order reduction or laziness guarantee that only those parts of a program are being evaluated that are needed for the result. While this is conceptually nice, the implementation of lazy evaluation comes at a potentially very high price. We try to look at languages that are based on a strict evaluation but who can nevertheless disregard terms that are known not to contribute to the overall result.

b) Transfinite Arrays; streaming as a special form of arrays.

Streaming can be seen as computing with arrays of infinite size. Using the notion of Ordinal numbers as indices allows for a new form of multi-dimensional streaming. There are many challenges in this context.

c) Arrays with higher-dimensional shapes; expressing proximities between the axes of a higher-dimensional array :-)

The shape of any n-dimensional array can be expressed by a shape vector. This implies a linear order of the indices. However, in several cases of program optimisation it would be convenient to express higher levels of neighbourhood. An example for this is the blocking of a 2-dimensional matrix. In this project, we look into higher-dimensional shapes that allow indices themselves to be multi-dimensional arrays.

d) Increased Type Safety; looking into state of the art type systems such as dependent types or intersection types to improve static guarantees for array languages.

This project looks into ways of expressing and verifying domain restrictions in the context of shape polymorphic array operations.


In case you are interested in any of these or related topics, please talk to Sven-Bodo Scholz M1.006