Low Level Libraries

Frequently use lower level libraries: numerical, multi processing.

• Blas - Basic Linear Algebra Subprograms

  • Intel Math Kernel Library

• Eigen

  • Inside Eigen: https://eigen.tuxfamily.org/dox/TopicInsideEigenExample.html

• OpenMP:

  • wikipedia: application programming interface (API) that supports multi-platform shared memory multiprocessing programming in C, C++, and Fortran, on many platforms
  • intel's guide

• NCCL: Optimized primitives for collective multi-GPU communication

  • documentation

  • operations (ring algorithms):

    • all-reduce
    • all-gather
    • reduce-scatter
    • reduce
    • broadcast