“CUDA (Compute Unified Device Architecture) is a parallel computing platform and application programming interface (API) model created by Nvidia” wikipedia
cudnn GPU-accelerated library of primitives for deep neural networks
cublas
apex: a pytorch extension for mixed precision and distributed training