Profiling PyTorch models for NVIDIA GPUs

PyTorch Profiler represents first Profiler tool from PyTorch project. It delivers the simplest experience available to date where users don't need to install any packages to profile their models and with the help of Tensorboard plugin provides out of the box analyses of the model bottlenecks including Step time breakdown shart - unique functionality that wasn't available to the users before. This should make it accessible to a wide variety of users of PyTorch who until today hasn't attempted to profile their models due to the complexity of the task. In addition to that, for experienced users Profiler provides trace view which illustrates GPU kernel execution details and their relationphip to the corresponding PyTorch operations that triggered them, including high level context of these operations such as training and optimizer steps. These features should make it very interesting topic and exciting news for both beginner and experienced users of PyTorch, basically for everyone.