Open Internet by MindsNet

Accurately Profiling PyTorch Training without Interfering with GPU Performance

Profiling PyTorch training poses a measurement problem where the act of measurement can alter the run's behavior. Synchronization points, like those introduced by `torch.cuda.synchronize()`, can change the performance characteristics of an otherwise asynchronous CUDA workload.

Computing & Technology, Computer Science, Machine Learning