CUDAGraphs in Pytorch 2.0 - compiler - PyTorch Dev Discussions
https://dev-discuss.pytorch.org/t/cudagraphs-in-pytorch-2-0/1428
WEBAug 9, 2023 · TL;DR. New Cudagraph Implementation improves HuggingFace Perf 12%, and Memory from .88% to 1.13% . If you are using torch.compile, especially to lower the entire model, cudagraphs may provide speedups. Even if the model has dynamism ! Try: torch.compile(mode="reduce-overhead") CUDAGraph Background.
DA: 99 PA: 86 MOZ Rank: 2