RuntimeError: CUDA out of memory. How can I set max_split_size_mb?
I found this problem running a neural network on Colab Pro+ (with the high RAM option). RuntimeError: CUDA out of memory. Tried to allocate 8.00 GiB (GPU 0; 15.90 GiB total capacity; 12.04 GiB already allocated; 2.72 GiB free; 12.27 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF I have already decreased the batch size to 2. I upload the data using the h5py format. At this point, I assume the only thing I can try is setting the max_split_size_mb. I could not find anything about how I can implement the max_split_size_mb. The PyTorch documentation was not clear to me.
The max_split_size_mb configuration value can be set as an environment variable. The exact syntax is documented, but in short: The behavior of caching allocator can be controlled via environment variable PYTORCH_CUDA_ALLOC_CONF. The format is PYTORCH_CUDA_ALLOC_CONF=<option>:<value>,<option2>:<value2>… Available options: … max_split_size_mb prevents the allocator from splitting blocks larger than this size (in MB). This can help prevent fragmentation and may allow some borderline workloads to complete without running out of memory. Performance cost can range from ‘zero’ to ‘substantial’ depending on allocation patterns. Default value is unlimited, i.e. all blocks can be split. The memory_stats() and memory_summary() methods are useful for tuning. This option should be used as a last resort for a workload that is aborting due to ‘out of memory’ and showing a large amount of inactive split blocks. … So, you should be able to set an environment variable in a manner similar to the following: Windows: set 'PYTORCH_CUDA_ALLOC_CONF=max_split_size_mb:512' Linux: export 'PYTORCH_CUDA_ALLOC_CONF=max_split_size_mb:512' This will depend on what OS you're using - in your case, for Google Colab, you might find this question helpful.
Get this solution programmatically \u2014 free, no authentication.
curl https://depscope.dev/api/error/a68af24430af320b1c438aea04904db76d16e21b67fd0e8df53acc1966580cc0