(proxy) [jalal@goku apex]$ export PATH=/usr/local/cuda-11./bin:$PATH
(proxy) [jalal@goku apex]$ export LD_LIBRARY_PATH=/usr/local/cuda-11.1/lib64:$LD_LIBRARY_PATH
(proxy) [jalal@goku apex]$ pip uninstall torchaudio torch torchvision
Found existing installation: torchaudio 0.9.0
Uninstalling torchaudio-0.9.0:
Would remove:
/scratch3/venv/proxy/lib/python3.8/site-packages/torchaudio-0.9.0.dist-info/*
/scratch3/venv/proxy/lib/python3.8/site-packages/torchaudio/*
Proceed (Y/n)? y
Successfully uninstalled torchaudio-0.9.0
Found existing installation: torch 1.9.0+cu111
Uninstalling torch-1.9.0+cu111:
Would remove:
/scratch3/venv/proxy/bin/convert-caffe2-to-onnx
/scratch3/venv/proxy/bin/convert-onnx-to-caffe2
/scratch3/venv/proxy/lib/python3.8/site-packages/caffe2/*
/scratch3/venv/proxy/lib/python3.8/site-packages/torch-1.9.0+cu111.dist-info/*
/scratch3/venv/proxy/lib/python3.8/site-packages/torch/*
Proceed (Y/n)? y
Successfully uninstalled torch-1.9.0+cu111
Found existing installation: torchvision 0.10.0+cu111
Uninstalling torchvision-0.10.0+cu111:
Would remove:
/scratch3/venv/proxy/lib/python3.8/site-packages/torchvision-0.10.0+cu111.dist-info/*
/scratch3/venv/proxy/lib/python3.8/site-packages/torchvision.libs/libcudart.05b13ab8.so.11.0
/scratch3/venv/proxy/lib/python3.8/site-packages/torchvision.libs/libjpeg.ceea7512.so.62
/scratch3/venv/proxy/lib/python3.8/site-packages/torchvision.libs/libnvjpeg.d463bc90.so.11
/scratch3/venv/proxy/lib/python3.8/site-packages/torchvision.libs/libpng16.7f72a3c5.so.16
/scratch3/venv/proxy/lib/python3.8/site-packages/torchvision.libs/libz.1328edc3.so.1
/scratch3/venv/proxy/lib/python3.8/site-packages/torchvision/*
Proceed (Y/n)? y
Successfully uninstalled torchvision-0.10.0+cu111
(proxy) [jalal@goku apex]$ pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html
Looking in links: https://download.pytorch.org/whl/torch_stable.html
Collecting torch==1.9.0+cu111
Using cached https://download.pytorch.org/whl/cu111/torch-1.9.0%2Bcu111-cp38-cp38-linux_x86_64.whl (2041.3 MB)
Collecting torchvision==0.10.0+cu111
Using cached https://download.pytorch.org/whl/cu111/torchvision-0.10.0%2Bcu111-cp38-cp38-linux_x86_64.whl (23.2 MB)
Collecting torchaudio==0.9.0
Using cached torchaudio-0.9.0-cp38-cp38-manylinux1_x86_64.whl (1.9 MB)
Requirement already satisfied: typing-extensions in /scratch3/venv/proxy/lib/python3.8/site-packages (from torch==1.9.0+cu111) (3.10.0.2)
Requirement already satisfied: numpy in /scratch3/venv/proxy/lib/python3.8/site-packages (from torchvision==0.10.0+cu111) (1.21.2)
Requirement already satisfied: pillow>=5.3.0 in /scratch3/venv/proxy/lib/python3.8/site-packages (from torchvision==0.10.0+cu111) (8.3.2)
Installing collected packages: torch, torchvision, torchaudio
Successfully installed torch-1.9.0+cu111 torchaudio-0.9.0 torchvision-0.10.0+cu111
(proxy) [jalal@goku apex]$ pip install -v --disable-pip-version-check --no-cache-dir > --global-option="--cpp_ext" --global-option="--cuda_ext" ./
/scratch3/venv/proxy/lib/python3.8/site-packages/pip/_internal/commands/install.py:229: UserWarning: Disabling all use of wheels due to the use of --build-option / --global-option / --install-option.
cmdoptions.check_install_build_global(options)
DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.
pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555.
Running command python setup.py egg_info
torch.__version__ = 1.9.0+cu111
running egg_info
creating /scratch/tmp/pip-pip-egg-info-balgcp4g/apex.egg-info
writing /scratch/tmp/pip-pip-egg-info-balgcp4g/apex.egg-info/PKG-INFO
writing dependency_links to /scratch/tmp/pip-pip-egg-info-balgcp4g/apex.egg-info/dependency_links.txt
writing top-level names to /scratch/tmp/pip-pip-egg-info-balgcp4g/apex.egg-info/top_level.txt
writing manifest file '/scratch/tmp/pip-pip-egg-info-balgcp4g/apex.egg-info/SOURCES.txt'
reading manifest file '/scratch/tmp/pip-pip-egg-info-balgcp4g/apex.egg-info/SOURCES.txt'
writing manifest file '/scratch/tmp/pip-pip-egg-info-balgcp4g/apex.egg-info/SOURCES.txt'
/scratch/tmp/pip-req-build-rqcjg2ac/setup.py:67: UserWarning: Option --pyprof not specified. Not installing PyProf dependencies!
warnings.warn("Option --pyprof not specified. Not installing PyProf dependencies!")
Running command /scratch3/venv/proxy/bin/python3.8 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/scratch/tmp/pip-req-build-rqcjg2ac/setup.py'"'"'; __file__='"'"'/scratch/tmp/pip-req-build-rqcjg2ac/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' --cuda_ext install --record /scratch/tmp/pip-record-jpa0j20t/install-record.txt --single-version-externally-managed --compile --install-headers /scratch3/venv/proxy/include/site/python3.8/apex
torch.__version__ = 1.9.0+cu111
/scratch/tmp/pip-req-build-rqcjg2ac/setup.py:67: UserWarning: Option --pyprof not specified. Not installing PyProf dependencies!
warnings.warn("Option --pyprof not specified. Not installing PyProf dependencies!")
Compiling cuda extensions with
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2018 NVIDIA Corporation
Built on Sat_Aug_25_21:08:01_CDT_2018
Cuda compilation tools, release 10.0, V10.0.130
from /usr/local/cuda-10.0/bin
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/scratch/tmp/pip-req-build-rqcjg2ac/setup.py", line 159, in <module>
check_cuda_torch_binary_vs_bare_metal(CUDA_HOME)
File "/scratch/tmp/pip-req-build-rqcjg2ac/setup.py", line 99, in check_cuda_torch_binary_vs_bare_metal
raise RuntimeError("Cuda extensions are being compiled with a version of Cuda that does " +
RuntimeError: Cuda extensions are being compiled with a version of Cuda that does not match the version used to compile Pytorch binaries. Pytorch binaries were compiled with Cuda 11.1.
In some cases, a minor-version mismatch will not cause later errors: https://github.com/NVIDIA/apex/pull/323#discussion_r287021798. You can try commenting out this check (at your own risk).
ERROR: Command errored out with exit status 1: /scratch3/venv/proxy/bin/python3.8 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/scratch/tmp/pip-req-build-rqcjg2ac/setup.py'"'"'; __file__='"'"'/scratch/tmp/pip-req-build-rqcjg2ac/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' --cuda_ext install --record /scratch/tmp/pip-record-jpa0j20t/install-record.txt --single-version-externally-managed --compile --install-headers /scratch3/venv/proxy/include/site/python3.8/apex Check the logs for full command output.
(proxy) [jalal@goku apex]$
Comments