MKL > 2018.3 doesn't work with NVIDIA nvblas

Platform is Ubuntu 16.04.5. GPU NVIDIA1070 cuda-9.2 python3.5

I use python with nvblas support by compiling numpy against intel MKL. It works with 2018.1 and 2018.2 but GPU is not used for MKL 2018.3 and 2019.0

Here is the experiment (have tried with different versions of numpy same issue)

LD_PRELOAD=/usr/local/cuda/targets/x86_64-linux/lib/libnvblas.so python3

NVBLAS_CONFIG_FILE environment variable is set to '/home/bernard/.config/nvblas.conf'

Python 3.5.2 (default, Nov 23 2017, 16:37:01)

[GCC 5.4.0 20160609] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> import numpy as np

>>> np.__version__ '1.15.2'

>>> np.show_config()

lapack_opt_info: define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]

library_dirs = ['/opt/intel/mkl/lib/intel64/']

include_dirs = ['/opt/intel/mkl/include']

libraries = ['mkl_rt', 'pthread'] blas_mkl_info: define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]

library_dirs = ['/opt/intel/mkl/lib/intel64/']

include_dirs = ['/opt/intel/mkl/include']

libraries = ['mkl_rt', 'pthread']

lapack_mkl_info: define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]

library_dirs = ['/opt/intel/mkl/lib/intel64/']

include_dirs = ['/opt/intel/mkl/include']

libraries = ['mkl_rt', 'pthread'] blas_opt_info: define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]

library_dirs = ['/opt/intel/mkl/lib/intel64/']

include_dirs = ['/opt/intel/mkl/include'] libraries = ['mkl_rt', 'pthread']

>>> a = np.random.rand(10000, 10000)

>>> b = np.random.rand(10000, 10000)

>>> a@b

now check with the command: nvidia-smi

For MKL <= 2018.3 you can see the gpu is being used 100%, with MKL > 2018.3 it is 0% so the gpu is not used,