TypeError: CudaNdarrayType only supports dtype float32 for now.

[srimalj]

Hi

I'm just trying to run example given in the readme file.

Everything works until I get to the point where ./scripts/train.py is called where I get this error when I try to train it on the sample data.

$ python train.py config/shapenet10.py shapenet10_train.tar
...
TypeError: CudaNdarrayType only supports dtype float32 for now. Tried using dtype float64 for variable None

I'm working on a Mac with Python 2.7.11. I have theano and lasagne installed.

Am I missing something here?

Thanks in advance

[dimatura]

Hmm. This may be a new issue. What's going is that all variables should be float32; somehow a float64 variable seems to have crept in. Can you let me know what version of numpy, theano and lasagne you are using? Also a more detailed traceback would be useful. I expect the problem can be solved by adding an .astype(np.float32) somewhere, but I need to know where.

[srimalj]

Here are the versions:

Name: numpy
Version: 1.11.1

Name: Theano
Version: 0.8.2

Name: Lasagne
Version: 0.1

and

Python 2.7.12

As a cross check of my installation of lasagne and theano, I ran the following lasagne example on

https://github.com/Lasagne/Lasagne/blob/master/examples/mnist.py

my laptop and there weren't any issues.

Also simple theano code from the tutorials seem to work.

Some of the tests from the theano tests did report errors.

python -c "import theano; theano.test()" | tee theano-test.txt

I've attached the last bit of the output.
theano-test.txt
I'm not sure if that would affect voxnet though.

BTW I don't have CUDA and my macbook has an AMD Radeon video card - I don't have OpenCL setup with Theano either.

I would rather simply use the CPU (no GPU acceleration) for the moment.

My theano settings has the default device set to CPU as per,
$ python -c 'import theano; print(theano.config)' | less

I'm not fully aware of the internals here but I was just wondering why voxnet still complains of a TypeError: CudaNdarrayType ... given that theano is set to use the CPU.

Is there any option or setting in voxnet to disable GPU acceleration?

Thanks.

[KazNX]

I have the same issue with the following verisons and callstack:
numpy: 1.9.2
Theano: 0.8.2
Lasagne: 0.1

Traceback (most recent call last):
File "train.py", line 179, in
main(args)
File "train.py", line 132, in main
tfuncs, tvars = make_training_functions(cfg, model)
File "train.py", line 28, in make_training_functions
out = lasagne.layers.get_output(l_out, X)
File "/apps/python/2.7.9/lib/python2.7/site-packages/lasagne/layers/helper.py", line 185, in get_output
all_outputs[layer] = layer.get_output_for(layer_inputs, *_kwargs)
File "~/voxnet/voxnet/layers.py", line 217, in get_output_for
contiguous_filters = gpu_contiguous(filters)
File "/apps/python/2.7.9/lib/python2.7/site-packages/theano/gof/op.py", line 611, in call
node = self.make_node(_inputs, *_kwargs)
File "/apps/python/2.7.9/lib/python2.7/site-packages/theano/sandbox/cuda/basic_ops.py", line 3963, in make_node
input = as_cuda_ndarray_variable(input)
File "/apps/python/2.7.9/lib/python2.7/site-packages/theano/sandbox/cuda/basic_ops.py", line 47, in as_cuda_ndarray_variable
return gpu_from_host(tensor_x)
File "/apps/python/2.7.9/lib/python2.7/site-packages/theano/gof/op.py", line 611, in call
node = self.make_node(_inputs, **kwargs)
File "/apps/python/2.7.9/lib/python2.7/site-packages/theano/sandbox/cuda/basic_ops.py", line 140, in make_node
dtype=x.dtype)()])
File "/apps/python/2.7.9/lib/python2.7/site-packages/theano/sandbox/cuda/type.py", line 93, in init
(self.class.name, dtype, name))
TypeError: CudaNdarrayType only supports dtype float32 for now. Tried using dtype float64 for variable None

[Maswor]

The issues seems to be that the author explicitly call

from theano.sandbox.cuda.basic_ops import gpu_contiguous
from theano.sandbox.cuda.blas import GpuCorr3dMM

in voxnet/voxnet/layers.py
So people like us, without an Nvidia GPU is screwed. Please fix this, thank!

[KazNX]

The issue can be resolved using a .theanorc file in the home directory.

.theanorc
[global]
floatX = float32
device = gpu

[Maswor]

I believe KazNX method is forcing the script to run on GPU and let Theano fall back to use CPU. I still AttributeError: 'module' object has no attribute 'downsample', I believe because author using code that explicitly requires Nvidia cards

python train.py config/shapenet10.py shapenet10_train.tar
WARNING (theano.sandbox.cuda): CUDA is installed, but device gpu is not available  (error: Unable to get the number of gpus available: CUDA driver version is insufficient for CUDA runtime version)
2016-08-15 09:11:34,194 INFO| Metrics will be saved to metrics.jsonl
2016-08-15 09:11:34,194 INFO| Compiling theano functions...
Traceback (most recent call last):
  File "train.py", line 179, in <module>
    main(args)
  File "train.py", line 132, in main
    tfuncs, tvars = make_training_functions(cfg, model)
  File "train.py", line 28, in make_training_functions
    out = lasagne.layers.get_output(l_out, X)
  File "/Users/apple/anaconda/lib/python2.7/site-packages/lasagne/layers/helper.py", line 191, in get_output
    all_outputs[layer] = layer.get_output_for(layer_inputs, **kwargs)
  File "/Users/apple/voxnet/voxnet/layers.py", line 244, in get_output_for
    out = max_pool_3d(input, self.pool_shape)
  File "/Users/apple/voxnet/voxnet/max_pool_3d.py", line 72, in max_pool_3d
    op = T.signal.downsample.DownsampleFactorMax((ds[1], ds[2]), ignore_border)
AttributeError: 'module' object has no attribute 'downsample'

[srimalj]

I get the error:

NameError: global name 'HostFromGpu' is not defined

with the above

.theanorc

[global]
floatX = float32
device = gpu

[acaglayan]

You can resolve with overriding your .theanorc defaults by:
$ THEANO_FLAGS='floatX=float32,device=gpu0,lib.cnmem=1' python train.py config/shapenet10.py shapenet10_train.tar

[stephencuinujia]

I changed but it doesn't work.... I put the .theanorc at my home directory (not working directory), but still get errors (TypeError: CudaNdarrayType only supports dtype float32 for now. Tried using dtype float64 for variable None), any help would be greatly appreciated!!!

[DBobkov]

I had the same issue as KazNX and I was able to resolve it using the following theano flag options:

THEANO_FLAGS='floatX=float32' python train.py config/shapenet10.py shapenet10_train.tar

It gave errors if I also specified the cnmem as acaglayan suggested:

cnmemInit call failed

I also had the error with "HostFromGpu", it is due to the fact that it does not find GPU functionality. You can resolve it as follows (assumes your cuda is installed inside /usr/local/cuda-8.0/ - adapt accordingly)

export PATH=/usr/local/cuda-8.0/bin:$PATH # based on https://github.com/dnouri/kfkd-tutorial/issues/8
export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64/:$LD_LIBRARY_PATH

I reiterate on putting the entire script together

sudo apt-get install python-numpy python-scipy python-dev python-pip python-nose g++ libopenblas-dev git
sudo pip install Theano
sudo pip install Lasagne
sudo pip install path.py
# You need CUDA, I assume it is inside /usr/local/cuda-8.0/
echo "You need CUDA for this, check that your cuda is inside /usr/local/cuda-8.0/. If it is not the case, press CTRL-C and then adapt this script accordingly!!!"
read -n 1
export PATH=/usr/local/cuda-8.0/bin:$PATH # based on https://github.com/dnouri/kfkd-tutorial/issues/8
export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64/:$LD_LIBRARY_PATH
git clone https://github.com/dimatura/voxnet.git
cd voxnet
sudo pip install --editable .
cd scripts
./download_shapenet10.sh
THEANO_FLAGS='floatX=float32' python train.py config/shapenet10.py shapenet10_train.tar

[claell]

Same problem here. @dimatura can this be updated, so it works again? Or could you specify more clearly which versions of Theano and Lasagne it worked with? That might also help.

[claell]

The suggestion from @DBobkov helped, I guess, so thanks. Unfortunately there are now other issues, probably coming from Theano. I guess this is pretty much unuseable with current versions of the needed packages and using old ones does not help, either because it might cause compatibility issues with newer versions of CUDA for example.