Datatype must provide an itemsize

[PaulsBecks]

Hey,

I'm stuck with a problem using bayesian_optimization with the ''direct'' algorithm in python 3.5

The stacktrace yields:

Call-back cb_fcn_in_direct__user__routines failed.
Traceback (most recent call last):
.
.
.
 File "env/lib/python3.5/site-packages/RoBO-0.2.1-py3.5.egg/robo/maximizers/direct.py", line 58, in maximize
  File "env/lib/python3.5/site-packages/DIRECT/__init__.py", line 206, in solve
    cdata
ValueError: data type must provide an itemsize

I have checked the itemsize of iidata, ddata, cdata and the bounds, but all provide an itemsize.

Thank you, Paul

[aaronkl]

There are some issues with DIRECT and python3.5. Could you try to run it with python 3.4 or python 3.6 to see if the error still occurs?

[BrutishGuy]

Hi there,
I recently also experienced this error. I am on python 3.4, on Ubuntu 14.04. I have followed the installation instructions posted: So, I installed libeigen3-dev, swig, and gfortran using sudo apt-get, and then ran "for req in $(cat requirements.txt); do pip install $req; done". Then I did "python setup.py install" as per instructions. Sadly I still get the following, which was in another issue as well I saw (this is running the Black-box function optimization using RoBO example on the docs):

DIRECT Version 2.0.4
 Problem Dimension n                    :      1
 Eps value                              :   0.1000E-03
 Epsilon is constant.
 Maximum number of f-evaluations (maxf) :    400
 Maximum number of iterations (MaxT)    :    200
 Value of f_global                      :  -0.1000+101
 Global percentage wanted               :   0.1000E-01
 Volume percentage wanted               :  -0.1000E+01
 Measure percentage wanted              :  -0.1000E+01
 Jones original DIRECT algorithm is used.
Bounds on variable x 1    :      0.00000 <= xi <=      6.00000
---------------------------------------------------------------------------
Call-back cb_fcn_in_direct__user__routines failed.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/victor/Documents/HyperLearn/RoBO/robo/fmin/bayesian_optimization.py", line 133, in bayesian_optimization
    x_best, f_min = bo.run(num_iterations)
  File "/home/victor/Documents/HyperLearn/RoBO/robo/solver/bayesian_optimization.py", line 167, in run
    new_x = self.choose_next(self.X, self.y, do_optimize)
  File "/home/victor/Documents/HyperLearn/RoBO/robo/solver/bayesian_optimization.py", line 245, in choose_next
    x = self.maximize_func.maximize()
  File "/home/victor/Documents/HyperLearn/RoBO/robo/maximizers/direct.py", line 58, in maximize
    maxf=self.n_func_evals)
  File "/home/victor/anaconda3/envs/hyperlearn/lib/python3.4/site-packages/DIRECT/__init__.py", line 202, in solve
    cdata
ValueError: data type must provide an itemsize

Another interesting issue is that the above snippet says DIRECT version 2.0.4 but after doing an imoprt DIRECT followed by DIRECT.__version__, it gives me version 1.0. Not even remotely sure what might be causing this. Switching to Random or CMAES in the bayesian optimization call works fine though. Thanks for any help!

[aaronkl]

Hi, after some digging into it I think that anaconda has some issues with the compiled frotran code of Direct. Unfortunately, I haven't found a solution to fix it.

We are right now reimplementing some parts of RoBO to get rid of all of these out-dated dependencies such as Direct or George.
Meanwhile I would suggest to use Random or CMAES, which shouldn't be much worse than Direct. I'll also implement a wrapper around scipy.optimize in the next week.

Sorry of any inconveniences.

[Nankaiming]

I use the CMAES and it shows that "The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()".

[jsosulski]

@BrutishGuy The version shown by DIRECT.__version__ is the version of the python wrapper library. The log that is printed, i.e. DIRECT Version 2.0.4 is the version of the actual DIRECT algorithm implemented in Fortran.

I experienced the same error and was able to solve it by using numpy in version 1.12.1. Additionally, i removed direct from the requirements files and compiled it on my own system with gfortran and against the old numpy.

If you leave direct in the requirements file, RoBO tries to use the version downloaded with pip. For some reason that version is not working on my machine.

[mukherjees]

By downgrading numpy to 1.12.1 as suggested by @BrutishGuy I was able to get things working on my desktop (macOS 10.13, Python 3.6).

Unfortunately, on my particular test GPU configuration (AWS instance with the Amazon DL AMI), Tensorflow does not work unless I use the latest version of numpy, so I couldn't downgrade my numpy to 1.12.1. Instead, I tried to swap the pydirect package for scipydirect which is a new fork of pydirect. Note that scipydirect implements a minimizer as opposed to a maximizer and calls DIRECT.solve() through scipydirect.minimize() with arguments l and u of DIRECT.solve() combined into the single argument bounds so robo/maximizers/direct.py has to be changed accordingly. Unfortunately, even after all these changes, I get (for my specific optimization) repeated error messages from DIRECT (the Fortran function) like the following

No feasible point found in  200 iterations and  1101 function evaluations

and I can never find a better value of the objective function than the initial one.

[tinutjoy]

Anu update on this issue? Scipydirect is failing.

Edit:
This works with python 3.6.

Cheers!