profile likelihood successive optimizations fail on Rastrigin

[beniz]

to reproduce:

./test_functions -fname rastrigin -dim 10 -x0 5 -lambda 400 -sigma0 5 -le -seed 1234

yields

WARNING - profile likelihood linesearch: optimization error -15
WARNING - profile likelihood linesearch: optimization error -15
WARNING - profile likelihood linesearch: optimization error -15
WARNING - profile likelihood linesearch: optimization error -15
...
dim 0 in [-0.380559,0.184843] with error [-0.380559,0.184843] / status=[-15,-15]
x=([   0.184843 -0.00323412  -0.0160101  0.00422739    0.010278   0.0214382  -0.0220671  -0.0547048    0.109611  -0.0142954],[   -0.380559   0.00108891  0.000162276   0.00359662   0.00277633 -0.000746278   0.00457592   0.00180682 -0.000621765  -0.00118962])
fvalue=(9.23829,17.4681)

where fvalue should be in between 0.09 and 0.11 instead.

Reducing lambda to 200 succeeds, though the optimization misses the global minimum.

From investigation, it seems the successive optimizations around the minimum in order to capture the bounds do fail with error -15, i.e. CONDITIONCOV.

[beniz]

Now compute profile likelihood by solving a series of lower dimensional problems, and rotating the problem with eigenvectors for line search. This combination clears the problem on Rastrigin and appears to work well on other test functions as well.

FTR, the triggering of condition cov stopping criteria was due to very low eigenvalue corresponding to dimension in which a parameter value was fixed.

Still need to re-test the contour function in order to see if there's any impact on it.

[beniz]

This is now definitely fixed:

./test_functions -fname rastrigin -ftarget 1e-8 -dim 10 -x0 5 -sigma0 5 -le -lambda 400 -le_maxiters 50

Correctly yields:

confidence intervals:
dim 0 in [-0.0222937904687184,0.0222985951716267] with error [-0.0222961928201725,0.0222961928201725] / status=[1,1] / fvalue=(0.0984855236030171,0.0984431556679226)
dim 1 in [-0.0224131261948192,0.0224126919591372] with error [-0.0224129090769782,0.0224129090769782] / status=[1,1] / fvalue=(0.0994981447406786,0.0994942957316596)
dim 2 in [-0.0223523757973581,0.022349748127916] with error [-0.0223510619626371,0.0223510619626371] / status=[1,1] / fvalue=(0.0989603808684052,0.098937153395255)
dim 3 in [-0.0223435612383877,0.022343958550947] with error [-0.0223437598946674,0.0223437598946674] / status=[1,1] / fvalue=(0.098885985586378,0.0988824746532941)
dim 4 in [-0.0230637211099416,0.0230623319896506] with error [-0.0230630265497961,0.0230630265497961] / status=[1,1] / fvalue=(0.105335452829451,0.105348120512147)
dim 5 in [-0.0215893955590527,0.021584367727493] with error [-0.0215868816432728,0.0215868816432728] / status=[1,1] / fvalue=(0.0922870203099535,0.0923299541057503)
dim 6 in [-0.0221659848293329,0.0221682219904595] with error [-0.0221671034098962,0.0221671034098962] / status=[1,1] / fvalue=(0.0973194935859212,0.0973391074246095)
dim 7 in [-0.0213192453759016,0.0213188388213376] with error [-0.021322010911055,0.0213160732861842] / status=[1,1] / fvalue=(0.0900337321101574,0.0900371609554966)
dim 8 in [-0.0232089383123021,0.0232068677948753] with error [-0.0232079030535887,0.0232079030535887] / status=[1,1] / fvalue=(0.106676566482941,0.10665756722109)
dim 9 in [-0.0217146215296334,0.0217084374077041] with error [-0.0217115294686687,0.0217115294686687] / status=[1,1] / fvalue=(0.093402493432361,0.0933493827654175)

for a 0.1 deviation from optima