The default gtol parameter of lsqrsolve might be too large.
Reported by Michael BAUDIN
Originally assigned to Paul BIGNIER
The default gtol parameter of lsqrsolve might be too large in some cases.
We have experiments for which this default tolerance was too large, so that the the output X was not accurate at all. The defails of these experiments are reported here :
http://wiki.scilab.org/PerformancesScilabMoreGarbowHillstromBenchmark#Failure_of_lsqrsolve_without_J
There is no way to define a default tolerance which satisfies both contradictory needs:
* the tolerance should be small, to guarantee convergence to an accurate X,
* the tolerance should be large, to guarantee a reasonably small number of iterations in the case where the optimization is difficult.
It is not possible to satisfy both needs at the same time.
According to the "User Guide for MINPACK-1", section 2.3, the tolerance on G is used in the G-convergence test:
"The G-convergence test measures the angle between the residual vector and the columns of the Jacobian matrix and thus can be expected to fail if either F(x*)=0 or any column of F'x(*) is zero. Also, note that there is no clear relationship between G-convergence and either X-convergence or F-convergence. Furthermore, the G-convergence test detects other critical points, namely maximizers and saddle points; therefore, termination with G-convergence should be examined carefully."
In the Minpack library, there are two version of the Levenberg-Marquardt:
* LMDER is the full-featured version: http://www.netlib.org/minpack/lmder.f
* LMDER1 is the simplified version: http://www.netlib.org/minpack/lmder1.f
LMDER has 3 tolerances : xtol, ftol and gtol while LMDER1 only has 1 tolerance tol. Hence, we might mimic the settings used in the LMDER1 routine, that is, the settings which are suggested by the authors of the module.
These settings are :
maxfev = 100*(n + 1)
ftol = tol
xtol = tol
gtol = zero
However, the default value in lsqrsolve is gtol=1.e-5:
* This does not correspond to a consistent value for doubles.
* This does not correspond to the settings suggested by LMDER1.
We suggest that the default value of gtol should be set to zero.
If this value does not allow to pass most of the tests in the More-Garbow-Hillstrom test set, we may increase it to sqrt(%eps)~1.e-8, but not larger. This would imply that at least half of the significant digits are satisfied in the G-convergence test.