# C/C++ Minpack

## What is Minpack?

This is the official description of Minpack, from the original ReadMe file: Minpack includes software for solving nonlinear equations and nonlinear least squares problems. Five algorithmic paths each include a core subroutine and an easy-to-use driver. The algorithms proceed either from an analytic specification of the Jacobian matrix or directly from the problem functions. The paths include facilities for systems of equations with a banded Jacobian matrix, for least squares problems with a large amount of data, and for checking the consistency of the Jacobian matrix with the functions.

The original authors of the FORTRAN version are Jorge More', Burt Garbow, and Ken Hillstrom from Argonne National Laboratory, and the code can be obtained from Netlib.

Minpack is probably the best open-source implementation of the Levenberg-Marquardt algorithm (in fact, it is even better, since it adds to L-M automatic variables scaling). There is another open-source L-M implementation in C/C++, levmar by Manolis Lourakis, but unfortunately is is released under the GPL, which restricts its inclusion in commercial software. Minpack is licensed under a BSD-like license (available in the distribution).

## What about CMinpack?

In July 2002 (before levmar), Manolis Lourakis (lourakis at ics forth gr) released a C version of Minpack, called CMinpack, obtained from the FORTRAN version using f2c and some limited manual editing. However, this version had several problems, which came from the FORTRAN version:

1. All the function prototypes were following the original FORTRAN call conventions, so that all parameters are passed by reference (if a function needs an int parameter, you have to pass a pointer to this int).
2. There were lots of static variables in the code, thus you could not optimize a function which required calling Minpack to be evaluated (a minimization-of-minimization problem for example): The Minpack code is not reentrant.
3. If the function to be optimized has to use extra parameters or data (this is the case most of the time), the only way to access them was though global variables, which is very bad, especially if you want to use the same function with different data in different threads: The Minpack code is not MT-Safe.
4. There was no C/C++ include file.
5. Examples and tests were missing from the distribution, although there are some FORTRAN examples in the documentation.

## Why is C/C++ Minpack better?

I took a dozen of hours to rework all these problems, and came out with a pure C version of Minpack, with has standard (ISO C99) parameters passing, is fully reentrant, multithread-safe, and has a full set of examples and tests:

1. Input variables are now passed by value, output variables are passed by reference. The keyword "const" is used as much as possible for constant arrays. The return value of each function is now used to get the function status (it was obtained via the IFLAG or INFO parameter in Minpack).
2. All non-const static variables were removed, and the code was tested after that. Luckily, Minpack didn't use the nastiest feature in FORTRAN: all local variables are static, so that a function can behave differently when you call it several times.
3. The function to be minimized and all the Minpack functions now take an extra "void*" argument, which can be used to pass any pointer-to-struct or pointer-to-class, and you can put all you extra parameters and data in that struct. Just cast this pointer to the appropriate pointer type in your function, and there they are! There is no need for global variables anymore. Be careful if you access the same object from different threads, though (a solution is to protect this extra data with a mutex).
4. The Debian project did a C include file for Minpack. It still needed some work (add consts and C++ compatibility), so I did this work, and used the include file for the FORTRAN version as the base for my C/C++ version.
5. The Debian project also translated all the FORTRAN examples to C. I worked from these to produce examples which also call my C/C++ version of Minpack instead of the FORTRAN version. Also included in the distribution are reference output files produced by the test runs (for comparison).

If you use C/C++ Minpack for a publication, you should cite it as:

    @misc{cminpack,
      title={C/C++ Minpack},
      author={Devernay, Fr{\'e}d{\'e}ric},
      year={2007},
      howpublished = "\url{http://devernay.github.io/cminpack}",
    }

## Distribution

The distribution contains:

- The CMinpack code, where static and global variables were removed (it is thus reentrant, but not MT-Safe).
- The C/C++ Minpack code (reentrant and MT-Safe).
- C and C++-compatible include files for Minpack or CMinpack (minpack.h) and C/C++ Minpack (cminpack.h), and Unix Makefiles.
- Full original documentation, translated to HTML, and all the examples, tests, and reference test results, so that you can check if the code runs properly on your machine before really using it.
- The extra covariance function "covar", with sample uses in the tlmder and tlmdir examples. Note that the result of covar has to be scaled by some factor, as shown in the source file examples/tlmderc.c (look for the documentation of the NAG function E04YCF for further explanations).

It is distributed under the original Minpack license (see the file CopyrightMINPACK.txt in the distribution).

### Download

- cminpack-1.3.11.tar.gz (latest version)
- cminpack-1.3.10.tar.gz
- cminpack-1.3.9.tar.gz
- cminpack-1.3.8.tar.gz
- cminpack-1.3.7.tar.gz
- cminpack-1.3.6.tar.gz
- cminpack-1.3.5.tar.gz
- cminpack-1.3.4.tar.gz
- cminpack-1.3.3.tar.gz
- cminpack-1.3.2.tar.gz
- cminpack-1.3.1.tar.gz
- cminpack-1.2.2.tar.gz
- cminpack-1.2.1.tar.gz
- cminpack-1.2.0.tar.gz
- cminpack-1.1.5.tar.gz
- cminpack-1.1.4.tar.gz
- cminpack-1.1.3.tar.gz
- cminpack-1.1.1.tar.gz
- cminpack-1.0.4.tar.gz
- cminpack-1.0.3.tar.gz
- cminpack-1.0.2.tar.gz
- cminpack-1.0.1.tar.gz

GitHub repository: https://github.com/devernay/cminpack

### Building CMinpack

CMinpack can be built with CMake. By default, CMake will build a single precision library named `cminpacks`, and a double precision library named `cminpack`. You can choose to build only one of the single, double, or long double precision variants by setting `CMINPACK_PRECISION` to one of "s", "d", or "ld" respectively (e.g. `cmake -DCMINPACK_PRECISION=d ..`).

The source distribution also contains a Makefile which can be used to build cminpack for double precision (the default, `make` or `make double`) for single-precision (`make float`), half-precision (`make half`), long double precision (`make longdouble`), and even CUDA (`make cuda`), or using LAPACK for linear algebra (`make lapack`). Unit tests are also provided, but results may depend on the platform (`make checkdouble checkfloat checkhalf checklongdouble checklapack`).

### Using CMinpack

The CMinpack calls have the same name as the FORTRAN functions, in lowercase (e.g. `lmder(...)`). See the links to the documentation below, or take a look at the simple examples in the `examples` directory of the distribution. The simple examples are named after the function they call: `tlmder.c` is the simple example for `lmder`.

If you want to use the single precision CMinpack, you should define __cminpack_float__ before including `cminpack.h`. __cminpack_half__ has to be defined for the half-precision version (and the code needs to be compiled with a C++ compiler).

The single-precision versions of the functions are prefixed by "s" (as in "`slmder(...)`"), and the half-precision are prefixed by "h".

CMinpack defines __cminpack_real__ as the floating point type, and the `__cminpack_func__()` macro can be used to call CMinpack functions independently of the precision used (as in the examples). However, you shouldn't use these macros in your own code, since your code is probably designed for a specific precision, and you should prefer calling directly `lmder(...)` or `slmder(...)`.

## Documentation

- The original documentation for MINPACK
- The Debian GNU/Linux manual pages for MINPACK
- MINPACK on Wikipedia
- B.S. Garbow, K.E. Hillstrom and J.J. Moré: Implementation Guide for MINPACK-1. ANL-80-68 (July 1980)
- J.J. Moré, B.S. Garbow and K.E. Hillstrom: User Guide for MINPACK-1. ANL-80-74 (August 1980) : Chapters 1-3, Chapter 4
- Documentation for the test functions (the fourteen nonlinear equations problems defined in `vecfcn.f/.c` and `vecjac.f/.c`, the eighteen nonlinear least squares problems defined in `ssqfcn.f/.c` and `ssqjac.f/.c`, and the eighteen nonlinear unconstrained minimization problems defined in `objfcn.f`, `grdfcn.f` and `hesfcn.f`) can be found in the following papers:
  - Moré, J. J., Garbow, B. S., and Hillstrom, K. E. (1981). Testing Unconstrained Optimization Software. ACM Transactions on Mathematical Software (TOMS), 7(1), 17-41. doi:10.1145/355934.355936
  - Moré, J. J., Garbow, B. S., and Hillstrom, K. E. (1981). Algorithm 566: FORTRAN Subroutines for Testing Unconstrained Optimization Software [C5],[E4]. ACM Transactions on Mathematical Software (TOMS), 7(1), 136-140. doi:10.1145/355934.355943
  - Averbukh, V. Z., Figueroa, S., and Schlick, T. (1992). HESFCN| A FORTRAN Package of Hessian Subroutines for Testing Nonlinear Optimization Software. New York University, New York, NY.
  - Averbukh, V. Z., Figueroa, S., and Schlick, T. (1994). Remark on algorithm 566. ACM Transactions on Mathematical Software (TOMS), 20(3), 282-285. doi:10.1145/192115.192128

## References

### LMDER/LMDIF

- J.J. Moré, "The Levenberg-Marquardt Algorithm, Implementation and Theory", in Numerical Analysis, G.A. Watson, Editor, Lecture Notes in Mathematics 630, Springer-Verlag, 1977. doi:10.1007/BFb0067700

### HYBRJ/HYBRD

- M. J. D. Powell, A Hybrid Method for Nonlinear Equations. Numerical Methods for Nonlinear Algebraic Equations, P. Rabinowitz, editor. Gordon and Breach, 1970.

## Simulating box constraints

Note that box constraints can easily be simulated in C++ Minpack, using a change of variables in the function (that hint was found in the lmfit documentation).

For example, say you want `xmin[j] < x[j] < xmax[j]`, just apply the following change of variable at the beginning of `fcn` on the variables vector, and also on the computed solution after the optimization was performed:

      for (j = 0; j < 3; ++j) {
        real xmiddle = (xmin[j]+xmax[j])/2.;
        real xwidth = (xmax[j]-xmin[j])/2.;
        real th =  tanh((x[j]-xmiddle)/xwidth);
        x[j] = xmiddle + th * xwidth;
        jacfac[j] = 1. - th * th;
      }

This change of variables preserves the variables scaling, and is almost the identity near the middle of the interval.

Of course, if you use `lmder`, `lmder1`, `hybrj` or `hybrj1`, the Jacobian must be also consistent with that new function, so the column of the original Jacobian corresponding to `x1` must be multiplied by the derivative of the change of variable, i.e `jacfac[j]`.

Similarly, each element of the covariance matrix must be multiplied by `jacfac[i]*jacfac[j]`.

For examples on how to implement this in practice, see the portions of code delimited by "#ifdef BOX_CONSTRAINTS" in the following source files: `tlmderc.c`, `thybrj.c`, `tchkderc.c`.

## Equivalence table with other libraries

The following table may be useful if you need to switch to or from another library.

Equivalence table between MINPACK and NAG, NPL, SLATEC, levmar and GSL
MINPACK | NAG | NPL | SLATEC | levmar | GSL
--- | --- | --- | --- | --- | ---
lmdif | E04FCF | LSQNDN | DNLS1 | dlevmar_dif |
lmdif1 | E04FYF | LSNDN1 | DNLS1E | dlevmar_dif |
lmder | E04GDF/E04GBF | LSQFDN | DNLS1 | dlevmar_der | gsl_multifit_fdfsolver_lmsder
lmder1 | E04GZF | LSFDN2 | DNLS1E | dlevmar_der |
lmstr | * | * | DNLS1 |  |
hybrd | C05NCF | * | DNSQ |  | gsl_multiroot_fsolver_hybrids
hybrd1 | C05NBF | * | DNSQE |  |
hybrj | C05PCF | * | DNSQ |  | gsl_multiroot_fdfsolver_hybridsj
hybrj1 | C05PBF | * | DNSQE |  |
covar | E04YCF | * | DCOV |  | gsl_multifit_covar
chkder | C05ZAF |  |  |  |

## Other MINPACK implementations

- Modernized Fortran MINPACK is a F90 rewrite with a C API and Python bindings.
- by Charles Bouillaguet: minpack-1.1 is another C version with better, more portable, testing.
- by John Burkardt: C++, FORTRAN77, FORTRAN90.
- by Steve Verrill: Java (documentation, problems).
- by Alan Miller: FORTRAN90.
- Quantlib has a C++ MINPACK hidden deep inside (C++, header, tutorial).
- GPL-licensed implementations of MINPACK algorithms in C are availaible from the GNU Scientific Library.
- Python library scipy, module `scipy.optimize.leastsq`,
- IDL, add-on MPFIT.
- R has the minpack.lm package.
- Eigen has an unsupported nonlinear optimization module based on cminpack.
- Ceres Solver is not derived from MINPACK, but is probably the best available alternative, with lots of features (New BSD License).

## History

- 1.3.10 (11/09/2024): Fix BLAS support and disable BLAS by default.
- 1.3.9 (28/05/2024): Mainly CMake fixes.
- 1.3.8 (02/09/2021): CMake now builds by default the single-, double-, and extended-precision versions. Avoid promoting to doubles in all operations for the single-precision version.
- 1.3.7 (09/12/2020): Makefile cleanups. Cmake-related fixes. Add Appveyor CI. Add support for single-precision CBLAS and LAPACK.
- 1.3.6 (24/02/2017): Fix FreeBSD build. CMake: install CMinpackConfig.cmake rather than FindCMinpack.cmake, and add option USE_BLAS to compile with blas.
- 1.3.5 (28/05/2016): Add support for compiling a long double version (Makefile only). CMake: static libraries now have the suffix `_s`.
- 1.3.4 (28/05/2014): Add FindCMinpack.cmake cmake module. If you use the cmake install, finding CMinpack from your `CMakeLists.txt` (especially on Windows) is as easy as `find_package(CMinpack)`.
- 1.3.3 (04/02/2014): Add documentation and examples abouts how to add box constraints to the variables. Continuous integration using Travis CI
- 1.3.2 (27/10/2013): Minor change in the CMake build: also set SOVERSION.
- 1.3.1 (02/10/2013): Fix CUDA examples compilation, and remove non-free files.
- 1.3.0 (09/06/2012): Optionally use LAPACK and CBLAS in lmpar, qrfac, and qrsolv. Added "make lapack" to build the LAPACK-based cminpack and "make checklapack" to test it (results of the test may depend on the underlying LAPACK and BLAS implementations). On 64-bits architectures, the preprocessor symbol __LP64__ must be defined (see cminpackP.h) if the LAPACK library uses the LP64 interface (i.e. 32-bits integer, vhereas the ILP interface uses 64 bits integers).
- 1.2.2 (16/05/2012): Update Makefiles and documentation (see "Using CMinpack" above) for easier building and testing.
- 1.2.1 (15/05/2012): The library can now be built as double, float or half versions. Standard tests in the "examples" directory can now be lauched using "make check" (to run common tests, including against the float version), "make checkhalf" (to test the half version) and "make checkfail" (to run all the tests, even those that fail).
- 1.2.0 (14/05/2012): Added original FORTRAN sources for better testing (type "make" in directory fortran, then "make" in examples and follow the instructions). Added driver tests lmsdrv, chkdrv, hyjdrv, hybdrv. Typing "make alltest" in the examples directory will run all possible test combinations (make sure you have gfortran installed).
- 1.1.5 (04/05/2012): cminpack now works in CUDA, thanks to Jordi Bataller Mascarell, type "make" in the "cuda" subdir (be careful, though: this is a straightforward port from C, and each problem is solved using a single thread). cminpack can now also be compiled with single-precision floating point computation (define __cminpack_real__ to float when compiling and using the library). Fix cmake support for CMINPACK_LIB_INSTALL_DIR. Update the reference files for tests.
- 1.1.4 (30/10/2011): Translated all the Levenberg-Marquardt code (lmder, lmdif, lmstr, lmder1, lmdif1, lmstr1, lmpar, qrfac, qrsolv, fdjac2, chkder) to use C-style indices.
- 1.1.3 (16/03/2011): Minor fix: Change non-standard strnstr() to strstr() in genf77tests.c.
- 1.1.2 (07/01/2011): Fix Windows DLL building (David Graeff) and document covar in cminpack.h.
- 1.1.1 (04/12/2010): Complete rewrite of the C functions (without trailing underscore in the function name). Using the original FORTRAN code, the original algorithms structure was recovered, and many goto's were converted to if...then...else. The code should now be both more readable and easier to optimize, both for humans and for compilers. Added lmddrv and lmfdrv test drivers, which test a lot of difficult functions (these functions are explained in Testing Unconstrained Optimization Software by Moré et al.). Also added the pkg-config files to the cmake build, as well as an "uninstall" target, contributed by Geoffrey Biggs.
- 1.0.4 (18/10/2010): Support for shared library building using CMake, thanks to Goeffrey Biggs from AIST and Radu Bogdan Rusu from Willow Garage. Shared libraries can be enabled using cmake options, as in: `cmake -DSHARED_LIBS=ON -DBUILD_EXAMPLES=OFF path_to_sources`
- 1.0.3 (18/03/2010): Added CMake support. XCode build is now Universal (i386+ppc). Added tfdjac2_ and tfdjac2c examples, which test the accuracy of a finite-differences approximation of the Jacobian. Bug fix in tlmstr1 (signaled by Thomas Capricelli).
- 1.0.2 (27/02/2009): Added Xcode and Visual Studio project files
- 1.0.1 (17/12/2007): bug fix in covar() and covar_(), the computation of tolr caused a segfault (signaled by Timo Hartmann).
- 1.0.0 (24/04/2007): Initial revision.

## Future work

There is now a very powerful alternative to MINPACK, which is the Ceres Solver. You may want to consider using Ceres for any new project.

The main feature that's missing on cminpack is the possibility to add constraints on variables. Simple boundary constraints should be enough, as implemented in ALGLIB or MPFIT, and they can easily be implemented using the hack above (section "Simulating box constraints").

levmar also has linear constraints, but they should not be necessary since linear constraints can be changed to box constraints by a simple change of variables. If you really need nonlinear constraints, and no reparameterization of variables (which may be able to linearize these constraints), you should consider using NLopt instead of cminpack.

Please file a GitHub issue for any suggestion or request.