Add a #define flag to enable nested autodiff in model_base class

[WardBrian]

Submission Checklist

• Run unit tests: ./runTests.py src/test/unit
• Run cpplint: make cpplint
• Declare copyright holder and open-source license: see below

Summary

The model base class exposes overloads to all the log_prob functions for double and math::var. It would be useful to expose math::fvar<var> overloads to allow higher-order autodiff from base class instances, but this does not work with all Stan models (particularly ODE models and other uses of implicit functions).

This PR introduces opt-in only overloads. This allows downstream users, who want to provide model Hessians or Hessian-vector products, to still use stan::model::model_base as a type. Currently, the Hessian functionality in (e.g.) stan::model::hessian relies on a template type which can never be filled by stan::model::model_base; after this PR it would be possible if the define STAN_MODEL_AD_HESSIAN is used.

These signatures are identical to the previously existing ones except for their scalar type.

To make this even more concrete, in an interface like bridgestan, we can use this same flag to switch from using finite difference Hessians to full-AD for models which support it.

Intended Effect

The ability to use model_base instances in stan::model::hessian on an opt-in basis.

Side Effects

Hopefully none.

Documentation

I'm not sure where this would be best to document

Copyright and Licensing

Please list the copyright holder for the work you are submitting (this will be you or your assignee, such as a university or company):

Simons Foundation

By submitting this pull request, the copyright holder is agreeing to license the submitted work under the following licenses:

• Code: BSD 3-clause (https://opensource.org/licenses/BSD-3-Clause)
• Documentation: CC-BY 4.0 (https://creativecommons.org/licenses/by/4.0/)

[bob-carpenter]

This looks great, but I have two questions:

1. I would prefer something like FVAR_VAR instead of AD_HESSIAN, because this signature will be used for any second order derivatives, not just Hessians.
2. Is there any way to test this? Does any of the other functionality have tests here? I'm thinking something simple like making sure the template-free version get the same answer for some inputs.

[WardBrian]

@bob-carpenter agreed it is better to use a different name.

I’m not sure how to test this - I don’t know if our test framework lets me specify a #define for only some tests, but I can look. In principle, if the fvar versions did disagree, that would be a bug in Stan math or stanc3, not in these signatures

[rok-cesnovar]

Looks good to me. Regarding tests, I agree with Brian's statement: "In principle, if the fvar versions did disagree, that would be a bug in Stan math or stanc3, not in these signatures".

[bob-carpenter]

I don’t know if our test framework lets me specify a #define for only some tests

I suggest asking @serban-nicusor-toptal for help. I suspect it's already possible because we probably need it for testing all the threading and GPU stuff, which are also controlled by environment variables.

[rok-cesnovar]

You can also always add another Github Action test job, specify whatever you need to and run a few tests. Like we do here https://github.com/stan-dev/math/blob/cd60e75a4e63fa5c321ed6effcf825adfbde92c0/.github/workflows/header_checks.yml#L66

[WardBrian]

I've added tests to the existing file which tested the other signatures, and a new GHA which runs just the src/test/unit/model/ folder with the STAN_MODEL_FVAR_VAR define

[rok-cesnovar]

Great! Merge once Jenkins gives the green light.