The approach is pretty straightforward and very similar to the CuDNN integration:

- Add some CMake magic to detect NNPACK if it's installed on the user's
  system.
- Add some Makefile.config options
- Add some thread-local state (a threadpool for NNPACK to caffe::Caffe).
- Add an NNPACK `Engine` parameter to Convolution/Pooling/InnerProduct
  layers.
- In LayerFactory, ifdef around whether NNPACK is enabled and dispatch
  to NNPACK*Layer or *Layer depending on the Engine parameter.
- In NNPACK*Layer, ensure the NNPACK preconditions hold and call the
  appropriate nnpack_*  function, otherwise fall back to *Layer.

I'm not sure how common it is to be running CNN inference on Haswell and
beyond CPUs for Caffe users, but if it is it might be a nice boost for
them.

- Moved headers for NNPACK layers from src/caffe/layers to include/caffe/layers
- Detect situations not handled by nnp_max_pooling_output from its return code,
rather than hard-code them in the integration code.