[RFC] Version 0.90 release candidate

[hcho3]

You can try the 0.90 release candidate by downloading the following:

• Linux (64-bit) Python: https://xgboost-ci.net/job/xgboost/job/PR-4475/1/artifact/python-package/dist/xgboost-0.90-py2.py3-none-any.whl
• Windows (64-bit) Python: https://xgboost-ci.net/job/xgboost-win64/job/PR-4475/1/artifact/python-package/dist/xgboost-0.90-py2.py3-none-any.whl
  (Don't worry about the file names here, they will be renamed)

v0.90 (2019.05.18)

XGBoost Python package drops Python 2.x (#4379, #4381)

Python 2.x is reaching its end-of-life at the end of this year. Many scientific Python packages are now moving to drop Python 2.x.

XGBoost4J-Spark now requires Spark 2.4.x (#4377)

• Spark 2.3 is reaching its end-of-life soon. See discussion at XGBoost 0.90 Roadmap #4389.
• Consistent handling of missing values ([jvm-packages] handle NaN as missing value explicitly #4309, [BREAKING][jvm-packages] fix the non-zero missing value handling #4349, [jvm-packages] fix compatibility problem of spark version #4411): Many users had reported issue with inconsistent predictions between XGBoost4J-Spark and the Python XGBoost package. The issue was caused by Spark mis-handling non-zero missing values (NaN, -1, 999 etc). We now alert the user whenever Spark doesn't handle missing values correctly ([jvm-packages] handle NaN as missing value explicitly #4309, [BREAKING][jvm-packages] fix the non-zero missing value handling #4349). See the tutorial for dealing with missing values in XGBoost4J-Spark. This fix also depends on the availability of Spark 2.4.x.

Roadmap: better performance scaling for multi-core CPUs (#4310)

• Poor performance scaling of the hist algorithm for multi-core CPUs has been under investigation (Call for contribution: improve multi-core CPU performance of 'hist' #3810). Optimizations of pre-processing for 'hist' tree method #4310 optimizes quantile sketches and other pre-processing tasks. Special thanks to @SmirnovEgorRu.

Roadmap: Harden distributed training (#4250)

• Make distributed training in XGBoost more robust by hardening Rabit, which implements the AllReduce primitive. In particular, improve test coverage on mechanisms for fault tolerance and recovery. Special thanks to @chenqin.

New feature: Multi-class metric functions for GPUs (#4368)

• Metrics for multi-class classification have been ported to GPU: merror, mlogloss. Special thanks to @trivialfis.
• With supported metrics, XGBoost will select the correct devices based on your system and n_gpus parameter.

New feature: Scikit-learn-like random forest API (#4148, #4255, #4258)

• XGBoost Python package now offers XGBRFClassifier and XGBRFRegressor API to train random forests. See the tutorial. Special thanks to @canonizer

New feature: use external memory in GPU predictor (#4284, #4396, #4438, #4457)

• It is now possible to make predictions on GPU when the input is read from external memory. This is useful when you want to make predictions with big dataset that does not fit into the GPU memory. Special thanks to @rongou, @canonizer, @sriramch.

  dtest = xgboost.DMatrix('test_data.libsvm#dtest.cache')
  bst.set_param('predictor', 'gpu_predictor')
  bst.predict(dtest)

• Coming soon: GPU training (gpu_hist) with external memory

New feature: XGBoost can now handle comments in LIBSVM files (#4430)

• Special thanks to @trivialfis and @hcho3

New feature: Embed XGBoost in your C/C++ applications using CMake (#4323, #4333, #4453)

• It is now easier than ever to embed XGBoost in your C/C++ applications. In your CMakeLists.txt, add xgboost::xgboost as a linked library:

  find_package(xgboost REQUIRED)
  add_executable(api-demo c-api-demo.c)
  target_link_libraries(api-demo xgboost::xgboost)

  XGBoost C API documentation is available. Special thanks to @trivialfis

Performance improvements

• Use feature interaction constraints to narrow split search space (Use feature interaction constraints to narrow search space for split candidates #4341, Simplify bound checking in feature interaction constraints #4428)
• Additional optimizations for gpu_hist (Optimisations for gpu_hist. #4248, Further optimisations for gpu_hist. #4283)
• Reduce OpenMP thread launches in gpu_hist ( Combine thread launches into single launch per tree for gpu_hist #4343)
• Additional optimizations for multi-node multi-GPU random forests. (Improved multi-node multi-GPU random forests. #4238)
• Allocate unique prediction buffer for each input matrix, to avoid re-sizing GPU array (Allow unique prediction vector for each input matrix #4275)
• Remove various synchronisations from CUDA API calls (Remove various synchronisations from cuda API calls, instrument monitor #4205)
• XGBoost4J-Spark
  • Allow the user to control whether to cache partitioned training data, to potentially reduce execution time ([jvm-packages] add configuration flag to control whether to cache transformed training set #4268)

Bug-fixes

• Fix node reuse in hist (Fix node reuse. #4404)
• Fix GPU histogram allocation (Fix Histogram allocation. #4347)
• Fix matrix attributes not sliced (Fix matrix attributes not sliced #4311)
• Revise AUC and AUCPR metrics now work with weighted ranking task (In AUC and AUCPR metrics, detect whether weights are per-instance or per-group #4216, Make AUCPR work with multiple query groups #4436)
• Fix timer invocation for InitDataOnce() in gpu_hist (minor fix: log InitDataOnce() only when it is actually called #4206)
• Fix R-devel errors (Fix errors on R-devel #4251)
• Make gradient update in GPU linear updater thread-safe (Fix multi-GPU test failures #4259)
• Prevent out-of-range access in column matrix (Address #4042: Prevent out-of-range access in column matrix #4231)
• Don't store DMatrix handle in Python object until it's initialized, to improve exception safety (Don't store DMatrix handle until it's initialized. #4317)
• XGBoost4J-Spark
  • Fix non-deterministic order within a zipped partition on prediction ([BLOCKING][jvm-packages] fix non-deterministic order within a partition (in the case of an upstream shuffle) on prediction  #4388)
  • Remove race condition on tracker shutdown ([jvm-packages] Fix "Rabit returns with exit code 3" problem #4224)
  • Allow set the parameter maxLeaves. ([xgboost4j-spark] Allow set the parameter "maxLeaves". #4226)
  • Allow partial evaluation of dataframe before prediction ([jvm-packages] allow partial evaluation of dataframe before prediction #4407)
  • Automatically set maximize_evaluation_metrics if not explicitly given ([jvm-packages] Automatically set maximize_evaluation_metrics if not explicitly given in XGBoost4J-Spark #4446)

API changes

• Deprecate reg:linear in favor of reg:squarederror. ( Deprecate reg:linear' in favor of reg:squarederror'. #4267, Change obj name to reg:squarederror in learner. #4427)
• Add attribute getter and setter to the Booster object in XGBoost4J ([jvm-packages] support operating attributes of booster #4336)

Maintenance: Refactor C++ code for legibility and maintainability

• Fix clang-tidy warnings. (Fix clang-tidy. #4149)
• Remove deprecated C APIs. (Remove deprecated C APIs. #4266)
• Use Monitor class to time functions in hist. (Use Monitor in quantile hist. #4273)
• Retire DVec class in favour of c++20 style span for device memory. (Retire DVec class in favour of c++20 style span for device memory. #4293)
• Improve HostDeviceVector exception safety (- make test fails to find testxgboost, if a build directory is created #4301)

Maintenance: testing, continuous integration, build system

• Major refactor of CMakeLists.txt (Refactor CMake scripts. #4323, C API example #4333, Correctly determine cuda version #4453): adopt modern CMake and export XGBoost as a target
• Major improvement in Jenkins CI pipeline ([RFC] Improving continuous integration #4234)
  • Migrate all Linux tests to Jenkins ([CI] Refactor Jenkins CI pipeline + migrate all Linux tests to Jenkins #4401)
  • Builds and tests are now de-coupled, to test an artifact against multiple versions of CUDA, JDK, and other dependencies ([CI] Refactor Jenkins CI pipeline + migrate all Linux tests to Jenkins #4401)
  • Add Windows GPU to Jenkins CI pipeline ([CI] Add Windows GPU to Jenkins CI pipeline #4463, [CI] Add Python and C++ tests for Windows GPU target #4469)
• Support CUDA 10.1 (support cuda 10.1 #4223, Downgrade Jenkins agent to CUDA 9.2, to fix broken NCCL2 installation #4232, Mark CUDA 10.1 as unsupported. #4265, Add cuda 10.1 support #4468)
• Python wheels are now built with CUDA 9.0, so that JIT is not required on Volta architecture ([CI] Build XGBoost wheels with CUDA 9.0 #4459)
• Integrate with NVTX CUDA profiler (Remove various synchronisations from cuda API calls, instrument monitor #4205)
• Add a test for cpu predictor using external memory (add a test for cpu predictor using external memory #4308)
• Refactor tests to get rid of duplication (refactor tests to get rid of duplication #4358)
• Remove test dependency on craigcitro/r-travis, since it's deprecated ([r-package] cut CI-time dependency on craigcitro/r-travis (fixes #4348) #4353)
• Add files from local R build to .gitignore (added files from local R build to gitignore #4346)
• Make XGBoost4J compatible with Java 9+ by revising NativeLibLoader ([jvm-packages] Fixing the NativeLibLoader on Java 9+ #4351)
• Jenkins build for CUDA 10.0 (jenkins build for cuda 10.0 #4281)
• Remove remaining silent and debug_verbose in Python tests (Remove remaining silent and debug_verbose. #4299)
• Use all cores to build XGBoost4J lib on linux ([jvm-packages] use all cores to build on linux #4304)
• Upgrade Jenkins Linux build environment to GCC 5.3.1, CMake 3.6.0 ([CI] Upgrade to GCC 5.3.1, CMake 3.6.0 #4306)
• Make CMakeLists.txt compatible with CMake 3.3 (Make CMakeLists.txt compatible with CMake 3.3 #4420)
• Add OpenMP option in CMakeLists.txt (add USE_OPENMP option in CMakeLists.txt #4339)
• Get rid of a few trivial compiler warnings (Get rid of a few trivial compiler warnings. #4312)
• Add external Docker build cache, to speed up builds on Jenkins CI ([CI] Add external Docker build cache #4331, [CI] Optimize external Docker build cache #4334, [CI] Properly cache Docker containers for R tests #4458)
• Fix Windows tests ([CI] Fix Windows tests #4403)
• Fix a broken python test (fix broken python test #4395)
• Use a fixed seed to split data in XGBoost4J-Spark tests, for reproducibility ([jvm-packages] fix the split of input #4417)
• Add additional Python tests to test training under constraints (Add additional Python tests for training under constraints #4426)
• Enable building with shared NCCL. (Enable building with shared NCCL. #4447)

Usability Improvements, Documentation

• Document limitation of one-split-at-a-time Greedy tree learning heuristic (Document limitation of one-split-at-a-time Greedy tree learning heuristic #4233)
• Update build doc: PyPI wheel now support multi-GPU (Update build doc: PyPI wheel now support multi-GPU #4219)
• Fix docs for num_parallel_tree (Fix docs for num_parallel_tree #4221)
• Fix document about colsample_by* parameter (Fix document about colsample_by* parameter #4340)
• Make the train and test input with same colnames. (Make the train and test input with same colnames. #4329)
• Update R contribute link. (Update R contribute link. #4236)
• Fix travis R tests (Fix travis R tests #4277)
• Log version number in crash log in XGBoost4J-Spark ([jvm-packages] logging version number #4271, [jvm-packages] do not filter shared library files #4303)
• Allow supression of Rabit output in Booster::train in XGBoost4J ([jvm-packages] Allow supression of Rabit output in Booster::train in xgboost4j #4262)
• Add tutorial on handling missing values in XGBoost4J-Spark ([jvm-packages] Tutorial on handling missing values #4425)
• Fix typos (fixed typos in R package docs #4345, debug(typo-fix) #4393, typo #4432, max_digits10 guarantees float decimal roundtrip #4435)
• Added language classifier in setup.py (Added language classifier for python #4327)
• Added Travis CI badge (Add travis logo to README badge #4344)
• Add BentoML to use case section ([Docs]Update demo readme's use case section with BentoML #4400)
• Remove subtly sexist remark (Update gaming example in documentation #4418)
• Add R vignette about parsing JSON dumps (Dev/xgboost from json vignette #3980 #4439)

Acknowledgement

Contributors: Nan Zhu (@CodingCat), Adam Pocock (@Craigacp), Daniel Hen (@Daniel8hen), Jiaxiang Li (@JiaxiangBU), Rory Mitchell (@RAMitchell), Egor Smirnov (@SmirnovEgorRu), Andy Adinets (@canonizer), Jonas (@elcombato), Harry Braviner (@harrybraviner), Philip Hyunsu Cho (@hcho3), Tong He (@hetong007), James Lamb (@jameslamb), Jean-Francois Zinque (@jeffzi), Yang Yang (@jokerkeny), Mayank Suman (@mayanksuman), jess (@monkeywithacupcake), Hajime Morrita (@omo), Ravi Kalia (@project-delphi), @ras44, Rong Ou (@rongou), Shaochen Shi (@shishaochen), Xu Xiao (@sperlingxx), @sriramch, Jiaming Yuan (@trivialfis), Christopher Suchanek (@wsuchy), Bozhao (@yubozhao)

Reviewers: Nan Zhu (@CodingCat), Adam Pocock (@Craigacp), Daniel Hen (@Daniel8hen), Jiaxiang Li (@JiaxiangBU), Laurae (@Laurae2), Rory Mitchell (@RAMitchell), Egor Smirnov (@SmirnovEgorRu), @alois-bissuel, Andy Adinets (@canonizer), Chen Qin (@chenqin), Harry Braviner (@harrybraviner), Philip Hyunsu Cho (@hcho3), Tong He (@hetong007), @jakirkham, James Lamb (@jameslamb), Julien Schueller (@jschueller), Mayank Suman (@mayanksuman), Hajime Morrita (@omo), Rong Ou (@rongou), Sara Robinson (@sararob), Shaochen Shi (@shishaochen), Xu Xiao (@sperlingxx), @sriramch, Sean Owen (@srowen), Sergei Lebedev (@superbobry), Yuan (Terry) Tang (@terrytangyuan), Theodore Vasiloudis (@thvasilo), Matthew Tovbin (@tovbinm), Jiaming Yuan (@trivialfis), Xin Yin (@xydrolase)

@dmlc/xgboost-committer

[terrytangyuan]

Thanks! LGTM

[CodingCat]

found some format problem in doc #4476,

ran some integration tests, results are consistent with previous versions

expecting some users came and ask why my xgboost4j-spark throw exceptions in 0.9 (breaking changes on missing value handling)

[hcho3]

Will merge after tests pass