[Roadmap] 1.1.0 Roadmap

[hcho3]

@dmlc/xgboost-committer please add your items here by editing this post. Let's ensure that

• Each item has to be associated with a ticket
• Major design/refactoring are associated with a RFC before committing the code
• Blocking issue must be marked as blocking
• Breaking change must be marked as breaking

For other contributors who have no permission to edit the post, please comment here about what you think should be in 1.1.0.

• Add Survival analysis objective: Accelerated Failure Time (Add Accelerated Failure Time loss for survival analysis task #4763)
• Optimized ApplySplit and UpdatePredictCache functions on CPU (Optimized ApplySplit and UpdatePredictCache functions on CPU #5244). This concludes the 'hist' optimization work by @SmirnovEgorRu. Roadmap: CPU optimizations - 'hist' method #5104
• Refactored setup.py (Rewrite setup.py. #5271, Fix CMake + MSVC build with setuptools. #5280)
• Gradient based sampling for GPU Hist (Gradient based sampling for external memory mode on GPU #5093)
• Wide dataset quantile performance improvement (Wide dataset quantile performance improvement #5306)
• Thread safe in-place prediction (Thread safe, inplace prediction. #5389).
• JSON serialization of R XGBoost object ([R] Basic implementation for R JSON serialization. #5123)
• DMatrix refactoring (WIP) (Predict on Ellpack. #5327, Move prediction cache to Learner. #5220, Pass shared pointer instead of raw pointer to Learner. #5302, Remove update prediction cache from predictors. #5312, Remove SimpleCSRSource #5315, Refactor SparsePageSource, delete cache files after use #5321). RFC [RFC] Possible DMatrix refactor #4354, Roadmap [Roadmap] DMatrix refactoring #5143
• Pass feature count from JVM to native layer ([jvm-packages] Add JVM_CHECK_CALL. #5199, Add number of columns to native data iterator. #5202, [jvm-packages]add feature size for LabelPoint and DataBatch #5303)
• Deterministic GPU Histogram building (Model reproduciblility with histogram tree method. #5023).
• Use configured header in CMake (Ensure that configured dmlc/build_config.h is picked up by Rabit and XGBoost #5514)
• Parameter validation for scikit-learn and R (Enable parameter validation for skl. #5477, Enable parameter validation for R. #5569)

[hcho3]

Proposal: this time, let us not wait too long until the next release.

[hcho3]

@JohnZed @datametrician

[avinashbarnwal]

Please Add Survival analysis objective: Accelerated Failure Time #4763

[datametrician]

LGTM especially #5023. Thanks @hcho3!

[hcho3]

@avinashbarnwal See the first item in the list.

[tqchen]

@hcho3 would be great if we can look into apache/tvm#4953

[hcho3]

@tqchen I will look.

[RAMitchell]

@CodingCat can please you help us resolve the last JVM issue for the release?

[trivialfis]

Reverting the status for DMatrix refactoring.

[hcho3]

@trivialfis Is DMatrix refactor blocking 1.1.0 release?

[trivialfis]

@hcho3 I just want #5504 before 1.1, the PR fixes prediction on device dmatrix which is part of dmatrix refactoring. Also, the weighted sketching is not yet implemented for device dmatrix, but this is not blocking.

[trivialfis]

Also please let me give a deeper look into #5285 . Will continue profiling in coming days.

[hcho3]

Got it. I am now reviewing #5123.

[trivialfis]

Labelled #5529 as breaking.

[hcho3]

@tqchen I cannot reproduce the issue in apache/tvm#4953.

[trivialfis]

I cannot reproduce the issue in apache/tvm#4953.

@tqchen Neither can I.

@hcho3 I'm happy to make the next release.

[hcho3]

#5209 is marked as blocking.

[trivialfis]

@hcho3 All blocking bugs are closed. Can we branch out? What can I help with the release process?

[hcho3]

@trivialfis If you can provide a summary of your contribution that would be great. I will create a new branch.

[trivialfis]

@hcho3 Here is a list of PRs that are related to me, I omitted some trivial changes.

Python

• Rewritten setup.py for aligning with Python package convension. (Rewrite setup.py. #5271, Fix CMake + MSVC build with setuptools. #5280)
• [Breaking] Now the scikit-learn featured package is called
  xgboost[scikit-learn] instead of xgboost[sklearn] (Use scikit-learn in extra dependencies. #5310).
• Enable parameter validation for scikit-learn interface. (Enable parameter validation for skl. #5477)
• Fix booster checks in scikit-learn interface. (Fix checking booster. #5505)
• Document update for cache and parameter validation. (Update Python doc. [skip ci] #5517)
• Fix scikit-learn nan tag for accepting inputs containing nan. (Fix skl nan tag. #5538)
• Assert matching length of evaluation inputs in scikit-learn interface. (Assert matching length of evaluation inputs. #5540)

R

• R can now call xgb.config to get a JSON representation of internal
  configuration from booster.
• Full raw serialization with model and parameters, which is used by bst$raw. ([R] Basic implementation for R JSON serialization. #5123)
• Fix r interaction constraints when number of features is greater than 1e5. (Fix r interaction #5543)
• Fix dropped booster attributes by restoring attributes in
  xgb.Booster.complete. ([R] Restore attributes in complete. #5573)
• Enable parameter validation. (Enable parameter validation for R. #5569)

JVM

• Add JVM_CHECK_CALL macro in JVM C++ wrapper, this avoids some segfaults when
  dmlc::Error is thrown in C++. ([jvm-packages] Add JVM_CHECK_CALL. #5199)
• Pass number of features from JVM to C++. Lower level code now doesn't have to guess the
  data shape. (Add number of columns to native data iterator. #5202)

C++

• Refactor prediction cache. (Pass shared pointer instead of raw pointer to Learner. #5302, Move prediction cache to Learner. #5220, Remove update prediction cache from predictors. #5312)

  Now XGBoost caches all DMatrix, and release the cache once DMatrix is expired.
  This way users no longer have to delete the booster before deleting DMatrix.
  Also the caching logic is simplified.

• Run GPU prediction on Ellpack page, which is part of the DMatrix refactoring. (Predict on Ellpack. #5327, Use ellpack for prediction only when sparsepage doesn't exist. #5504)

• Various fixes for pruner. (Fix pruner. #5335)

• Deterministic GPU histogram. Regression and classification are now deterministic for GPU
  Hist tree method. (POC implementation of deterministic GPU histogram. #5361)

• [maintenance] Split up LearnerImpl. (Split up LearnerImpl. #5350)

• Check whether current updater can modify a tree, avoids using wrong updaters. (Check whether current updater can modify a tree. #5406)

• Force GPU compressed buffer to be 4 bytes aligned to fix running
  cuda-memcheck. (Force compressed buffer to be 4 bytes aligned. #5441)

• [maintenance] Refactor tests with data generator. (Refactor tests with data generator. #5439)

• Reduce span check overhead. (Reduce span check overhead. #5464)

• [maintenance] Upgrade clang-tidy on CI, fixing all previously not detected errors in
  header files (Upgrade clang-tidy. #5469)

• Fix GPU tree statistic. Requires setting leaf stat when expanding tree. (Requires setting leaf stat when expanding tree. #5501)

• Remove distcol updater. (Remove distcol updater. #5507)

• [maintenance] Unify max nodes. (Unify max nodes. #5497)

• [Breaking] Remove makefiles. (Remove makefiles. #5513)

• Fix loading binary model by adding a header. (Write binary header. #5532)

• Fix slice and get info for classification and Accelerated failure time training. (Fix slice and get info. #5552)

• Fix non-openmp build. (Fix non-openmp build. #5566)

• Group aware GPU sketching, now GPU sketching algorithm can handle per-group
  weight. (Group aware GPU sketching. #5551)

• Fix configuration status with loading binary model. (Fix configuration not run. #5562)

Dask

• Integrate DMLC_TASK_ID for rabit initialization for better logging messages. ([dask] Use DMLC_TASK_ID. #5415)
• Fix prediction result when the order of partitions on different workers are not
  consistent. ([dask] Order the prediction result. #5416)
• Honor nthreads from dask worker. ([dask] Honor nthreads from dask worker. #5414)
• Accept inputs other than DaskDMatrix for prediction, now dask package can return a
  series when input is a dataframe. ([dask] Accept other inputs for prediction. #5428)
• Fix ignored missing value for dask scikit-learn wrapper. ([dask] Fix missing value for scikit-learn interface. #5435)

General

• Thread safe, inplace prediction. (Move thread local entry into Learner. #5396, Thread safe, inplace prediction. #5389, [dask] dask cudf inplace prediction. #5512)

  Now users can use inplace_predict on Python (including dask) and C for thread safe, lock free
  prediction on both CPU and GPU inputs.

• [Breaking] silent parameter is completely removed. Setting it will no-longer have any
  effect.

• [Breaking] Set output margin to True for custom objective. ([Breaking] Set output margin to True for custom objective. #5564)

  Now both R and Python interface custom objectives get un-transformed prediction
  outputs.

CLI

• Fix model dump. (Fix cli dump model. #5485)
• Fix CLI model IO. (Fix cli model IO. #5535)
• Remove hard coded seed. (Don't set seed on CLI interface. #5563)

[hcho3]

I'll create new release branch after #5577.

[hcho3]

All blockers have been addressed. I will start a new release branch.

[hcho3]

1.1.0 is now released.