Skip to content

sllynn/spark-xgboost

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

27 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

spark-xgboost

Build

Spark users can use XGBoost for classification and regression tasks in a distributed environment through the excellent XGBoost4J-Spark library. As of July 2020, this integration only exposes a Scala API. A PR is open on the main XGBoost repository to add a Python equivalent, but this is still in draft.

This repository contains the Python wrapper components from that PR. By building and installing the appropriate .whl (see 'releases' in this repository), PySpark users can directly use this wrapper with the XGBoost library.

Tested with Databricks Runtime for ML, versions 6.6 (Spark 2.4.5, XGBoost 0.9) and 7.1 (Spark 3.0.0, XGBoost 1.0)

See the notebook in /examples for a practical illustration of usage.