From f50cffbc150f81f7a7fe9b03b6f060a2b73740e8 Mon Sep 17 00:00:00 2001 From: Andrew Lamb Date: Wed, 1 May 2024 06:36:16 -0400 Subject: [PATCH 1/3] [Website]: Blog announcing DataFusion top level project --- _posts/2025-05-03-datafusion-tlp.md | 95 +++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+) create mode 100644 _posts/2025-05-03-datafusion-tlp.md diff --git a/_posts/2025-05-03-datafusion-tlp.md b/_posts/2025-05-03-datafusion-tlp.md new file mode 100644 index 000000000000..11b2038893c0 --- /dev/null +++ b/_posts/2025-05-03-datafusion-tlp.md @@ -0,0 +1,95 @@ +--- +layout: post +title: "Announcing Apache Arrow DataFusion is now Apache DataFusion" +date: "2024-05-01 00:00:00" +author: pmc +categories: [subprojects] +--- + + + + + +## Introduction + +TLDR; [Apache Arrow] DataFusion --> [Apache DataFusion] + +The Arrow PMC and newly created DataFusion PMC are happy to announce that as of +April 16, 2024 the Apache Arrow DataFusion subproject is now a top level +[Apache Software Foundation] project. + +[Apache Arrow]: https://arrow.apache.org/ +[Apache DataFusion]: https://datafusion.apache.org/ +[Apache Software Foundation]: https://www.apache.org/ + +## Background + +Apache DataFusion is a fast, extensible query engine for building high-quality +data-centric systems in Rust, using the Apache Arrow in-memory format. + +When DataFusion was [donated to the Apache Software Foundation] in 2019, the +DataFusion community was not large enough to stand on its own and the Arrow +project agreed to help support it. The community has grown significantly since +2019, benefiting immensely from being part of Arrow and following [The Apache +Way]. + +[donated to the Apache Software Foundation]: https://arrow.apache.org/blog/2019/02/04/datafusion-donation/ +[The Apache Way]: https://www.apache.org/theapacheway/ + +## Why now? + +The community [discussed graduating to a top level project publicly] for almost +a year, as the project seemed ready to stand on its own and would benefit from +more focused governance. For example, earlier in DataFusion's life many +contributed to both [arrow-rs] and DataFusion, but as DataFusion has matured many +contributors, committers and PMC members focused more and more exclusively on +DataFusion. + +[discussed graduating to a top level project publicly]: https://github.com/apache/datafusion/discussions/6475 +[arrow-rs]: https://github.com/apache/arrow-rs + +## Looking forward + +The future looks bright. There are now [10s of known projects built with +DataFusion], and that number continues to grow. We recently held our [first in +person meetup] passed [5000 stars] on GitHub, [wrote a paper that was accepted +at SIGMOD 2024], and began work on [Comet], an [Apache Spark] accelerator +[initially donated by Apple]. + + +[10s of known projects built with DataFusion]: https://datafusion.apache.org/user-guide/introduction.html#known-users +[first in person meetup]: https://github.com/apache/datafusion/discussions/8522 +[5000 stars]: https://github.com/apache/datafusion/stargazers +[wrote a paper that was accepted at SIGMOD 2024]: https://github.com/apache/datafusion/issues/8373#issuecomment-2025133714 +[Comet]: https://github.com/apache/datafusion-comet +[Apache Spark]: https://spark.apache.org/ +[initially donated by Apple]: https://arrow.apache.org/blog/2024/03/06/comet-donation/ + +## How to Get Involved + +If you are interested in joining the community, we would love to have you join +us. Get in touch using [Communication Doc] and learn how to get involved in the +[Contributor Guide]. We welcome everyone to try DataFusion on their +own data and projects and let us know how it goes, contribute suggestions, +documentation, bug reports, or a PR with documentation, tests or code. + + +[communication doc]: https://datafusion.apache.org/contributor-guide/communication.html +[Contributor Guide]: https://datafusion.apache.org/contributor-guide/index.html From 1588fd533d130c1635ce5a29da72932afa7d1383 Mon Sep 17 00:00:00 2001 From: Andrew Lamb Date: Wed, 1 May 2024 06:41:11 -0400 Subject: [PATCH 2/3] Add note of thanks and last post --- _posts/2025-05-03-datafusion-tlp.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/_posts/2025-05-03-datafusion-tlp.md b/_posts/2025-05-03-datafusion-tlp.md index 11b2038893c0..9c19d8c0a20e 100644 --- a/_posts/2025-05-03-datafusion-tlp.md +++ b/_posts/2025-05-03-datafusion-tlp.md @@ -73,6 +73,11 @@ person meetup] passed [5000 stars] on GitHub, [wrote a paper that was accepted at SIGMOD 2024], and began work on [Comet], an [Apache Spark] accelerator [initially donated by Apple]. +Thank you to everyone in the Arrow community who helped DataFusion grow and +mature over the years, and we look forward to continuing our collaboration as +projects. All future blogs and announcements will be posted on the [Apache +DataFusion] website. + [10s of known projects built with DataFusion]: https://datafusion.apache.org/user-guide/introduction.html#known-users [first in person meetup]: https://github.com/apache/datafusion/discussions/8522 @@ -82,7 +87,7 @@ at SIGMOD 2024], and began work on [Comet], an [Apache Spark] accelerator [Apache Spark]: https://spark.apache.org/ [initially donated by Apple]: https://arrow.apache.org/blog/2024/03/06/comet-donation/ -## How to Get Involved +## Get Involved If you are interested in joining the community, we would love to have you join us. Get in touch using [Communication Doc] and learn how to get involved in the From 9eab57dbb8d87c1ec544e70e20be13db76451f8e Mon Sep 17 00:00:00 2001 From: Andrew Lamb Date: Tue, 7 May 2024 06:03:20 -0400 Subject: [PATCH 3/3] Update date --- ...-datafusion-tlp.md => 2025-05-07-datafusion-tlp.md} | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) rename _posts/{2025-05-03-datafusion-tlp.md => 2025-05-07-datafusion-tlp.md} (97%) diff --git a/_posts/2025-05-03-datafusion-tlp.md b/_posts/2025-05-07-datafusion-tlp.md similarity index 97% rename from _posts/2025-05-03-datafusion-tlp.md rename to _posts/2025-05-07-datafusion-tlp.md index 9c19d8c0a20e..5aaac5ee4cd4 100644 --- a/_posts/2025-05-03-datafusion-tlp.md +++ b/_posts/2025-05-07-datafusion-tlp.md @@ -1,7 +1,7 @@ --- layout: post title: "Announcing Apache Arrow DataFusion is now Apache DataFusion" -date: "2024-05-01 00:00:00" +date: "2024-05-07 00:00:00" author: pmc categories: [subprojects] --- @@ -33,7 +33,7 @@ TLDR; [Apache Arrow] DataFusion --> [Apache DataFusion] The Arrow PMC and newly created DataFusion PMC are happy to announce that as of April 16, 2024 the Apache Arrow DataFusion subproject is now a top level -[Apache Software Foundation] project. +[Apache Software Foundation] project. [Apache Arrow]: https://arrow.apache.org/ [Apache DataFusion]: https://datafusion.apache.org/ @@ -42,7 +42,7 @@ April 16, 2024 the Apache Arrow DataFusion subproject is now a top level ## Background Apache DataFusion is a fast, extensible query engine for building high-quality -data-centric systems in Rust, using the Apache Arrow in-memory format. +data-centric systems in Rust, using the Apache Arrow in-memory format. When DataFusion was [donated to the Apache Software Foundation] in 2019, the DataFusion community was not large enough to stand on its own and the Arrow @@ -91,9 +91,9 @@ DataFusion] website. If you are interested in joining the community, we would love to have you join us. Get in touch using [Communication Doc] and learn how to get involved in the -[Contributor Guide]. We welcome everyone to try DataFusion on their +[Contributor Guide]. We welcome everyone to try DataFusion on their own data and projects and let us know how it goes, contribute suggestions, -documentation, bug reports, or a PR with documentation, tests or code. +documentation, bug reports, or a PR with documentation, tests or code. [communication doc]: https://datafusion.apache.org/contributor-guide/communication.html