sparklyr, an LF AI Foundation Incubation Project, has released version 1.2.0 and we’re excited to see a great release with contributions from several members of the community. sparklyr is an R Language package that lets you analyze data in Apache Spark, the well-known engine for big data processing, while using familiar tools in R. The R Language is widely used by data scientists and statisticians around the world and is known for its advanced features in statistical computing and graphics.
In version 1.2.0, sparklyr adds a variety of improvements, including:
- sparklyr now supports Databricks Connect
- A number of interop issues with Spark 3.0.0-preview were fixed
- The `registerDoSpark` method was implemented to allow Spark to be used as a `foreach` parallel backend in Sparklyr (see registerDoSpark.Rd)
- And more…A complete list of changes can be found in sparklyr 1.2.0 section of the NEWS.md file: sparklyr-1.2.0
The power of open source projects is the aggregate contributions originating from different community members and organizations that collectively help drive the advancement of the projects and their roadmaps. The sparklyr community is a great example of this process and was instrumental in producing this release. A special THANK YOU goes out to the following community members for their contributions of commits and pull request reviews!
- Andy Zhang (Databricks)
- Yitao Li (RStudio)
- Javier Luraschi (RStudio)
- Hossein Falaki (Databricks)
- Lu Wang (Databricks)
- Samuel Macedo (IFPE)
To learn more about the sparklyr 1.2.0 release, check out the full release notes. Want to get involved with sparklyr? Be sure to join the sparklyr-Announce and sparklyr Technical-Discuss mailing lists to join the community and stay connected on the latest updates.
Congratulations to the sparklyr team and we look forward to continued growth and success as part of the LF AI Foundation! To learn about hosting an open source project with us, visit the LF AI Foundation website.
sparklyr Key Links
LF AI Resources
- Learn about membership opportunities
- Explore the interactive landscape
- Check out our technical projects
- Join us at upcoming events
- Read the latest announcements on the blog
- Subscribe to the mailing lists
- Follow us on Twitter or LinkedIn
- Access other resources on LF AI’s GitHub or Wiki
