Creating High Quality RAG Applications with Databricks
Retrieval-Augmented-Generation (RAG) has quickly emerged as a powerful way to incorporate proprietary, real-time data into Large Language Model (LLM) applications. Today we are...
Page 1
Free Dolly: Introducing the World's First Truly Open Instruction-Tuned LLM
Two weeks ago, we released Dolly , a large language model (LLM) trained for less than $30 to exhibit ChatGPT-like human interactivity (aka...
Cortex Labs is Joining Databricks to Accelerate Model Serving and MLOps
As enterprises grow their investments in data platforms, they increasingly want to go beyond using data for internal analytics and start integrating predictions...
Evolving the Databricks brand
Some brands start out as, well, brands. A lot of work goes into the concept and painting the picture before the business is...
Apache Spark 2015 Year In Review
To learn more about Apache Spark, attend Spark Summit East in New York in Feb 2016 . 2015 has been a year of...
Announcing Apache Spark 1.6
To learn more about Apache Spark, attend Spark Summit East in New York in Feb 2016 . Today we are happy to announce...
Announcing an Apache Spark 1.6 Preview in Databricks
Today we are happy to announce the availability of an Apache Spark 1.6 preview package in Databricks. The Apache Spark 1.6.0 release is...
Spark Survey 2015 Results are now available
We ran the Spark Survey 2015 this summer to gain insights on how organizations are using Apache Spark. The results of this year’s...
Announcing Apache Spark 1.5
The inaugural Spark Summit Europe will be held in Amsterdam this October. Check out the full agenda and get your ticket before it...
Diving into Apache Spark Streaming's Execution Model
With so many distributed stream processing engines available, people often ask us about the unique benefits of Apache Spark Streaming . From early...
Joint Blog Post: Bringing ORC Support into Apache Spark
This is a joint blog post with our partner Hortonworks. Zhan Zhang is a member of technical staff at Hortonworks, where he collaborated...
Announcing Apache Spark 1.4
Today I’m excited to announce the general availability of Apache Spark 1.4! Spark 1.4 introduces SparkR, an R API targeted towards data scientists...
Announcing Apache Spark 1.3!
Today I’m excited to announce the general availability of Apache Spark 1.3! Apache Spark 1.3 introduces the widely anticipated DataFrame API, an evolution...
Apache Spark: A review of 2014 and looking ahead to 2015 priorities
2014 has been a year of tremendous growth for Apache Spark. It became the most active open source project in the Big Data...
"Learning Spark" book available from O'Reilly
Today we are happy to announce that the complete Learning Spark book is available from O’Reilly in e-book form with the print copy...
Announcing Apache Spark Packages
Today, we are happy to announce Apache Spark Packages ( http://spark-packages.org ), a community package index to track the growing number of open source packages and libraries that work with Apache Spark. Spark Packages makes it easy for users to find, discuss, rate, and install packages for any version of Spark, and makes it easy for developers to contribute packages.
Announcing Apache Spark 1.2
We at Databricks are thrilled to announce the release of Apache Spark 1.2! Apache Spark 1.2 introduces many new features along with scalability...
Apache Spark 1.1: The State of Spark Streaming
With Apache Spark 1.1 recently released, we’d like to take this occasion to feature one of the most popular Spark components - Spark...
Announcing Apache Spark 1.1
Today we’re thrilled to announce the release of Apache Spark 1.1! Apache Spark 1.1 introduces many new features along with scale and stability improvements. This post will introduce some key features of Apache Spark 1.1 and provide context on the priorities of Spark for this and the next release.
Announcing Apache Spark 1.0
Today, we’re very proud to announce the release of Apache Spark 1.0 . Apache Spark 1.0 is a major milestone for the Spark...
Apache Spark 0.9.0 Released
Our goal with Apache Spark is very simple: provide the best platform for computation on big data. We do this through both a...
Apache Spark 0.8.1 Released
We are happy to announce the release of Apache Spark 0.8.1. In addition to performance and stability improvements, this release adds three new...