Databricks, the data, and AI company announced that Delta Engine is available and that Redash will be acquired. These new capabilities make the use of its Unified Data Analytics platform for data science, machine learning, and a wide range of data analytics use cases faster and more comfortable for data teams. Delta Engine is a high-performance cloud data lakes query engine, and Redash is an open-source dashboarding and visualization co-operation for data scientists and analysts to carry out data exploration. In this article, we talk about Databricks and what revolution it could create by acquiring Redash. Data Science is a popular career choice for many. Wish to become a data science expert? Find the best online data science courses available and choose the one that suits you the best and start your Data Science journey today!
Table of Contents
- About Databricks
- Delta Engine- An Overview
- Acquisition of Redash by Databricks
- Conclusion
There are various types of data science certifications, and the best part is that you can become a data science developer from the comfort at your home. Let’s move further and see more about Databricks, Delta Engine, and the acquisition of Redash.
About Databricks
Databricks is an Artificial Intelligence and data company. Thousands of organizations worldwide – including H&M, Comcast, Condé Nast, Nationwide – rely on the open and unified Databricks platform for data engineering, machine learning, and analytics. With offices around the globe, Databricks is venture-backed and headquartered in San Francisco. Established by the original creators of Apache SparkTM, Delta Lake, and MLflow, Databricks is on a mission to help data teams solve the toughest challenges in the world.
Delta Engine- An Overview
Delta Engine is tailored for its application with Delta Lake, the popular open-source, structured transaction layer that provides data lakes with quality and reliability. Organizations can now construct curated data lakes that include structured and semi-structured data and run all their analytics on high-quality, fresh cloud data. Delta Lake was released by Databricks in 2017 and donated in 2019 to the Linux Foundation. Comcast, Condé Nast, Nielsen, FINRA, Shell, and numerous other organizations have adopted Delta Lake since its introduction.
Acquisition of Redash by Databricks
To provide easy-to-use dashboarding and visualization abilities on the curated data lakes, Databricks has acquired Redash, the company behind the successful open source project Redash. Redash allows data scientists and SQL analysts to eliminate the complexity of moving data into other analytical systems. Together, these enhancements facilitate organizations to adopt a single, simplified data management cloud architecture, helping them significantly reduce costs and complexity and speed up data teams’ productivity. Additionally, they are a response to the emerging design pattern “Lakehouse,” which many enterprise organizations adopt to bring structured transactions, quality, and performance to their cloud data lakes. The announcements were made at the Spark + AI Summit, which held with virtually over 60,000 data community members, from over 100 countries.
Most organizations that want to do data warehousing and data science use multiple architectures. Data is stuck in organizational silos. It is defined by closed and proprietary systems that slow down organizations and make it more challenging to reach high-quality decisions because the information is fragmented and out-of-date. Curated cloud data lakes provide organizations with a way to run analytics on all their latest data, including data science and machine learning. The introduction of Delta Engine and the acquisition of Redash are significant steps forward in helping organizations build these high-quality, curated data lakes, which some call’ lakehouses.’
Delta Engine Enables Fast Query Performance on Delta Lake
Traditional data analytics on semi-structured and structured data require very rapid performance to keep pace with the business. Historically, organizations have duplicated data in their data lakes across various data warehouses and operating systems because data query and analysis tools are not suitable for quick query execution. Managing this architectural complexity presents challenges, including fragmented and inconsistent data silos, and significantly increased costs.
The new Delta Engine for Delta Lake by Databricks allows fast query execution for data analytics and data science, without removing data from the data lake. The high-performance query engine was built from the ground up to take advantage of modern cloud hardware to speed up queries. With this improvement, customers of Databricks can move to a unified data analytics platform that can support any case of data use and result in meaningful operating efficiencies and cost savings.
Redash makes data collecting easy for data scientists and analysts
The Redash open source project has been created to help the data teams make sense of their data. Data scientists and SQL analysts can quickly gather a wide array of data sources into thematic dashboards, including operational databases, data lakes, and Delta Lake. Results can be viewed in a wide variety of formats, such as charts, cohorts, and funnels, and can be easily shared across an organization or outside users.
Conclusion
At thousands of organizations, millions of users are already using Redash to develop insights and actionable data. The open-source project was created by a passionate developer community built by more than 300 contributors from around the world since the project was launched in 2013. The open-source Redash project can be used with Databricks today using a free connector. In the coming months, as predicted by data science experts, Redash will be fully integrated into the Unified Data Analytics Platform and the Databricks workspace of Databricks. It will also take advantage of capabilities such as the Delta Engine.