Monday, October 23, 2017

Databricks bets on deep learning for future cloud data analytics

Addition of deep learning and deep dreaming capabilities mark the opening of new age in the data analytics space.

From Brussels: Open source data analytics company Databricks has today introduced the addition of deep learning support to its cloud-based Apache Spark platform marking a new beginning in the artificial intelligence (AI) space.

The announcement was made by Ali Ghodsi, CEO and co-founder of Databricks during the company’s Spark Summit 2016 being held in Brussels, Belgium, this week.

The addition of deep learning functionalities adds a new emerging technology layer to the company’s platform by adding GPU support, Ghodsi explained.

In addition, the announcement will help to integrate deep learning libraries to the Databricks’ big data platform, extending its capabilities to enable the rapid development of deep learning models.

Kavitha Mariappan, VP of marketing at Databricks, told Data Economy: “Databricks is committed to building the most advanced data platform to help companies create value with Apache Spark. As leading technologists seek to utilize Spark for deep learning, we are proud to be the first to provide native support for end-to-end deep learning.”

In his presentation, CEO Ghodsi used a picture of Boris Johnson where he was able to perform deep learning analytics and through the software’s AI capabilities he performed a throughout analysis of the picture identifying all the important characteristics associated with the image.

During the demo, Ghodsi also used deep dreaming capabilities, which is an early stage of neural network processing that enhances patterns in images via algorithmic pareidolia.

As Ghodsi explained, this deep dreaming capabilities create an hallucinogenic vision of the image giving it basic features and attributes which will help to identify the use case.

He said that data scientists looking to combine deep learning with big data, whether it is recognising handwriting, translating speech between languages, or distinguishing between malignant and benign tumors, can now utilise Databricks for every stage of their workflow, from data wrangling to model tuning.

The company claimed to be the first to integrate these “diverse workloads in a fast, secure, and easy-to-use Apache Spark platform in the cloud”.

Deep learning itself is considered one of the most disrupting technologies in the AI machine learning space. The technology bases its knowledge on a set of algorithms that in their turn try to model different layers of a given data asset, be it an image, object and so on.

According to the ‘2016 Spark Survey’, machine learning usage in production saw a 38% increase since 2015, making it one of Spark’s key growth areas.

Ghodsi said: “We are proud to enable organisations to achieve better results in their mission-critical applications and are always looking ahead at the latest technologies – such as deep learning – to provide the Spark community with the most flexible, approachable big data toolset.”

Now watch how Spark 2.0 works