AtScale and Databricks first started partnering together in early 2022. At the time, Databricks SQL was relatively new, and many of their customers were still leveraging optional presentation layers to support Business Intelligence workloads.

The partnership began to gain momentum. AtScale was a way to democratize Lakehouse for non-technical business users and remove the need for unnecessary data movement into proprietary formats. This was extremely valuable to organizations looking to accelerate their Lakehouse journey. With the help of Soham Bhatt and Franco Patano, We went to market with this message to great success. 

With the partnership up and running, we started collecting customer feedback and developing our better-together story. It was clear that most of your use cases fell into two categories. 

• Migrate Legacy SQL Server Analysis Services (SSAS) – While Datbricks customers were eager to get their hands on DBSQL, it was clear that many were still maintaining an investment in on-prem SSAS cubes. The infrastructure and engineering hours required to support these cubes are massive, but a combination of AtScale + Databrick provided a superior alternative on limitless amounts of data in the cloud. This use case was discussed in more detail in a blog with Swaroop Oggu and I. (Modernize Your SSAS Architecture with Databricks and AtScale)
• Semantic Lakehouse – Even when a customer doesn’t use an optional presentation layer, ensuring that all Business Intelligence users consume data from the Lakehouse consistently and with governance is challenging. The complexity grows exponentially when organizations have multiple BI tools that speak SQL, DAX, and MDX that all want to extract and cache data in proprietary formats. Implementing a tool-agnostic Semantic Layer on top of Lakehouse that leverages DBSQL and Unity Catalog solves this problem. Soham Bhatt, Kyle Hale, and I discussed this use case in more detail. (Building a Semantic Lakehouse With AtScale and Databricks)

This idea of a “Semantic Lakehouse” began to resonate with our joint customers. We went to market with it, and the “Father of the Data Warehouse” also took notice in a joint webinar with AtScale, Databricks, and other industry experts in a series of webinars.

• How to Use a Data Lakehouse for AI & BI at Scale – A panel discussion featuring Bill Inmon, Soham Bhatt of Databricks, plus industry experts from Inspire Brands and DataPrime.
• How to Scale Business Intelligence on your Lakehouse with a Semantic Layer – AtScale CTO/Founder David Mariani speaking at the Databricks World Tour Event in NYC.
• How to Use a Semantic Layer and Data Lakehouse to Scale Analytics for Everyone – David is joined by Franco Patano to discuss Modern Data Warehousing on the Lakehouse.
• How to Scale Business Intelligence on your Data Lakehouse – David is joined by Bill Inmon and Vihag Gupta of Databricks to discuss Business Intelligence with Databricks SQL.

Operation ‘Semantic Lakehouse’

We went from 0 to 100 real quick, and it culminated in AtScale being named the Databricks Emerging Partner of the Year in 2023! We started doubting down on the partnership by deepening our technical integration by being one of the first ISVs available on Databricks Partner Connect and being a launch partner of Databricks Marketplace. 

The buzz around the Semantic Lakehouse was real, and to capitalize on the opportunity, our teams started to verticalize our joint go-to-market motion to better align with our customer’s needs and interests. 

Semantic Lakehouse for Manufacturing 

Data volumes in the Manufacturing industry are exploding and are expected to grow 200-500% in the next five years, outpacing other industries by 2-4x. With data being the most valuable asset a manufacturer has to help overcome rising costs and Supply Chain disruption, this vertical quickly became one of our strongest, which resulted in being a launch partner of the official Databricks Lakehouse for Manufacturing.

• Building a Semantic Lakehouse for Supply Chain and Manufacturing – Shiv Trisal, Global Manufacturing Industry Lead at Databricks, Bill Inmon and David Mariani sit down to discuss some of the key challenges facing Manufacturers and how a Semantic Lakehouse enables real-time decision-making and granular analysis, leveraging all data for accurate results
• Transparency, visibility, data: Optimizing the Manufacturing Supply Chain with a Semantic Lakehouse – Shiv Trisal, Bala Amavasai, our friends at Tredence, and I discuss how to implement a “Semantic Model Repository” to promote shared models with a consistent and compliant view across the supply chain, users can create data products that meet the needs of each supply chain domain all while working on a single source of truth.
• Revolutionizing Data Analysis: The Shift to Databricks and AtScale – Corning discusses how they deliver multi-dimensional views, OLAP capabilities, and native Excel integration directly on the Lakehouse with AtScale + Databricks to improve our time to insight and decision-making process, all while ensuring a seamless user experience.
• Why a Semantic Layer is Critical in Manufacturing – I put together a little blog highlighting the above and some other observations based on conversations with customers, prospects, and solution integrators.

Semantic Lakehouse for Retail & Consumer Goods

One trend in Retail & Consumer Goods is that decision-making is being pushed to the frontline, closer to where the problem occurs. In order to ensure these non-technical business users have a guided experience with their organizational data, there’s been massive interest in exploring Semantic Layers as a tool to achieve this. 

• How AtScale’s Semantic Layer Enables Trusted, Real-Time Customer-360 for Top Retail Companies – A blog by Albert Zhou of AtScale discussing how one of the world’s largest apparel brands is achieving a unified view of customer behavior by implementing a Semantic Layer with the Data Intelligence platform.
• Rediscovering Semantic Models in the Retail Industry – Bryan Smith, Global Head of Industry Solutions RCG at Databricks, spoke at our Semantic Layer Summit in 2023 to discuss the resurgence of Semantic Layers in Retail and how they are used to get timely insights into the hands of decision-makers. 
• Enabling Insights with Prebuilt Semantic Models with Databricks – Bryan Smith returns for a second year to speak at our Semantic Layer Summit in 2024 on pre-built industry Semantic Models. He walks through a practical example of leveraging the Crisp retail data model. (Available on Github!) 

Semantic Lakehouse for FinServ, Capital Markets, and Insurance

Given our integration with Databricks and how our solution removes the need for data movement while augmenting existing security and governance frameworks, it’s no surprise that we have seen great success in the regulated industries. Also, One of the primary drivers for that is how popular Excel is as a decision support system. These decision-makers may not be proficient in SQL, but they know Excel inside out. AtScale bridges that gap and turns every Excel user into a Data intelligence-powered decision-maker. 

• Building a Semantic Lakehouse for Insurance: Marcela Granados, Global Insurance Leader at Databricks; Anindita Mahapatra, Lead Solutions Architect at Databricks; Bill Inmon; and David Mariani discuss Insurance trends and how the Semantic Layer enables business users to access data and insights.

• Building a Semantic Lakehouse for Capital Markets – Jordan Kramer, Senior Solutions Engineer at Databricks, and David Mariani discuss a portfolio management use case and how different personas can interact with a consistent view of business metrics and definitions across traditional BI tools such as Excel, but also modern tools such a Databricks AI/BI Dashboards. 

Semantic Lakehouse for Fun

Sometimes, a use case doesn’t fall into an industry bucket; instead, you just want to nerd out and talk about something technical. That has undoubtedly happened along the way.

• Leveraging the Power of the Data Intelligence Platform and the Semantic Layer with AtScale and Databricks – Pulkit Chada, Architect and Databricks & NY Times Best Selling Author of ‘Data Engineering with Databricks’ sits down with David Mariani to discuss the Data Intelligence category.
• Changing the Game for OLAP and Business Intelligence with a Semantic Lakehouse – Denny Lee, Senior Staff Developer Advocate at Databricks, and David Mariani tell some war stories from over a decade wgo when they were working together on the worlds biggest SSAS cube.
• Generative BI: Leverage the AtScale Semantic Layer and Databricks Genie for Enhanced Decision-Making – Joseph Hobbs, Senior Solutions Architect and Semantic Model Guru at Databricks, joined David Mariani to discuss the importance of Semantics in GenAI.
• Getting started with Databricks and Atscale – Yousseff Mrini, Senior Solutions Engineer at Databricks and host of NextGenLakehouse, hosts David Mariani on his show to discuss AtScale on Databricks.
• Democratizing Your Lakehouse to Excel Users with Databricks SQL and AtScale – Plus the most significant use case of all time.

What’s next for the Semantic Lakehouse in the age of GenAI? 

The final section of this blog is more forward-looking and explains why the AtScale and Databricks partnership will continue to go from strength to strength. It has to do with our shared vision and why the Semantic Lakehouse is here to stay.

Databricks is the open, performant, and collaborative Data Intelligence platform. AtScale is the open, performant, and collaborative Semantic Layer. Databricks is built on open standards and continues to innovate, having just recently open-sourced Unity Catalog. AtScale is following in their footsteps by announcing the open-source release of the Semantic Modeling Language (SML), a universal standard designed to promote interoperability and foster a vibrant community of semantic model builders.

While other vendors continue to lock data into proprietary formats and force users to adopt specific architectures and tools to interact with that data, both AtScale and Databricks have a shared point of view and advocate for open standards and interoperability. This is why you may have seen some of the following announcements and content over the last few months. 

• AtScale is a launch partner of Unity Catalog Metrics – By standardizing metric definitions, Unity Catalog Metrics allows data teams to work with the same semantics and underlying data, ensuring that all teams use consistent definitions. This promotes trust and reliability in the data.
• Open Source Semantics: Setting the Standard for Interoperability with Databricks and AtScale – Dael Williamson, Field CTO of Databricks, and David Mariani sit down to discuss the importance of open standards in the age of GenerativeAI and how important it is to unlock the full potential of these applications. 
• Setting the Standards for Semantics in AI and BI – Dael and David sit down for a long-form conversation and dig into the technical integration between AtScale and AI/BI. 
• AtScale and Databricks AI/BI Genie Demo – David walks through a benchmark report highlighting how crucial semantic context is for text-to-SQL applications to achieve sufficient accuracy and domain specificity.
• Why you should consider using an Open source Semantic layer – David returns to the NextGenLakehouse to discuss the benefits of open source semantics with Quentin Ambard.

Our visions are aligned. Clear Eyes, Full Heart, Can’t Lose. Check out what’s next in the subsequent blog, Semantic Lakehouse for AI/BI.