July 9, 2024
How Semantic Layers Make GenAI 3X More Accurate Than Direct SQLAt AtScale, we believe in applying data to solve key challenges in the modern data stack for the enterprise. Bringing together key data personas to align on objectives, data, and definitions is crucial for the successful deployment of analytics and represents a key tenant of our platform and products.
Data science and machine learning (ML) represent key new cornerstones of the modern analytics stack. We’ve learned there are still some key aspects that encumber successful application of ML into production environments:
- Getting Access to Data: Properly trained ML models require not just large datasets from its source, but also the context and definitions that dictate its structure. Data scientists need the same view of data as other data personas.
- Creating Performant, Compliant Models: Data scientists have to find a way to manage the versions/extracts of data they use for model training, maintain governance standards, and scale the attributes of data to create models that satisfy the needs of the business and IT teams.
- Applying and Trusting AI: Models will not move past the experimentation phase if they do not have a well-understood value to instill confidence in business stakeholders.
AI-Link, a python-based connector enables data science users to tap into the power of the AtScale semantic layer to:
- Understand and Prepare Data for ML: Use the same view of data and the same definitions that the business defines — removing the need for extracts, copies, and misinterpretation of the context of data. AtScale and AI-Link will help automate pain points related to both data and feature engineering while maintaining data governance standards.
- Feature Engineering — The Right Way: Transform, encode, and scale business-defined features and store those in the AtScale semantic layer or other commercial feature stores to remove replicated work among data science teams.
- Bridge the Gap Between AI and BI Write-Back ML Predictions: Using lightweight dataframes, ML model predictions and metadata can be published in BI tools so the business can understand and trust the impact of the model in production.
We believe that by extending the power of the semantic layer to all data personas and users, we will be able to address core challenges that prohibit the creation and scaling of performant ML.
We also believe that seeing is believing.
We are Excited to Announce the Availability of the AI-Link Lab Environment
What is the AI-Link Lab Environment?
A free, unlimited access two-week trial of AI-Link with:
- Flexible deployment of AI-Link as a notebook
- Test data and a curated semantic (data) model to interact with
- Pre-built Tableau dashboard that can be used to present historical and ML-prediction data
How Do I Get Started?
We will make sure that you have access to the lab environment and associated resources, all you will need to get started is:
- Python knowledge (and any preferred libraries)
- Jupyter or other Notebook environment
- ML Modeling environment (framework or AutoML)
Please email zach.eslami@atscale.com to get started.
Power BI/Fabric Benchmarks