March 7, 2019
Cloud Cost Optimization: How to Reduce Cloud Data Warehouse CostsTechnology is risky.
I understand risk. It was risky starting a company. It was risky for our amazing investors to invest in us. Risk is part of life and cannot be avoided because with risk comes the potential for reward.
Sometimes you get the bear; sometimes the bear gets you.
Unfortunately, the bear got a couple of our competitors recently. With all due respect, full stack BI vendors that offer visualization & server stacks have hit some real headwinds as of late, Arcadia and Jethro Data have gone the way of the buffalo, while Alteryx took mercy on ClearStory (whose outcome was anything but).
Future-Proofing Your Data Architecture (or, How Do You Get the Bear?)
__Never make permanent decisions on temporary feelings. __
When we started AtScale over six years ago, Hadoop was the ‘new hotness’—the shiny new technology that was going to solve All The Big Data Problems. We knew Big Data was real, and we saw Hadoop as a very interesting and difficult platform to support. However, we believed making a 100% commitment to a then-unproven technology could be a fatal mistake. We knew from experience that it wasn’t enough to prove we could do it, we needed to do it right 6 years ago while positioning for the future.
*I remember many days early on in the history of the company when we didn’t write code but made the architectural decisions that still drives the company today. I’ve heard architecture described as the “decisions you wish you had gotten right at the beginning of the project”. That statement resonates. *
- We made an architectural decision that we would not move data out of the Big Data ecosystem.
- We made an architectural decision that we would not build a database.
- We made an architectural decision that we would automate the requisite data engineering that allows analytics to operate on Big Data.
- We made an architectural decision that we would be a programmable, embeddable analytics platform.
- We made an architectural decision that we would support native business Intelligence front ends such as Excel, Tableau and PowerBI rather than building our own front end.
Today, four out of these five decisions run counter to our competition and as a result, we’re very happy with our decisions. So, yes, architectural decisions can have a considerable effect on the viability of your business.
Frictionless Cloud Adoption for Enterprise Customers
We are passionate about helping enterprise companies tackle the obstacles with ease that come with digital transformation—the transition from small data to Big Data. The transition from on-prem to the cloud and the transition from Hadoop to a new platform.
AtScale’s architectural decisions ultimately enable business agility – including a 100% first-year credit from your Arcadia, Jethro or ClearStory purchase on an AtScale subscription. As you contemplate a move and make an investment to modernize your analytics stack, understanding how the platforms differ is important.
- AtScale supports complex multidimensional use cases.
- AtScale has best-in-class performance on data of any size.
- AtScale supports standard connection APIs such as JDBC, ODBC, and MDX.
- AtScale security features are second to none.
- AtScale works with all your favorite tools.
- AtScale will optimize your spend with automatically managing workloads.
It’s All About Performance, Security, and Agility
The biggest differences between AtScale and the rest are twofold.
First, AtScale has invested heavily in maximizing the performance, security and agility functionality in our engine rather than building a BI visualization tool.
Second, AtScale isn’t building a new engine. Competitors have forked Impala or built their own database and storage mechanism. The world doesn’t need yet another SQL on Hadoop engine—the 8+ that exist are enough. The winners in that war have been crowned. Building a full data warehouse is not for the faint of heart. The most recent success story is our wonderful partner Snowflake who raised close to a billion dollars to build a data warehouse and relies on partners for the business intelligence functionality.
Platform Independence Will Rule the Day
AtScale’s vision for the future of data was based on empowering and supercharging enterprise investment in traditional databases, Hadoop big data ecosystems, and cloud-based next-generation data warehousing technologies such as BigQuery and Snowflake. There is a natural avoidance to transitioning away from traditional data platforms that companies have heavily invested in; however, the real priority should be alleviating vendor lock-in while exploring cloud-based data platform opportunities.
As organizations evaluate EDW modernization efforts there should be thoughtful architectural consideration as these projects typically require multiple years and millions of dollars – the scariest lock-in in 2019 is your public cloud decision.
For Arcadia customers who have investments in the Hadoop ecosystem, AtScale eliminates lock-in because we enable Impala (core, standard, Cloudera Impala) as well as Hive, Spark, and Presto giving you more choice and versatility to build data infrastructure exactly as you want it. If you are currently on a Hadoop distribution and are thinking about moving to the cloud (and you should be) we’ve specifically addressed this. AtScale runs natively on all of the top cloud data warehouses with specific high-end optimizations for both cost and performance on BigQuery, Snowflake, Redshift, and Azure SQL Datawarehouse. With AtScale, you gain the ability to start your implementation on Hadoop and then quickly and seamlessly migrate your analytics workloads to the cloud of your choice.
For Jethro customers, AtScale gives you better performance without the need to move data while leveraging a more mature security and governance model and support for industry standard BI tools. AtScale breaks the hard-wire of analytics tools to the data warehouse and enables you to transparently shift 100% of your data warehouse as requirements change and the market develops—with absolutely no impact on your end users.
How AtScale Wins at Modernizing Your Data Architecture
Virtualization
Choose to invest in intelligent data virtualization—the ability to visually create a network of databases, relate the data across cloud, on-prem and hybrid cloud and present the data transparently to end users as though it was one massive, self-optimizing autonomous analytics warehouse.
Security
Never skimp on security. Seasoned enterprise veterans conceived AtScale, and therefore data security was considered job one. AtScale pursued the concepts of ‘Zero Trust’ before it was a formal concept. AtScale has numerous patents on security features ranging from MFA to native platform security support and True Delegation.
No Data Movement
Reduce complexity. Don’t add to it. Data has mass, and moving mass takes energy. AtScale has migrated customers with a variety of strategies from multiple legacy platforms and the great news is that a significant amount of work required to implement these products is ‘like-for-like’ with AtScale.
Moving is Easy with AtScale
With Arcadia, Jethro or Clearstory, we’ve got you covered. There is no restart. You are almost at the goal line with AtScale because we can modernize your architecture while maximizing your investment and prevent future vendor lock-in—quickly and cost-effectively.
Photo by drmakete lab on Unsplash
Power BI/Fabric Benchmarks