10 December 2020 12:50
Firebolt says its cloud data warehouse as a service delivers high-performance experiences with terabytes to petabytes of data and enables analysts, employees and end customers to gain fast, cost-efficient insights. "Firebolt is a complete redesign of the data warehouse for the era of the cloud and data lakes," writes Firebolt co-founder and CEO Eldad Farkash. "Our aim is to enable organizations to deliver an incredible data analytics experience regardless of the size and usage patterns of a company's data without having to constantly be worried about performance and costs." The solution is built for AWS with an efficient SQL query engine, Firebolt says, and decouples storage and compute, allowing users to access multiple isolated compute resources on the same database "While companies can store massive amounts of data, most organizations are only able to analyze a fraction of that big data, and often find themselves looking at stale data that does not reflect the current state of their business," Farkash said in a statement on Wednedsay. "For companies to flourish today, they need to move fast, and they should not be forced to make data compromises to achieve only a small part of the business value that their data holds. Oren Zeev of Zeev Ventures said Firebolt created a SaaS solution "that changes the analytics experience over big data sets." "The pace of innovation in the big data space has lagged the explosion in data growth, rendering most data warehousing solutions too slow, too expensive, or too complex to scale," Zeev added, claiming that Firebolt is taking cloud data warehousing to the next level by "offering the world's most powerful analytical engine." Enterprises and companies can analyze multi-terabyte/petabyte data sets easily "at significantly lower costs and provide a truly interactive user experience to their employees, customers or anyone who needs to access the data," Zeev said.
"The biggest change is a new file format called the triple F format," or F3, he says. "The purpose of this format is to tackle what we believe are the biggest problems and gaps in efficiency and speed when it comes to cloud data warehouse. The company doesn't support other public cloud object stores (although that may change in the future). "The way to tackle S3 is to rethink the way we prune data in S3," says Farkash, who was a co-founder and CTO at SiSense before co-founding Firebolt. and our goal is to increase the size of files over time so we can exploit compression and coding in a much better way because the data is ordered in that file." With larger, ordered files, Firebolt was able to apply a novel indexing approach, called sparse indexing.
That, in turn, allows Firebolt to be able to download much less data from S3 into RAM and still have a reasonable shot at having the data that the user is looking for. Firebolt pairs this sparse indexing in the F3 format with a just-in-time, vectorized query engine. (The Vectorwise and Snowflake query engines can also attribute their lineage to CWI.) But according to Farkash, the way that Firebolt implements all of these elements is fairly unique. "Whenever a query starts cold, the first thing users will notice is that they scan much less data," he says. Firebolt doesn't just download the file from S3 and cache it in RAM, Farkash says. As data is streamed into the S3 lake, say from Kafka or Kinesis, Firebolt is continuously re-ordering it in its cache, and preparing it for analysis. "This is a real-time, streaming environment where the data gests chunked and F3 format files are being generated. Do I wish to have near real time so every record that I insert will automatically be available for querying, versus I'm just doing a copy of a hundred TB of data and I want that to be as optimized as possible." Once the data is ingested, then the elasticity of the Firebolt service kicks in. Different users can be doing different things on the same data, Farkash says. Tableau users will no longer need to work on an extract of the data, because they will be able to essentially load all of their data into Firebolt, according to Farkash. (The biggest use case so far is a 800TB of compressed data, but Farkash indicated that it could go higher.) With a single engine that can handle the biggest workloads, Firebolt has the potential to significantly simplify the analytics architecture that companies are using, according to Farkash. "That's why we have baked into our SQL front-end, the query optimizer, a lot of new optimizations that are really intended to deal with interactive, ad-hoc, star schemas, multi-fact table schema situations where people can't just pre-join the data." It works best on normalized data, according to Farkash. Having predictable costs in a cloud data warehouse, and being able to scale up workloads in a predicable, is one of the design parameters of Firebolt. "The purpose of Firebolt is to detach you from that constant feeling of having to calculate and ask your boss whether you can spin up engine to work some queries," Farkash says. They need to be able to decide whether a specific data point or a specific report or specific query should cost $1 or $10." But according to Farkash, the minimum advantage over other cloud data warehouses is 3x to 4x. The bigger and more complex the data, the faster Firebolt goes, Farkash says. "The more filters you add, the more complexity you add to the query, you can get to 100 to 1,000x faster," he says. Farkash says he has not seen a cloud data warehouse that can exceed Firebolt's performance. "For typical use case, non ELT analytics, Firebolt will really melt the snow, seriously melt it, in a very up and in your face way," he says. Now it's selling access to its data warehouse running on AWS. The Tel Aviv company also today announced the completion of a $37 million round of venture financing led by Zeev Ventures, TLV Partners, Bessemer Venture Partners, and Angular Ventures. Firebolt, a Tel Aviv, Israel-based provider of a cloud data warehouse, purpose-built for high performance analytics, raised $37m in funding. Co-founded by CEO Eldad Farkash and COO Saar Bitner, Firebolt provides a cloud data warehouse that delivers a sub-second interactive analytics experience with terabytes to petabytes of data. It enables analysts, employees and end customers to gain the insights they need without having to wait for the results or worry about costs.