Screen Size Comparison

Lets you compare display size

Loan Calculator

Calculate your loan amortization schedule

BigQuery BI Engine - how to measure your savings?

BigQuery BI Engine - it promises a lot, but what's the actual benefit?
Google Cloud Platform markets BI Engine as "a fast, in-memory analysis service" - in other words, it is a flexible caching service that helps you to increase the speed of your business-intelligence service (like Data Studio, but also reduce costs.

You can check how to set up BI Engine reservation here.
While it is important to remember that's BI engine reservation is not free, it can actually help you save money on top of improving query speed. Once the BI Engine reservation is configured to use, and some queries already hit your data warehouse you can easily check how it's performing using JOBS_BY_* views.


SELECT
  creation_time,
  total_bytes_processed,
  total_bytes_billed,
  CAST(5.0*(total_bytes_billed/POWER(2,40)) as numeric) as query_cost_usd,
  CAST(5.0*((total_bytes_processed-total_bytes_billed)/POWER(2,40)) as numeric) as bi_engine_savings_usd,
  job_id,
  bi_engine_statistics
FROM
  `region-eu`.INFORMATION_SCHEMA.JOBS_BY_PROJECT
WHERE
  creation_time > TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 2 DAY)
  AND bi_engine_statistics.bi_engine_mode IS NOT NULL
  /* uncomment below to get only jobs
     with specific BI engine status,
     one of: FULL, PARTIAL, DISABLED*/
  /* AND bi_engine_statistics.bi_engine_mode = 'FULL' */

By using INFORMATION_SCHEMA view it is easy to get the query history, including the total_bytes_processed and total_bytes_billed. Convert it to terabytes, get the difference between the two, multiply by 5.0 to get dollar values and you're ready to go.
It is important to analyze that data, check what queries are commonly used and how the BI Engine is performing - be creative when using this data and it will literally pay-off.

stay curious.

Add new comment