Optional Metric Time Selection in Google BigQuery and Snowflake Updates

Explore the latest updates including optional metric time selection in Google BigQuery and Snowflake, and enhancements to the Data Quality engine.

By

Melanie

Updated on

August 3, 2024

As a data engineer, you probably know the importance of maintaining high-quality data to ensure accurate analytics and decision-making which in turn leads to building trust with your stakeholders. That's why we are excited to announce the latest update to our Data Quality engine, which will ensure that you can make full use of our monitoring and notification system to get ahead of potential incidents that may affect your business.

Volume and Freshness Monitoring Without Metric Time Selection

Enable monitoring with metric time selection. Just toggle it on, that easy.

We understand that you need to be able to monitor data changes and ensure data quality without having to specify a metric time selection. That's why we've introduced Volume and Freshness monitoring for Google Big Query and Snowflake connections without requiring a metric time selection. This update makes it easier for you to enable monitoring across these connections, try it out in our Bulk Table Configuration. Learn more here.

Expanded Metric Time Column Options

Select a date as a metric time.

For all data connections, we have also expanded our metric time column options to include date columns as metric time columns for all data sources, not just datetime columns. We received feedback from teams that this feature would be useful in tracking tables that may not have datetime columns. Selecting a date field restricts scan frequency selection to 24 hours only.

It's important for us to know a metric time column (with exception of Google Big Query and Snowflake) to know the window to scan your tables, to prevent a large database cost to you. Learn more here.

Schema Change Monitoring Enabled on All Tables

Schema drift is now enabled by default.

We understand that schema changes can occur frequently and unexpectedly, which is why we have enabled schema change monitoring on all tables by default. You will receive notifications automatically whenever a schema change occurs, ensuring that you're aware of any potential issues and work on it before it affects the downstream. You can toggle this notification on or off in the Bulk Table Config or Asset Details section.

UI Improvements for Better User Experience

We have made significant UI improvements to our Bulk Table Config and Asset Table Config features to provide a better user experience for our users, including updating icons for metric time selection and adding descriptive placeholder texts to make it easier for you to navigate and select the appropriate monitoring options. We also added clearer tooltips to help you understand the pre-requisites for selecting a metric time before adding a field monitor.

Bug Fixes and Improvements

We included several bug fixes and other improvements to enhance the reliability and flexibility of our system. Our goal is to provide the best possible user experience for data teams, and we hope that these updates will make your job easier and more efficient.

If you have any questions or feedback, please do not hesitate to contact our support team. You can also view our Docs Changelog to see the nitty-gritty details of the fixes we've done since our v1.0.

Table of Contents

Read other blog articles

Grow with our latest insights

Sneak peek from the data world.

Thank you! Your submission has been received!
Talk to a designer

All in one place

Comprehensive and centralized solution for data governance, and observability.

decube all in one image