New in Google BigQuery: Schema Selection & Enhanced Asset Details

Learn about Google BigQuery's new schema selection and enhanced asset details, plus insights on Langchain for language models.

By

Melanie

Updated on

October 7, 2024

New settings for schema selection in Google Big Query

For Google Big Query connections, you can now specify which schemas are to be scanned and included to have greater control of which assets will be included into decube's Data Quality and Data Catalog.

Now, when you add a new connection to Google Big Query, you will need to test your connection first. If the connection is successful, you now have the choice to either scan and include all schemas within the connection or only scan a specific selection on schemas.

We've also added a new selection here for us to handle new schemas being added to your data source. You can either let us:

Add new schemas automatically as we pick them up in the scan (and thus will appear in your Catalog without any additional effort to add them on your end), or;
Never add them automatically. The new schema would still appear in the list when you head over to Modify Data Source and you can manually add it from there.

Example of list of schemas that can be selected. — Example of list of schemas. You can select those to be included into our scan.

First time setting up your connection? Check our our docs here for more information.

‍

Asset Details has a new look

Example of merchants table. You can now add owners onto the assets!

We've updated the Asset Details to now have improved icons for your data sources, asset types and attributes. Enrich your asset details by adding Data Owners and Business Owners onto your critical data assets. Owners are responsible to maintain the data assets and act as the contact point for the status of the data asset.

We'll be adding the ability for users to tag columns within the data asset (eg. adding PII tags), and also add documentation to each asset for a complete data cataloging experience for your organization. Check out our plan on our roadmap here.

‍

Update your user profile

Didn't like the name you used when you first signed up? Just go ahead and change it in our new tab Update Profile in the My Account page!

‍

You can now edit your name and change your profile image.

‍

... and other improvements.

We've also enhanced the following:

Incident Details graph now show the min/max of the Expected vs the Actual for Volume graphs.

You can now save time by updating your table-level tests (Freshness, Volume, Schema drift) from the Data Quality page directly.

‍

You can also view our Docs Changelog to see the nitty-gritty details of the fixes we've done since our v1.0.

Read other blog articles

What is Data Consistency? Types, Challenges, Examples, and Best Practices

Understand the different types of data consistency, key challenges in distributed systems, and explore examples and best practices to ensure reliable data across pipelines and databases with Decube.

By

Jatin Solanki

November 19, 2024

Introducing Decube Copilot's Autocomplete: Smarter Data Curation

Empower data governance teams by automating descriptions, reducing manual efforts, and enhancing data curation workflows.

By

Melanie Yong

November 19, 2024

Data Observability Use Cases

Explore the transformative power of data observability use cases - from data quality monitoring to anomaly detection and lineage tracking for effective data management.

By

Jatin Solanki

November 14, 2024