Share on facebook
Share on linkedin
Share on twitter
Share on reddit

Organizing BigQuery — How to group your queries into folders

This is the ultimate trick to organizing your queries in Google BigQuery.

If you use BigQuery extensively, then as your data projects pile up you’ll likely accumulate a long list of queries along the way (or already have). 

Suddenly, your Saved Queries transform into one long list of unrelated queries.

And as a result, the more queries you save, the harder it becomes to navigate and find the one you want.

The benefits of keeping organized at work are obvious. In the same vein, its a habit you should maintain with your queries as well.

That’s why in this post, you’ll learn how to create an organized system for your BigQuery queries using superQuery‘s web-based IDE.

So if you’re interested in working more productively in BigQuery, read on.

How BigQuery users organize queries today

First, let’s examine the most common ways BigQuery users organize queries today:

  1. Placing queries into BigQuery’s Saved Queries
  2. Saving them into a Git repo with a folder structure
  3. Pasting queries into .txt files and saving to your computer with some folder structure

But an ideal query organization capability would have the best of all three options:

  •  Be easily accessible in the environment in which you are working (Option #1)
  •  Include the capability to place queries into separate folders (Options #2 and #3)

Moreover, with these options you would have to update / save your queries manually (unless you use superQuery’s GitHub integration). 

For instance, clicking “Save Query” every time there is an update, or switching back and forth between your query editor and Git.

Let’s say you wrote queries related to three different topics — product analytics, marketing channel performance, and e-commerce data — and wanted to organize them in BigQuery. 

 

This is how that would look like in BigQuery’s Saved Queries:

Long list of saved queries in Google BigQuery
Saved Queries can very quickly turn into an uncategorized mess

Certainly, it would be ideal to group those queries based on topic while being able to access them from the same environment you’re writing the queries from.

Now let’s explore how to do just  that with superQuery’s Boards feature.

Here’s how it works.

A better way to organize your queries

Boards are like folders for organizing your BigQuery queries by data project, department, or however you’d like. Think of them as a filing & management system for your queries.

Instead of having a single Saved Queries folder, you have multiple Boards for grouping related queries together.

The image below shows how you would use boards to organize queries related to the topics from our example mentioned earlier.

A list of three boards in superQuery (red arrows), with the names of the queries inside each board.

Transform boards into dashboards.

Queries stored in boards represent the building blocks for a dashboard as well.

With one click, you can build on top of your query results, transforming each tab into widgets that make up a dashboard. 

See how it works below.

Sharing boards with colleagues

By default, all of your queries and boards are private. However, you can easily share your boards with others if you’d like. 

Shared users can do the following in your board:

  • View the SQL of the queries.**
  • Copy the SQL and paste it into their own board.
  • Access the dashboard view of your board.
  • Refresh the dashboard (any query affected by the changed value will be charged to the billing project of the board creator)
  • Modify your variable(s) values if any exist.

** We are working on real-time collaboration capabilities (i.e. Google Docs) for users in shared boards. Stay tuned on that. 

Give Boards a go

With Boards, all of your queries are properly organized in one place — where you need them the most.

BigQuery users that switch to grouping queries with Boards are more productive and far more efficient than before.

Now that you know how to use Boards, give them a try yourself.

It takes only three clicks to get a superQuery account and start using Boards, and the other productivity-enhancing features the IDE supports.

  • © 2019 superQuery. All rights reserved.

We use 🍪 to ensure you get the best experience.