How to Set Up Your First BigQuery Project

In this post, you’ll learn what BigQuery is, understand its capabilities, and set up a project in Google Cloud which we will later use to practice using BigQuery for loading, querying, and analyzing data.

This is the first article in a series about BigQuery: Google’s fully managed, serverless, and AI-friendly data platform for managing, querying, and analyzing large datasets.

What is BigQuery?

BigQuery is a server-less data analysis platform that has been fully developed and managed by Google. It has been specifically envisaged to manage and process large volumes of structured (tabular) data, allowing users to run SQL queries rapidly and efficiently. Because of its full integration into Google Cloud Platform (GCP), BigQuery eliminates the burden of managing servers and their underlying infrastructure. Moreover, its flexible and scalable architecture makes BigQuery an appealing option for organizations that need to analyze vast amounts of data (terabytes, petabytes) in real time.

Below are listed some of the main capabilities and functionalities of BigQuery:

• Efficient querying: Thanks to its distributed processing engine, BigQuery allows to execute SQL-like queries against large datasets in just seconds.
• Scalable data storage: BigQuery supports columnar storage and optimized compression, thereby reducing costs and boosting performance.
• Integration with a wealth of Google tools: The integration between BigQuery and other Google tools in the cloud couldn’t be easier. Popularly used tools like Google Sheets, Data Studio, Vertex AI, and Cloud ML Engine, can be used alongside BigQuery to facilitate the creation of advanced data analysis solutions.
• Security, robustness, and compliance: BigQuery implements advanced security features, such as data encryption protocols, and it is compliant with the major data protection regulations worldwide.

This table succinctly summarizes the differences between BigQuery and other conventional database management tools.

Unlike other conventional database management tools, using BigQuery does not require setting up or managing servers or manually optimizing queries. This will save us a lot of time in the project and API setup steps below. At the end of the day, this feature allows teams to focus their efforts on analyzing data, without worrying about the underlying computing infrastructure.

Setting up a Google Cloud project

First of all, we’ll need access to the Google Cloud Console. If you do not have your Google account registered in Google Cloud yet and want to use the free tier, click on “Start free trial”. After filling in the required information, you’ll gain access to the Google Cloud console.

Google Cloud projects are the centerpiece for creating and utilizing the range of services offered by the cloud provider. Setting up a project in Google Cloud involves a couple of easy steps:

• Click on the navigation menu icon (triple horizontal bars) in the upper-left corner, then go to “IAM & Admin” and “Manage resources”.
• You’ll be asked for a project name, and an optional location (we can leave the “No organization” option as the default one). For the project name, let’s call our project “BigQuery Project”.
• Optionally, at this stage you have the option of changing the project ID assigned by default to our project. Importantly, if you do not change your project ID at this point, it will become immutable and cannot be changed later. Just make sure the default project ID resonates with you.

And that’s it! Our Google Cloud project has now been created.

Enabling BigQuery API

One more setup process is needed before fully exploring BigQuery: we’ll need to enable its API. To do this, go to the navigation menu on the left-hand side of the console again, and this time click on “APIs and Services”, followed by “Library”. Use the search box to type “BigQuery API” and scroll down the list of results to find the namesake API, as shown in the screenshot below.

Click on the item, and click on “Enable” to enable the API. If the “Enable” button is not available, you’ll see the “Manage” button instead. No need to do anything in that case, since you already got the API enabled earlier on. If this is your first BigQuery project, however, chances are you’ll need to activate the API.

We are done with the initial setup! In the next post of this series, we’ll cover data loading into BigQuery using the Google Cloud project we just created: stay tuned.

Iván Palomares Carrascosa is a leader, writer, speaker, and adviser in AI, machine learning, deep learning & LLMs. He trains and guides others in harnessing AI in the real world.

1. Google Cybersecurity Certificate – Get on the fast track to a career in cybersecurity.

2. Google Data Analytics Professional Certificate – Up your data analytics game

3. Google IT Support Professional Certificate – Support your organization in IT