Configuring BigQuery Operations

To use the BigQuery connector, add the <bigquery.init> or <bigquery.getAccessTokenFromServiceAccount> element in your configuration before carrying out any other BigQuery operations. 

The BigQuery API requires all requests to be authenticated as a user or a service account. For more information, see https://cloud.google.com/bigquery/authentication and about the service account authentication see https://developers.google.com/identity/protocols/OAuth2ServiceAccount

<bigquery.init>
    <apiUrl>{$ctx:apiUrl}</apiUrl>
    <accessToken>{$ctx:accessToken}</accessToken>
    <clientSecret>{$ctx:clientSecret}</clientSecret>
    <clientId>{$ctx:clientId}</clientId>
    <refreshToken>{$ctx:refreshToken}</refreshToken>
    <registryPath>{$ctx:registryPath}</registryPath>
    <fields>{$ctx:fields}</fields>
    <prettyPrint>{$ctx:prettyPrint}</prettyPrint>
    <quotaUser>{$ctx:quotaUser}</quotaUser>
    <userIp>{$ctx:userIp}</userIp>
</bigquery.init>

Properties

• apiUrl: The base endpoint URL of the BigQuery API.
• accessToken: OAuth token for BigQuery API.
• clientId:  Client Id for BigQuery API.
• clientSecret: Client Secret for BigQuery API.
• refreshToken: RefreshToken token for BigQuery API.
• registryPath: Registry path to save the access token.
  
• fields: List of fields to be returned in the response.
• callback: Name of the JavaScript callback function that handles the response. Used in JavaScript JSON-P requests.
• apiKey: API key. Required, unless you provide an OAuth 2.0 token.
• prettyPrint: Returns response with indentations and line breaks. Returns the response in a human-readable format if true.
• quotaUser: Alternative to userIp. Lets you enforce per-user quotas from a server-side application even in cases when the user's IP address is unknown.
• userIp: IP address of the end user for whom the API call is being made. Lets you enforce per-user quotas when calling the API from a server-side application.
• ifMatch: Etag value to use for returning a page of list values if the values have not changed.
• ifNoneMatch: Etag value to use for returning a page of list values if the values have changed.

or

You can using only the below getAccessTokenFromServiceAccount  to get the access token and do all the other operations.

<bigquery.getAccessTokenFromServiceAccount>
    <apiUrl>{$ctx:apiUrl}</apiUrl>
    <keyStoreLocation>{$ctx:keyStoreLocation}</keyStoreLocation>
    <serviceAccount>{$ctx:serviceAccount}</serviceAccount>
    <scope>{$ctx:scope}</scope>
    <accessTokenRegistryPath>{$ctx:accessTokenRegistryPath}</accessTokenRegistryPath>
</bigquery.getAccessTokenFromServiceAccount>

Properties

• apiUrl: The base endpoint URL of the BigQuery API.
• keyStoreLocation: The location where the p12 key file is located.
• serviceAccount: The value of the service Account.
• scope: The space delimited scope to access the API.
• accessTokenRegistryPath: The registry path to store the access token(This is an optional parameter).
  

Sample request

Following is a sample REST request that can be handled by the getAccessTokenFromServiceAccount operation.

{
    "apiUrl":"https://www.googleapis.com",
	"keyStoreLocation":"/home/hariprasath/Desktop/bigQuery/p12/Data Services - Non Production.p12",
	"serviceAccount":"service-account-bq.com",
    "scope":"https://www.googleapis.com/auth/bigquery",
	"accessTokenRegistryPath":"connectors/BigQuery/accessToken"
}

<bigquery.getAccessTokenFromAuthorizationCode>
    <apiUrl>{$ctx:apiUrl}</apiUrl>
    <authorizationCode>{$ctx:authorizationCode}</authorizationCode>            
	<redirectUrl>{$ctx:redirectUrl}</redirectUrl>
    <clientSecret>{$ctx:clientSecret}</clientSecret>
    <clientId>{$ctx:clientId}</clientId>
    <registryPath>{$ctx:registryPath}</registryPath>
</bigquery.getAccessTokenFromAuthorizationCode>

{
  "authorizationCode": "4/crg1KE2zMBMQPJTrYct-wszP0j-J8Zfzd5adVz2ci7I",
  "apiUrl": "https://www.googleapis.com",
  "redirectUrl": "https://localhost",
  "clientId": "504627865627-kdni8r2s10sjddfgXzqb4bvnba.apps.googleusercontent.com",
  "clientSecret": "ChlbHI_T7zssXXTRYuqj_-TM",
  "registryPath": "connectors/bq"
}

<bigquery.getAccessTokenFromRefreshToken>
	<apiUrl>{$ctx:apiUrl}</apiUrl>
    <clientSecret>{$ctx:clientSecret}</clientSecret>
    <clientId>{$ctx:clientId}</clientId>
    <refreshToken>{$ctx:refreshToken}</refreshToken>
</bigquery.getAccessTokenFromRefreshToken>

{
  "apiUrl": "https://www.googleapis.com",
  "clientId": "504627865627-kdni8r2s10sjcgd4v6stthdaqb4bvnba.apps.googleusercontent.com",
  "refreshToken": "1/uWful-diQNAdk-alDUa6ixxxxxxxx-LpJIikEQ2sqA",
  "clientSecret": "ChlbHI_T7zssXXTRYuqj_-TM"
}

Now that you have connected to BigQuery, use the information in the following topics to perform various operations with the connector. 

• Working with Datasets in BigQuery
• Working with Jobs in BigQuery
• Working with Projects in BigQuery
• Working with Table Data in BigQuery
• Working with Tables in BigQuery