Skip to content

Samples for migration from datalab's BigQuery module to google-cloud-bigquery#1995

Merged
alixhami merged 8 commits into
masterfrom
pydatalab-migration
Feb 11, 2019
Merged

Samples for migration from datalab's BigQuery module to google-cloud-bigquery#1995
alixhami merged 8 commits into
masterfrom
pydatalab-migration

Conversation

@alixhami

@alixhami alixhami commented Feb 5, 2019

Copy link
Copy Markdown
Contributor

No description provided.

@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Feb 5, 2019
@alixhami alixhami requested a review from tswast February 8, 2019 19:43

@tswast tswast left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

I still smile when I see those "doomed" datasets. 😈

schema = bq.Schema.from_data(dataframe)
table = bq.Table(
'{}.monty_python'.format(dataset_id)).create(schema=schema)
table.insert(dataframe) # Starts steaming insert of data

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Streaming insert is not the same thing as a load job. Does datalab not have a way to create a load job from a dataframe?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was the only way I could find

@alixhami alixhami merged commit 95bf145 into master Feb 11, 2019
@alixhami alixhami deleted the pydatalab-migration branch February 11, 2019 18:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla: yes This human has signed the Contributor License Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants