cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Data Ingestion from GCP

delson
New Contributor II

Hi,

I'm ingesting data from GCP to Databricks, and I think I've noticed a bug in that any datatables that have a numerical starting character are not ingested at all.

Has anyone else experienced this?

Please let me know if there is a way around this apart from re-naming the datatables (I have many)

4 REPLIES 4

szymon_dybczak
Contributor III

Hi @delson ,

Could you give us some examples? And when you're saying GCP you mean bigquery?

delson
New Contributor II

Hi Slash, thanks for getting back to me

So for instance - I have data tables such as "20240901_demographics_data_v1" which I'm trying to move from BQ

Other data tables that don't include a date (or other numerical characters) at the front are being ingested but datatables with the date naming convention at the start are not being brought through

Hi @delson ,

Thanks for clarification. Could you try to escape problematic table name using backtick character?

`20240901_demographics_data_v1`

Hi Slash - Thanks for your response

Unfortunately, due to BQ's capabilities I'm not able to insert backticks to data table names without manually changing each of the hundreds of data tables I have

Currently, I'm ingesting the data in an automated fashion using a BQ connection through Databricks in the Catalog / Catalog Explorer

Nonetheless, thanks - hopefully another option is possible ๐Ÿ™‚

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group