Data Ingestion from GCP
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-03-2024 04:03 AM
Hi,
I'm ingesting data from GCP to Databricks, and I think I've noticed a bug in that any datatables that have a numerical starting character are not ingested at all.
Has anyone else experienced this?
Please let me know if there is a way around this apart from re-naming the datatables (I have many)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-03-2024 04:14 AM
Hi @delson ,
Could you give us some examples? And when you're saying GCP you mean bigquery?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-03-2024 04:37 AM
Hi Slash, thanks for getting back to me
So for instance - I have data tables such as "20240901_demographics_data_v1" which I'm trying to move from BQ
Other data tables that don't include a date (or other numerical characters) at the front are being ingested but datatables with the date naming convention at the start are not being brought through
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-03-2024 04:44 AM
Hi @delson ,
Thanks for clarification. Could you try to escape problematic table name using backtick character?
`20240901_demographics_data_v1`- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-03-2024 07:42 AM
Hi Slash - Thanks for your response
Unfortunately, due to BQ's capabilities I'm not able to insert backticks to data table names without manually changing each of the hundreds of data tables I have
Currently, I'm ingesting the data in an automated fashion using a BQ connection through Databricks in the Catalog / Catalog Explorer
Nonetheless, thanks - hopefully another option is possible 🙂
