by
sher
• Valued Contributor II
- 1587 Views
- 4 replies
- 3 kudos
Hi all,Is there any way to implement delta sharing in databricks to snowflake direct connect ?
- 1587 Views
- 4 replies
- 3 kudos
Latest Reply
I don't think that Snowflake has implemented the ability to read from a table via Delta Sharing as of December 2023. Please reach out to your Snowflake representatives and urge them to consider this feature from their side.
Alternatively, you can qu...
3 More Replies
- 3211 Views
- 2 replies
- 0 kudos
Hi,I am trying to get the raw size (total size) of delta table. I could get delta table size from DeltaTable api but that gives only latest version size. I need to find the actual S3 size the tables takes on S3.Is there any way, to find the S3 size ...
- 3211 Views
- 2 replies
- 0 kudos
Latest Reply
Hi Kaniz,Thank you for your suggestions. As per my understanding, the "snapshot.sizeInBytes" gives only current snapshot size. But I am looking for total size (all versions) of the table on S3.
1 More Replies
- 1232 Views
- 4 replies
- 0 kudos
Hello everyone,I have a workflow that starts by reading the CDF data for a change data feed.The syntax is exactly the following : (spark.readStream .format("delta") .option("readChangeFeed", "true") .option("startingVersion", 10) .table("my.str...
- 1232 Views
- 4 replies
- 0 kudos
Latest Reply
Hi, this seems to be related to the issue I've been getting around the same time here: Change data feed from target tables of APPLY CHANG... - Databricks - 54436Would be great to get an explanation for the sudden change in behaviour.
3 More Replies
- 2174 Views
- 3 replies
- 0 kudos
Hi,I wanted to ask if anyone is using Databricks SQL Warehouse as backend for small to large scale web application? What are your thoughts about it, specially what Databricks team thinks of it ?Kind Regards,A
- 2174 Views
- 3 replies
- 0 kudos
Latest Reply
Hey AhsanKhawaja,Yeah, I've played around with using Databricks SQL Warehouse as a backend for a web app. It's got its perks, but like anything, it depends on your needs.From my experience, it's slick for handling big data and crunching numbers. If y...
2 More Replies
by
Jules
• New Contributor
- 301 Views
- 0 replies
- 0 kudos
Hi,We are using Databricks bundles to deploy our DBT project. Everything is set up to deploy and run as a Service Principal.The DBT job is connected to an Azure DevOps repository. The problem is that we cannot find a way to properly authenticate the ...
- 301 Views
- 0 replies
- 0 kudos
by
NLearn
• New Contributor II
- 410 Views
- 2 replies
- 0 kudos
For one of the requirements of project, I want to save default language of notebook into variable based on notebook path mentioned dynamically.For eg: if first notebook given by user in widget is having default language as Python then variable value...
- 410 Views
- 2 replies
- 0 kudos
Latest Reply
Hi @NLearn, To change the default language of a notebook in Databricks, you can select File -> Change default cell language. This will affect all the cells in the notebook that use the same language as the default one.
You can also use magic commands...
1 More Replies
- 443 Views
- 1 replies
- 0 kudos
Dear Databricks community members:SymptomReceived the error for a delta load, after a successful initial load with a Unity Catalog Volume as a data source.org.apache.spark.sql.streaming.StreamingQueryException: [STREAM_FAILED] Query [id = xxx, runId...
- 443 Views
- 1 replies
- 0 kudos
Latest Reply
Hi @harvey-c, The file system path of the data source has changed from dbfs:/ to abfss:// after a previous successful load. This might confuse the Spark streaming query and cause it to fail with a wrong file system exception.
One possible solution ...
by
GijsM
• New Contributor
- 513 Views
- 1 replies
- 0 kudos
Hi,I work for a small company, we're mostly focussing on small retail and e-commerce customers. We provides data analysis and automated data connections between their platforms. Most of our datasets are things like order data, google ads click data, ...
- 513 Views
- 1 replies
- 0 kudos
Latest Reply
Hi,
Thanks for the information, there is a lot to unpack and some assumptions that need to be made without fully understanding the details, so here are a few thoughts:
If the cluster start times longer because of the libraries you're installing, can ...
by
Fz1
• New Contributor III
- 4545 Views
- 5 replies
- 3 kudos
I have DLT tables created under the hive_metastore with external data stored in ADL gen2.The ADL blob storage is mounted into /mnt/<storage-account>The tables are successfully created and accessible from my notebooks, as well the ADL storage.I have c...
- 4545 Views
- 5 replies
- 3 kudos
by
Phani1
• Valued Contributor
- 562 Views
- 1 replies
- 0 kudos
Hi Team,How can expose data stored in delta table through API like exposing sql data through .net api?
- 562 Views
- 1 replies
- 0 kudos
Latest Reply
You can use the SQL Statement Execution API.At energinet.dk we have created this open-source .NET client, which we use internally in the company.
- 2359 Views
- 3 replies
- 3 kudos
We use typesafe (scala) to read configuration values from hocon files.When not using Unity, we read the configuration files from /dbfs/... works fine.However, with Unity, usage of dbfs is frowned upon.So I started looking into alternatives.And unfor...
- 2359 Views
- 3 replies
- 3 kudos
Latest Reply
In the end we will continue to use dbfs. Maybe in the future when volumes are supported by scala io we can re-evaluate, but for now dbfs seems the way to go.
2 More Replies
- 1475 Views
- 1 replies
- 6 kudos
response = requests.request("POST", url, verify=False, headers=headers, data=payload)
SSLError: HTTPSConnectionPool(host='dcs.adobedc.net', port=443): Max retries exceeded with url: /collection/d99e6dfcffb0b5aeaec2cf76cd3bc2b9e9c414b0c74a528d13dd39...
- 1475 Views
- 1 replies
- 6 kudos
- 394 Views
- 0 replies
- 0 kudos
Hi Team,My Databricks Certified Data Engineer Associate exam got suspended on 17th December and it is in progress state.I was there continuously in front of the camera and suddenly the alert appeared, and support person asked me to show the desk and ...
- 394 Views
- 0 replies
- 0 kudos
- 748 Views
- 2 replies
- 0 kudos
Hello Team,I had a disappointing experience while attempting my first DataBricks certification. Abruptly, the proctor asked me to show my desk, and after complying. Eventually, they suspended my exam, citing excessive eye movement and other practices...
- 748 Views
- 2 replies
- 0 kudos
Latest Reply
@mkrish28 I'm sorry to hear you had this experience. Thank you for logging at ticket with the support team. They have informed me they have rescheduled your exam. Good luck!
1 More Replies
- 4473 Views
- 8 replies
- 1 kudos
After making some changes in my feature branch, I have committed and pushed (to Azure Devops) some work (note I have not yet raised a PR or merge to any other branch). Many of the files I committed are data files and so I would like to reverse the co...
- 4473 Views
- 8 replies
- 1 kudos
Latest Reply
I'm also curious about this question - does anyone have an answer? Being able to use the full repertoire of git commands inside Databricks would be quite useful.
7 More Replies
