- 8781 Views
- 3 replies
- 3 kudos
Resolved! Notebooks & Jobs Migration from one workspace to another
Hi allI have a need to migrate just notebooks & jobs from one workspace to another. Is there an utility to do so?
- 8781 Views
- 3 replies
- 3 kudos
- 3 kudos
@NOOR BASHA SHAIK you can also try databricks connect in addition to @Artem Sheiko information that was provided .
- 3 kudos
- 2377 Views
- 2 replies
- 2 kudos
Create python modules for both repos and workspace
We are using the "databricks_notebook" terraform resource to deploy our notebooks into the "Workspace" as part of our CICD run, and our jobs run notebooks from the workspace. For development we clone the repo into "Repos". At the moment the only modu...
- 2377 Views
- 2 replies
- 2 kudos
- 2 kudos
You can create your own Python package and host it in Azure Artifacts. https://learn.microsoft.com/en-us/azure/devops/artifacts/quickstarts/python-packages?view=azure-devops
- 2 kudos
- 1101 Views
- 0 replies
- 0 kudos
Why am I not able to view all table properties?
We have a live streaming table created using the commandCREATE OR REFRESH STREAMING LIVE TABLE foo TBLPROPERTIES ( "pipelines.autoOptimize.zOrderCols" = "c1,, c2, c3, c4", "delta.randomizeFilePrefixes" = "true" );But when I run the show table propert...
- 1101 Views
- 0 replies
- 0 kudos
- 6375 Views
- 7 replies
- 3 kudos
Resolved! Community Reward points
Hi Data bricks team , It is informed that all the lifetime points on community will be retired by June 2023. Is it the same for brightsites reward store also .Will the brightsites get restored or lost along with community life time points ? @Vidula...
- 6375 Views
- 7 replies
- 3 kudos
- 3 kudos
@Priyadarshini G So, we are updating our community platform to provide a better experience for our users and members. As part of the updates, we plan on introducing a new recognition protocol that will better celebrate members for their contribution...
- 3 kudos
- 7239 Views
- 1 replies
- 0 kudos
Solution for ConnectException error: This is often caused by an OOM error that causes the connection to the Python REPL to be closed. Check your query's memory usage.
When ever I am trying to run and load multiple files in single dataframe for processing (overall file size is more than 15 gb in single dataframe at the end of the loop, my code is crashing everytime with the below error...ConnectException error: Thi...
- 7239 Views
- 1 replies
- 0 kudos
- 0 kudos
@Satish Agarwal It seems your system memory is not sufficient to load the 15GB file. I believe you are using Python Pandas data frame for loading 15GB file and not using Spark. Is there any particular reason that you cannot use Spark for this.
- 0 kudos
- 1581 Views
- 0 replies
- 0 kudos
Very often, we need to know how many files my table path contains and the overall size of the path for various optimizations. In the past, I had to wr...
Very often, we need to know how many files my table path contains and the overall size of the path for various optimizations. In the past, I had to write my own logic to accomplish this.Delta Lake is making life easier. See how simple it is to obtain...
- 1581 Views
- 0 replies
- 0 kudos
- 2848 Views
- 1 replies
- 0 kudos
%run is not working
I have created two notebooks in my workspace one(A) is having function and other one(B) is having the main code.I am trying to use the %run magic function to use the functions available in notebook A from notebook B. Both notebooks are on the same pa...
- 2848 Views
- 1 replies
- 0 kudos
- 0 kudos
@SK ASIF ALI It's not supported in community if you are using premium version then this should workCan you please send the snippet of the error message? #DAIS2023
- 0 kudos
- 6540 Views
- 0 replies
- 0 kudos
Is Delta Live available in Azure Gov Cloud?
I stood a premium databricks instance but don't have the delta tab visible.
- 6540 Views
- 0 replies
- 0 kudos
- 2664 Views
- 0 replies
- 0 kudos
java.lang.SecurityException: Could not verify permissions for DeleteFromTable ERROR IN EXTERNAL SYNAPSE TABLE
Hi guys, I am trying to delete an external table in Databricks that is connected to a Synapse table (via the com.databricks.spark.sqldw connector) but got the error shown below:Apart from that, when I try to execute an DELETE and INSERT actions (like...
- 2664 Views
- 0 replies
- 0 kudos
- 1249 Views
- 0 replies
- 0 kudos
Is there a way to manually update the cuda required file in the db runtime? There are some rather annoying bugs still in TF 2.11 that have been fixed ...
Is there a way to manually update the cuda required file in the db runtime?There are some rather annoying bugs still in TF 2.11 that have been fixed in TF 2.12.Sadly the latest DB runtime 13.1 (beta) only supports the older TF 2.11 even tho 2.12 was ...
- 1249 Views
- 0 replies
- 0 kudos
- 1352 Views
- 0 replies
- 0 kudos
I'm getting a strange error whenever I try to execute some unittest tests of a class I have written in python. The error is 'DummyMod' object has no attribute...I don't understand this error and need some assistance to sort it out.
The class is written using Python 3.10 and relies on unittest framework for the tests. I'm running it on DB version 13.0 ML.
- 1352 Views
- 0 replies
- 0 kudos
- 6427 Views
- 2 replies
- 3 kudos
Resolved! Why I did not receive the databricks lakehouse fundamentals accreditation badge?
I have passed the test but did not receive the Badge. I also didn't receive any email. Thank you for you attention.
- 6427 Views
- 2 replies
- 3 kudos
- 3 kudos
Hi @Pedro Medeiros Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answer...
- 3 kudos
- 6545 Views
- 1 replies
- 0 kudos
While trying to save the spark dataframe to delta table is taking too long
While working on video analytics task I need to save the image bytes to the delta table earlier extracted into the spark dataframe. While I want to over write a same delta table over the period of complete task and also the size of input data differs...
- 6545 Views
- 1 replies
- 0 kudos
- 0 kudos
can you check the spark UI, to see where the time is spent?It can be a join, udf, ...
- 0 kudos
- 6714 Views
- 0 replies
- 0 kudos
How to set the ABFSS URL for Azure Databricks Init Scripts that have spaces in directory names?
I want to use an Init Script on ADLS Gen2 location for my Azure Databricks 11.3 and 12.2 clusters. The init_script.sh is placed in a directory that has spaces in it:https://storageaccount1.blob.core.windows.net/container1/directory%20with%20spaces/su...
- 6714 Views
- 0 replies
- 0 kudos
- 6382 Views
- 1 replies
- 1 kudos
Resolved! How to create a raw data (with filter_by) to pull query history from now to 5 mins ago
Hi Team, Is it possible I can use "query_start_time_range" filter from the api call to get the query data only from now to 5 mins ago?Im using telegraf to call query history api but it looks like Im reaching the max return and I can't find how to use...
- 6382 Views
- 1 replies
- 1 kudos
- 1 kudos
Have you checked this https://docs.databricks.com/api-explorer/workspace/queryhistory/list you can list the queries based on time range as well. So you can try passing the fields in the filter_by parameter. Then pass the value as (current time - 5 m...
- 1 kudos
| User | Count |
|---|---|
| 1616 | |
| 787 | |
| 383 | |
| 349 | |
| 287 |
