- 1479 Views
- 1 replies
- 0 kudos
Using Community Edition instance for Customer Academy’s DE Learning Plan
Hi!So I’ve been wondering since I started with the Data Engineering Learning Plan on the Customer Academy, should I go with my Community Edition Databricks, or I should go with creating a premium edition on either a cloud provider or the website.Than...
- 1479 Views
- 1 replies
- 0 kudos
- 0 kudos
Hey, @Kaniz. May I recommend deleting the post if it is not in the right place or changing the forum so I get the proper response?
- 0 kudos
- 1928 Views
- 0 replies
- 0 kudos
DLT to push data instead of a pull
I am relatively new to Databricks, and from my recent experience it appears that every step in a DLT Pipeline, we define each LIVE TABLES (be it streaming or not) to pull data upstream.I have yet to see an implementation where data from upstream woul...
- 1928 Views
- 0 replies
- 0 kudos
- 5435 Views
- 1 replies
- 1 kudos
Resolved! Unable to provide access in unity catalog using SQL commands
I am trying to provide access in unity catalog using the SQL commands.I am following the below documentation:https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/manage-privileges/It suggests to create SQL commands in belo...
- 5435 Views
- 1 replies
- 1 kudos
- 1 kudos
Try this. For some reason the quotes are crazy when using these commands. GRANT USAGE ON CATALOG `uda_dev` TO `your-group` GRANT SELECT ON SCHEMA uda_dev.default TO `your-group` (without quotes for the schema)
- 1 kudos
- 2099 Views
- 0 replies
- 0 kudos
Issues with Runtime 15.1/15.2Beta in shared access mode
We have been using runtime 14.2, share mode for our computing cluster in Databrick for quite some time. We are now trying to upgrade to python 3.11 for some dependencies mangement, thereby requiring us to use runtime 15.1/15.2 as runtime 14.2 only ...
- 2099 Views
- 0 replies
- 0 kudos
- 1391 Views
- 1 replies
- 0 kudos
Databricks workflow jobs run is taking Double time in EU Region
We have a scheduled job in Databricks workflow, This Job run is taking aroud 5 hours Previously before 1 month it was tasking on 2.5 hours. Can any one tell what may be the reason behind this. Note: There is no change has been made in this period of ...
- 1391 Views
- 1 replies
- 0 kudos
- 0 kudos
You can check if you are using spot instances on your Job Cluster.btw. if you are using Azure West Europe is on very high demand and sometimes it takes time to provision compute.But it should be matter of minutes, not hours.Check maybe if your data v...
- 0 kudos
- 13090 Views
- 3 replies
- 0 kudos
Resolved! How to pass variables to a python file job
Hi everyone,It's relatively straight forward to pass a value to a key-value pair in notebook job. But for the python file job however, I couldn't figure out how to do it. Does anyone have any idea?Have been tried out different variations for a job wi...
- 13090 Views
- 3 replies
- 0 kudos
- 0 kudos
Thanks so much for this! By the way, is there a way to do it with the JSON interface? I am struggling to get the parameters if entered in this way
- 0 kudos
- 1722 Views
- 1 replies
- 0 kudos
Chat Bot with Azure blob and databricks
Hi Team, I am thinking to start a chat bot application for teams to query data from Azure blob and data bricks tables in python programming language.Please help me out on how i can start and which tools i can use for this requirement.Thanks in advanc...
- 1722 Views
- 1 replies
- 0 kudos
- 0 kudos
@Nagrjuna , that's a great idea! Although we do not know about your use case completely, I am sure you would definitely fall in love with our AI/ML Products. To create a Python chat bot application that can pull data from Azure Blob Storage and Datab...
- 0 kudos
- 2620 Views
- 1 replies
- 1 kudos
How to configure github credentials for a service principal NOT using Azure
I want to have a service principal run a job that uses a notebook in our github. We are AWS not Azure. How do I configure git credentials for the service principal? Does this use deploy keys?
- 2620 Views
- 1 replies
- 1 kudos
- 1 kudos
Awesome details thank you for your help.
- 1 kudos
- 2349 Views
- 1 replies
- 1 kudos
Resolved! Workspace FileNotFoundExecption
I have a model created with catboost and exported in onnx format in workspace and I want to download that model to my local machine.I tried to use the Export that is in the three points to the right of the model, but the model weighs more than 10 Mb ...
- 2349 Views
- 1 replies
- 1 kudos
- 1 kudos
you need put file to FileStorehttps://docs.databricks.com/en/dbfs/filestore.html#save-a-file-to-filestore
- 1 kudos
- 2134 Views
- 1 replies
- 0 kudos
VS Code integration with Python Notebook and Remote Cluster
Hi, I'm trying to work on VS code remotely on my machine instead of using the Databricks environment on my browser. I have went through documentation to set up the Databricks. extension and also setup Databricks Connect but don't feel like they work ...
- 2134 Views
- 1 replies
- 0 kudos
- 3370 Views
- 1 replies
- 0 kudos
What happened to the ephemeral notebook links????? and the job ids????
Hey Databricks, Why did you remove the ephemeral notebook links and job Ids from the parallel runs? This has created a huge gap for us. We can no longer view the ephemeral notebooks, and also the Jobids are missing from the output. Waccha doing?...
- 3370 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi Kaniz, It's funny you mention these things - we are doing some of those - the problem now is that the JobId is obscured from the output meaning we can't tell which ephemeral notebook goes with which JobId. It looks like the ephemeral notebook ...
- 0 kudos
- 5453 Views
- 0 replies
- 0 kudos
Updating Databricks SQL Warehouse using Terraform
We can Update SQL Warehouse manually in Databricks.Click SQL Warehouses in the sidebarIn Advanced optionsWe can find Unity Catalog toggle button there! While Updating Existing SQL Warehouse in Azure to enable unity catalog using terraform, I couldn'...
- 5453 Views
- 0 replies
- 0 kudos
- 5834 Views
- 5 replies
- 3 kudos
OSError: [Errno 78] Remote address changed
Hello:)as part of deploying an app that previously ran directly on emr to databricks, we are running experiments using LTS 9.1, and getting the following error: PythonException: An exception was thrown from a UDF: 'pyspark.serializers.SerializationEr...
- 5834 Views
- 5 replies
- 3 kudos
- 3 kudos
Hi @liormayn , I can understand. I see the fix went on 20 March 2024, you would have to restart the clusters. Thanks!
- 3 kudos
- 2850 Views
- 1 replies
- 0 kudos
Py4JError: An error occurred while calling o992.resourceProfileManager
Hello I am trying to run the SparkXGBoostRegressor and I am getting the following error:SpoilerPy4JError: An error occurred while calling o992.resourceProfileManager. Trace: py4j.security.Py4JSecurityException: Method public org.apache.spark.resource...
- 2850 Views
- 1 replies
- 0 kudos
- 3250 Views
- 0 replies
- 0 kudos
Stream to stream join NullPointerException
I have a DLT pipeline running in continous mode. I have a stream to stream join which runs for the first 5hrs but then fails with a Null Pointer Exception. I need assistance to know what I need to do to handle this. my code is structured as below:@dl...
- 3250 Views
- 0 replies
- 0 kudos
