- 903 Views
- 1 replies
- 0 kudos
displaying unity catalog metadata and other information in sharePoint
Is there any connectors or api which we can use to display metadata information stored in Unity catalog to business users using SharePoint.
- 903 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Lucifer As metadata are stored in system schema as table that means you can use databricks to extract the data from databricks and display it to Sharepoint Docs - Statement Execution API: Run SQL on warehouses | Databricks on AWS
- 0 kudos
- 826 Views
- 0 replies
- 0 kudos
DLT to push data instead of a pull
I am relatively new to Databricks, and from my recent experience it appears that every step in a DLT Pipeline, we define each LIVE TABLES (be it streaming or not) to pull data upstream.I have yet to see an implementation where data from upstream woul...
- 826 Views
- 0 replies
- 0 kudos
- 2907 Views
- 1 replies
- 1 kudos
Resolved! Unable to provide access in unity catalog using SQL commands
I am trying to provide access in unity catalog using the SQL commands.I am following the below documentation:https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/manage-privileges/It suggests to create SQL commands in belo...
- 2907 Views
- 1 replies
- 1 kudos
- 1 kudos
Try this. For some reason the quotes are crazy when using these commands. GRANT USAGE ON CATALOG `uda_dev` TO `your-group` GRANT SELECT ON SCHEMA uda_dev.default TO `your-group` (without quotes for the schema)
- 1 kudos
- 1057 Views
- 0 replies
- 0 kudos
Issues with Runtime 15.1/15.2Beta in shared access mode
We have been using runtime 14.2, share mode for our computing cluster in Databrick for quite some time. We are now trying to upgrade to python 3.11 for some dependencies mangement, thereby requiring us to use runtime 15.1/15.2 as runtime 14.2 only ...
- 1057 Views
- 0 replies
- 0 kudos
- 1915 Views
- 3 replies
- 1 kudos
Cluster policy type
Hi guys,I am creating a cluster policy through json. "runtime_engine": {"type": "fixed""value": "PHOTON"}When I run the above code... PHOTON option is getting enabled but graying out... What would I specify in type field so that the photon option sho...
- 1915 Views
- 3 replies
- 1 kudos
- 1 kudos
@thrinadhReddy This section includes a reference for each of the available policy types. There are two categories of policy types: fixed policies and limiting policies. Fixed policies prevent user configuration on an attribute. The two types of fixed...
- 1 kudos
- 674 Views
- 1 replies
- 0 kudos
Databricks workflow jobs run is taking Double time in EU Region
We have a scheduled job in Databricks workflow, This Job run is taking aroud 5 hours Previously before 1 month it was tasking on 2.5 hours. Can any one tell what may be the reason behind this. Note: There is no change has been made in this period of ...
- 674 Views
- 1 replies
- 0 kudos
- 0 kudos
You can check if you are using spot instances on your Job Cluster.btw. if you are using Azure West Europe is on very high demand and sometimes it takes time to provision compute.But it should be matter of minutes, not hours.Check maybe if your data v...
- 0 kudos
- 7254 Views
- 3 replies
- 0 kudos
Resolved! How to pass variables to a python file job
Hi everyone,It's relatively straight forward to pass a value to a key-value pair in notebook job. But for the python file job however, I couldn't figure out how to do it. Does anyone have any idea?Have been tried out different variations for a job wi...
- 7254 Views
- 3 replies
- 0 kudos
- 0 kudos
Thanks so much for this! By the way, is there a way to do it with the JSON interface? I am struggling to get the parameters if entered in this way
- 0 kudos
- 889 Views
- 1 replies
- 0 kudos
Chat Bot with Azure blob and databricks
Hi Team, I am thinking to start a chat bot application for teams to query data from Azure blob and data bricks tables in python programming language.Please help me out on how i can start and which tools i can use for this requirement.Thanks in advanc...
- 889 Views
- 1 replies
- 0 kudos
- 0 kudos
@Nagrjuna , that's a great idea! Although we do not know about your use case completely, I am sure you would definitely fall in love with our AI/ML Products. To create a Python chat bot application that can pull data from Azure Blob Storage and Datab...
- 0 kudos
- 1631 Views
- 0 replies
- 0 kudos
Unable to grant catalog access to service principal
Hi everyone,I created a service principals called TestServicePrincipal. I tried to grant the catalog access to the service principals, but the error mentioned that it could not find principal with name TestServicePrincipal. If I grant the access to s...
- 1631 Views
- 0 replies
- 0 kudos
- 1532 Views
- 1 replies
- 1 kudos
Resolved! Workspace FileNotFoundExecption
I have a model created with catboost and exported in onnx format in workspace and I want to download that model to my local machine.I tried to use the Export that is in the three points to the right of the model, but the model weighs more than 10 Mb ...
- 1532 Views
- 1 replies
- 1 kudos
- 1 kudos
you need put file to FileStorehttps://docs.databricks.com/en/dbfs/filestore.html#save-a-file-to-filestore
- 1 kudos
- 1751 Views
- 1 replies
- 0 kudos
What happened to the ephemeral notebook links????? and the job ids????
Hey Databricks, Why did you remove the ephemeral notebook links and job Ids from the parallel runs? This has created a huge gap for us. We can no longer view the ephemeral notebooks, and also the Jobids are missing from the output. Waccha doing?...
- 1751 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi Kaniz, It's funny you mention these things - we are doing some of those - the problem now is that the JobId is obscured from the output meaning we can't tell which ephemeral notebook goes with which JobId. It looks like the ephemeral notebook ...
- 0 kudos
- 2892 Views
- 5 replies
- 3 kudos
OSError: [Errno 78] Remote address changed
Hello:)as part of deploying an app that previously ran directly on emr to databricks, we are running experiments using LTS 9.1, and getting the following error: PythonException: An exception was thrown from a UDF: 'pyspark.serializers.SerializationEr...
- 2892 Views
- 5 replies
- 3 kudos
- 3 kudos
Hi @liormayn , I can understand. I see the fix went on 20 March 2024, you would have to restart the clusters. Thanks!
- 3 kudos
- 2386 Views
- 0 replies
- 0 kudos
Stream to stream join NullPointerException
I have a DLT pipeline running in continous mode. I have a stream to stream join which runs for the first 5hrs but then fails with a Null Pointer Exception. I need assistance to know what I need to do to handle this. my code is structured as below:@dl...
- 2386 Views
- 0 replies
- 0 kudos
- 2832 Views
- 4 replies
- 2 kudos
Resolved! How to choose a compute, and how to find alternatives for the current compute being used?
We are using a compute for an Interactive Cluster in Production which incurs X amount of cost. We want to know what are the options available to use with near about the same processing power as the current compute but incur a cost of Y, which is less...
- 2832 Views
- 4 replies
- 2 kudos
- 2 kudos
Hello @Ikanip , You can utilize the Databricks Pricing Calculator to estimate costs. For detailed information on compute capacity, please refer to your cloud provider's documentation regarding Virtual Machine instance types.
- 2 kudos
- 969 Views
- 0 replies
- 0 kudos
Databricks Running Jobs and Terraform
What happens to a currently running job when a workspace is deployed again using Terraform? Are the jobs paused/resumed, or are they left unaffected without any down time? Searching for this specific scenario doesn't seem to come up with anything and...
- 969 Views
- 0 replies
- 0 kudos
