Hi all,When attempting to connect to Databricks with Spark ODBC using the regular host ip and port, everything is successful. However, we need to send the connection through an internal proxy service that re-maps the server's endpoint to a local port...
Hi,I am trying to connect AWS Databricks to PowerBI using service principal. Below are the steps I followed:1. Created Service Principal in Identity and Access2. I went to Permission setting page under Settings>Advanced and added this new service pri...
Thanks for the reply, I got the solution. I was missing adding SP account in SQL Warehouse permission setting. After adding it, my PBI report is working fine.
Hi Team,How can I make a SQL Endpoint web api in Databricks that can be used in denodo?Regards,Janga
I want to get the JSON definition of the "create part" of the job. I have the job id and job name. I am using databricks notebook for this. I can the get the "GET" json api definition but not able able to get the "CREATE" part json definition which I...
Hi,Can you let me know what are the challenges and how to mitigate while migrating Hive objects to Databricks. I could not get any information on this. Can you please provide.
I have this usecase wherein i am supposed to pass a dynamic query to get data from source I have tried the query option but its giving error SparkConnectGrpcException: (com.microsoft.sqlserver.jdbc.SQLServerException) Incorrect syntax near the keywor...
Hi thanks for your reply,I have used foreign catalouge to fetch required data from information schema then i am creating the dynamic query in databricks and then passing in query this is working for me! @Retired_mod
I'm not sure if this is the right place to post my question. If not, please let me know where I should post my question. I want to download large files from the web from Databricks' master(driver) node. For example, I fetch a file over 150GB via API ...
Hi @himanmon,If you 100% sure that you can't download this file to storage account configured with unity catalog and you want it directly on driver node local storage, then why can't you just increase local disk space by choosing a larger instance ty...
Hi Team,In Azure Databricks, we currently use Service Principal when creating Mount Points to Azure storage ( ADLS Gen1, ADLS Gen 2 and Azure Blob Storage).As part of S360 action to eliminate SPN secrets, we were asked to move to SPN+certificate / MS...
@ramesitexp Yes @szymon_dybczak is correct for now only valid option is below : OAuth 2.0 with a Microsoft Entra ID service principalShared access signatures (SAS)Account keys For now we are using OAuth 2.0 with a Microsoft Entra ID service principal...
HiI have multiple json files stored in my ADLS2 and I want to create a table in which will directly read all the data from ADLS without mounting the files. When I create the table, i cannot select all the data How can i achieve this.ADLS Path : /dwh/...
I am trying to use OAuth user-to-machine (U2M) authentication from azure databricks CLI.When I run databricks auth login --host ,I get a web browser open and get authentication sucessfull message and My profile also save successfully with auth-type...
Hi @Aria , Good Day! Which CLI version you are using here? Can you try to update the CLI version to a newer version by referring to this document : https://learn.microsoft.com/en-us/azure/databricks/dev-tools/cli/install#--homebrew-update-for-linux-...
Hi Can I able to create unity catalog using free trail account?
Hi @Spyro_3 ,Yeah, you should be able to create unity catalog. Trial version allows you to create premium workspace which is required for unity catalog. Notice, that to setup metastore you also need to have Global Administrator permission (if we are ...
I'm encountering an issue with the installation of Python packages from a Private PyPI mirror, specifically when the package contains dependencies and the installation is on Databricks clusters - Cluster libraries | Databricks on AWS. Initially, ever...
Hi @hugodscarvalho ,I am also at this point, where the transitive dependencies (available in jfrog) are not getting installed in my job cluster. Could you please elaborate a bit on what exactly needed to be changed in the JFrog setup for this to work...
When using Databrick's model serving to query Llama3, I noticed the endpoint URL is my databricks instance. Does that still mean data is sent to databricks process by databricks? If so, does databricks keep/use any of the data sent to model serving ...
Related question, when databricks process requests for foundational model, I noticed the latency is pretty small, wondering what kind of processing power is used on databricks side? I am interested in hosting model ourselves so wondering what type of...
I have a Delta table that I’m trying to index but when I try to create a vector search index with either the UI or the Python SDK, it only indexes 1 row out of my 3000 rows. I have tried using different vector search endpoints. I have verified the fo...
I am currently working in a databricks notebook and using an ipywidgets.Output to display a pandas Dataframe. Because spark.DataFrame cannot be displayed in an ipywidgets.Output widget, I have been using:import pandas as pd import numpy as np import ...
I have this issue also, and the listed steps do not resolve it.
