- 2427 Views
- 3 replies
- 0 kudos
Adding NFS storage as external volume (Unity)
Can anyone share experience (or point me to another reference) that describes how to configure Azure Blob storage which has NFS enabled as an external volume to Databricks ?I've succeeded in adding SMB storage to Databricks but (if I understand prope...
- 2427 Views
- 3 replies
- 0 kudos
- 0 kudos
hi @phguk could you share how you managed to create an external volume referencing to an azure fileshare ?are you using Unity catalog for this ? it was my understanding this is not possible.
- 0 kudos
- 502 Views
- 0 replies
- 1 kudos
Denodo with Databricks
Hi Team,How can I make a SQL Endpoint web api in Databricks that can be used in denodo?Regards,Janga
- 502 Views
- 0 replies
- 1 kudos
- 489 Views
- 0 replies
- 0 kudos
Hadoop Hive migration to Databricks
Hi,Can you let me know what are the challenges and how to mitigate while migrating Hive objects to Databricks. I could not get any information on this. Can you please provide.
- 489 Views
- 0 replies
- 0 kudos
- 796 Views
- 1 replies
- 1 kudos
Resolved! How to pass a dynamic query to source server from databricks
I have this usecase wherein i am supposed to pass a dynamic query to get data from source I have tried the query option but its giving error SparkConnectGrpcException: (com.microsoft.sqlserver.jdbc.SQLServerException) Incorrect syntax near the keywor...
- 796 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi thanks for your reply,I have used foreign catalouge to fetch required data from information schema then i am creating the dynamic query in databricks and then passing in query this is working for me! @Retired_mod
- 1 kudos
- 1135 Views
- 5 replies
- 0 kudos
S60 Eliminate SPN secrets - Connect Azure Databricks to ADLS Gen2 , Gen1 via custom AD token
Hi Team,In Azure Databricks, we currently use Service Principal when creating Mount Points to Azure storage ( ADLS Gen1, ADLS Gen 2 and Azure Blob Storage).As part of S360 action to eliminate SPN secrets, we were asked to move to SPN+certificate / MS...
- 1135 Views
- 5 replies
- 0 kudos
- 0 kudos
@ramesitexp Yes @szymon_dybczak is correct for now only valid option is below : OAuth 2.0 with a Microsoft Entra ID service principalShared access signatures (SAS)Account keys For now we are using OAuth 2.0 with a Microsoft Entra ID service principal...
- 0 kudos
- 346 Views
- 1 replies
- 0 kudos
Free trail account
Hi Can I able to create unity catalog using free trail account?
- 346 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Spyro_3 ,Yeah, you should be able to create unity catalog. Trial version allows you to create premium workspace which is required for unity catalog. Notice, that to setup metastore you also need to have Global Administrator permission (if we are ...
- 0 kudos
- 1517 Views
- 7 replies
- 3 kudos
Writing a single huge dataframe into Azure SQL Database using JDBC
Hi All,I am currently trying to read data from a materialized view as a single dataframe which contains around 10M of rows and then write it into an Azure SQL database. However, I don't see the spark job moving a bit even an hour is passed. I have al...
- 1517 Views
- 7 replies
- 3 kudos
- 3 kudos
Hi @yeungcase , Thank you for reaching out to our community! We're here to help you. To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your feedba...
- 3 kudos
- 965 Views
- 3 replies
- 2 kudos
Resolved! When does the cost of JOB COMPUTE start to be calculated?
I'm trying to run a workflow with job compute.Job compute needs to be pending for about 5 to 7 minutes before executing the workflow. I think it takes time to find a suitable instance in the cloud, configure the environment, install libraries, etc.An...
- 965 Views
- 3 replies
- 2 kudos
- 2 kudos
There is a big discrepancy to take note of here. It is the difference between Databricks job costs (DBUs) and billing for the cloud provider (the actual VM doing the the work). Billing for the Databricks job starts when the Spark context is being ini...
- 2 kudos
- 759 Views
- 1 replies
- 1 kudos
Error with: %run ../Includes/Classroom-Setup-SQL
Hi Guys,Just started the ASP 2.1L Spare SQL Lab and I get this error, when I run the first setup SQL command:%run ../Includes/Classroom-Setup-SQLThe execution of this command did not finish successfullyPython interpreter will be restarted.Python inte...
- 759 Views
- 1 replies
- 1 kudos
- 1 kudos
You can create a cluster compatible with the notebooks. So It will work.
- 1 kudos
- 1665 Views
- 2 replies
- 0 kudos
Azure Synapse vs Databricks
Hi team,Could you kindly provide your perspective on the cost and performance comparison between Azure Synapse and Databricks SQL Warehouse/serverless, as well as their respective use cases? Thank you.
- 1665 Views
- 2 replies
- 0 kudos
- 0 kudos
Agree with @mhiltner, it doesn't make sense to compare it with Synapse, as it's literally dead. You most likely want to compare it to Fabric instead. Fabric is highly under development, but IMHO it still lacks behind other Data/AI solutions. No catal...
- 0 kudos
- 33560 Views
- 3 replies
- 3 kudos
Resolved! How to import excel on databricks
To import an Excel file into Databricks, you can follow these general steps:1. **Upload the Excel File**:- Go to the Databricks workspace or cluster where you want to work.- Navigate to the location where you want to upload the Excel file.- Click on ...
- 33560 Views
- 3 replies
- 3 kudos
- 3 kudos
The question here is how to read the multi-excel files based on path.The mentioned solution interacts with one file only, do we have the ability to read all the Excel files in the folder?
- 3 kudos
- 802 Views
- 3 replies
- 2 kudos
Resolved! Try Databricks sign up failed
Hi, I am trying to use Databricks with the community edition. However, when I tried to create an account, the sign-up failed after I completed the puzzle.
- 802 Views
- 3 replies
- 2 kudos
- 4948 Views
- 6 replies
- 0 kudos
Streaming xls files Using Auto Loader
Hello,Is there a way to read .xls files using auto loader or is there any workaround since excel files are not supported by the auto loader per the following document?https://docs.databricks.com/en/ingestion/auto-loader/options.htmlThanks.
- 4948 Views
- 6 replies
- 0 kudos
- 0 kudos
I am facing the same issue--I have a stream that I'd like to use autoloader on with an .xlsx. Is there any update to any workarounds on this issue?
- 0 kudos
- 828 Views
- 0 replies
- 0 kudos
File Not Found Error while reading pickle file
Hello, thereI have a pickle file uploaded in a mounted location in databricks ( /dbfs/mnt/blob/test.pkl). I am trying to read this pickle file using the below python snippetwith open(path + "test.pkl", "rb") as f: bands = pickle.load(f)But it t...
- 828 Views
- 0 replies
- 0 kudos
- 4555 Views
- 2 replies
- 0 kudos
Resolved! Using private package, getting ERROR: No matching distribution found for myprivatepackage
My project's setup.py filefrom setuptools import find_packages, setup PACKAGE_REQUIREMENTS = ["pyyaml","confluent-kafka", "fastavro", "python-dotenv","boto3", "pyxlsb", "aiohttp", "myprivatepackage"] LOCAL_REQUIREMENTS = ["delta-spark", "scikit-lea...
- 4555 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi, Does this look like a dependency error? All the dependencies are packed in the whl? Also, could you please confirm if all the limitations are satified? Refer: https://docs.databricks.com/en/compute/access-mode-limitations.html
- 0 kudos
