cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Phani1
by Valued Contributor II
  • 3912 Views
  • 5 replies
  • 1 kudos

Azure Synapse vs Databricks

 Hi team,Could you kindly provide your perspective on the cost and performance comparison between Azure Synapse and Databricks SQL Warehouse/serverless, as well as their respective use cases? Thank you.

  • 3912 Views
  • 5 replies
  • 1 kudos
Latest Reply
Witold
Honored Contributor
  • 1 kudos

@Suncat There hasn't been any major changes for than a year: https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/apache-spark-version-support E.g. I don't believe we will see support for Spark 3.5 at all. At least, apparently it's support...

  • 1 kudos
4 More Replies
Phani1
by Valued Contributor II
  • 717 Views
  • 2 replies
  • 0 kudos

Solace to Azure Data Lake Storage

 Hi Team,What is the most effective method for performing data ingestion from Solace to Azure Data Lake Storage (ADLS) utilizing an Azure Databricks notebook? Any recommendations would be greatly appreciated.Regards,Phani 

  • 717 Views
  • 2 replies
  • 0 kudos
Latest Reply
SravanThotakura
New Contributor II
  • 0 kudos

Here is the sample script to invoke the connectorval struct_stream = spark.readStream.format("solace").option("host", "").option("vpn", "").option("username", "").option("password", "").option("queue", "").option("connectRetries", 3).option("reconnec...

  • 0 kudos
1 More Replies
EricMa
by New Contributor III
  • 6070 Views
  • 20 replies
  • 4 kudos

Mounting Data IOException

Hello,I am currently taking a course from Coursera for data science using SQL. For one of our assignments we need to mount some data by running a script that has been provided to us by the class. When I run the script I receive the following error. I...

IOException.jpg IOException_Classroom-Setup.jpg
  • 6070 Views
  • 20 replies
  • 4 kudos
Latest Reply
raghdafaris
New Contributor II
  • 4 kudos

Hello all, we came up with a solution: to download the data directly instead of mounting it. The community version is limited, and we don't have access to S3 unless we create our own aws account, load the data there, and then mount our account on dat...

  • 4 kudos
19 More Replies
shkelzeen
by New Contributor II
  • 1826 Views
  • 3 replies
  • 1 kudos

Databricks JDBC driver multi query in one request.

Can I run multi query in one command using databricks JDBC driver and would databricks execute one query faster then running multi queries in one script?  

  • 1826 Views
  • 3 replies
  • 1 kudos
Latest Reply
NandiniN
Databricks Employee
  • 1 kudos

Yes, you can run multiple queries in one command using the Databricks JDBC driver.The results will be displayed in separate tables. When you run the multiple queries, they are all still individual queries. Running multiple queries in a script will no...

  • 1 kudos
2 More Replies
AGnewbie
by New Contributor
  • 195 Views
  • 1 replies
  • 1 kudos

Required versus current compute setup

To run demo and lab notebooks, I am required to have the following Databricks runtime(s): 15.4.x-cpu-ml-scala2.12 but the compute in my setup is of the following runtime version, will that be an issue? 11.3 LTS (includes Apache Spark 3.3.0, Scala 2.1...

  • 195 Views
  • 1 replies
  • 1 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 1 kudos

Hello @AGnewbie, Firstly, regarding the Databricks runtime: your compute setup is currently running version 11.3 LTS, which will indeed be an issue as the specified version is not present in your current runtime. Hence, you need to update your runtim...

  • 1 kudos
GerardAlexander
by New Contributor
  • 390 Views
  • 1 replies
  • 0 kudos

Creating Unity Catalog in Personal AZURE Portal Account

Seeking advice on the following:1. Given that I have a Personal - and not an Organization-based - AZURE Portal Account,    2. that I can see I am Global Admin and have Admin Role in Databricks,         3. then why can I not get "Manage Account" for a...

  • 390 Views
  • 1 replies
  • 0 kudos
Latest Reply
Takuya-Omi
Valued Contributor II
  • 0 kudos

@GerardAlexander Try signing in to the Account Console (https://accounts.azuredatabricks.net/login) using a user account with the appropriate permissions, rather than accessing it from the workspace.If you are unable to sign in, the following resourc...

  • 0 kudos
spd_dat
by New Contributor II
  • 552 Views
  • 0 replies
  • 0 kudos

Can AWS workspaces share subnets?

The docs state:"You can choose to share one subnet across multiple workspaces or both subnets across workspaces."as well as:"You can reuse existing security groups rather than create new ones."and on this page:"If you plan to share a VPC and subnets ...

  • 552 Views
  • 0 replies
  • 0 kudos
FanMichelle0729
by New Contributor II
  • 441 Views
  • 5 replies
  • 0 kudos

Serveless compute does need has cloud accout(AWS、Google 、Azure)

I am a Databricks beginner, and I would like to ask if the Compute created in the Databricks account , it means also exists in the cloud account (e.g., AWS)? If the AWS account is deactivated, the existing compute will not be usable. This is what I h...

  • 441 Views
  • 5 replies
  • 0 kudos
Latest Reply
Takuya-Omi
Valued Contributor II
  • 0 kudos

@FanMichelleTW No, Databricks recommends using serverless compute, and you can use serverless compute as well.To do so, open a notebook and check the top-right corner to see if a serverless compute option is in a Ready state. If it is, simply select ...

  • 0 kudos
4 More Replies
nathanmle
by New Contributor
  • 198 Views
  • 1 replies
  • 0 kudos

Where to find Jupyter Notebook course materials for Get Started with Databricks for Generative AI

Hello, I can't seem to find any way to gain access to the Jupyter Notebook demo source of "Get Started with Databricks for Generative AI" course.  Please help.  Thank you kindly in advance. 

  • 198 Views
  • 1 replies
  • 0 kudos
Latest Reply
Advika_
Databricks Employee
  • 0 kudos

Hello @nathanmle! We are sorry to inform you that we are no longer offering notebooks or the DBC files for the self-paced courses due to recent changes.If you’re interested in working on labs in a provided Databricks environment, you can purchase the...

  • 0 kudos
kiko_roy
by Contributor
  • 1179 Views
  • 3 replies
  • 0 kudos

Getting access token error when connecting from azure databricks to GCS bucket

I am creating a data frame by reading a table's data residing in Azure backed unity catalog. I need to write the dataframe or file to GCS bucket. I have configured the spark cluster config using the GCP service account json values. Also tried uploadi...

  • 1179 Views
  • 3 replies
  • 0 kudos
Latest Reply
KristiLogos
Contributor
  • 0 kudos

@kiko_roy  unfortunately that didnt' work. the error is stating its trying to get the access token from metadata server, I wonder why from the metadata server?

  • 0 kudos
2 More Replies
lukaseder
by New Contributor II
  • 301 Views
  • 2 replies
  • 0 kudos

How to fetch nested data structures in Databricks using JDBC

I've asked the question also here on stack overflow When using nested data structures in Databricks (e.g. `ARRAY` or `ROW`) using JDBC, it appears that the results can be fetched as JSON `String` values, e.g.: try (Statement s = connection.createStat...

  • 301 Views
  • 2 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Hi @lukaseder, This looks to be a bug, but will get more details about it internally. The driver is relatively new. Have you tried with another version, for instance JDBC: 2.6.40

  • 0 kudos
1 More Replies
AChang
by New Contributor III
  • 5478 Views
  • 2 replies
  • 2 kudos

Model Serving Endpoint keeps failing with SIGKILL error

I am trying to deploy a model in the serving endpoints section, but it keeps failing after attempting to create for an hour. Here are the service logs:Container failed with: 9 +0000] [115] [INFO] Booting worker with pid: 115[2023-09-15 19:15:35 +0000...

Get Started Discussions
model registry
Model serving
serving endpoint
serving endpoints
  • 5478 Views
  • 2 replies
  • 2 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 2 kudos

Hello @AChang, This is a common issue when the memory requirements of your model exceed the available memory on your current compute resources.   Moving to a larger compute instance with more memory can help accommodate the memory requirements of yo...

  • 2 kudos
1 More Replies
enhancederroruk
by New Contributor II
  • 3027 Views
  • 3 replies
  • 2 kudos

Chrome/Edge high memory usage for Databricks tabs.

Is it normal for Databricks tabs to be using such high memory?The Chrome example I just got a screenshot of was this (rounded up/down)...3 x Databricks tabs for one user, sized at6gb, 4.5gb, and 2gbTotal = 12.5gbI know it gets higher than this too, I...

  • 3027 Views
  • 3 replies
  • 2 kudos
Latest Reply
OliverD
New Contributor II
  • 2 kudos

Boosting. I have the same issue and it's getting out of hand.

  • 2 kudos
2 More Replies
Phani1
by Valued Contributor II
  • 460 Views
  • 1 replies
  • 0 kudos

Databricks On-Premises or in Private Cloud

Hi All,Is it possible to store/process the data on-premises or in a private cloud with Databricks? Will this choice affect costs and performance? Please advise, as the customer wants the data stored on-premises or in a private cloud for security reas...

  • 460 Views
  • 1 replies
  • 0 kudos
Latest Reply
Takuya-Omi
Valued Contributor II
  • 0 kudos

@Phani1 Databricks does not provide a product that can be directly installed and self-managed on on-premises or private cloud environments. Instead, Databricks primarily operates as a managed service on public cloud platforms such as AWS, Azure, and ...

  • 0 kudos

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Labels
Top Kudoed Authors