cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Fkebbati
by New Contributor II
  • 904 Views
  • 1 replies
  • 0 kudos

Ingress/Egress private endpoint

Hello , We have configured our Databricks environment with private endpoint connections injected into our VNET, which includes two subnets (public and private). We have disabled public IPs and are using Network Security Groups (NSGs) on the subnet, a...

  • 904 Views
  • 1 replies
  • 0 kudos
Latest Reply
szymon_dybczak
Esteemed Contributor III
  • 0 kudos

Hi @Fkebbati ,There always be some costs related to data transfer between those account. Let's have a look at private link pricing page. So it's expected, but MS likes to hide this kind of information  

  • 0 kudos
Phani1
by Valued Contributor II
  • 13188 Views
  • 4 replies
  • 2 kudos

Masking techniques for more PII columns

Hi Databricks Team,We would appreciate it if you could inform us about the situations when Column-Masking, Row-Level Filtering, and Attributed-Based Masking should be utilized, as well as the recommended technique for handling large data volumes cont...

  • 13188 Views
  • 4 replies
  • 2 kudos
Latest Reply
NandiniN
Databricks Employee
  • 2 kudos

Agree with @Meghla-C , https://databricks.aha.io/ideas/ideas/DB-I-7941 this was the feature request and if you see the status, it is in preview.

  • 2 kudos
3 More Replies
ladyleet
by New Contributor
  • 696 Views
  • 1 replies
  • 1 kudos

Best way to find Databricks Certified Professionals?

Hi all! We have a few Databricks certified folks on our team, but we are looking for more! What is the best way to find a list of or know who is certified? We are looking for North America / Europe / LATAM / South America. I am part of the Microsoft ...

  • 696 Views
  • 1 replies
  • 1 kudos
Latest Reply
Rafael-Sousa
Contributor II
  • 1 kudos

Hey @ladyleet ,There is the community of Databricks certified: Databricks Certified Credential Holder Directory.

  • 1 kudos
Amrit23
by New Contributor
  • 1154 Views
  • 1 replies
  • 0 kudos

Amazon MSK integration with Databricks

Hello Everyone,I am a beginner in the world of Data bricks. I am trying to achieve a use case which involves : consuming messages from Amazon MSK and creating a Delta table in Data bricks.I need to get some insights on what all accesses are supposed ...

  • 1154 Views
  • 1 replies
  • 0 kudos
Latest Reply
NandiniN
Databricks Employee
  • 0 kudos

Hey @Amrit23 , Databricks Assistant can help you write the code. You need to use spark.readStream() to access the stream.  https://docs.databricks.com/en/connect/streaming/kafka.html And for the Access Permissions: Amazon MSK: Ensure you have the nec...

  • 0 kudos
SachinJanani
by New Contributor II
  • 1405 Views
  • 1 replies
  • 0 kudos

Resolved! Databricks SQL as alternative to Spark thrift server

We are currently using Spark as our SQL engine with Thrift Server but are evaluating Databricks Serverless SQL as a potential alternative. We have a few specific questions:Does Databricks Serverless SQL support custom Spark extensions?Can we configur...

Get Started Discussions
Databricks SQL
databrickssql
  • 1405 Views
  • 1 replies
  • 0 kudos
Latest Reply
NandiniN
Databricks Employee
  • 0 kudos

Hi @SachinJanani , 1 - Databricks Serverless SQL does not support custom Spark extensions (Advanced Spark Configs/Libraries etc). This is because the serverless environment is designed to be highly optimized and managed by Databricks, which limits th...

  • 0 kudos
NC
by New Contributor III
  • 4292 Views
  • 4 replies
  • 0 kudos

GDAL on Databricks Cluster Runtime 12.2 LTS

I need gdal in my course work.After reading this post, I used init script as follows to install gdal into runtime 12.2 LTS  dbutils.fs.put("/databricks/scripts/gdal_install.sh",""" #!/bin/bash sudo add-apt-repository ppa:ubuntugis/ppa sudo apt-get up...

  • 4292 Views
  • 4 replies
  • 0 kudos
Latest Reply
Matt_C
New Contributor II
  • 0 kudos

Hi, in case anyone is still struggling here. I found I could not get the init script approach to work, but if I just run a shell command to install gdal at the start of my notebook it works fine. You might note, however, that this installs gdal versi...

  • 0 kudos
3 More Replies
juliemoore
by New Contributor
  • 1592 Views
  • 1 replies
  • 0 kudos

Problems with Big Data Solutions and Databricks- Any advice?

Hello everyone,I am currently facing several challenges related to big data solutions, particularly with the Databricks. As many of you may know, Databricks is a powerful platform for data engineering and analytics, but I have encountered some signif...

  • 1592 Views
  • 1 replies
  • 0 kudos
Latest Reply
gchandra
Databricks Employee
  • 0 kudos

Your problem statement is too generic. If your company is facing this, you can reach out to your SA; they will help you. If it's a personal project, then mention what you are trying in detail with cluster size, what you are trying to integrate with, ...

  • 0 kudos
bvraravind
by New Contributor II
  • 1822 Views
  • 1 replies
  • 0 kudos

Resolved! Unable to access Azure blob storage with SAS token

I am following Microsoft documentation to connect from Databricks workspace to Azure blob storage. but it is not working. Any help is greatly appreciated. Below is the codespark.conf.set("fs.azure.account.auth.type.<storage-account>.dfs.core.windows....

  • 1822 Views
  • 1 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Hi @bvraravind, The error you are encountering is due to an incorrect configuration setting in your code. The error message indicates that the configuration fs.azure.account.auth.type.<storage-account>.dfs.core.windows.net is not recognized Verify th...

  • 0 kudos
Tejsharma
by New Contributor
  • 3372 Views
  • 1 replies
  • 0 kudos

Troubleshooting the Error "Credential was not sent or was of an unsupported type for this API"

I previously worked on Databricks Asset Bundle (DAB) using a Service Principal token, and it was successful. However, when I attempted it again now, I encountered an error.Error: failed to compute file content for {{.project_name}}/databricks.yml.tmp...

  • 3372 Views
  • 1 replies
  • 0 kudos
Latest Reply
Walter_C
Databricks Employee
  • 0 kudos

Which type of token are you currently using, is it an Oauth token or Obo token? Have you generated a new token for testing?

  • 0 kudos
MC97
by New Contributor
  • 911 Views
  • 1 replies
  • 0 kudos

Update on CTE

So I am reflecting a business logic from on prem to azure databricks . what on prem did is created the table and after that updated . I have to construct that as a single query . Example Create or replace table table1with CTE 1 as () ,CTE 2 as (selec...

  • 911 Views
  • 1 replies
  • 0 kudos
Latest Reply
VZLA
Databricks Employee
  • 0 kudos

An actual "Update", it may not be possible, but have you consider and will something like this work for you? This is simulating updates within the query without actual UPDATE statements: CREATE OR REPLACE TABLE table1 AS WITH CTE1 AS ( -- Your in...

  • 0 kudos
dbx_687_3__1b3Q
by New Contributor III
  • 5545 Views
  • 5 replies
  • 5 kudos

Impersonating a user

How do I impersonate a user? I can't find any documentation that explains how to do this or even hint that it's possible.Use case: I perform administrative tasks like assign grants and roles to catalogs, schemas, and tables for the benefit of busines...

  • 5545 Views
  • 5 replies
  • 5 kudos
Latest Reply
NandiniN
Databricks Employee
  • 5 kudos

DB-I-8117 this one is mentioned to be considered for future so adding votes for sure will help.

  • 5 kudos
4 More Replies
JolM
by New Contributor II
  • 1168 Views
  • 1 replies
  • 1 kudos

Resolved! is there a way for us to see billing usage per catalog?

is there a way for us to see billing usage per catalog? I'm using 14days trial period for now.. would it be available in Premium?

  • 1168 Views
  • 1 replies
  • 1 kudos
Latest Reply
gchandra
Databricks Employee
  • 1 kudos

Billing system table provides cost by notebook, jobs, and clusters. If catalog-to-job/cluster/notebook relation is maintained, then catalog-based usage can be determined. https://docs.databricks.com/en/admin/system-tables/billing.html

  • 1 kudos
abueno
by Contributor
  • 8485 Views
  • 3 replies
  • 8 kudos

Resolved! Find and replace

Hi,Is there a "Find and replace" option to edit SQL code?  I am not referring to the "replace" function but something similar to Control  + shift + F in Snowflake or Control + F in MS Excel.

  • 8485 Views
  • 3 replies
  • 8 kudos
Latest Reply
DBKENGR
New Contributor III
  • 8 kudos

is there an option to find-replace just within a cell instead of entire notebook?

  • 8 kudos
2 More Replies
hetrasol
by New Contributor III
  • 4145 Views
  • 7 replies
  • 0 kudos

Resolved! Unable to start browser for databricks certification

Hello, I  have registered for databricks certified data engineering associate exam. One of the requirements to give the exam is The exam is set for Sunday 6th October, 2024 but the browser installation (psi secure bridge browser) does not work. .Reac...

hetrasol_0-1728079492073.png
  • 4145 Views
  • 7 replies
  • 0 kudos
Latest Reply
TaiNguyen
New Contributor II
  • 0 kudos

Hi @hetrasol ,I'm a Windows user. After installation, I just got the Lockdown Browser OEM instead of the PSI browser, as you mentioned above. Can you help to instruct again on how to install these browsers

  • 0 kudos
6 More Replies

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Labels