cancel
Turn on suggestions
Showing results for 
Search instead for 
Did you mean: 
Help
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Turn on suggestions
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
khaansaab
by • New Contributor II
  • 960 Views
  • 1 replies
  • 1 kudos

Resolved! read file from local machine(my computer) and create a dataframe

I want to create a notebook and add a widget which will allow the user to select a file from local machine(my computer) and read the contents of the file and create a dataframe. is it possible? and how? in dbutils.widgets i dont have any options for ...

Get Started Discussions
Reply
  • 960 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
szymon_dybczak
Esteemed Contributor III
  • 1 kudos

Hi @khaansaab ,Currently, there is now out of the box mechanism that will allow you to that. As a workaround,you can create a UC volume and tell your users to upload files into that volume. Then you can create notebook that will have file_name parame...

  • 1 kudos
thethirtyfour
by • New Contributor III
  • 4353 Views
  • 2 replies
  • 1 kudos

Resolved! Configure Databricks in VSCode through WSL

Hi,I am having a hard time configuring my Databricks workspace when working in VSCode via WSL. When following the steps to setup Databricks authentication I am receiving the following error on the Step 5 of "Step 4: Set up Databricks authentication"....

Get Started Discussions
Reply
  • 4353 Views
  • 2 replies
  • 1 kudos
View Replies
Latest Reply
RaulMoraM
New Contributor II
  • 1 kudos

What worked for me was NOT opening the browser using the pop-up (which generated the 3-legged-OAuth flow error), but clicking on the link provided by the CLI (or copy paste the link on the browser)

  • 1 kudos
1 More Replies
JohnJustus
by • New Contributor III
  • 10321 Views
  • 3 replies
  • 0 kudos

Space in Column names when writing to Hive

All,I have the following code.df_Warehouse_Utilization = (    spark.table("hive_metastore.dev_ork.bin_item_detail")    .join(df_DIM_Bins,col('bin_tag')==df_DIM_Bins.BinKey,'right')    .groupby(col('BinKey'))    .agg(count_distinct(when(col('serial_lo...

Get Started Discussions
Reply
  • 10321 Views
  • 3 replies
  • 0 kudos
View Replies
Latest Reply
KandyKad
New Contributor III
  • 0 kudos

Hi,I have faced this issue a few times. When we are overwriting the dataframes to hive catalog in databricks, it doesn't naturally allow for column names to have spaces or special characters. However, you can add an option statement to bypass that ru...

  • 0 kudos
2 More Replies
NagarajuT
by • New Contributor
  • 775 Views
  • 1 replies
  • 0 kudos

Connect to SQL Developer using Custom JDBC

Hello,I'm trying to connect databricks SQL to the SQL Developer using custom JDBC.I'm getting errorjdbc:databricks:<server>:443;HttpPath=<HttpPath>;UID=token;PWD=<password> RegardsNaga

Get Started Discussions
Reply
  • 775 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
ShaliniC
New Contributor II
  • 0 kudos

Hi,We are trying to test if we can connect sql developer to databricks. did it work for you?Regards, shalini

  • 0 kudos
tyorisoo
by • New Contributor III
  • 1012 Views
  • 6 replies
  • 0 kudos

Unity Catalog About Metastore

Registered on 2024/10 from AWS marketplace.We have created a customer management VPC and manually created the workspace.No specific metastore settings were made when the workspace was created.In the catalog screen of the account console,unity catalog...

Get Started Discussions
Reply
  • 1012 Views
  • 6 replies
  • 0 kudos
View Replies
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Hello @tyorisoo,I hope you are doing well!Metastore manages metadata, not catalog information, schema information, table information, function information, access control information, etc. In the current state, the metastore configuration is not done...

  • 0 kudos
5 More Replies
Surajv
by • New Contributor III
  • 1764 Views
  • 2 replies
  • 1 kudos

What is the quota limit for using create user token api?

Hi Community, I was going through this doc: https://docs.databricks.com/api/workspace/tokens/create to and got to know, that there is a quota limit to how many token one can generate using the api: POST /api/2.0/token/create, having breached the thre...

Get Started Discussions
Reply
  • 1764 Views
  • 2 replies
  • 1 kudos
View Replies
Latest Reply
Alberto_Umana
Databricks Employee
  • 1 kudos

Hello@Surajv, Q1: What is and how to find out the quota limit? The quota limit for creating user tokens via the API (POST /api/2.0/token/create) is essential to manage token usage. Each user can have multiple personal access tokens in a Databricks wo...

  • 1 kudos
1 More Replies
benito
by • New Contributor
  • 586 Views
  • 1 replies
  • 0 kudos

Databricks Initial Costs AWS

I have a new premium account. I set up a cost dashboard (see attached) after I create a new workspace using AWS Quickstart, where I see some costs. Why do I have this If I am not using Databricks at all? How can I save the costs?

databricks.png
Get Started Discussions
Reply
  • 586 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
Walter_C
Databricks Employee
  • 0 kudos

Are you seeing this data from the Usage tab in the Account console? Does it allow you to filter it by SKU?

  • 0 kudos
Phani1
by • Valued Contributor II
  • 11056 Views
  • 4 replies
  • 2 kudos

Masking techniques for more PII columns

Hi Databricks Team,We would appreciate it if you could inform us about the situations when Column-Masking, Row-Level Filtering, and Attributed-Based Masking should be utilized, as well as the recommended technique for handling large data volumes cont...

Get Started Discussions
masking
Reply
  • 11056 Views
  • 4 replies
  • 2 kudos
View Replies
Latest Reply
NandiniN
Databricks Employee
  • 2 kudos

Agree with @Meghla-C , https://databricks.aha.io/ideas/ideas/DB-I-7941 this was the feature request and if you see the status, it is in preview.

  • 2 kudos
3 More Replies
NC
by • New Contributor III
  • 3734 Views
  • 4 replies
  • 0 kudos

GDAL on Databricks Cluster Runtime 12.2 LTS

I need gdal in my course work.After reading this post, I used init script as follows to install gdal into runtime 12.2 LTS  dbutils.fs.put("/databricks/scripts/gdal_install.sh",""" #!/bin/bash sudo add-apt-repository ppa:ubuntugis/ppa sudo apt-get up...

Get Started Discussions
Reply
  • 3734 Views
  • 4 replies
  • 0 kudos
View Replies
Latest Reply
Matt_C
New Contributor II
  • 0 kudos

Hi, in case anyone is still struggling here. I found I could not get the init script approach to work, but if I just run a shell command to install gdal at the start of my notebook it works fine. You might note, however, that this installs gdal versi...

  • 0 kudos
3 More Replies
juliemoore
by • New Contributor
  • 917 Views
  • 1 replies
  • 0 kudos

Problems with Big Data Solutions and Databricks- Any advice?

Hello everyone,I am currently facing several challenges related to big data solutions, particularly with the Databricks. As many of you may know, Databricks is a powerful platform for data engineering and analytics, but I have encountered some signif...

Get Started Discussions
Reply
  • 917 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
gchandra
Databricks Employee
  • 0 kudos

Your problem statement is too generic. If your company is facing this, you can reach out to your SA; they will help you. If it's a personal project, then mention what you are trying in detail with cluster size, what you are trying to integrate with, ...

  • 0 kudos
JolM
by • New Contributor II
  • 889 Views
  • 1 replies
  • 1 kudos

Resolved! is there a way for us to see billing usage per catalog?

is there a way for us to see billing usage per catalog? I'm using 14days trial period for now.. would it be available in Premium?

Get Started Discussions
Reply
  • 889 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
gchandra
Databricks Employee
  • 1 kudos

Billing system table provides cost by notebook, jobs, and clusters. If catalog-to-job/cluster/notebook relation is maintained, then catalog-based usage can be determined. https://docs.databricks.com/en/admin/system-tables/billing.html

  • 1 kudos
hetrasol
by • New Contributor III
  • 3216 Views
  • 7 replies
  • 0 kudos

Resolved! Unable to start browser for databricks certification

Hello, I  have registered for databricks certified data engineering associate exam. One of the requirements to give the exam is The exam is set for Sunday 6th October, 2024 but the browser installation (psi secure bridge browser) does not work. .Reac...

hetrasol_0-1728079492073.png
Get Started Discussions
Reply
  • 3216 Views
  • 7 replies
  • 0 kudos
View Replies
Latest Reply
TaiNguyen
New Contributor II
  • 0 kudos

Hi @hetrasol ,I'm a Windows user. After installation, I just got the Lockdown Browser OEM instead of the PSI browser, as you mentioned above. Can you help to instruct again on how to install these browsers

  • 0 kudos
6 More Replies
ff-paulo-barbos
by • New Contributor
  • 2174 Views
  • 2 replies
  • 0 kudos

Spark Remote error when connecting to cluster

Hi, I am using the latest version of pyspark and I am trying to connect to a remote cluster with runtime 13.3.My doubts are:- Do i need databricks unity catalog enabled?- My cluster is already in a Shared policy in Access Mode, so what other configur...

Get Started Discussions
Reply
  • 2174 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
Debayan
Databricks Employee
  • 0 kudos

Hi, Is your workspace is already unity catalog enabled? Also, did you go through the considerations for enabling workspace for unity catalog? https://docs.databricks.com/en/data-governance/unity-catalog/enable-workspaces.html#considerations-before-yo...

  • 0 kudos
1 More Replies

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Labels
Top Kudoed Authors
View all
li.common.scroll-to.top