cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

rohithmalla
by New Contributor
  • 540 Views
  • 1 replies
  • 0 kudos

Snowflake Data Formatting Issue

I'm loading snowflake data to delta tables in databricks, few columns in snowflake data have datatype as Number (20,7) after loading to delta table it is taking as decimal (20,7), for example, if the value is 0.0000000 in snowflake then it is showing...

  • 540 Views
  • 1 replies
  • 0 kudos
Latest Reply
-werners-
Esteemed Contributor III
  • 0 kudos

explicit casting seems like the way to go.First try with one column, to see if that solves your issue.If so, you can write a function that casts all decimal columns to a certain precision, something like this:def convert_decimal_precision_scale(df, p...

  • 0 kudos
Mumrel
by Contributor
  • 1237 Views
  • 4 replies
  • 1 kudos

Why is importing python code supported in Repos but not in Workspaces ?

Hi, we currently use a one repo approach which does not require a local development environment (we utilize azure dev ops and nutter for automated tests). We also have shared code accross pipelines and started with %run-sytle modularization and have ...

  • 1237 Views
  • 4 replies
  • 1 kudos
Latest Reply
-werners-
Esteemed Contributor III
  • 1 kudos

the why is most probably because of different development tracks/teams between workspace and repos.If they will consilidate in functionality?  Can't tell, only Databricks knows that; but it seems reasonable to assume the files will also be added to w...

  • 1 kudos
3 More Replies
DeltaTrain
by New Contributor II
  • 831 Views
  • 3 replies
  • 0 kudos

hive_metastore Access Control by different cluster type

Hello Databricks Community,I'm reaching out with a query regarding access control in the hive_metastore. I've encountered behavior that I'd like to understand better and potentially address.To illustrate the situation:I've set up three users for test...

DeltaTrain_0-1691616911858.png DeltaTrain_1-1691617650542.png
  • 831 Views
  • 3 replies
  • 0 kudos
Latest Reply
DeltaTrain
New Contributor II
  • 0 kudos

Hi @Debayan, thank you for your reply.  with hive_metastore, still I cannot get the level of isolation, which means that if anyone activates the Single node cluster, she/he can see all the catalog, schema, and table. However, with Unity catalog appli...

  • 0 kudos
2 More Replies
NC
by New Contributor III
  • 2197 Views
  • 3 replies
  • 0 kudos

GDAL on Databricks Cluster Runtime 12.2 LTS

I need gdal in my course work.After reading this post, I used init script as follows to install gdal into runtime 12.2 LTS  dbutils.fs.put("/databricks/scripts/gdal_install.sh",""" #!/bin/bash sudo add-apt-repository ppa:ubuntugis/ppa sudo apt-get up...

  • 2197 Views
  • 3 replies
  • 0 kudos
Latest Reply
Debayan
Esteemed Contributor III
  • 0 kudos

Hi, I think I understood the issue, for the no module found error sometimes you need to clean the apt list , something like below: %shrm -r /var/cache/apt/archives/* /var/lib/apt/lists/*sudo apt-get cleansudo apt-get updatesudo add-apt-repository -y ...

  • 0 kudos
2 More Replies
Seongmin
by New Contributor
  • 810 Views
  • 2 replies
  • 0 kudos

What's LLM?

What's LLM for generative AI?

  • 810 Views
  • 2 replies
  • 0 kudos
Latest Reply
annianni
New Contributor II
  • 0 kudos

LLMs are large language models like Google Bard or OpenAI or ChatGPT, etc. "It is a type of artificial intelligence (AI) algorithm that uses deep learning techniques and massively large data sets to understand, summarize, generate and predict new con...

  • 0 kudos
1 More Replies
mallesh2088
by New Contributor II
  • 981 Views
  • 1 replies
  • 1 kudos

Compute cluster, SQL warehouse not starting - AWS

Hey,I am using "AWSQuickstartCloudformationLambda" to create a workspace in my trial environment and on the AWS side the stack create succeeded but on the Databricks side compute cluster is not starting. The error says:Aws Authorization Failure:Failu...

  • 981 Views
  • 1 replies
  • 1 kudos
Latest Reply
Debayan
Esteemed Contributor III
  • 1 kudos

Hi, the error suggests there is some error with the credentials. You can check the credentials chain to see where aws is looking for the credentials. https://blog.leapp.cloud/aws-sdk-unable-to-locate-credentials.You can refer to another article: http...

  • 1 kudos
AleksandraFrolo
by New Contributor III
  • 590 Views
  • 1 replies
  • 0 kudos

WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally.

Hiiiii,I want to scratch a text from the web page with simple code:from selenium import webdriver # Set options to make browsing easier #Function is responsible to create a driver and return that driver. def get_driver(): chrome_options = webdr...

  • 590 Views
  • 1 replies
  • 0 kudos
Latest Reply
Debayan
Esteemed Contributor III
  • 0 kudos

Hi, Looks like the issue is with your browser, could you please try to change the browser and confirm?Please tag @Debayan with your next comment, which will get me notified. Thanks!

  • 0 kudos
375721
by New Contributor II
  • 1548 Views
  • 4 replies
  • 0 kudos

webscraping with Firefox driver failing - Python code

Getting below exception while running web scraping script, stuck at installing firefox from snap.   Getting apparmor permission issue and tried to reinstall it and still getting the same. please check the error message section. Dependencies:Python li...

  • 1548 Views
  • 4 replies
  • 0 kudos
Latest Reply
Debayan
Esteemed Contributor III
  • 0 kudos

Hi, This is an issue with the dependencies of different packages. Please try as below: apt get update && apt get upgrade; apt install firefox;Please tag @Debayan with your next comment which will notify me. 

  • 0 kudos
3 More Replies
Phani1
by Valued Contributor
  • 4111 Views
  • 1 replies
  • 0 kudos

Run a job with an Azure service principal

We had a problem in pointing out the Service Principal in the “Run as” option of workflow. Below are the details about the issue we are facing.Issue Details:I had created a Workflow in Azure Databricks workspace to automate the Data movement process....

Phani1_0-1692256727118.png Phani1_1-1692256735431.png
Get Started Discussions
Service Principal
  • 4111 Views
  • 1 replies
  • 0 kudos
Latest Reply
Debayan
Esteemed Contributor III
  • 0 kudos

Hi, Could you please confirm if the service principle is already added to the workspace via admin console? https://docs.databricks.com/en/administration-guide/users-groups/service-principals.html#assign-a-service-principal-to-a-workspace-using-the-ac...

  • 0 kudos
gg_047320_gg_94
by New Contributor II
  • 803 Views
  • 2 replies
  • 0 kudos

Update metastore bucket

Hello,Our Databricks workspace is currently hosted on AWS account1, while the meta store bucket is configured within a separate AWS account2. We want to transfer this bucket to account1 to eliminate our reliance on account2. How to configure this buc...

  • 803 Views
  • 2 replies
  • 0 kudos
Latest Reply
Debayan
Esteemed Contributor III
  • 0 kudos

Hi, You can use: https://github.com/databrickslabs/migrate/blob/master/METASTORE.md# for the metastore data migration. Please tag @Debayan with your next comment which will get me notified. 

  • 0 kudos
1 More Replies
vanessafvg
by New Contributor III
  • 465 Views
  • 0 replies
  • 0 kudos

Using SQL to create delta live tables but column headers from csv have spaces

I know this has probably been asked many times, although I have struggled to find the answer using sql CREATE STREAMING LIVE TABLE bronze_site_usage COMMENT "ingest latest snapshot from api"TBLPROPERTIES ("layer" = "bronze") AS SELECT * FROM cloud_fi...

  • 465 Views
  • 0 replies
  • 0 kudos
jgrycz
by New Contributor III
  • 5907 Views
  • 7 replies
  • 0 kudos

Response for list job runs API request doesn't have next/prev_page_token field

HI,When I do GET request to obtain list of job runs using `/api/2.1/jobs/runs/list` there is no `prev_page_token`, `next_page_token` fields in the response despite of having `has_more: True`. 

Screenshot 2023-08-03 at 09.23.13.png
  • 5907 Views
  • 7 replies
  • 0 kudos
Latest Reply
jgrycz
New Contributor III
  • 0 kudos

@Hubert-Dudek @Kaniz_Fatma could you confirm that behavior is an issue? if yes, can I report it anywhere?Otherwise can I report it as a feature request (to get all runs)? If yes, do you have any service to report it?

  • 0 kudos
6 More Replies
Hubert-Dudek
by Esteemed Contributor III
  • 1126 Views
  • 1 replies
  • 3 kudos

Revolutionizing Data Interaction

The new way of working with Databricks allows users to easily connect their notebook to a SQL serverless warehouse, write code in plain English, and generate output with the press of Space-Ctrl-Shift.

ezgif-5-4da5f77fec.gif
  • 1126 Views
  • 1 replies
  • 3 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 3 kudos

Hi @Hubert-Dudek ,  Thank you for sharing the exciting developments in working with Databricks!  It's great to hear that Databricks continuously enhances the user experience and enables more efficient and intuitive workflows. Your updates are valuabl...

  • 3 kudos
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!

Labels
Top Kudoed Authors