cancel
Showing results for 
Search instead for 
Did you mean: 
Community Platform Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

AcrobaticMonkey
by New Contributor II
  • 437 Views
  • 1 replies
  • 1 kudos

Instance profiles are not working in Shared access mode

I’m trying to fetch billing data from an AWS account using boto3 to assume a role that has access to this information. This operation works fine in No Isolation and Single User access modes, but it fails in Shared access mode. Since I need to store t...

  • 437 Views
  • 1 replies
  • 1 kudos
Latest Reply
gchandra
Databricks Employee
  • 1 kudos

Instance Profiles are open for all, they won't work in Shared access mode. Using Storage credentials, create  External Locations. Cleaner way to Govern the access.

  • 1 kudos
RahulChaubey
by New Contributor III
  • 2916 Views
  • 4 replies
  • 1 kudos

Resolved! Can we get notebook owner using notebook path as parameter in api ?

I need to get the notebook owner using api or some other way by passing notebook path as parameter.

  • 2916 Views
  • 4 replies
  • 1 kudos
Latest Reply
nkumar18
New Contributor II
  • 1 kudos

Hi @RahulChaubey Did you find any solution to your problem?

  • 1 kudos
3 More Replies
Kinger
by New Contributor
  • 2246 Views
  • 2 replies
  • 0 kudos

Associating a Git Credential with a Service Principal using Terraform Provider (AWS)

I am attempting to create a Databrick Repo in a workspace via Terraform. I would like the Repo and the associated Git Credential to be associated with a Service Principal. In my initial run, the Terraform provider is associated with the user defined ...

  • 2246 Views
  • 2 replies
  • 0 kudos
Latest Reply
nicole_lu_PM
Databricks Employee
  • 0 kudos

Hi Kinger and Debi-Moha, Do the steps in the "Use a service principal with Databricks Git folders" documentation work for you?  Specifically for Terraform: https://docs.databricks.com/en/repos/ci-cd-techniques-with-repos.html#terraform-integration Th...

  • 0 kudos
1 More Replies
datastones
by Contributor
  • 1472 Views
  • 6 replies
  • 3 kudos

Resolved! Data loss after writing a transformed pyspark dataframe to delta table in unity catalog

Hey guys, after some successful data preprocessing without any errors, i have a final dataframe shape with the shape of ~ (200M, 150). the cluster i am using has sufficient ram + cpus + autoscaling, all metrics look fine after the job was done.The pr...

Community Platform Discussions
Data Processing
help
  • 1472 Views
  • 6 replies
  • 3 kudos
Latest Reply
datastones
Contributor
  • 3 kudos

@szymon_dybczak i could resolve it now! basically, i broke the process down into further subprocesses, for each sub process, i cached and wrote them all into delta table (without overwritting), the next subprocess needs to read data in the delta tabl...

  • 3 kudos
5 More Replies
DavidKxx
by Contributor
  • 2135 Views
  • 3 replies
  • 3 kudos

Display data as multi-line in dashboard table

I am displaying a table in a notebook dashboard.  One column of the data is conceptually a list of strings.  I can originate or convert the list as whatever format would be useful (as a string representing a JSON array, as an ARRAY struct, etc.). I w...

solution1.png solution2.png
  • 2135 Views
  • 3 replies
  • 3 kudos
Latest Reply
filipniziol
Contributor III
  • 3 kudos

Hi @DavidKxx ,What you can do is convert your array to into an HTML formatted string with bullet points.Here is the code: # Sample data with an array column data = [ (1, ['Apple', 'Banana', 'Cherry']), (2, ['Dug', 'Elephant']), (3, ['Fish...

  • 3 kudos
2 More Replies
DavidKxx
by Contributor
  • 572 Views
  • 1 replies
  • 0 kudos

Word wrap in dashboards

When I'm displaying a Table-style visualization in a notebook dashboard, is there a setting I can apply to a text column so that it automatically word-wraps text longer than the display width of the column?For example, in the following dashboard disp...

solution3.png
  • 572 Views
  • 1 replies
  • 0 kudos
Latest Reply
filipniziol
Contributor III
  • 0 kudos

Hi @DavidKxx ,That is quite similar question to one about displaying array as bullet list. Since you were successful in implementing displayHTML, what do you think about doing similar in this case? # Sample DataFrame with long text data = [ (1, '...

  • 0 kudos
DBXUser7
by New Contributor
  • 318 Views
  • 0 replies
  • 0 kudos

Databricks integration with Canada Post

hello everyone,I want to validate whether the postal code in my data is a valid postal code as per Canada postal code directory. Assuming, we have subscription to Canada Post APIs, how can we bring the postal code data into Databricks?Thanks

  • 318 Views
  • 0 replies
  • 0 kudos
Ahasan_Ulla
by New Contributor
  • 289 Views
  • 0 replies
  • 0 kudos

setting defaultValue for runtime_engine policy

Hi,I want to set runtime_engine default value to STANDARD but a user can also select PHOTON if they want. Something like below. Can anyone please verify if that works?"runtime_engine": {"type": "allowlist","values": ["STANDARD","PHOTON"],"defaultValu...

  • 289 Views
  • 0 replies
  • 0 kudos
valentino
by New Contributor II
  • 742 Views
  • 3 replies
  • 0 kudos

Lost untracked and gitignore files when push new commit

Hi guys, our team uses Databricks clusters to develop with Python using Jupyter Notebooks.Recently we had a serious problem in our repos folder ( mainly Jupyter Notebook and Python scripts ).After my teammate committed in the branch, all the tracked ...

  • 742 Views
  • 3 replies
  • 0 kudos
Latest Reply
valentino
New Contributor II
  • 0 kudos

Yes we have a subscription. Please tell me which information you need to check. I want to dive in this problem. If possible, pls contact through my email.

  • 0 kudos
2 More Replies
Patrick-Data
by New Contributor II
  • 16299 Views
  • 4 replies
  • 0 kudos

Connecting live google sheets data to Databricks

Hi! So we have a live google sheets data that gets updated on an hourly/daily basis, and we want to bring it to databricks as a live/scheduled connection for further analysis, together with other tables and views present there. Do you have any sugges...

  • 16299 Views
  • 4 replies
  • 0 kudos
Latest Reply
Patrick-Data
New Contributor II
  • 0 kudos

Thanks @Ajay-Pandey ! Appreciate your reply. I am new to Databricks, apologies, but I wonder if it's possible to put this live data into a table under a specific catalog-schema? Such that the table will reflect the live data in google sheet?

  • 0 kudos
3 More Replies
manupmanoos
by New Contributor III
  • 2960 Views
  • 2 replies
  • 0 kudos

How to save the best model checkpoi through the epochs of a deep learning network through callbacks?

I have create a neural network and I am training the model with the code as below.  The code fails to write to the databricks file storage. is there any other way to write the checkpoint to databricks storage or to an s3 bucket directly?custom_early_...

  • 2960 Views
  • 2 replies
  • 0 kudos
Latest Reply
MeakhMalik
New Contributor II
  • 0 kudos

I'm experiencing difficulty saving to nearby capacity in Data bricks DBFS due to an "invalid activity" message. In spite of the fact that I can save models to S3 with my AWS accreditations, I want to locally save the best model designated spot during...

  • 0 kudos
1 More Replies
prajwal-c
by New Contributor II
  • 1291 Views
  • 5 replies
  • 1 kudos

My Databricks Exam got suspended

Hi Databricks Team,My Certification exam got suspended today, I started my exam as normal and then my exam was put on hold quoting "team from support needs to talk with you", I connected with the support team and showed my Exam area and everything ve...

  • 1291 Views
  • 5 replies
  • 1 kudos
Latest Reply
prajwal-c
New Contributor II
  • 1 kudos

Hi @Cert-TeamOPS ,My exam was rescheduled today and I gave my exam and still it got suspended, it's very sad that it's happening again to meI have complied with all the rules set by the exam.The question and options are on the complete left of the sc...

  • 1 kudos
4 More Replies
Chughes1408
by New Contributor II
  • 664 Views
  • 3 replies
  • 1 kudos

#Table object name limits

Hi, is there a limit on the number of characters a table object name can be? If so, please provide the source of where this information can be found.

  • 664 Views
  • 3 replies
  • 1 kudos
Latest Reply
filipniziol
Contributor III
  • 1 kudos

Hi @Chughes1408 ,The table name can have up to 255 characters:The source:Names - Azure Databricks - Databricks SQL | Microsoft Learn

  • 1 kudos
2 More Replies
trivedi5678
by New Contributor III
  • 393 Views
  • 1 replies
  • 0 kudos

Databricks Data engineering associates exam got suspended need urgent help

I am writing to request a review of my recently suspended exam. I believe that my situation warrants reconsideration, and I would like to provide some context for your understanding.I applied for Databricks Certified: Data Engineer Associate certific...

  • 393 Views
  • 1 replies
  • 0 kudos
Latest Reply
Cert-TeamOPS
Databricks Employee
  • 0 kudos

Hello @trivedi5678 , Thank you for reaching out to Databricks Community support! We know how frustrating it must be for you. Thank you for filing a ticket with our support team. Please allow the support team 24-48 hours for a resolution. In the meant...

  • 0 kudos
Deepak648
by New Contributor
  • 417 Views
  • 1 replies
  • 0 kudos

Unable to sign up for databricks coummunity edition.

HI, I a getting error an error has occured .Please try again later while creating data bricks community edition account steps i followed are1. provided the datails like name , email etc,2. in the next page upon clicking get started with coummunity ed...

Screenshot 2024-09-22 105909.png
  • 417 Views
  • 1 replies
  • 0 kudos
Latest Reply
gchandra
Databricks Employee
  • 0 kudos

I tried creating with personal email, and it worked smoothly. Please try it in incognito mode if you have any extensions that are stopping it from working. 

  • 0 kudos

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Top Kudoed Authors