cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

WillHeyer
by New Contributor II
  • 1870 Views
  • 1 replies
  • 2 kudos

Resolved! Best Practices for PowerBI Connectivity w/ Partner Connect. Access Token w/ Service Principal, Databricks Username w/ Service account, or OAuth?

I'm aware all are possible methods but are all equal? Or is the matter trivial? Thank you so much!

  • 1870 Views
  • 1 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

@Will Heyer​ :The best method for Power BI connectivity with Partner Connect depends on your specific use case and requirements. Here are some factors to consider for each method:Access Token with Service Principal: This method uses a client ID and s...

  • 2 kudos
DavideCagnoni
by Contributor
  • 3556 Views
  • 1 replies
  • 4 kudos

Resolved! How to use multi-cursor and rectangular selection for notebooks and query editor in Linux ?

The documentation explains how to use multicursor in notebooks. However, it only says it for Windows and MacOS. The Windows way would work in Linux (Ubuntu) up to a few days ago but it does not work now anymore.

  • 3556 Views
  • 1 replies
  • 4 kudos
Latest Reply
Anonymous
Not applicable
  • 4 kudos

@Davide Cagnoni​ :Multicursor support in Databricks notebooks is implemented using the Ace editor, which is a web-based code editor. Therefore, the behavior of multicursor support may depend on the specific browser and operating system you are using....

  • 4 kudos
kDev
by New Contributor
  • 4798 Views
  • 1 replies
  • 1 kudos

UnauthorizedAccessException: PERMISSION_DENIED: User does not have READ FILES on External Location

Our jobs have been running fine so far w/o any issues on a specific workspace. These jobs read data from files on Azure ADLS storage containers and dont use the hive metastore data at all.Now we attached the unity metastore to this workspace, created...

  • 4798 Views
  • 1 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

@kumar mahadevan​ :Based on the error message you received, it seems like the user or service principal that is running the Databricks job does not have the necessary read permissions on the Azure ADLS storage containers.First, double-check that the ...

  • 1 kudos
vnc001
by New Contributor
  • 852 Views
  • 1 replies
  • 1 kudos

Resolved! Clusters API 2.0 - Unable to execute cluster events api

Details: I keep getting "Missing required field: cluster_id" even though you can see it is supplied. Is this a bug? or I am missing something? I am testing this in postman. Error: {"error_code":"INVALID_PARAMETER_VALUE","message":"Missing required fi...

image
  • 852 Views
  • 1 replies
  • 1 kudos
Latest Reply
SUMI1
New Contributor III
  • 1 kudos

Hi guysI'm sorry to hear that the Clusters API 2.0 and cluster event execution are giving you trouble. I advise getting in touch with the support staff for guidance on quickly fixing the problem.

  • 1 kudos
Phani1
by Valued Contributor
  • 733 Views
  • 1 replies
  • 1 kudos

Resolved! DLT best practices

Hi Team,Could you please recommend the best practices to implement the delta live tables?Regards,Phanindra

  • 733 Views
  • 1 replies
  • 1 kudos
Latest Reply
Ryan_Chynoweth
Honored Contributor III
  • 1 kudos

Hi Phani, what exactly are you looking for with best practices? At a high level:Always provide an external storage location (S3, ADLS, GCS) for your pipelineUse Auto Scaling! Python imports can be leverage to reuse code With regards to providing a st...

  • 1 kudos
NOOR_BASHASHAIK
by Contributor
  • 1070 Views
  • 1 replies
  • 2 kudos

Resolved! Azure Databricks PATs expire even before validity

Hi all,we have this issue in our environment - even thought we give 365 days validity for Databricks PATS generation, the PATs expire every now and then. Is there any problem with the command we use : curl --location --request POST 'https://<<HOST_NA...

  • 1070 Views
  • 1 replies
  • 2 kudos
Latest Reply
karthik_p
Esteemed Contributor
  • 2 kudos

@NOOR BASHA SHAIK​ It looks you are providing 365 days, can you please post your response. if you won't provide any lifetime then it should be valid indefinitely. can you please add 90 days validity and test

  • 2 kudos
Chinu
by New Contributor III
  • 450 Views
  • 1 replies
  • 1 kudos

API to get Databricks Status AWS.

Hi, Do you have an api endpoint to call to get the databricks status for AWS?Thanks,

  • 450 Views
  • 1 replies
  • 1 kudos
Latest Reply
karthik_p
Esteemed Contributor
  • 1 kudos

@Chinu Lee​ you have webhook/slack that can be used to fetch status https://docs.databricks.com/resources/status.html#webhookare you specifically looking for your account workspace/above one

  • 1 kudos
marcin-sg
by New Contributor III
  • 529 Views
  • 1 replies
  • 2 kudos

Create (account wide) groups without account admin permissions

The use case is quite simple: each environment - databricks workspace (prod, test, dev) will be created by a separate service principal (which for isolation purpose should not have account wide admin permission) with terraform, but will belong to the...

  • 529 Views
  • 1 replies
  • 2 kudos
Latest Reply
marcin-sg
New Contributor III
  • 2 kudos

Another thing would be also to assign workspace to a metastore without account admin permission - for similar reason.

  • 2 kudos
Anonymous
by Not applicable
  • 2748 Views
  • 6 replies
  • 2 kudos

Resolved! Delta Sharing - Unity Catalog difference

Delta Sharing and Unity catalog both have elements of data sharing. Can you please explain when one would use Delta sharing vs Unity Catalog?

  • 2748 Views
  • 6 replies
  • 2 kudos
Latest Reply
DBXC
Contributor
  • 2 kudos

Based on the Databricks reply from the post below: "Unity Catalog does not currently support separating data by workspace or Azure subscription. As you noted, data from all catalogs within a region can be accessed by any workspace within that region,...

  • 2 kudos
5 More Replies
deficiant_codge
by Contributor II
  • 3085 Views
  • 10 replies
  • 11 kudos

Resolved! Can't able to run following queries

I cannot able to run following queriesALTER TABLE iot_events ADD ATTRIBUTE pii ON emailALTER TABLE users ADD ATTRIBUTE pii ON phoneGRANT SELECT ON DATABASE iot_data HAVING ATTRIBUTE NOT IN (pii) TO product_managersand GRANT SELECT ON iot_events TO ...

  • 3085 Views
  • 10 replies
  • 11 kudos
Latest Reply
karthik_p
Esteemed Contributor
  • 11 kudos

@Kaniz Fatma​ can anyone from data Bircks help on why attribute-based access control function is not working in unity catalog @Rahul Mishra​ below commands ALTER TABLE iot_events ADD ATTRIBUTE pii ON emailALTER TABLE users ADD ATTRIBUTE pii ON phoneG...

  • 11 kudos
9 More Replies
goldentown
by New Contributor III
  • 1816 Views
  • 1 replies
  • 2 kudos

Resolved! The Jupiter note-book doesn't update imports after updating the .py file

Please help. Here's an example:I have one .py file and one .ipynb, and the .py file contains the test function, but after adding the new function test1, it doesn't appear in .ipynb. Even after re-running the .py file and reimporting it in .ipynb. How...

  • 1816 Views
  • 1 replies
  • 2 kudos
Latest Reply
goldentown
New Contributor III
  • 2 kudos

%load_ext autoreload%autoreload 2

  • 2 kudos
PearceR
by New Contributor III
  • 3480 Views
  • 2 replies
  • 1 kudos

Resolved! custom upsert for delta live tables apply_changes()

Hello community :).I am currently implementing some pipelines using DLT. They are working great for my medalion architecture for landed json in bronze -> silver (using apply_changes) then materialized gold views ontop.However, I am attempting to crea...

  • 3480 Views
  • 2 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

@Robert Pearce​ :It is possible to achieve the desired behavior using apply_changes in Databricks Delta Lake. You can use the merge operation to merge data from your source into your target Delta table, and then use whenMatchedUpdate to update the id...

  • 1 kudos
1 More Replies
ROB2
by New Contributor
  • 547 Views
  • 1 replies
  • 0 kudos

clickfortranslation.com

What are the Different Types of Industry Specific Translation Services Manual  ?GET TO KNOW MORE CLICK THE LINK BELOW.https://clickfortranslation.com/manual-translation.php

Best Manual Translation Services by Click For Translation
  • 547 Views
  • 1 replies
  • 0 kudos
Latest Reply
Joe_Suarez
New Contributor III
  • 0 kudos

Do you provide all types of industry-specific translation services? I'm asking because many translation companies pretend to provide all industry-specific translation services, but the final result is far from being a qualitative and professional one...

  • 0 kudos
ivanychev
by Contributor
  • 972 Views
  • 2 replies
  • 2 kudos

Does anyone run Databricks jobs using Docker + ARM (Graviton) instances?

Graviton instances do not support Container services on paper (https://docs.databricks.com/clusters/graviton.html#unsupported-features) but if you try to build Docker ARM image and run it on Graviton, it will work. Does anyone use this combination in...

  • 972 Views
  • 2 replies
  • 2 kudos
Latest Reply
Debayan
Esteemed Contributor III
  • 2 kudos

Graviton is not supported by Databricks Container Services. How are you planning to run it on Databricks? Please tag @Debayan​ with your next comment so that I will get notified. Thank you!

  • 2 kudos
1 More Replies
shawnbarrick
by New Contributor III
  • 1062 Views
  • 2 replies
  • 2 kudos

Resolved! How to resolve SAT driver errors

I was able to follow the SAT setup instructions, but ran into the same error whether I ran it "manually" or via terraform. The initialization seemed to run fine. Can anyone suggest any steps to troubleshoot this?

Screenshot 2023-05-01 at 10.19.23 AM
  • 1062 Views
  • 2 replies
  • 2 kudos
Latest Reply
shawnbarrick
New Contributor III
  • 2 kudos

Thanks - I also spoke with Arun, who was very helpful. Our databricks admin users all require an okta login, which is causing the error. We're looking into a "break glass" admin user for this purpose.

  • 2 kudos
1 More Replies
Labels
Top Kudoed Authors