cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

vsharma
by New Contributor
  • 1639 Views
  • 3 replies
  • 0 kudos

Where to move cluster-init scripts after latest message (Storing initialization scripts on DBFS is being deprecated. We recommend using a different storage location)?

Recently Databricks has started showing "Storing initialization scripts on DBFS is being deprecated. We recommend using a different storage location" . Is there an alternative of still keep using DBFS or do we need to move to ABSFS ? I could not find...

  • 1639 Views
  • 3 replies
  • 0 kudos
Latest Reply
vkeziv
New Contributor II
  • 0 kudos

It is suggested that we can use Workspace but databricks CLI not supporting importing shell scripts, but we can import shell script using web page? 

  • 0 kudos
2 More Replies
esauesp_co
by New Contributor III
  • 2488 Views
  • 5 replies
  • 1 kudos

Resolved! My jobs and cluster were deleted in a suspicious way

I want to know what happen with my cluster and if I can recover it.I entered to my Databricks account and I didn't found my jobs and my cluster. I couldn't find any log of the deleted cluster because the log is into the cluster interface. I entered t...

  • 2488 Views
  • 5 replies
  • 1 kudos
Latest Reply
Sid_databricks
New Contributor II
  • 1 kudos

Dear folks,When the tables has been deleted, then why I am unable to create the table with same name.It continiously giving me error"DeltaAnalysisException: Cannot create table ('`spark_catalog`.`default`.`Customer_Data`'). The associated location ('...

  • 1 kudos
4 More Replies
Akshay_127877
by New Contributor II
  • 19299 Views
  • 7 replies
  • 1 kudos

How to open Streamlit URL that is hosted by Databricks in local web browser?

I have run this webapp code on Databricks notebook. It works properly without any errors. With databricks acting as server, I am unable open this link on my browser for this webapp.But when I run the code on my local IDE, I am able to just open the U...

image
  • 19299 Views
  • 7 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Akshay Aravinnakshan​ Thank you for posting your question in our community!Your input matters! Help our community thrive by coming back and marking the most helpful and accurate answers. Together, we can make a difference!

  • 1 kudos
6 More Replies
RohitKulkarni
by Contributor
  • 819 Views
  • 3 replies
  • 3 kudos

Databricks Libaries

Hello Team,I am trying to use the below libraries in data bricks .But they are not supporting.import com.microsoft.spark.sqlanalyticsfrom com.microsoft.spark.sqlanalytics.Constants import ConstantsPlease advise the correct Libraries nameRegardsRohit

  • 819 Views
  • 3 replies
  • 3 kudos
Latest Reply
Kaniz
Community Manager
  • 3 kudos

Hi @Rohit Kulkarni​ , We haven't heard from you on the last response from @Akash Bhat​​, and I was checking back to see if their suggestions helped you. Or else, If you have any solution, please share it with the community as it can be helpful to oth...

  • 3 kudos
2 More Replies
BasavarajAngadi
by Contributor
  • 1545 Views
  • 4 replies
  • 1 kudos

Resolved! Question on Transaction logs and versioning in data bricks ?

Hi Experts ,No doubt data bricks supports ACID properties. What when it comes to versioning how much such versions will data bricks captures ? For Example : If i do any DML operations on top of Delta tables every time when i do it captures the tran...

  • 1545 Views
  • 4 replies
  • 1 kudos
Latest Reply
stefnhuy
New Contributor III
  • 1 kudos

Hey,As a data enthusiast myself, I find this topic quite intriguing. Data Bricks indeed does a fantastic job in supporting ACID properties, ensuring data integrity, and allowing for versioning.To address BasavarajAngadi's question, Data Bricks effici...

  • 1 kudos
3 More Replies
jdobken
by New Contributor III
  • 2445 Views
  • 8 replies
  • 11 kudos

As the Databricks account manager; I cannot login: "Your user already belongs to a Databricks account"

On GCP I subscribed to Databricks in one project within the organization.Then I canceled this subscription and subscribed to Databricks in another project.When I try to login onto the newly subscribed databricks with google SSO:> There was an error s...

Screenshot 2023-06-07 at 11.32.30
  • 2445 Views
  • 8 replies
  • 11 kudos
Latest Reply
Anonymous
Not applicable
  • 11 kudos

I can see the issue might be related to organizations or billing accounts. The new Databricks project I tried creating was on a different organization/billing-account than the test Databricks subscription I created a month back.I went back to the ori...

  • 11 kudos
7 More Replies
stephansmit
by New Contributor III
  • 6014 Views
  • 3 replies
  • 11 kudos

How do I access the account console of Databricks in Azure?

To create a Unity metastore the docs refer me to the account console in Databricks, see:https://docs.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/create-metastoreHowever when I go to manage account, I get redirected to select wo...

  • 6014 Views
  • 3 replies
  • 11 kudos
Latest Reply
Anonymous
Not applicable
  • 11 kudos

Please refer here - https://community.databricks.com/s/question/0D58Y000098lIqgSAE/unity-catalog-azure-account-console-how-to-accessYou must be an Azure Databricks account admin.The first Azure Databricks account admin must be an Azure Active Directo...

  • 11 kudos
2 More Replies
Hubert-Dudek
by Esteemed Contributor III
  • 2008 Views
  • 3 replies
  • 25 kudos

Bamboolib with databricks, low-code programming is now available on #databricks Now you can prepare your databricks code without ... coding. Low code ...

Bamboolib with databricks, low-code programming is now available on #databricksNow you can prepare your databricks code without ... coding. Low code solution is now available on Databricks. Install and import bamboolib to start (require a version of ...

Picture2 Picture3 bamboolib Picture4
  • 2008 Views
  • 3 replies
  • 25 kudos
Latest Reply
Palkers
New Contributor II
  • 25 kudos

I have tried to load parquet file using bamboolib menu, and getting below error that path does not existI can load the same file without no problem using spark or pandas using following pathciti_pdf = pd.read_parquet(f'/dbfs/mnt/orbify-sales-raw/Wide...

  • 25 kudos
2 More Replies
Skv
by New Contributor II
  • 1458 Views
  • 2 replies
  • 1 kudos

Resolved! Snowflake query with time travel not working from Databricks while reading into Dataframe.

I am trying to read the changes data from snowflake query into the dataframe using Databricks.Same query is working in snowflake but not in Databricks. Both sides timezones and format are same for the timestamp. I am trying to implement changetrackin...

  • 1458 Views
  • 2 replies
  • 1 kudos
Latest Reply
sher
Valued Contributor II
  • 1 kudos

you are format is wrong that's why you got an errortry thisSELECT * FROM TestTable CHANGES(INFORMATION => DEFAULT) AT(TIMESTAMP => TO_TIMESTAMP_TZ('2023-05-03 00:43:34.885','YYYY-MM-DD HH24:MI:SS.FF')) 

  • 1 kudos
1 More Replies
Phani1
by Valued Contributor
  • 374 Views
  • 1 replies
  • 0 kudos

Databricks Cross cloud (Databricks on AWS and Storage is on Azure)

We have a Databricks account on AWS and storage is on Azure Cloud (ADLS), because of cross-platform do we face any security and performance-related issues also is this feasible and supported? could you please provide your suggestions on this?

  • 374 Views
  • 1 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Janga Reddy​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.

  • 0 kudos
gtyhchang
by New Contributor II
  • 586 Views
  • 2 replies
  • 1 kudos

pandas issue

We identify a potential bug in either DBFS or Pandas that when writting a dataframe using Pandas `to_csv`, `to_parquet`, `to_pickle` etc to a mounted ADLS location with read-only service principle didn't throw permission deny exceptions. However, met...

  • 586 Views
  • 2 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Yung-Hang Chang​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Th...

  • 1 kudos
1 More Replies
Susheel
by New Contributor II
  • 490 Views
  • 1 replies
  • 0 kudos

We have concern with respect to the feasibility of installing the Databricks runtime supported version on-premise. This is because we are planning to...

We have concern with respect to the feasibility of installing the Databricks runtime supported version on-premise. This is because we are planning to migrate the existing Azure Databricks application code to an on-premise environment based on the cl...

  • 490 Views
  • 1 replies
  • 0 kudos
Latest Reply
Susheel
New Contributor II
  • 0 kudos

Any update?

  • 0 kudos
Aanchal
by New Contributor III
  • 1123 Views
  • 4 replies
  • 2 kudos

Resolved! Unable to lunch cluster in Databricks as my azure subscription has been disabled

Azure subscription- disabledDatabricks subscription- free trial 13 day leftDatabricks host- AzureThe cluster is not getting created as my Azure subscription has been disabled after a month of free trial. However, Databricks subscription has still got...

  • 1123 Views
  • 4 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Aanchal Soni​ We haven't heard from you since the last response from @Tyler Retzlaff​ ​, and I was checking back to see if her suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be helpful to o...

  • 2 kudos
3 More Replies
gustavomcarmo-h
by New Contributor III
  • 1480 Views
  • 5 replies
  • 2 kudos

Resolved! Is there a way to list the dlt maintenance jobs through the API?

After creating the delta pipeline, I would like to get details from the dlt maintenance job automatically created by Databricks, like the scheduled time when the dlt maintenance tasks will be executed. However, it seems the Job API 2.1 doesn't cover ...

  • 1480 Views
  • 5 replies
  • 2 kudos
Latest Reply
gustavomcarmo-h
New Contributor III
  • 2 kudos

Hi @Debayan Mukherjee​ ,Actually the Databricks Jobs API documentation has not been fixed yet. The parameter `job_type` should be included in the list endpoint request documentation. Please do this in order to avoid unnecessary questions here in the ...

  • 2 kudos
4 More Replies
Labels