cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Kevin_Kang
by New Contributor III
  • 2385 Views
  • 3 replies
  • 0 kudos

Resolved! Account Registration Help for Partner-Academy

Hi there,I used my company email to register an account for customer-academy.databricks.com a while ago. And I need to create an account in the partner-academy.databricks.com using the same company email.However, when I tried for partner-academy, it ...

  • 2385 Views
  • 3 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Geunhee Kang​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thank...

  • 0 kudos
2 More Replies
DenisMcd
by New Contributor
  • 2512 Views
  • 2 replies
  • 0 kudos

"Endpoint not found for /2.0/sql/statements/"

Hey everyone! I´m trying to access table row using databricks api. Using Insomnia or postman, to test and the error are the same: { "error_code": "ENDPOINT_NOT_FOUND", "message": "Endpoint not found for /2.0/sql/statements/"}Below is my request:(for ...

  • 2512 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Denis Macedo​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers ...

  • 0 kudos
1 More Replies
Papa_PSL
by New Contributor
  • 1659 Views
  • 2 replies
  • 1 kudos

I want to import a notebook

Want to import a note book, facing this following error:Importation fails: could not deserialize: Unkown language htm

  • 1659 Views
  • 2 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @BITAIGOT Guy-Serge​ We haven't heard from you since the last response from @Debayan Mukherjee​ â€‹, and I was checking back to see if her suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be hel...

  • 1 kudos
1 More Replies
MarsSu
by New Contributor II
  • 3239 Views
  • 3 replies
  • 3 kudos

Resolved! Does driver node of job compute have HA?

I would like to confirm and discuss HA mechanism about driver node of job compute. Because we can image driver node just like master node of cluster. In AWS EMR, we can setup 2 master node so that one of master node failed, another master node can re...

  • 3239 Views
  • 3 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Mars Su​ We haven't heard from you since the last response from @Werner Stinckens​ and @karthik p​ â€‹, and I was checking back to see if her suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be...

  • 3 kudos
2 More Replies
nistrate
by New Contributor III
  • 7034 Views
  • 2 replies
  • 3 kudos

Resolved! Revoking cluster creation permission for non-admin users in Databricks workspace

I'm currently building a Databricks workspace for my organization and I would like to revoke the cluster creation permission for all users except admins or a specific group (e.g., Data Analysts). Could someone please guide me on how to achieve this?

  • 7034 Views
  • 2 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Nicolae Istrate​ We haven't heard from you since the last response from @Werner Stinckens​ , and I was checking back to see if her suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be helpful ...

  • 3 kudos
1 More Replies
Ankith
by New Contributor
  • 4137 Views
  • 2 replies
  • 1 kudos

Resolved! How to enable spark.shuffle.compress in spark 3.3.0 or above versions?

when I try to set that I got a following error, appreciate your comments, thanks you in advance.

image.png
  • 4137 Views
  • 2 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Ankith Patlolla​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answe...

  • 1 kudos
1 More Replies
Bbren
by New Contributor
  • 3112 Views
  • 2 replies
  • 1 kudos

Resolved! Handling of millions of xml in json files

Hi all, i have some questions related to the handling of many smalls files and possible improvements and augmentations. We have many small xml files. These files are previously processed by another system that puts them in our datalake, but as an add...

  • 3112 Views
  • 2 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Bauke Brenninkmeijer​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best ...

  • 1 kudos
1 More Replies
Mumrel
by Contributor
  • 6394 Views
  • 4 replies
  • 0 kudos

Resolved! Usage of forum: How to find the threads where I commented on and my bookmarks

Hi quick questions, 1) the other day I commented on a thread and cannot find it. Is there a feature to find all my posts? I cannot find it under my profile.2) I find interesting threads and want to watch (in the sense: email me when updates occure) t...

  • 6394 Views
  • 4 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Jan St.​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your ...

  • 0 kudos
3 More Replies
avivshafir
by New Contributor II
  • 1015 Views
  • 0 replies
  • 1 kudos

SQL endpoint querying table changes using a greater timestamp or version than the last table commit

Reading table changes using a greater timestamp or version than the last table commit throws an error and can be changed using a flag timestampOutOfRange.enabled,My issue is that I use an SQL endpoint and I don't see any way of providing this spark f...

  • 1015 Views
  • 0 replies
  • 1 kudos
Rishitha
by New Contributor III
  • 1880 Views
  • 2 replies
  • 2 kudos

Resolved! Normalizing data from autoloader

I have data on s3 and i'm using autoloader to load the data. My json docs have fields which are array of structures. When I don't specify any schema the whole data is stored as strings even the array of structures are just a blob of string making it ...

  • 1880 Views
  • 2 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Rishitha Reddy​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us s...

  • 2 kudos
1 More Replies
naga_databricks
by Contributor
  • 3327 Views
  • 2 replies
  • 1 kudos

Using Init scripts using DBX

I specify init scripts in my deployment.conf, as below: basic-static-cluster: &basic-static-cluster new_cluster: spark_version: "13.0.x-scala2.12" num_workers: 1 node_type_id: "n2-highmem-2" init_scripts: - worksp...

  • 3327 Views
  • 2 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Naga Vaibhav Elluru​ We haven't heard from you since the last response from @Debayan Mukherjee​ â€‹, and I was checking back to see if her suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be he...

  • 1 kudos
1 More Replies
AnkitG
by New Contributor III
  • 7213 Views
  • 4 replies
  • 1 kudos

Resolved! Disable iFraming prevention not working

For my own workspace disable iFraming prevention stopped working on 9th May 2023. Till Friday it was working fine and I have already disabled the iFraming prevention.Is there something not right with the databricks platform?

  • 7213 Views
  • 4 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Ankit Goyal​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so w...

  • 1 kudos
3 More Replies
ppang
by New Contributor III
  • 6253 Views
  • 1 replies
  • 0 kudos

Resolved! Job cluster failed to start with custom docker image

Using Databricks Container Services, we have created two custom docker image: one based on nvidia/cuda:11.8.0-runtime-ubuntu22.04 and another based on databricksruntime/standard:12.2-LTS. In either case, we got this error with no specific diagnostics...

  • 6253 Views
  • 1 replies
  • 0 kudos
Latest Reply
ppang
New Contributor III
  • 0 kudos

I managed to get databricksruntime/standard:12.2-LTS to run in Databricks. However, for the CUDA image (nvidia/cuda:11.8.0-runtime-ubuntu22.04), I have only managed to get it to run with Databricks runtime 10.4 LTS. Does anyone know if Databricks run...

  • 0 kudos
JordanYaker
by Contributor
  • 3589 Views
  • 1 replies
  • 0 kudos

Batch Doesn't Exist Failure

I have a job that's been working perfectly fine since I deployed it earlier this month. Last night, however, one of the tasks within the job started failing with the following error:java.lang.IllegalStateException: batch 4 doesn't exist at org.apac...

  • 3589 Views
  • 1 replies
  • 0 kudos
Latest Reply
JordanYaker
Contributor
  • 0 kudos

I tried FSCK REPAIR just on the chance that it would work and it had no effect.

  • 0 kudos

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels