cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

gud4eve
by New Contributor III
  • 2702 Views
  • 1 replies
  • 0 kudos

Resolved! Scala app getting NullPointerException while migrating from DBR 7.3 to 9.1 (and above)

We are migrating our Scala jobs from AWS EMR (6.2.1 and Spark version - 3.0.1) to Lakehouse and few of our jobs are failing due to NullPointerException. We tried in Databricks Runtime 7.3 LTS, it is working fine. Because it had same spark version 3.0...

  • 2702 Views
  • 1 replies
  • 0 kudos
Latest Reply
gud4eve
New Contributor III
  • 0 kudos

In one of my code statements, I updated scala Boolean to java.lang.Boolean and this is working fine now. May be in new newer Spark versions, null in scala Boolean isn't supported.

  • 0 kudos
DianGermishuiz1
by New Contributor III
  • 2746 Views
  • 4 replies
  • 0 kudos

What is the purpose of the databricks-workspace-stack-lambdazipsbucket S3 bucket created on Databricks AWS Account Provisioning?

An S3 bucket with the prefix "databricks-workspace-stack-lambdazipsbucket" was created by default when I created my AWS Databricks account. It is set to public access. It has one zip file in it called "lambda.zip". What is the purpose of this S3 buck...

  • 2746 Views
  • 4 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Dian Germishuizen​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best ans...

  • 0 kudos
3 More Replies
haxker
by New Contributor
  • 2836 Views
  • 5 replies
  • 2 kudos

Unable to Login and Unable to reset Password . Password reset link keeps on loading even after entering passwords

https://community.cloud.databricks.com/login.htmlUnable to Login and Unable to reset Password . Password reset link keeps on loading even after entering passwords 

  • 2836 Views
  • 5 replies
  • 2 kudos
Latest Reply
Miguel_AngelJim
New Contributor II
  • 2 kudos

I have the same problem, when you press the reset password button it stays loading indefinitely and does not confirm that it has been changed

  • 2 kudos
4 More Replies
Tracy_
by New Contributor II
  • 10580 Views
  • 5 replies
  • 0 kudos

Incorrect reading csv format with inferSchema

Hi All,There is a CSV with a column ID (format: 8-digits & "D" at the end).When trying to read a csv with .option("inferSchema", "true"), it returns the ID as double and trim the "D". Is there any idea (apart from inferSchema=False) to get correct ...

image.png
  • 10580 Views
  • 5 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @tracy ng​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your...

  • 0 kudos
4 More Replies
Naeem_K
by New Contributor III
  • 1893 Views
  • 3 replies
  • 1 kudos

Resolved! Data Engineer Associate Certificate and badge not yet received

@Kaniz Fatma​ I have cleared the certification exam on 26th January 2023, but still haven't received the certificate. I had given the exam with a different mail ID but I'm not receiving any emails from Databricks to that mail ID.Kindly help me resolv...

  • 1893 Views
  • 3 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Naeemah Khatib​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us s...

  • 1 kudos
2 More Replies
QPeiran
by New Contributor III
  • 2730 Views
  • 3 replies
  • 5 kudos

How to exit the entire job in the orchestration scenario?

Hi, can anybody answer this question I posted on StackOverflow? https://stackoverflow.com/questions/73314048/databricks-how-to-exit-the-entire-job-in-the-notebooks-orchestration-scenario

  • 2730 Views
  • 3 replies
  • 5 kudos
Latest Reply
CarterM
New Contributor III
  • 5 kudos

@Vidula Khanna​ @Vidula Khanna​ We are experiencing the same issue in our Workflows and I was wondering if there has been any update.We need the functionality to call a method similar to `dbutils.notebook.exit` in a notebook that will cancel the exec...

  • 5 kudos
2 More Replies
Michael_Marquis
by New Contributor II
  • 3702 Views
  • 1 replies
  • 3 kudos

How can I change the font size on ipywidgets in a Databricks notebook?

I'm trying to create a simple UI for a notebook using the recently implemented support for ipywidgets, but I'm having a hard time figuring out how to change certain style attributes like font size and color in widgets that should accept those style p...

  • 3702 Views
  • 1 replies
  • 3 kudos
Latest Reply
Miguel_Suarez
Databricks Employee
  • 3 kudos

Hey Michael,The example you're trying to run is for ipywidgets 8, we currently have ipywidgets 7-which has fewer button customizations. I believe the only font customization available in 7 is "font_weigh t" (no space). I hope this helps.Best,Miguel

  • 3 kudos
Anonymous
by Not applicable
  • 1398 Views
  • 4 replies
  • 11 kudos

Hi team, Could everyone please help explain about the Charity and Charity Name in "Data + AI Summit 2023: VIP ALL-Access Pass" in the Reward...

Hi team,Could everyone please help explain about the Charity and Charity Name in "Data + AI Summit 2023: VIP ALL-Access Pass" in the Reward Store? I don't know what it means.Thanks and Best Regards,Jensen Nguyen

Image
  • 1398 Views
  • 4 replies
  • 11 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 11 kudos

Adding @Kaniz Fatma​ @Suteja Kanuri​ for visibility

  • 11 kudos
3 More Replies
Anonymous
by Not applicable
  • 632 Views
  • 1 replies
  • 2 kudos

What happened to the Databricks Marketplace?

I recently watched a Data + AI keynote from 2022 and it was announced there and in the below article. What is the status of this? I do not see it on my account; I am currently on a trial. If it is there, how do I get to it?https://www.databricks.com/...

  • 632 Views
  • 1 replies
  • 2 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 2 kudos

adding @Kaniz Fatma​ and @Vidula Khanna​ for visibility on this question

  • 2 kudos
FirozAhmad
by New Contributor II
  • 958 Views
  • 1 replies
  • 1 kudos

how to convert my Academy to partner Account comapany mail.

I have a account of Databricks Academy account but unable to login same id and password to Partner Academy Account .

  • 958 Views
  • 1 replies
  • 1 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 1 kudos

Adding @Vidula Khanna​ and @Kaniz Fatma​ for visibility

  • 1 kudos
FJ
by Contributor III
  • 1614 Views
  • 1 replies
  • 2 kudos

Instructor-led course for Advanced data engineering course

I enrolled myself through the partner academy for Advanced Data Engineering with Databricks (North America) (Instructor-Led course) Just received this message saying my registration for Advanced Data Engineering with Databricks (North America) has be...

  • 1614 Views
  • 1 replies
  • 2 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 2 kudos

Adding @Vidula Khanna​ and @Kaniz Fatma​ for visibility

  • 2 kudos
Hubert-Dudek
by Esteemed Contributor III
  • 1886 Views
  • 1 replies
  • 10 kudos

databricks SQL now supports getting the SQLSTATE code of a query to identify errors. You can use the e.getSqlState() method in a try/catch block to ge...

databricks SQL now supports getting the SQLSTATE code of a query to identify errors. You can use the e.getSqlState() method in a try/catch block to get the five-character code that indicates the success or failure of an SQL command. I am still thinki...

Untitled
  • 1886 Views
  • 1 replies
  • 10 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 10 kudos

Thank you for sharing @Hubert Dudek​ 

  • 10 kudos
databicky
by Contributor II
  • 5138 Views
  • 1 replies
  • 0 kudos

how to create borders in excel by python

how to create borders in excel by python like the following format.​in the ex column if i entered the value as data means it should be center between those rows.​​​

  • 5138 Views
  • 1 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

@Mohammed sadamusean​ :You can use the openpyxl library in Python to create borders in Excel. Here is an example code snippet that creates a border around a range of cells and centers the text in a specific column:import openpyxl from openpyxl.styles...

  • 0 kudos
429957
by New Contributor
  • 901 Views
  • 1 replies
  • 0 kudos

DeltaColumnMappingUnsupportedException' when performing 'Full refresh all' on DLT pipeline

Trigger:Perform 'Full refresh all' on a DLT pipeline (new or existing). The existing DLT table already existed beforehand.Issue:Getting the error 'DeltaColumnMappingUnsupportedException' during "Setting up tables" stage.```com.databricks.sql.transact...

  • 901 Views
  • 1 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

@Raeger Tay​ :The error message indicates that a schema change has been detected while changing the column mapping mode. It seems like you are trying to change the column mapping mode from the default (position) to the "name" mode, which maps columns...

  • 0 kudos
KVNARK
by Honored Contributor II
  • 2333 Views
  • 1 replies
  • 5 kudos

accessing power bi dataset using MDX query using windows is working but the same not working using python Linux server.

trying to access the SSAS POIWER BI dataset using MDX query from python LInux server. We are hitting roadblock. The existing setup works as expected in windows system due to adodb.dll but unable to connect in Linux. Any help would be much appreciated...

  • 2333 Views
  • 1 replies
  • 5 kudos
Latest Reply
Anonymous
Not applicable
  • 5 kudos

@KVNARK .​ :One potential solution would be to use an open-source MDX library for Python that can connect to SSAS, such as OLAP-XMLA for Python. This library can be used to execute MDX queries against a SSAS server, including Power BI datasets.Here's...

  • 5 kudos

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels