cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

gud4eve
by New Contributor III
  • 4743 Views
  • 1 replies
  • 0 kudos

Resolved! Scala app getting NullPointerException while migrating from DBR 7.3 to 9.1 (and above)

We are migrating our Scala jobs from AWS EMR (6.2.1 and Spark version - 3.0.1) to Lakehouse and few of our jobs are failing due to NullPointerException. We tried in Databricks Runtime 7.3 LTS, it is working fine. Because it had same spark version 3.0...

  • 4743 Views
  • 1 replies
  • 0 kudos
Latest Reply
gud4eve
New Contributor III
  • 0 kudos

In one of my code statements, I updated scala Boolean to java.lang.Boolean and this is working fine now. May be in new newer Spark versions, null in scala Boolean isn't supported.

  • 0 kudos
DianGermishuiz1
by Databricks Partner
  • 4925 Views
  • 4 replies
  • 0 kudos

What is the purpose of the databricks-workspace-stack-lambdazipsbucket S3 bucket created on Databricks AWS Account Provisioning?

An S3 bucket with the prefix "databricks-workspace-stack-lambdazipsbucket" was created by default when I created my AWS Databricks account. It is set to public access. It has one zip file in it called "lambda.zip". What is the purpose of this S3 buck...

  • 4925 Views
  • 4 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Dian Germishuizen​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best ans...

  • 0 kudos
3 More Replies
haxker
by New Contributor
  • 4390 Views
  • 5 replies
  • 2 kudos

Unable to Login and Unable to reset Password . Password reset link keeps on loading even after entering passwords

https://community.cloud.databricks.com/login.htmlUnable to Login and Unable to reset Password . Password reset link keeps on loading even after entering passwords 

  • 4390 Views
  • 5 replies
  • 2 kudos
Latest Reply
Miguel_AngelJim
New Contributor II
  • 2 kudos

I have the same problem, when you press the reset password button it stays loading indefinitely and does not confirm that it has been changed

  • 2 kudos
4 More Replies
Tracy_
by Databricks Partner
  • 17660 Views
  • 5 replies
  • 0 kudos

Incorrect reading csv format with inferSchema

Hi All,There is a CSV with a column ID (format: 8-digits & "D" at the end).When trying to read a csv with .option("inferSchema", "true"), it returns the ID as double and trim the "D". Is there any idea (apart from inferSchema=False) to get correct ...

image.png
  • 17660 Views
  • 5 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @tracy ng​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your...

  • 0 kudos
4 More Replies
QPeiran
by New Contributor III
  • 4545 Views
  • 3 replies
  • 5 kudos

How to exit the entire job in the orchestration scenario?

Hi, can anybody answer this question I posted on StackOverflow? https://stackoverflow.com/questions/73314048/databricks-how-to-exit-the-entire-job-in-the-notebooks-orchestration-scenario

  • 4545 Views
  • 3 replies
  • 5 kudos
Latest Reply
CarterM
New Contributor III
  • 5 kudos

@Vidula Khanna​ @Vidula Khanna​ We are experiencing the same issue in our Workflows and I was wondering if there has been any update.We need the functionality to call a method similar to `dbutils.notebook.exit` in a notebook that will cancel the exec...

  • 5 kudos
2 More Replies
Michael_Marquis
by New Contributor II
  • 5387 Views
  • 1 replies
  • 3 kudos

How can I change the font size on ipywidgets in a Databricks notebook?

I'm trying to create a simple UI for a notebook using the recently implemented support for ipywidgets, but I'm having a hard time figuring out how to change certain style attributes like font size and color in widgets that should accept those style p...

  • 5387 Views
  • 1 replies
  • 3 kudos
Latest Reply
Miguel_Suarez
Databricks Employee
  • 3 kudos

Hey Michael,The example you're trying to run is for ipywidgets 8, we currently have ipywidgets 7-which has fewer button customizations. I believe the only font customization available in 7 is "font_weigh t" (no space). I hope this helps.Best,Miguel

  • 3 kudos
Anonymous
by Not applicable
  • 2098 Views
  • 4 replies
  • 11 kudos

Hi team, Could everyone please help explain about the Charity and Charity Name in "Data + AI Summit 2023: VIP ALL-Access Pass" in the Reward...

Hi team,Could everyone please help explain about the Charity and Charity Name in "Data + AI Summit 2023: VIP ALL-Access Pass" in the Reward Store? I don't know what it means.Thanks and Best Regards,Jensen Nguyen

Image
  • 2098 Views
  • 4 replies
  • 11 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 11 kudos

Adding @Kaniz Fatma​ @Suteja Kanuri​ for visibility

  • 11 kudos
3 More Replies
Anonymous
by Not applicable
  • 1074 Views
  • 1 replies
  • 2 kudos

What happened to the Databricks Marketplace?

I recently watched a Data + AI keynote from 2022 and it was announced there and in the below article. What is the status of this? I do not see it on my account; I am currently on a trial. If it is there, how do I get to it?https://www.databricks.com/...

  • 1074 Views
  • 1 replies
  • 2 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 2 kudos

adding @Kaniz Fatma​ and @Vidula Khanna​ for visibility on this question

  • 2 kudos
FirozAhmad
by Databricks Partner
  • 1623 Views
  • 1 replies
  • 1 kudos

how to convert my Academy to partner Account comapany mail.

I have a account of Databricks Academy account but unable to login same id and password to Partner Academy Account .

  • 1623 Views
  • 1 replies
  • 1 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 1 kudos

Adding @Vidula Khanna​ and @Kaniz Fatma​ for visibility

  • 1 kudos
FJ
by Contributor III
  • 2449 Views
  • 1 replies
  • 2 kudos

Instructor-led course for Advanced data engineering course

I enrolled myself through the partner academy for Advanced Data Engineering with Databricks (North America) (Instructor-Led course) Just received this message saying my registration for Advanced Data Engineering with Databricks (North America) has be...

  • 2449 Views
  • 1 replies
  • 2 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 2 kudos

Adding @Vidula Khanna​ and @Kaniz Fatma​ for visibility

  • 2 kudos
Hubert-Dudek
by Databricks MVP
  • 2648 Views
  • 1 replies
  • 10 kudos

databricks SQL now supports getting the SQLSTATE code of a query to identify errors. You can use the e.getSqlState() method in a try/catch block to ge...

databricks SQL now supports getting the SQLSTATE code of a query to identify errors. You can use the e.getSqlState() method in a try/catch block to get the five-character code that indicates the success or failure of an SQL command. I am still thinki...

Untitled
  • 2648 Views
  • 1 replies
  • 10 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 10 kudos

Thank you for sharing @Hubert Dudek​ 

  • 10 kudos
databicky
by Contributor II
  • 8555 Views
  • 1 replies
  • 0 kudos

how to create borders in excel by python

how to create borders in excel by python like the following format.​in the ex column if i entered the value as data means it should be center between those rows.​​​

  • 8555 Views
  • 1 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

@Mohammed sadamusean​ :You can use the openpyxl library in Python to create borders in Excel. Here is an example code snippet that creates a border around a range of cells and centers the text in a specific column:import openpyxl from openpyxl.styles...

  • 0 kudos
429957
by Databricks Partner
  • 1671 Views
  • 1 replies
  • 0 kudos

DeltaColumnMappingUnsupportedException' when performing 'Full refresh all' on DLT pipeline

Trigger:Perform 'Full refresh all' on a DLT pipeline (new or existing). The existing DLT table already existed beforehand.Issue:Getting the error 'DeltaColumnMappingUnsupportedException' during "Setting up tables" stage.```com.databricks.sql.transact...

  • 1671 Views
  • 1 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

@Raeger Tay​ :The error message indicates that a schema change has been detected while changing the column mapping mode. It seems like you are trying to change the column mapping mode from the default (position) to the "name" mode, which maps columns...

  • 0 kudos
KVNARK
by Honored Contributor II
  • 3896 Views
  • 1 replies
  • 5 kudos

accessing power bi dataset using MDX query using windows is working but the same not working using python Linux server.

trying to access the SSAS POIWER BI dataset using MDX query from python LInux server. We are hitting roadblock. The existing setup works as expected in windows system due to adodb.dll but unable to connect in Linux. Any help would be much appreciated...

  • 3896 Views
  • 1 replies
  • 5 kudos
Latest Reply
Anonymous
Not applicable
  • 5 kudos

@KVNARK .​ :One potential solution would be to use an open-source MDX library for Python that can connect to SSAS, such as OLAP-XMLA for Python. This library can be used to execute MDX queries against a SSAS server, including Power BI datasets.Here's...

  • 5 kudos
Indra
by New Contributor
  • 2755 Views
  • 1 replies
  • 0 kudos

Performance issue with Simba ODBC Driver to perform simple insert command to Deltalake

Hi,Our team is using Simba ODBC to perform data loading to Deltalake, and For a table with 3 columns it took around 55 seconds to insert 15 records. How to improve transactional loading into Deltalake? is there any option from the Simba ODBC driver t...

  • 2755 Views
  • 1 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

@Indra Limena​ :There are several ways to improve transactional loading into Delta Lake:Use Delta Lake's native Delta JDBC/ODBC connector instead of a third-party ODBC driver like Simba. The native connector is optimized for Delta Lake and supports b...

  • 0 kudos
Labels