Databricks Community

Pradeep_Namani

We are running a logic in an Azure Databricks notebook using Python with Spark. Initially, we read data from ADLS and load it into a global temporary table to perform data quality checks. We then recreate the same temporary table. Afterward, we use t...

Pradeep_Namani · 11-23-2022

The date field is getting changed while reading data from source .xls file to the dataframe. In the source xl file all columns are strings but i am not sure why date column alone behaves differentlyIn Source file date is 1/24/1947.In pyspark datafram...

Pradeep_Namani

The script is too large to paste in here, so please get in touch with me to to obtain it.

Pradeep_Namani · 11-24-2022

I have tried the option which we have give in above url but no use, still I am facing same issue

Pradeep_Namani · 11-23-2022

Thank you @Yogita Chavan for replying , but when I am reading file as csv it is showing all data in different format, I am attaching the screen shot

Pradeep_Namani · 11-17-2022

Hi Team, @Merca Ovnerud I am also facing same issue , below is the code snippet which I am using df=spark.read.format("com.crealytics.spark.excel").option("header","true").load("/mnt/dataplatform/Tenant_PK/Results.xlsx")I have a couple of date colum...

Databricks Community

User Stats

User Activity

Getting Different results when I am running the Global Templary table (Transform Query)

Date field getting changed when reading from excel file to dataframe in pyspark

Re: Getting Different results when I am running the Global Templary table (Transform Query)

Re: Date field getting changed when reading from excel file to dataframe in pyspark

Re: Date field getting changed when reading from excel file to dataframe in pyspark

Re: Date field getting changed when reading from excel file to dataframe