- 1693 Views
- 1 replies
- 1 kudos
- 1693 Views
- 1 replies
- 1 kudos
Latest Reply
- 1 kudos
Hi,Could you share more details on what you have tried? please provide more details.
- 1 kudos
Forum Posts
- 6760 Views
- 1 replies
- 5 kudos
Resolved! How to Overwrite Using pyspark's JDBC without loosing constraints on table columns
Hello,My table has primary key constraint on a perticular column, Im loosing primary key constaint on that column each time I overwrite the table , What Can I do to preserve it? Any Heads up would be appreciatedTried Belowdf.write.option("truncate", ...
- 6760 Views
- 1 replies
- 5 kudos
Latest Reply
- 5 kudos
@Abeeya . , Mode "truncate", is correct to preserve the table. However, when you want to add a new column (mismatched schema), it wants to drop it anyway.
- 5 kudos
- 3273 Views
- 2 replies
- 3 kudos
Resolved! issue with rounding selected column in "for in" loop
This must be trivial, but I must have missed something.I have a dataframe (test1) and want to round all the columns listed in list of columns (col_list)here is the code I am running:col_list = ['measure1', 'measure2', 'measure3'] for i in col_list:...
- 3273 Views
- 2 replies
- 3 kudos
Latest Reply
- 9926 Views
- 3 replies
- 2 kudos
Resolved! Issue in reading parquet file in pyspark databricks.
One of the source systems generates from time to time a parquet file which is only 220kb in size.But reading it fails."java.io.IOException: Could not read or convert schema for file: 1-2022-00-51-56.parquetCaused by: org.apache.spark.sql.AnalysisExce...
- 9926 Views
- 3 replies
- 2 kudos
Latest Reply
- 2 kudos
@nafri A - Howdy! My name is Piper, and I'm a community moderator for Databricks. Would you be happy to mark @Hubert Dudek's answer as best if it solved the problem? That will help other members find the answer more quickly. Thanks
- 2 kudos
- 30482 Views
- 4 replies
- 4 kudos
Resolved! Spark is not able to resolve the columns correctly when joins data frames
Hello all, I m using pyspark ( python 3.8) over spark3.0 on Databricks. When running this DataFrame join:next_df = days_currencies_matrix.alias('a').join( data_to_merge.alias('b') , [ days_currencies_matrix.dt == data_to_merge.RATE_DATE, days...
- 30482 Views
- 4 replies
- 4 kudos
Latest Reply
- 4 kudos
@Alessio Palma - Howdy! My name is Piper, and I'm a moderator for the community. Would you be happy to mark whichever answer solved your issue so other members may find the solution more quickly?
- 4 kudos