Databricks Community

Fred_F · 01-09-2023

Hi there,I've a batch process configured in a workflow which fails due to a jdbc timeout on a Postgres DB.I checked the JDBC connection configuration and it seems to work when I query a table and doing a df.show() in the process and it displays th...

Fred_F · 01-24-2023

Hi all,I finally apply the following workaround, adding a persist on the dataframe when fetching the data from DB: The data volume being small, it works. def get_table_from_db(self, table_name) -> pyspark.sql.DataFrame: df = self.spark.rea...

Fred_F · 01-11-2023

Hi,I tried all this configuration but still the same issue.We need to check the configuration at clod sql level.I keep you update on the solution we'll found.Thks

Fred_F · 01-09-2023

Hi @Rama Krishna N ,I use a class with its method get_table_from_db as follow:class ExtractData: def __init__(self, db: Enum, **kwargs): self.db_name = db.DB_NAME.value self.db = db self.kwargs = kwargs self.spa...

Databricks Community

User Stats

User Activity

JDBC connection timeout on workflow cluster

Re: JDBC connection timeout on workflow cluster

Re: JDBC connection timeout on workflow cluster

Re: JDBC connection timeout on workflow cluster