Databricks Community

Hi Guys, You're right the problem is with the child notebook.All my notebooks are failing at this point. I can't seem to be wining with solving this error

@Janga Reddy​ It's really hard to find a solution here without further investigating PBI Gateway performance/Data models etc.If Databricks completed the session in 9 mins then I assume that the issue could be with the performance of PBI datasets.

I would use a data factory to copy 4TB files as it has gigantic throughput. After completing a copy of everything, I would register as a table in the new metastore.

Hi @Husain Sadriwala​ ,Just a friendly follow-up. Did you receive your badge yet? please let us know if you still need help.

Question where do you want to download it to? If to cloud location, use regular DataFrameWriter. You can install, for example, Azure Storage Explorer on your computer. Some cloud storage you can even mount in your system as a folder or network share.

Hi, You can pass {job_id}} and {{run_id}} in Job arguments and print that information and save into wherever it is neededplease find below the documentation for the same:https://docs.databricks.com/data-engineering/jobs/jobs.html#task-parameter-varia...

Hi The issue was still persistent. We are trying to solve this by using docker image with preinstalled Selenium driver and chrome browser.RegardsDharmin

HI @Fred Foucart​ ,The above code looks good to me. Can you try with below code as well.spark.read\  .format("jdbc") \  .option("url", f"jdbc:postgresql://{host}/{database}") \  .option("driver", "org.postgresql.Driver") \  .option("user", username) ...