Databricks Community

Akshay_Petkar

I am preparing to migrate from EMR to Databricks and would like to know the best practices for this process. Is there a direct connector, such as a JDBC connector, available to facilitate the migration? Alternatively, would it be more effective to ex...

Akshay_Petkar

I have to migrate the data from Azure Synapse Analytics to Databricks. Could anyone share the different approaches to migrate data, and from those, which is the best approach to use?

Akshay_Petkar · 08-22-2024

I'm planning to connect SQL Server Management Studio (SSMS) with Databricks using Lakehouse Federation. I understand that there are some differences in the SQL dialects between SSMS and Databricks SQL. For instance, in SSMS, we use TOP 10 to limit th...

Akshay_Petkar · 07-26-2024

I'm encountering an issue while importing a .gz file containing JSON data into a Spark DataFrame in Databricks. The error indicates a column name conflict. Could you please advise on how to resolve this issue and handle duplicate column names during ...

Akshay_Petkar · 07-04-2024

When changes are made to a Databricks SQL table, a new version is created. If changes to the table are made using Spark or Python in a notebook and the table is overwritten, will a new version be created, or will it remain as version number 0?

Akshay_Petkar

Hi @TamD A materialized view looks for a permanent table. In your case, since the temporary view used to load the data is deleted after the session expires, the materialized view cannot find it when the session ends. For example, if you create a mate...

Akshay_Petkar

Hi @TamD You cannot create a materialized view on a temporary view because temporary views are session-based. Once the session expires, the temporary view will be dropped. To resolve this, you need to create either a regular view or a table, and then...

Akshay_Petkar · 05-27-2024

Hello @Raphael BalogoThank you for the detailed explanation. I am referring to the SQL Lakeview Dashboard, where we select visualizations on the canvas and add fields to them. I would like to know if there is a way to display the entire dataset in a ...

Databricks Community

User Stats

User Activity

Best Approach for Migrating from EMR to Databricks

Migrate Azure Synapse Analytics data to Databricks

SQL Differences When Using SSMS with Databricks Lakehouse Federation

Issue with Column Name Conflict While Importing .gz File into Spark DataFrame

Impact of Overwriting Databricks SQL Tables on Versioning

Re: SELECT from VIEW to CREATE a table or view

Re: SELECT from VIEW to CREATE a table or view

Re: Truncated Data on Lakeview Dashboard