Databricks Community

Ramakrishnan83 · 04-15-2024

Team,I did setup a SQL Warehouse Cluster to support request from Mobile devices through REST API. I read through the documentation of concurrent query limit which is 10. But in my scenario I had 5 small clusters and the query monitoring indicated the...

Ramakrishnan83 · 03-18-2024

Team ,I am trying understand how the parquet files and JSON under the delta log folder stores the data behind the scenesTable Creation:from delta.tables import *DeltaTable.create(spark) \.tableName("employee") \.addColumn("id", "INT") \.addColumn("na...

Ramakrishnan83 · 02-06-2024

Team,Initially our team created the databases with the environment name appended. Ex: cust_dev, cust_qa, cust_prod.I am looking to standardize the database name as consistent name across environments. I want to rename to "cust". All of my tables are ...

Ramakrishnan83 · 02-04-2024

Hi team,I am running a weekly purge process from databricks notebooks that cleans up chunk of records from my tables used for audit purposes. Tables are external tables. I need clarification on below items1.Should I need to run Optimize and Vacuum c...

Ramakrishnan83 · 12-20-2023

Team,I need to migrate only specific notebooks which has changes committed to be pulled from one repo to another repoEnvironment/Repo Setup:Master -> Dev -> Feature Branch -> Developer commits the code in Feature Branch -> Dev has the changes from D...

Ramakrishnan83 · 03-12-2024

Hi @Sid1805 ,I am in the same situation as you. I am looking for guidance how to setup the authentication for REST API to connect with Databricks SQL. I started with PAT and used the user id as "token" and pwd as PAT Token to connect with Databricks...

Ramakrishnan83 · 02-06-2024

Hi,I don't see SET Schema as keyword in Alter table statement. https://docs.databricks.com/en/sql/language-manual/sql-ref-syntax-ddl-alter-table.html . I tried your statement. alter table ****.table_row_count SET SCHEMA ****.table_row_countParseExcep...

Ramakrishnan83 · 02-06-2024

I would recommend using managed tables for table backups and tables used for data processing in the notebooks that can be dropped at the end of the process or kind of staging table. I have not explored how to copy a managed table from Dev to QA Envir...

Databricks Community

User Stats

User Activity

Intermittent SQL Failure on Databricks SQL Warehouse

Understanding Spark Architecture during Table Creation

Renaming the database Name in Databricks

Optimize and Vaccum Command

How to Migrate specific notebooks from one Azure Repo to another Azure Repo

Re: Azure Databricks SQL Execution API Authentication

Re: Renaming the database Name in Databricks

Re: Experiences using managed tables