topic upload file/table to delta table using CLI in Data Engineering

upload file/table to delta table using CLI

pshuk — Mon, 13 May 2024 15:09:13 GMT

Hi,

I am using CLI to transfer local files to Databricks Volume. At the end of my upload, I want to create a meta table (storing file name, location, and some other information) and have it as a table on databricks Volume. I am not sure how to create a table using CLI. can anyone point me in the right direction.

TIA.

Re: upload file/table to delta table using CLI

Ayushi_Suthar — Thu, 16 May 2024 20:08:09 GMT

Hi @pshuk , Greetings!

We understand that you are looking for a CLI command to create a Table but at this moment Databricks doesn't support CLI command to create the table but you can use SQL Execution API -https://docs.databricks.com/api/workspace/statementexecution/executestatement to run the DDL statements against a SQL warehouse, here is an example:

POST https://<DATABRICKS INSTANCE>/api/2.0/sql/statements Body - { "warehouse_id": "warehouse-ID", "statement": "CREATE TABLE <Catalog Name><Schema Name><Table Name>" } Authorisation - < PAT Token >

Note - While executing this API, the SQL warehouse should be in a running state or else the cmd will go into a PENDING state.

Please let me know if this helps and leave a like if you find this information useful. Follow-ups are appreciated.
Kudos
Ayushi

Re: upload file/table to delta table using CLI

pshuk — Fri, 17 May 2024 16:29:57 GMT

Thanks Ayushi. Quick question: How do I put this in python code?