I have my Databricks SQL Endpoint running, I have created many tables there using AWS S3 Delta. Now I want to query the Databricks SQL Endpoint from NodeJs. So It is possible ?I tried to find and researched a lot but didn't get any useful tutorial fo...
Hi @Krishna Wahi​, Just a friendly follow-up. Do you still need help, or @Bilal Aslam​'s response help you to find the solution? Please let us know.
Can you please share the Databricks JDBC Driver on Maven Central ?I see it available on : https://databricks.com/spark/jdbc-drivers-download . However I can’t find it on Maven Central to make use of it in automated tests connecting to Databricks infr...
The problem:We have a dataframe which is based on the query:SELECT *
FROM Very_Big_TableThis table returns over 4 GB of data, and when we try to push the data to Power BI we get the error message:ODBC: ERROR [HY000] [Microsoft][Hardy] (35) Error from...
Hey @Hila Galapo​ Hope everything is going good. Just wanted to check in if you were able to resolve your issue or do you need more help? We'd love to hear from you.Thanks!
Hi All,I plan to store the schema of my table in AWS glue schema registry. Is there any simple way to use it in Databricks Autoloader?My goal is to build a data pipeline with Autoloader for schema validation.
Hey there @Herry Ramli​ Hope all is well!Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution? Else please let us know if you need more help. We'd love to hear from you.Thanks!
Hi @Pierre MASSE​ Thank you so much for getting back to us. It's really great of you to send in the solution and mark the answer as best. We really appreciate your time.Wish you a great Databricks journey ahead!
From Databricks runtime 10.5 you can get metadata using the hidden _metadata column. Currently, the column contains input files information (file_path, file_name, file_size and file_modification_time)
I have created a job that contains a notebook that reads a file from Azure Storage. The file-name contains the date of when the file was transferred to the storage. A new file arrives every Monday, and the read-job is scheduled to run every Monday. I...
Hi @Karolin Albinsson​ , Just a friendly follow-up. Do you still need help, or @Hubert Dudek (Customer)​ 's response help you to find the solution? Please let us know.
Hi,I am trying to enable the Databricks SQL environment from the Community Edition workspace (using left menu pane). However, the options I see in the dropdown menu are or Data Science & Eng and Machine Learning workspaces/environments. Is Databricks...
+1 on what Andrew said. We are not currently planning on enabling Databricks SQL in Community. Please use a trial account on one of the supported clouds.
I am reading the data from a folder /mnt/lake/customer where mnt/lake is the mount path referring to ADLS Gen 2, Now I would like to rename a folder from /mnt/lake/customer to /mnt/lake/customeraddress without copying the data from one folder to ano...
I am new to databricks platform. what is the best way to keep data persistent so that once I restart the cluster I don't need to run all the codes again?So that I can simply continue developing my notebook with the cached data.I have created many dat...
Hey there @Vivek Ranjan​ Hope you are doing great!Just wanted to check in if you were able to resolve your issue or do you need more help? We'd love to hear from you.Thanks!
Hello guys, I'm having an issue when trying to get a row values from spark data frame.I have a DF with index column, and i need to be able to return a row based on index in fastest way possible .I tried to partitionBy index column, optimize with zor...
Using DBR 10 or later and I’m getting an error when running the following querySELECT * FROM delta.`s3://some_path`getting org.apache.spark.SparkException: Unable to fetch tables of db deltaFor 3.2.0+ they recommend reading like this:CREATE TEMPORAR...
Hi All,We are facing one un-usual issue while loading data into Delta table using Spark SQL. We have one delta table which have around 135 columns and also having PARTITIONED BY. For this trying to load 15 millions of data volume but its not loading ...
@Kaniz Fatma​ @Parker Temple​ I found an root cause its because of serialization. we are using UDF to drive an column on dataframe, when we are trying to load data into delta table or write data into parquet file we are facing serialization issue ....
