Databricks Community

Hi @Abhay_1002,  Using --py-files Argument: When submitting a Spark application, you can use the --py-files argument to add Python files (including .zip or .egg archives) to be distributed with your application1. However, this approach is typical...

Hi @Kaniz_Fatma! Sorry for a long wait...The problem is not the columns or the data itself, the UTF-8 option for csv is working fine. The issue is with table_names not being compatible it seems. If I run the query through Auto Loader outside DLT and ...

Hi @mderela, When dealing with Databricks bundles in a multi-environment setup, there are some best practices you can follow to ensure smooth execution and maintainable code. Let’s explore a couple of recommendations: Parameterization and Configu...

Hi, I have not tried it but based on the doc you have to go by this approach. ./environment/bin/pythonmust be replaced with the correct path. import os from pyspark.sql import SparkSession os.environ['PYSPARK_PYTHON'] = "./environment/bin/python" sp...

Hi @Nagarathna , I just tried it on a shared cluster and did not face any issue. What is the exact error that you are facing? Complete stacktrace might help. Just to confirm are you accessing the "/dbfs/mnt/datalake/input.json" from the same workspac...

I think we should raise a Request/Product Feedback. Not sure if it would be Databricks that would own it or Microsoft but you may submit feedback for Databricks here - https://docs.databricks.com/en/resources/ideas.html  

Hi @MOUNIKASIMHADRI ,   Workspace admins get ANY FILE granted by default. They can explicitly grant it to non-admin users. Hence as suggested in the kb,  GRANT SELECT ON ANY FILE TO `<user@domain-name>`  

Hidbx_687_3__1b3Q, Actually, I have seen impersonation, is this something that you are looking for? https://docs.gcp.databricks.com/en/dev-tools/google-id-auth.html#step-5-impersonate-the-google-cloud-service-account

Hi AlexG, I tested with the table content containing null and with empty data and it works as expected in the download option too. Here is an eg: CREATE TABLE my_table_null_test1 ( id INT, name STRING ); INSERT INTO my_table_null_test1 (id, name)...

Hello @DataBricks_Use1 , It would great if you could add the entire stack trace, as Jose mentioned. But there should be a "Caused by:" section below which would give you an idea of what's the reason for this failure and then you can work on that.  fo...

Hi Phani1, It would be a use case specific answer, so if it is possible I would suggest to work with the Solution Architect on this or share some more insights for a better guidance. When I say that, I just would want to understand would we really ne...

Hi @Koa, You’ve encountered a security concern related to Databricks and handling JWT tokens within notebooks.  Dashboard State Persistence: When you share a dashboard with another user (in this case, UserA), any updates made by that user will re...

Hi @Ruby8376 - can you please review the similar posts where the resolution is provided  https://community.databricks.com/t5/warehousing-analytics/databricks-sql-restful-api-to-query-delta-table/td-p/8617 https://www.databricks.com/blog/2023/03/07/da...

Hi @vbvasa,  The error message indicates that a GDAL API version must be specified. You can address this by providing a path to gdal-config using a GDAL_CONFIG environment variable or by using a GDAL_VERSION environment variable1.To set the GDAL_CONF...

Hi @ChristopherQ1,  OData (Open Data Protocol) is a standard for building and consuming RESTful APIs. It provides a consistent way to expose and consume data over the web.While OData can be used for data integration, it’s essential to evaluate whethe...