- 1526 Views
- 1 replies
- 0 kudos
How to get the table reference of cached query results?
Databricks caches query results for 24 hours. I would like to access the query results as if it is a table so that I can post process it. For example, do another query against it. The ask is similar to Snowflake RESULT_SCAN https://docs.snowflake.com...
- 1526 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @MAN LI​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 0 kudos
- 8217 Views
- 6 replies
- 3 kudos
Azure Storage Account inside Databricks cannot enable soft-delete.
Hello,When deploying any databricks workspace inside Azure, the storage account inside the databricks managed resource group is unable to apply any changes, including enabling soft-delete. Is there a way to enable it?Best regards,Jon
- 8217 Views
- 6 replies
- 3 kudos
- 2302 Views
- 1 replies
- 2 kudos
Take advantage of the Data + AI Summit Virtual Experience next week! Data + AI Summit is just a few days away! With data professionals from 155+ count...
Take advantage of the Data + AI Summit Virtual Experience next week!Data + AI Summit is just a few days away! With data professionals from 155+ countries already registered, this is truly the premier event for the global data, analytics and AI commun...
- 2302 Views
- 1 replies
- 2 kudos
- 2 kudos
Thank you for informing highlights of the upcoming sessions.
- 2 kudos
- 1853 Views
- 1 replies
- 0 kudos
- 1853 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Muhammad Umer​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 0 kudos
- 3641 Views
- 2 replies
- 4 kudos
docs.databricks.com
Databricks new Release : Full-page workspace browser includes ReposDatabricks plans to enable the full-page workspace browser experience that unifies Workspace and Repos by default. You’ll browse content in Databricks Repos alongside your workspace c...
- 3641 Views
- 2 replies
- 4 kudos
- 7466 Views
- 4 replies
- 2 kudos
Delta file question
Hi! Im using Autoloader to ingest Binary files into delta format. I have 7 binary files but delta generate 3 files and the format is part-0000, part-0001... Why generate this files with format part-000...
- 7466 Views
- 4 replies
- 2 kudos
- 2 kudos
Hi @Alejandro Piury Pinzón​ We haven't heard from you since the last response from @Lakshay Goel​ r​, and I was checking back to see if her suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be hel...
- 2 kudos
- 3578 Views
- 1 replies
- 1 kudos
Issues loading files csv files that contain BOM (Byte Order Mark) character
I keep getting and error when creating dataframe or steam from certain CSV files where the header contains BOM (Byte Order Mark) character This is the error message:AnalysisException: [RequestId=e09c7c8d-2399-4d6a-84ae-216e6a9f8f6e ErrorClass=INVALI...
- 3578 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @mohamed harraz​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 1 kudos
- 2138 Views
- 1 replies
- 0 kudos
CDM connector for Spark can't connect to Azure storage account
Hello,I'm trying to use the CDM connector for Spark, but I can't connecto to the Azure storage account when using the connector. I mounted a container of storage account with a SAS-token. When I'm trying to read CDM data from a (mounted) storage acco...
- 2138 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Martijn de Bruijn​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 0 kudos
- 4735 Views
- 2 replies
- 1 kudos
about .crc file in delta transaction log
why .crc file will create along with delta log files
- 4735 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Lakshmi devi​ , crc file is basically a checksum file that contains the stats for the respective version file. It is used for snapshot verification in the backend.
- 1 kudos
- 8545 Views
- 2 replies
- 1 kudos
Resolved! Constructor public org.apache.spark.SparkConf(boolean) is not whitelisted.
My code:from great_expectations.datasource import SparkDFDatasourcefrom pyspark.sql import SparkSessionsession_name = 'mk_spark_session'spark = SparkSession.builder.appName(session_name).getOrCreate()datasource = SparkDFDatasource(spark)query = "SELE...
- 8545 Views
- 2 replies
- 1 kudos
- 1 kudos
Are you using a high concurrency cluster? if you do, please try to run this code on standard cluster
- 1 kudos
- 8939 Views
- 2 replies
- 2 kudos
getContext() in dbutils.notebook not working in DBR 11.2 10.4 LTS Shared ModeIt's also working in no isolation Mode in DBR 11.2
We are trying to fetch notebook context in our Job logging workflow. current_context = dbutils.notebook.entry_point.getDbutils().notebook().getContext().toJson() We were able to access this in DBR 10.4 custom mode but in DBR 10.4 & 11.2 (Shared M...
- 8939 Views
- 2 replies
- 2 kudos
- 2 kudos
See also: https://community.databricks.com/s/question/0D58Y00009t95NHSAY/unity-catalog-shared-access-mode-dbutilsnotebookentrypointgetcontext-not-whitelisted
- 2 kudos
- 16939 Views
- 6 replies
- 0 kudos
Resolved! How to copy the content of a repos, in the workspace/shared, automatically, daily?
How to copy the content of a repos, in the workspace/shared, automatically, daily?Purpose here is to bring some notebooks, in shared, available to all workspace users, without requesting users to use REPOS.
- 16939 Views
- 6 replies
- 0 kudos
- 0 kudos
Alternatively, you can schedule a notebook with the following cell:%sh cp -r /Workspace/Repos/username/repo_name /Workspace/SharedPlease note that you'll have to update (i.e. pull) the repo manually if you've updated it somewhere else (although you c...
- 0 kudos
- 2441 Views
- 0 replies
- 1 kudos
Connect to warehouse vs connect to compute/cluster
When I create a warehouse, there are connection details which I was able to successfully use, but there are also connection details when I navigate in the console to Compute->configuration->Advanced options-> JDBC/ODBC What is the difference between ...
- 2441 Views
- 0 replies
- 1 kudos
- 8937 Views
- 3 replies
- 0 kudos
Linear Regression HELP! Pickle + Broadcast Variable Error
Hi there,I need some help with this example. We're trying to create a linearRegression model that can parallelize for thousands of symbols per date. When we run this we get a picklingError Any suggestions would be much appreciated!KError:PicklingErro...
- 8937 Views
- 3 replies
- 0 kudos
- 15447 Views
- 3 replies
- 9 kudos
Resolved! Delta save timestamp as timestamp with time zone
Hello! I have the following problem. I want to save the delta table and that table contains timestamp columns, but when I try to write that table with spark the timestamp columns become timestamp with the time zone. This is a problem in my case becau...
- 15447 Views
- 3 replies
- 9 kudos
- 9 kudos
Hi @Hubert Dudek​ ,When you have time, I describe my problem. Can you please check?
- 9 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 554 | |
| 349 | |
| 287 |
