- 7507 Views
- 6 replies
- 3 kudos
Azure Storage Account inside Databricks cannot enable soft-delete.
Hello,When deploying any databricks workspace inside Azure, the storage account inside the databricks managed resource group is unable to apply any changes, including enabling soft-delete. Is there a way to enable it?Best regards,Jon
- 7507 Views
- 6 replies
- 3 kudos
- 2085 Views
- 1 replies
- 2 kudos
Take advantage of the Data + AI Summit Virtual Experience next week! Data + AI Summit is just a few days away! With data professionals from 155+ count...
Take advantage of the Data + AI Summit Virtual Experience next week!Data + AI Summit is just a few days away! With data professionals from 155+ countries already registered, this is truly the premier event for the global data, analytics and AI commun...
- 2085 Views
- 1 replies
- 2 kudos
- 2 kudos
Thank you for informing highlights of the upcoming sessions.
- 2 kudos
- 1746 Views
- 1 replies
- 0 kudos
- 1746 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Muhammad Umer​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 0 kudos
- 3383 Views
- 2 replies
- 4 kudos
docs.databricks.com
Databricks new Release : Full-page workspace browser includes ReposDatabricks plans to enable the full-page workspace browser experience that unifies Workspace and Repos by default. You’ll browse content in Databricks Repos alongside your workspace c...
- 3383 Views
- 2 replies
- 4 kudos
- 6570 Views
- 4 replies
- 2 kudos
Delta file question
Hi! Im using Autoloader to ingest Binary files into delta format. I have 7 binary files but delta generate 3 files and the format is part-0000, part-0001... Why generate this files with format part-000...
- 6570 Views
- 4 replies
- 2 kudos
- 2 kudos
Hi @Alejandro Piury Pinzón​ We haven't heard from you since the last response from @Lakshay Goel​ r​, and I was checking back to see if her suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be hel...
- 2 kudos
- 3031 Views
- 1 replies
- 1 kudos
Issues loading files csv files that contain BOM (Byte Order Mark) character
I keep getting and error when creating dataframe or steam from certain CSV files where the header contains BOM (Byte Order Mark) character This is the error message:AnalysisException: [RequestId=e09c7c8d-2399-4d6a-84ae-216e6a9f8f6e ErrorClass=INVALI...
- 3031 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @mohamed harraz​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 1 kudos
- 2008 Views
- 1 replies
- 0 kudos
CDM connector for Spark can't connect to Azure storage account
Hello,I'm trying to use the CDM connector for Spark, but I can't connecto to the Azure storage account when using the connector. I mounted a container of storage account with a SAS-token. When I'm trying to read CDM data from a (mounted) storage acco...
- 2008 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Martijn de Bruijn​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 0 kudos
- 4376 Views
- 2 replies
- 1 kudos
about .crc file in delta transaction log
why .crc file will create along with delta log files
- 4376 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Lakshmi devi​ , crc file is basically a checksum file that contains the stats for the respective version file. It is used for snapshot verification in the backend.
- 1 kudos
- 8018 Views
- 2 replies
- 1 kudos
Resolved! Constructor public org.apache.spark.SparkConf(boolean) is not whitelisted.
My code:from great_expectations.datasource import SparkDFDatasourcefrom pyspark.sql import SparkSessionsession_name = 'mk_spark_session'spark = SparkSession.builder.appName(session_name).getOrCreate()datasource = SparkDFDatasource(spark)query = "SELE...
- 8018 Views
- 2 replies
- 1 kudos
- 1 kudos
Are you using a high concurrency cluster? if you do, please try to run this code on standard cluster
- 1 kudos
- 7904 Views
- 2 replies
- 2 kudos
getContext() in dbutils.notebook not working in DBR 11.2 10.4 LTS Shared ModeIt's also working in no isolation Mode in DBR 11.2
We are trying to fetch notebook context in our Job logging workflow. current_context = dbutils.notebook.entry_point.getDbutils().notebook().getContext().toJson() We were able to access this in DBR 10.4 custom mode but in DBR 10.4 & 11.2 (Shared M...
- 7904 Views
- 2 replies
- 2 kudos
- 2 kudos
See also: https://community.databricks.com/s/question/0D58Y00009t95NHSAY/unity-catalog-shared-access-mode-dbutilsnotebookentrypointgetcontext-not-whitelisted
- 2 kudos
- 15907 Views
- 6 replies
- 0 kudos
Resolved! How to copy the content of a repos, in the workspace/shared, automatically, daily?
How to copy the content of a repos, in the workspace/shared, automatically, daily?Purpose here is to bring some notebooks, in shared, available to all workspace users, without requesting users to use REPOS.
- 15907 Views
- 6 replies
- 0 kudos
- 0 kudos
Alternatively, you can schedule a notebook with the following cell:%sh cp -r /Workspace/Repos/username/repo_name /Workspace/SharedPlease note that you'll have to update (i.e. pull) the repo manually if you've updated it somewhere else (although you c...
- 0 kudos
- 2296 Views
- 0 replies
- 1 kudos
Connect to warehouse vs connect to compute/cluster
When I create a warehouse, there are connection details which I was able to successfully use, but there are also connection details when I navigate in the console to Compute->configuration->Advanced options-> JDBC/ODBC What is the difference between ...
- 2296 Views
- 0 replies
- 1 kudos
- 8585 Views
- 3 replies
- 0 kudos
Linear Regression HELP! Pickle + Broadcast Variable Error
Hi there,I need some help with this example. We're trying to create a linearRegression model that can parallelize for thousands of symbols per date. When we run this we get a picklingError Any suggestions would be much appreciated!KError:PicklingErro...
- 8585 Views
- 3 replies
- 0 kudos
- 14542 Views
- 3 replies
- 9 kudos
Resolved! Delta save timestamp as timestamp with time zone
Hello! I have the following problem. I want to save the delta table and that table contains timestamp columns, but when I try to write that table with spark the timestamp columns become timestamp with the time zone. This is a problem in my case becau...
- 14542 Views
- 3 replies
- 9 kudos
- 9 kudos
Hi @Hubert Dudek​ ,When you have time, I describe my problem. Can you please check?
- 9 kudos
- 3801 Views
- 2 replies
- 0 kudos
Running a K-means (.fit) gives error:Params must be either a param map or a list/tuple of param maps but got %s." % type(params)
am running a k-means algorithm. My feature are DoubleType and have no nulls, but I get : raise TypeError("Params must be either a param map or a list/tuple of param maps but got %s." % type(params). Anyone have any idea how to solve this?File /datab...
- 3801 Views
- 2 replies
- 0 kudos
- 0 kudos
I found the answer just by trying several things, although I do not understand exactly what the problem was. All I had to do was to cache the input data before fitting the model:assemble=VectorAssembler(inputCols=columns_input, outputCol='features')...
- 0 kudos
| User | Count |
|---|---|
| 1619 | |
| 790 | |
| 486 | |
| 349 | |
| 287 |
