- 2108 Views
- 2 replies
- 0 kudos
Resolved! databricks certified data engineer associate exam got suspended
Hi TeamI have given Databricks certified data engineer associate exam, but it got suspended before completion. I showed my exam area to proctor accordingly. I got message by proctor that my exam got suspended due to your failure to meet certain envir...
- 2108 Views
- 2 replies
- 0 kudos
- 0 kudos
@Divs23 The support team is working on your ticket now, please be sure to check your spam folder.
- 0 kudos
- 1555 Views
- 1 replies
- 0 kudos
Resolved! Databricks Certification exam Suspended. Need Assistance
I was taking online exam for Databricks Certified Data Analyst Associate on 06-Oct-2023 1:45PM. In between, they paused it and wanted to survey my whole room which they did, told me to clear the table of water bottle and laptop charger and then asked...
- 1555 Views
- 1 replies
- 0 kudos
- 0 kudos
@Sanjay96m Thank you for your patience, the support team is working through support tickets and will reach out to you shortlly.
- 0 kudos
- 1995 Views
- 0 replies
- 0 kudos
Change Data Feed - Selected Columns
Is there a way to configure CDF in delta table but only for selected columns and not all?
- 1995 Views
- 0 replies
- 0 kudos
- 3284 Views
- 1 replies
- 1 kudos
Resolved! Databricks 13.3 Runtime Unable to use Python OS library to access mounted GCP bucket
In Databricks 12.2, this Python code block lists out the contents of our ad_hoc folder in our mounted GCP bucket. import os os.listdir('/dbfs/mnt/hlm/ad_hoc/') For some reason in 13.3 this same code block throws a " No such file or directory" error.
- 3284 Views
- 1 replies
- 1 kudos
- 1 kudos
I've discovered the cause of this issue. The path is functional, but what I'm actually running into is a restriction due to the cluster set to "Shared" access mode:Cannot use R, RDD APIs, or clients that directly read the data from cloud storage, suc...
- 1 kudos
- 1740 Views
- 0 replies
- 0 kudos
ACID properties in delta?
How are locks maintained within a Delta Lake? For instance, lets say there are 2 simple tables, customer_details and say orders. Lets say I am running a job that will say insert an order in the orders table for say $100 for a specific customerId, it ...
- 1740 Views
- 0 replies
- 0 kudos
- 7779 Views
- 4 replies
- 1 kudos
Resolved! Read data from Azure SQL DB
I am trying to read data into a dataframe from Azure SQL DB, using jdbc. Here is the code I am using.driver = "com.microsoft.sqlserver.jdbc.SQLServerDriver" database_host = "server.database.windows.net" database_port = "1433" database_name = "dat...
- 7779 Views
- 4 replies
- 1 kudos
- 1588 Views
- 1 replies
- 1 kudos
Foreign catalogs
With the introduction of the Unity Catalog in databricks, many of us have become familiar with creating catalogs. However, did you know that the Unity Catalog also allows you to create foreign catalogs? You can register databases from the following s...
- 1588 Views
- 1 replies
- 1 kudos
- 1 kudos
Thank you for sharing @Hubert-Dudek !!!
- 1 kudos
- 1371 Views
- 1 replies
- 3 kudos
row-level concurrency
With the introduction of Databricks Runtime 14, you can now enable row-level concurrency using these simple techniques!
- 1371 Views
- 1 replies
- 3 kudos
- 3 kudos
Thank you for sharing this @Hubert-Dudek
- 3 kudos
- 1169 Views
- 0 replies
- 0 kudos
Window functions in Change Data Feed
Hello!I am currently exploring the possibility of implementing incremental changes in our company's ETL pipeline and looking into Change Data Feed option. There are a couple of challenges I'm uncertain about.For instance, we have a piece of logic lik...
- 1169 Views
- 0 replies
- 0 kudos
- 13946 Views
- 2 replies
- 1 kudos
Resolved! External location + Failure to initialize configuration for storage account
Hello,I created a storage credential and an external location. Test is ok, I'm able to browse it from the portal. I have a notebook to create a table :%sqlCREATE OR REPLACE TABLE myschema.mytable( data1 string, data2 string)USING DELTA LOCATION "abf...
- 13946 Views
- 2 replies
- 1 kudos
- 1169 Views
- 0 replies
- 0 kudos
Debugging options if you are using streaming, RDDs and SparkContext?
Hi all,I've been trying to make use of some of the more recent tools for debugging in Databricks: pdb in the Databricks web interface with the variable explorer described in this article.I've also been trying to debug locally using the VSCode extensi...
- 1169 Views
- 0 replies
- 0 kudos
- 13144 Views
- 1 replies
- 1 kudos
[INCONSISTENT_BEHAVIOR_CROSS_VERSION.PARSE_DATETIME_BY_NEW_PARSER]
Hi all,I'm trying to join 2 views in SQL editor for some analysis. I get the following error:[INCONSISTENT_BEHAVIOR_CROSS_VERSION.PARSE_DATETIME_BY_NEW_PARSER] You may get a different result due to the upgrading to Spark >= 3.0: Fail to parse '22/12/...
- 13144 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi Kaniz, I found the equivalent SQL code for this but it didn't seem to store the operation past the execution. I.e I would run the code to configure settings, then run the troublesome code afterwards and still get the same result. The problem has b...
- 1 kudos
- 14237 Views
- 2 replies
- 1 kudos
Z-ordering optimization with multithreading
Hi, I am wondering if multithreading will help with the performance for z-ordering optimization on multiple delta tables.We are periodically doing optimization on thousands of tables and it easily takes a few days to finish the job. So we are looking...
- 14237 Views
- 2 replies
- 1 kudos
- 22196 Views
- 4 replies
- 5 kudos
Pyflake errors when using %run
I am using %run command to import shared resources for each of my processes. Because it was the most easy way to import my common libraries. However, in that way, pyflake can't resolve the dependencies quite well. And I end up working in code with ma...
- 22196 Views
- 4 replies
- 5 kudos
- 5 kudos
You could use something like flake8 and customize the rules in the .flake8 file or ignore specific lines with #noqa. https://flake8.pycqa.org/en/latest/user/configuration.html
- 5 kudos
- 7285 Views
- 0 replies
- 0 kudos
Pandas 2.x availability
Hi All,I am wondering if Pandas 2.x will be available soon or is it an available option to install.I have a small job I built to manipulate some strings from a database table when technically did the job, but doesn't scale with older versions of pan...
- 7285 Views
- 0 replies
- 0 kudos
| User | Count |
|---|---|
| 1613 | |
| 770 | |
| 349 | |
| 286 | |
| 253 |
