cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
ivanychev
Contributor II
since ‎04-19-2022
‎09-20-2024

User Stats

  • 32 Posts
  • 1 Solutions
  • 0 Kudos given
  • 9 Kudos received

User Activity

Delta table takes too long to write due to S3 full scan

08-21-2024
DBR 14.3, Spark 3.5.0. We use AWS Glue Metastore.On August 20th some of our pipelines started timing out during write to a Delta table. We're experiencing many hours of driver executing post commit hooks. We write dataframes to delta with `mode=overw...

Corrupted Python installation on Python restart on DBR 13.3

02-22-2024
Hey there, we're using DBR 13.3 (no Docker) as general purpose cluster and init the cluster using the following init script:```#!/usr/bin/env bashexport DEBIAN_FRONTEND=noninteractiveset -euxo pipefailif [[ $DB_IS_DRIVER = "TRUE" ]]; thenecho "I am d...

community.databricks.com doesn't open in Poland?

02-22-2024
I and 3 of my friends can't open https://community.databricks.com/ from Poland. The page returns 403. Different LTE and internet providers

Spark doesn't register executors when new workers are allocated

12-11-2023
Our pipelines sometimes get stuck (example).Some workers get decommissioned due to spot termination and then the new workers get added. However, after (1) Spark doesn't notice new executors: And I don't know why. I don't understand how to debug this,...

Mount Workspace to Docker container

08-23-2023
Is there a way to mount Workspace folder (WSFS) to the Docker container if I'm using the Databricks Container Services ofr running a general purpose cluster?If I create a cluster without a Docker image, the `!ls` command in Databricks notebook return...
View more