Databricks Community

Yogasathyandrun

I'm prototyping a cluster cost / right-sizing advisor and wanted to get a reality-check from people running Databricks at real scale before I sink more time into it.The main thing I'm chasing is Photon fallback. Photon quietly drops to the JVM on uns...

Yogasathyandrun

The fact that print("Hello") eventually works but SELECT 1 never completes suggests the cluster may be running but not fully initialized for Spark workloads.A few things I’d check first:Cluster Event Log for any provisioning or startup errors.Spark U...

Yogasathyandrun

The labs referenced in the Data Engineer Learning Plan aren’t included in the free learning plan itself.Those hands-on labs are available through the Databricks Academy Labs subscription, which provides access to the lab-enabled versions of the cours...

Yogasathyandrun

Thank you for the recognition and for putting together these weekly roundups!I’ve learned a tremendous amount from the Databricks Community, and it’s great to be able to contribute back. Looking forward to continuing the discussions and learning from...

Yogasathyandrun

Based on the events you've shared, it does appear that the gateway is recognizing the configuration change (Tables removed: Gestiones) but is still attempting to process a previously initiated snapshot request for that table.A few things stand out:Th...

Yogasathyandrun

@Louis_Frolio — really appreciate this, it refocused the whole thing.Biggest takeaway: I'm flipping the framing to lead with Photon fallback / Photon ROI and treat cost + right-sizing as the baseline, since the system-tables dashboards and Overwatch...

Databricks Community

User Stats

User Activity

Detecting Photon fallback in-cluster + safe right-sizing from system tables

Re: Code on cluster runs idefinitely

Re: Need Labs Access for Data Engineering Learning Plan

Re: 🌟 Community Pulse: Your Weekly Roundup! June 15 – 21, 2026

Re: Lakeflow Connect - Pending ‘full refresh’ process that needs to be removed in gateway pipeline.

Re: Detecting Photon fallback in-cluster + safe right-sizing from system tables