Databricks Community

balajij8

Data Engineering 1.0 was largely about integrating, moving, consolidating massive data to support enterprise decisions. Enterprises built ETL pipelines, data warehouses, data lakes and orchestration platforms to bring data together reliably. Data Eng...

balajij8 · 06-30-2026

Real time capabilities in healthcare is a critical factor in care outcomes and operational efficiency. From streaming continuous vitals from IoT-enabled care monitors to routing instantaneous telemetry from wearable medical devices, modern healthcare...

balajij8 · 06-25-2026

Modern Enterprise Healthcare Lake bases have fundamentally transformed care data operations by seamlessly unifying high concurrency transactional workloads such as electronic records (EMR) syncing, streaming care vitals and persistent memory for gene...

balajij8 · 06-18-2026

The friction between modern operational edge applications and core analytical data systems is a massive architectural bottleneck in healthcare organizations. Building patient-facing mobile applications, synchronizing remote patient monitoring (RPM) w...

balajij8 · 06-11-2026

Healthcare organizations possess enormous volumes of care, operational and payer related data. Every care interaction generates information across care notes, diagnosis records, medication histories, imaging reports, claims systems and payer policies...

balajij8

@faruk Handling Oracle NUMBER in the ingestion pipelineThe Oracle JDBC driver maps an NUMBER (no precision, no scale) to DecimalType(38,10) as it must pick a fixed scale for a type system that has no equivalent to Oracle's unconstrained decimal. You ...

balajij8

@keshavmonga22 You can also check by writing logs to /tmp then upload using the SDK's Files API (w.files.upload() or w.files.upload_from()) as it generally bypasses FUSE entirely - uploads happen via REST calls directly to the object storage backend ...

balajij8

@Kushal_2612 The slowness you are seeing is generally when an external service submits queries against standard workspace compute or interactive clusters, which carry heavy execution overhead. For querying Unity Catalog Delta tables from an external ...

balajij8

@gowri_databrick Auto Loader automatically detects and handles new columns when ingesting files. When your source adds the email column, Auto Loader's checkpoint tracks the schema change and can either rescue unknown columns into _rescued_data (safe ...

balajij8

@Islam_hotiYou can keep separate workspaces per environment with one catalog namespace per env (prod.sales, staging.sales, dev.sales) managed through bundles DAB. Generate synthetic data for unit tests and use an anonymized subset of prod to the dev ...

Databricks Community

User Stats

User Activity

Data Engineering 3.0 - From Trusted Data Products to Context Layers

Real Time Healthcare Analytics with Databricks Zerobus and Lakehouse RT via Reyden

Fortifying Enterprise Healthcare Databricks Lakebase with the Security Triad

Zero Code REST Integration for Modern HealthCare Vitals via Databricks Lakebase Data API

HealthCare Prior Authorizations with Databricks Lakebase Vector Search

Re: Oracle NUMBER → DecimalType(38,10) on ingestion, and silver layer best practices

Re: shutil.copy from /local_disk0 to Unity Catalog Volume hangs for hours — recommended pattern for

Re: Agent outside databricks communication with databricks delta table

Re: Handling New Columns in a Databricks Data Pipeline

Re: How are you separating dev, staging and prod in Unity Catalog without duplicating everything?