Data Engineering
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Filtering files for query

Hi Team,While writing my data to datalake table I am getting 'filtering files for query', it would be stuck at writingHow can I resolve this issue

Muhammed by New Contributor II
  • 10 replies
  • 0 kudos

Dynamic Number of Tasks in Databricks Workflow

Do Databricks workflows support creating a workflow with a dynamic number of tasks?For example, let's say we have a DAG like this:T1 ->    T2(1) ->             T2(2) ->              .....                 -> T3             T2(n-1) ->             T2(n)...

how can I learn DataBricks

I am currently enrolled in data engineering boot camp. We go over various technologies azure , pyspark , airflow , Hadoop ,nosql,SQL, python. But not over something like databricks. I am in contact with lots of recent graduates who landed a job. Almo...

billfoster by New Contributor II
  • 9 replies
  • 4 kudos

Is databricks capable of housing OLTP and OLAP?

Hi data experts.I currently have an OLTP (Azure SQL DB) that keeps data only for the past 14 days. We use Partition switching to achieve that and have an ETL (Azure data factory) process that feeds the Datawarehouse (Azure Synapse Analytics). My requ...