Databricks Community

ashraf1395 · 09-05-2024

We have to deliver a Databricks Finops Assessment project. I am trying to write a proposal for it. I haven't done one before. I have created a general process of how the assessment will look like and then restructured it using gpt.Plz give your feedb...

ashraf1395 · 08-21-2024

I dont have the complete context of the issue.But Here it is what I know, a friend of mine facing this""I am fetching data from Oracle data in databricks using python.But every time i do it the schema gets changesso if the column is of type decimal f...

ashraf1395 · 08-14-2024

So previously I created source connections from looker with Databricks using my personal access token.I followed this databricks docs. https://docs.databricks.com/en/partners/bi/looker-studio.htmlBut from 10 July, I think basic authentication has bee...

ashraf1395 · 07-19-2024

Hi there,We are trying to setup a infra that ingest data from MySQL hosted on awa EC2 instance with pyspark and azure databricks and dump to the adls storage.Since databases has public accessibility disabled and how can I interact with MySQL from azu...

ashraf1395 · 07-16-2024

Hi there, This was my previous approach - I had a databricks notebook with a streaming table bronze level reading data from volumes which created a 2 downstream tables.- 1st A a materialised view gold level, another a table for storing ingestion_meta...

ashraf1395 · 07-19-2024

I used this approach - Querying the materialised view using databricks serverless SQL endpoint by connecting it with SQL connect. Its working right now. If I face any issues, I will write it into a normal table and delta share it.Thanks for your repl...

ashraf1395 · 07-12-2024

Hi there Team, I have completed this course - Data Engineering with Databricks on 12th July 14:00 IST. (within 10-24 July).So Am I now eligible for Databricks Data Engineering professional voucher.or do I need to complete Advanced Data Engineering wi...

ashraf1395 · 06-12-2024

Hi there @Tiwarisk,if this is the major issue @Tiwarisk wrote:I am writing a file using this but the data type of columns get changed while reading. You can explicitly specify your table schema like this from pyspark.sql.types import StructType, Stru...

ashraf1395 · 06-11-2024

Hi there @maskepravin02,We have once implemented this approach of two reading two different hive metasores, but it was not on AWS and GCP, maybe the docs can help.Though it is not recommended The best approach is to create separate spark applications...

ashraf1395 · 05-07-2024

I had another question. Though not related to this thread.Do databricks has any plan for startups, like they have normal free trial

Databricks Community

User Stats

User Activity

Databricks Finops Assessment

Schema issue while fetching data from oracle

Authentication Issue while connecting to Databricks using Looker Studio

Querying Mysql db from Azure databricks where public access is disabled

Reading a materialised view locally or using databricks api

Re: Reading a materialised view locally or using databricks api

Re: Databricks Learning Festival (Virtual): 10 July - 24 July 2024

Re: How can I preserve the data type of the delta tables while writing to Azure blob storage ?

Re: How can we connect to 2 different hive spark.hadoop.hive.metastore.uris

Re: Starting Serverless sql cluster on GCP