Hi, Cat! I’m applying for a position at Databricks and was hoping to get some current Brickster insights. I’ve been wanting to join the company for a while!! Thanks in advance
I would like to create the directory develop under Repos as part of the script, then link it to github and update it? How can I do this?
Hi, Adam:Repos CLI does not have specific functionality to create directories in Databricks Repos. Please check the following doc for more information: https://docs.databricks.com/dev-tools/cli/repos-cli.htmlYou cold use run databricks workspace mkd...
Announcing a new portfolio of Generative AI learning offerings on Databricks AcademyToday, we launched new Generative AI, including LLMs, learning offerings for everyone from technical and business leaders to data practitioners, such as Data Scientis...
Would like to extract data like ticket info, resolve time, etc., from ServiceNow in databricks.Not finding much information in community and appreciate your guidance on the same.
ServiceNow offers API capabilities. You can consume the ServiceNow API within a Databricks notebook to extract data from ServiceNow. Following is a suggested prompt to use with ChatGPT for example python code to connect to ServiceNow's api. PROMPT: ...
Could you please guide on how to create the DLT pipeline that directly reads the data from jdbc.When I created the DLT pipeline it give me error at Setting up table, If I ran interactively in notebooks it run successfully, but in non interactive mode...
What you try do to is not possible.dlt uses autoloader, not jdbcno jars (dlt is sql/python only)I'd skip DLT for this scenario and use an ordinary notebook, nothing wrong with that.
Hi, somebody to help me how to connect a delta table with a web app? I search to a delta-rs library but I can't obtain to make the connection.
Without downloading the files directly every time, you have to create a sql warehouse cluster and connect to it via jdbc connection. This way you just use the requests library in python (or an equal one in another language like axios for javascript) ...
In a normal notebook I would save metadata to my Delta table using the following code:( df.write .format("delta") .mode("overwrite") .option("userMetadata", user_meta_data) .saveAsTable("my_table") )But I couldn't find online how c...
In Delta lab you can set up User MetaData so i will give you some tips from delta import DeltaTable# Create or load your Delta tabledelta_table = DeltaTable.forPath(spark, "path_to_delta_table")# Define your user metadata myccpayuser_meta_data = {"ke...
Hello everybody,I am currently trying to run some performance tests on queries in Databricks on Azure. For my tests, I am using a Classic SQL Warehouse in the SQL Editor. I have created two views that contain the same data but have different structur...
They are probably executing the same query plan now that you say it. And yes, restarting the warehouse does theoretically works but it isnt a nice solution.I guess I will do some restarting and build averages to have a good comparison for now
Hi Ayapanday, ​How can check soure path in databricks​​
Hi @Govardhana Reddy​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feed...
Hi There,I have been trying to create an external table on Azure Databricks with below statement.df.write.partitionBy("year", "month", "day").format('org.apache.spark.sql.execution.datasources.parquet.ParquetFileFormat').option("path",sourcepath).mod...
Hi @Gaurishankar Sakhare​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best ...
If you mean a stream-static join, yes that is possible:https://learn.microsoft.com/en-us/azure/databricks/delta-live-tables/transform#--stream-static-joinsIf not, what exactly do you mean?
Hi guys,I have some notebooks with REPOS but I noticed that REPOS changed my notebook format to .py because of this my Azure Data Factory no longer recognizes the notebook (.py)Have any ideia to convert that .py to databricks format ?
that is odd. repos is merely another location (linked to git).You can copy/paste the code inside the py file into a notebook, or convert them using online tools or python libraries (like py2ipynb).
I am trying to read data from ElasticSearch(ES Version 8.5.2) using PySpark on Databricks (13.0 (includes Apache Spark 3.4.0, Scala 2.12)). The ecosystem is on AWS.I am able to run a curl command on the Databricks notebook to the ES ip:port and fetch...
I have the same problem, did you find any solution? thanks
Dear Community- Get ready to mark your calendars for the upcoming Databricks Community Social event! Happening on June 16th, 2023, this event promises to be the ultimate monthly gathering for everyone in the Databricks Community.Join us for an hour ...
Dear Community, Have you enrolled into the New Large Language Model Courses with edX yet?As Large Language Model (LLM) applications disrupt countless industries, generative AI is becoming an important foundational technology. The demand for LLM-based...
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group