I know you can use microsoft diagnostics to receive notifications such as commandSubmit or commandFinish in json. But is it possible to get these diagnostics in databricks itself using pyspark?I would like to get these diagnostics inside of a noteboo...
We have scheduled a dashboard to automatically refresh at some specific time.However, some visualizations in the dashboard don't get refreshed at the scheduled time.Checking the query logs, we found the source queries were properly executed, but the ...
Thank you for the answers.I'm using Databricks SQL environment, not the Data Science & Engineering one. And, I scheduled the dashboard following this guideline: https://docs.databricks.com/sql/user/dashboards/index.html#automatically-refresh-a-dashbo...
This may be a tricky question, so please bear with meIn a real life scenario, i have a dataframe (i'm using pyspark) called age, with is a groupBy of other 4 dataframes. I join these 4 so at the end I have a few million rows, but after the groupBy th...
Hi @Marcos Dias​ ,Frankly, I think we need more detail to answer your question:Are these 4 dataframes​ updated their data?How often you use the groupBy-dataframe?
Hi,I am using databricks through azure. I am trying to connect to remote oracle database using jdbc url. I am getting an error of no suitable driver found."java.sql.SQLException: No suitable driver"can somebody help mw with this?
Hi @vikas k​ Hope all is well! Does @Hubert Dudek​ response were able to resolve your issue, and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!
I have been trying to extract one date field from cosmos which looks like as below:"lastModifiedDate" : { "$date" : 1668443121840 }when the above field is extracted using Databricks it gets converted into a date format which looks like this...
Hi, I manually created a template dashboard with one widget. I wanted to clone this dashboard using the Create Dashboard API however I don't know what to put for the widget object. What I did was use the Retrieve API on the template dashboard, and th...
@Akash Bhat​ how do I clone a dashboard across workspaces? Being able to deploy dashboards (with widgets!) through the API is essential to set up proper Data Engineering workflows.
Hi I would like to know if anyone interested to volunteer in person Databricks meetup.Please share your thoughts, and we can talk further about the logistics Thank you
Hi all,​I have a question which is likely very fundamental. I am passing data from hive to snowflake using pyspark. My query is very simple - "select from table limit 100".​ The table I am querying is very large, but this query can be shown to the co...
I have an Azure service principle that is used for our CI/CD pipelines. We do not have access to the Databricks UI via user logins. Our github repos also require SSO PATs. How can I configure the git integration for the service principal so that I ca...
Hello,Can I restore a dbfs directory ?I dropped a database using an important directory Someone can help me please ?
Hey guys, I'm using petastorm to train DNN, First i convert spark df with make_spark_convertor and then open a reader on the materialized dataset.While i start training session only on subset of the data every thing works fine but when I'm using all...
Same error. This started a few days ago on notebooks that used to run fine in the past. Now, I cannot finish a notebook.I have already disabled almost all output being streamed to the result buffer, but the problem persists. I am left with <50 lines ...
I'm looking for some information regarding log4j vulnerability - if any databricks runtime should be changed manually by the client or when specific update will be applied.I know I can go through the docs by myself, finding out which log4j library is...
Thanks for sharing such important facts. I got the best security service provider website on the internet. They are excellence in the field of security.
Hello,I am getting inconsistent representation of long types.1661817599972 is the unix timestamp in milliseconds for Monday, August 29, 2022 11:59:59.972 PM GMTwhen I execute:`select 1661817599972 as t`the result is:166181759997 (last digit truncated...
Hi @Cosimo Felline​ Hope all is well!Does @Hubert Dudek (Customer)​ response answer your question?If you could resolve your issue, would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd...
Hi Everyone,I am getting the following error when running a SQL query and do not understand what it means or what can be done to resolve it. Any recommendations?View DDL:CREATE VIEW myschema.table ( accountId, agreementType, capture_file_name, ...
Hi @Michael Okulik​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Tha...
Lakehouse Fundamentals Accreditation badge not received. @Kaniz Fatma​ @Lindsay Olson​
Hi @Haobo Zhu​ and @Srihari P​ Just a friendly- follow-up that if you were able to resolve your certification issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell u...
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 570 | |
| 349 | |
| 287 |
