- 2614 Views
- 2 replies
- 2 kudos
Resolved! newly created account not working and when changing password it is getting hangged
My community user account is not working
- 2614 Views
- 2 replies
- 2 kudos
- 10671 Views
- 4 replies
- 3 kudos
Resolved! Writing PySpark DataFrame onto AWS Glue throwing error
I have followed the steps as mentioned in this blog : https://www.linkedin.com/pulse/aws-glue-data-catalog-metastore-databricks-deepak-rajak/ but when trying to saveAsTable(table_name), it is giving an error as IllegalArgumentException: Path must be ...
- 10671 Views
- 4 replies
- 3 kudos
- 3 kudos
Hey @Raghu Bharadwaj Tallapragada​ Just wanted to check in if you were able to resolve your issue or do you need more help? We'd love to hear from you.Thanks!
- 3 kudos
- 2362 Views
- 1 replies
- 2 kudos
- 2362 Views
- 1 replies
- 2 kudos
- 2 kudos
Continuous Jobs trigger mode has been introduced recently. To prevent unnecessary resource usage and reduce cost, Databricks automatically pauses a continuous job if there are more than five consecutive failures within a 24 hour period.
- 2 kudos
- 3908 Views
- 1 replies
- 0 kudos
- 3908 Views
- 1 replies
- 0 kudos
- 0 kudos
The UNPIVOT clause is now supported by Databricks SQL. Use the UNPIVOT clause to rotate columns of a table-valued expression into column values https://docs.databricks.com/sql/language-manual/sql-ref-syntax-qry-select-unpivot.html
- 0 kudos
- 18487 Views
- 8 replies
- 8 kudos
How to enable interactive Python matplotlib figures in DataBricks?
I want to make a matplolib interactive (I can zoom in/out, etc.) in databricks. This is achieved in Jupyter notebook by the following code: %matplotlib notebookHow to achieve this in databricks?Thank you
- 18487 Views
- 8 replies
- 8 kudos
- 8 kudos
Hi there, while facing a similar issue we switched to Altair python library and it works great with Databricks. (other options can be Bokeh or Plotly).
- 8 kudos
- 5145 Views
- 1 replies
- 2 kudos
Resolved! What to do when you face this error "Please delete your streaming query checkpoint and restart"
- 5145 Views
- 1 replies
- 2 kudos
- 2 kudos
The best way is to run a full refresh.
- 2 kudos
- 1868 Views
- 1 replies
- 2 kudos
- 1868 Views
- 1 replies
- 2 kudos
- 2 kudos
You will have to do a full refresh.
- 2 kudos
- 10363 Views
- 3 replies
- 0 kudos
How to trigger workflow job tasks from Autoloader
I have configured a File Notification Autoloader that monitors S3 bucket for binary files. I want to integrate autoloader with workflow job so that whenever a file is placed in S3 bucket, the pipeline job notebook tasks can pick-up new file and start...
- 10363 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @Saravanan Ponnaiah​ Hope everything is going great.Does @odoll odoll​ response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to hear from you.Thanks!
- 0 kudos
- 4152 Views
- 2 replies
- 1 kudos
Resolved! Font in Databricks Notebook is Greyed Out - Glitchy
The monospaced/code font in my databricks notebooks is greyed out, both in light and dark theme. I've tried playing with all the notebook settings, etc. and nothing will make the font 'normal'. I've tried Chrome and Edge, and the results are the same...
- 4152 Views
- 2 replies
- 1 kudos
- 1 kudos
I was having this exact same issue. I fixed it by uninstalling the Source Code Pro font from my Windows machine.
- 1 kudos
- 6857 Views
- 10 replies
- 1 kudos
DataBricks
How to find Mountpoints definitions
- 6857 Views
- 10 replies
- 1 kudos
- 1 kudos
Hi @Govardhana Reddy​ Glad to hear!Please mark the answer as best, it will be highly appreciable.Have a great day!Regards
- 1 kudos
- 5239 Views
- 4 replies
- 1 kudos
Resolved! How can I get date when autoloader processes the file
Hi,I am running autoloader which is running continuously and checks for new file every 1 minute. I need to store when file was received/processed but its giving me date when autoloader started. Here is my code.df = (spark .readStream .format("clo...
- 5239 Views
- 4 replies
- 1 kudos
- 1 kudos
Hi @Sanjay Jain​ , You can use the File Metadata column functionality to collect that information.Ref doc:- https://docs.databricks.com/ingestion/file-metadata-column.html
- 1 kudos
- 21965 Views
- 5 replies
- 0 kudos
Resolved! Can't install python library
I'm trying to install a python library but I'm not able, the status won't change from "pending". I get this message when I click on the library under the cluster's Libraries tab: "Library installation has been attempted on the driver node but has not...
- 21965 Views
- 5 replies
- 0 kudos
- 0 kudos
Ok, looks like I was able to solve my problem.First, I needed to install all the required libraries one by one. These are the followings:pandassixrequestspyspnegocryptographykrb5requests-kerberosAfter that I was able to install the webAPI library.
- 0 kudos
- 22194 Views
- 4 replies
- 3 kudos
Resolved! How can I add a duration in milliseconds to a timestamp?
Let's say I have a DataFrame with a timestamp and an offset column in milliseconds respectively in the timestamp and long format. E.g.from datetime import datetime df = spark.createDataFrame( [ (datetime(2021, 1, 1), 1500, ), (dat...
- 22194 Views
- 4 replies
- 3 kudos
- 3 kudos
Although @Lakshay Goel​'s solution works, we've been using an alternative approach, that we found to be a bit more readable:from pyspark.sql import Column, functions as f def make_dt_interval_sec(col: Column): return f.expr(f"make_dt_interval...
- 3 kudos
- 3428 Views
- 2 replies
- 0 kudos
voucher
Did you receive your voucher?
- 3428 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Kashish Khetarpaul​ Thank you for reaching out! Please submit a ticket to our Training Team here: https://help.databricks.com/s/contact-us?ReqType=training and our team will get back to you shortly.
- 0 kudos
- 5894 Views
- 1 replies
- 0 kudos
Databrick notebook to Azure SQL server connection using private ip because Public access is Denied in Azure SQL database, and Databrick and Azure SQL both in same subscription but different Virtual Network.
We have created private endpoint for Azure SQL database which has private ip. and by making host file entry in my system i am able to resolve Ip for Azure sql server from my system and connect to Server. but unable to connect from Azure Databrick not...
- 5894 Views
- 1 replies
- 0 kudos
- 0 kudos
If vnet injection is not used, the connection could be established by setting up an init script with azure private resolver as nameserver.​#!/bin/bashmv /etc/resolv.conf /etc/resolv.conf.origecho nameserver <your dns server ip> | sudo tee --append /e...
- 0 kudos
| User | Count |
|---|---|
| 1645 | |
| 793 | |
| 582 | |
| 349 | |
| 287 |
