- 16556 Views
- 7 replies
- 10 kudos
Resolved! Delta Table storage best practices
Hi!We have a project where we do some Data Engineering for a client. I implemented a scheduled batch processing of Databricks' autoloader (stream w/ availableNow) since they primarily have numerous file exports from several sources. We wanted to foll...
- 16556 Views
- 7 replies
- 10 kudos
- 10 kudos
Hi @Gimwell Young​,Just a friendly follow-up. Did any of the responses help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.
- 10 kudos
- 6223 Views
- 6 replies
- 4 kudos
Resolved! Hyperleaup to push data to Tableau Server
Would any care to share how they got the Hyperleaup library working I am currently stuck at an error at publish and cannot seem to find a solution. TypeError: publish() got an unexpected keyword argument 'file_path'I am %pip installing all the requir...
- 6223 Views
- 6 replies
- 4 kudos
- 4 kudos
Hi. Yes dashboard includes multiple filters but only uploaded dashboard on server without any other sheets. I am looking into the extract that other users have suggested. Thanks.
- 4 kudos
- 2712 Views
- 3 replies
- 0 kudos
- 2712 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @Hariharan Sambath​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you....
- 0 kudos
- 3007 Views
- 4 replies
- 0 kudos
How to prevent sql queries in 2 notebooks from reading the same row from a Table ?
I have an SQL query to select and update rows in a table. I do this in batches of 300 rows (select 300 , update the selected 300 , select new 300 and update the newly selected and so on..) I run this query in 2 different notebooks concurrently to spe...
- 3007 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @Krishna Zanwar​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Tha...
- 0 kudos
- 3561 Views
- 2 replies
- 2 kudos
Get file from SharePoint to copy into Azure blob storage
Hello Team,I am trying to copy the xlx files from sharepoint and move to the Azure blob storageUSERNAME = app_config_client.get_configuration_setting(key='BIAppConfig:SharepointUsername',label='BIApp').valuePASSWORD = app_config_client.get_configura...
- 3561 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @Rohit Kulkarni​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Tha...
- 2 kudos
- 2729 Views
- 2 replies
- 0 kudos
Create H3 hexagons Mosaic
I am new to using `Mosaic` on Databricks. The task is to create a Heatmap with the counts within Hexagons of H3 resolution. Since the dataset is quite large I am opting for Mosaic. As the first step I am trying to create the hexagons covering the giv...
- 2729 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Shanaka Pererea​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Th...
- 0 kudos
- 1984 Views
- 3 replies
- 2 kudos
Hey all, cross posting a medium article I wrote on writing DLT so it dynamically generates tables with less code. Let me know what you think!
Hey all, cross posting a medium article I wrote on writing DLT so it dynamically generates tables with less code. Let me know what you think!
- 1984 Views
- 3 replies
- 2 kudos
- 1927 Views
- 2 replies
- 1 kudos
Rowlock in databricks sql?
I am trying to get the commands for xlock, holdlock, rowlock, readpast in databricks sql.Looking to implement this
- 1927 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Krishna Zanwar​,We dont currently support these features, but you can create a feature request if you would like to have these features added to Databricks SQL. Please follow the instructions from here https://docs.databricks.com/resources/ideas....
- 1 kudos
- 3809 Views
- 2 replies
- 6 kudos
Resolved! Databricks Kafka Read Not connecting
I'm trying to read data from GCP kafka through azure databricks but getting below warning and notebook is simply not completing. Any suggestion please? WARN NetworkClient: Consumer groupId Bootstrap broker rack disconnectedPlease note I've properly c...
- 3809 Views
- 2 replies
- 6 kudos
- 6 kudos
Could you share the full error stack trace from your driver's logs? This is a Warning message, we need to take a look at the error level messages.
- 6 kudos
- 4283 Views
- 1 replies
- 3 kudos
dbutils.fs.ls is giving "null uri host This can be caused by unencoded / in the password string"
I'm trying to list number of files in s3 bucket. I've initially used "aws s3 ls <s3://>" to list the files and it worked. However, when trying to do the same using dbutils.fs.ls, I'm getting java.lang.NullPointerException: null uri host. This can be ...
- 4283 Views
- 1 replies
- 3 kudos
- 3 kudos
You might be encountering an issue with bucket naming. Which I'm also getting with a bucket named something.[0-9]https://issues.apache.org/jira/browse/HADOOP-17241
- 3 kudos
- 4254 Views
- 1 replies
- 3 kudos
Resolved! Forward Spark structured streaming metrics to Datadog
We have a spark streaming application written in Pyspark that we'd like to monitor with Datadog. By default, datadog collects a couple of streaming metrics like 'spark.structured_streaming.processing_rate' and 'spark.structured_streaming.latency'. Ho...
- 4254 Views
- 1 replies
- 3 kudos
- 3 kudos
@Liz Zhang​ , Please refer to the below documentation contain pyspark implementation of streamingQueryListener https://www.databricks.com/blog/2022/05/27/how-to-monitor-streaming-queries-in-pyspark.html
- 3 kudos
- 2325 Views
- 2 replies
- 0 kudos
How to install R GeoLift library on Databrickts
Hi, I am having problems installing the GeoLift library. I am proceeding as per the official instructions: https://facebookincubator.github.io/GeoLift/docs/GettingStarted/InstallingRThis is what I run in the notebook:1) I install this particular vers...
- 2325 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Ludmila Kuncarova​,I would like to share the following link to our docs https://docs.databricks.com/libraries/notebooks-r-libraries.html in this link you will be able to find more details on how to install R libraries.
- 0 kudos
- 2482 Views
- 2 replies
- 3 kudos
Azure Databricks SQL Warehouse connection issue
When trying to start SQL Warehouse from my Azure pay-as-you-go subscription, I'm getting error about not enough vCPUs provisioned. Documentation says to increase quota at Azure portal - but it requires knowing type of vCPUs to provision. What type of...
- 2482 Views
- 2 replies
- 3 kudos
- 3 kudos
Hi @Yuliya Quintela​,Just a friendly follow-up. Did Rostislaw's response help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.
- 3 kudos
- 12940 Views
- 9 replies
- 2 kudos
SQLAlchemy ORM Connection String Error
We tried to insert records to Delta table using ORM. It looks like only SQLAlchemy has option to connect to Delta table.We tried the following codefrom sqlalchemy import Column, String, DateTime, Integer, create_engine engine = create_engine("data...
- 12940 Views
- 9 replies
- 2 kudos
- 2 kudos
Hi @Frank Zhang​ , Please disregard the driver comment. The Python SQL Connector requires no driver. Just a pip install and you are good to go. The links you provided don't actually show a working example of using SQL Alchemy's ORM to connect to Data...
- 2 kudos
- 1792 Views
- 2 replies
- 0 kudos
Where to report a bug with Databricks ?
I have in issue in Pyspark.Pandas to report. Is there a github or some forum where I can register my issue?Here's the issue
- 1792 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Krishna Zanwar​ Does @Debayan Mukherjee​ response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to hear from you.Thanks!
- 0 kudos
| User | Count |
|---|---|
| 1616 | |
| 786 | |
| 369 | |
| 349 | |
| 287 |
