- 2164 Views
- 2 replies
- 1 kudos
How to get executors info by SDK (Python)
Hi guys,How I get executors information to my cluster by SDK (Python) have any idea ?Thank you
- 2164 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @William Scardua We haven't heard from you since the last response from @josephk and I was checking back to see if it helped you.Or else, If you have any solution, please share it with the community, as it can be helpful to others. Also, Please d...
- 1 kudos
- 8331 Views
- 4 replies
- 4 kudos
How to create a new group in the Databricks community? Dear esteemed community users, It is with great pleasure that we inform you of an important upd...
How to create a new group in the Databricks community?Dear esteemed community users,It is with great pleasure that we inform you of an important update regarding the creation of Groups on Community. As part of our continuous efforts to enhance your e...
- 8331 Views
- 4 replies
- 4 kudos
- 4 kudos
HI @Hubert Dudek and @Ratna Chaitanya Raju Bandaru : Thanks for pointing this out. This is going to be a design decision which we will take after looking into the ask carefully. Thanks for getting the conversation going. This really helps us.
- 4 kudos
- 10665 Views
- 5 replies
- 6 kudos
StreamCorruptedException, databricks-connect 9.1
Last week, around the 21st of march, we started having issues with databricks-connect (DBR 9.1 LTS). "databricks-connect test" works, but the following code snippet: from pyspark.sql import SparkSession spark = SparkSession.builder.getOrCreate() ...
- 10665 Views
- 5 replies
- 6 kudos
- 6 kudos
Hi @Jordi Dekker Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers ...
- 6 kudos
- 6332 Views
- 2 replies
- 1 kudos
DataFrame
How can we create empty dataframe in databricks and how many ways we can create dataframe?
- 6332 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Govardhana Reddy Hope everything is going great.Does @Suteja Kanuri's answer help? If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we can help you. Cheers!
- 1 kudos
- 4024 Views
- 3 replies
- 1 kudos
Outdated - Databricks Data Engineer associate v2 lesson DE 4.2
While following the video lesson and executing the notebook 4.2, I noticed that creating the CREATE Table "users_jdbc" command generates an EXTERNAL table, while the video and, notebook too, suggests it as being a Managed table.Here are some printscr...
- 4024 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @Tiago Barata Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thank...
- 1 kudos
- 20589 Views
- 4 replies
- 0 kudos
AppendDataExecV1 Taking a lot of time
Hi, I have a Pyspark job that takes about an hour to complete, when looking at the SQL tab on Spark UI I see this:Those processes run for more than 1 minute on a 60-minute process.This is Ganglia for that period (the last snapshot, will look into a l...
- 20589 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @Alejandro Martinez Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you...
- 0 kudos
- 13394 Views
- 3 replies
- 14 kudos
Resolved! No suitable driver error When configure the Databricks ODBC and JDBC drivers
Hi all,I've just encountered with this issue. Before I launched an My SQL database in RDS of AWS after use this simple code to create connection to it but it all fails with this error.Is there any additional step? or could anyone can take a look on i...
- 13394 Views
- 3 replies
- 14 kudos
- 14 kudos
Hello, It looks issue with JDBC URL. When I am trying to access the Azure SQL database. I was facing the same issue. So I have created JDBC URL as below and it went well.jdbc:sqlserver://<serverurl>:1433;database=<databasename>;user=<username>@<serve...
- 14 kudos
- 3200 Views
- 3 replies
- 0 kudos
Division Auto Truncates Decimal Even After Casting Inputs
Division of two numbers is auto truncating decimals and I can't get a more precise result.Example of things I've tried:10 / 60 => 0.17cast(10 as float) / cast(60 as float) => 0.17cast(cast(10 as float) / cast(60 as float) as float) => 0.17round(10 / ...
- 3200 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @Alex Python Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers y...
- 0 kudos
- 12836 Views
- 3 replies
- 0 kudos
about .dbc notebook
I can't be able to import .dbc notebook into my community edition. Please help.
- 12836 Views
- 3 replies
- 0 kudos
- 0 kudos
I imported .dbc notebook using url successfully but I can't be able to import using upload file option and I didn't get either any error message or anything when I tried to import using upload file option.
- 0 kudos
- 16762 Views
- 6 replies
- 5 kudos
Need a way to show the non-trimmed data while query a table ?
When querying a json data , the values are getting trimmed. I need to see the full data for that field, is there any way to do so ?
- 16762 Views
- 6 replies
- 5 kudos
- 5 kudos
Hi @Raj Sethi We haven't heard from you since the last response from @Lakshay Goel and @Vigneshraja Palaniraj , and I was checking back to see if her suggestions helped you.Or else, If you have any solution, please share it with the community, a...
- 5 kudos
- 12469 Views
- 3 replies
- 3 kudos
Resolved! Using "FOR XML PATH" in Spark SQL in sql syntax
I'm using spark version 3.2.1 on databricks (DBR 10.4 LTS), and I'm trying to convert sql server sql query to a new sql query that runs on a spark cluster using spark sql in sql syntax. However, spark sql does not seem to support XML PATH as a functi...
- 12469 Views
- 3 replies
- 3 kudos
- 3 kudos
Posting the solution that I ended up using:%sql DROP TABLE if exists UserCountry; CREATE TABLE if not exists UserCountry ( UserID INT, Country VARCHAR(5000) ); INSERT INTO UserCountry SELECT L.UserID AS UserID, CONCAT_WS(',', co...
- 3 kudos
- 4331 Views
- 3 replies
- 1 kudos
How to get url of the dashboard corresponding to a notebook?
I have several notebooks and their corresponding dashboards. I was able to get the url of the notebooks via the workspace api by concatenating the static part of the url with the object ids, but I am struggling to find a way to do the same for their ...
- 4331 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @Akshaya Choudhary Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best ans...
- 1 kudos
- 3945 Views
- 2 replies
- 1 kudos
Error on locating FileStore on Databricks DBFS
Hello, I'm having a strange error while locating the FileStore folder on DBFS.Following the error message:dbutils.fs.ls('dbfs:/Filestore')java.io.FileNotFoundException: File /6239660819762462/Filestore does not exist. ExecutionError ...
- 3945 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Marco Fina Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers yo...
- 1 kudos
- 2243 Views
- 2 replies
- 0 kudos
KinesisSource generates empty microbatches when there is no new data.
Is it normal for KinesisSource to generate empty microbatches when there is no new data in Kinesis? Batch 1 finished as there were records in kinesis and BatchId 2 started. BatchId 2 was running but then BatchId 3 started . Even though there was no m...
- 2243 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Pranathi Girish Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answe...
- 0 kudos
- 4003 Views
- 2 replies
- 0 kudos
What is the best practice for tracing databricks - observe and writestream data record flow
Trying to connect dots on method below through a new event on Azure eventhub, storage, partition, avro records (those I can monitor) to my delta table? How do I trace observe, writeStream and the trigger? ... elif TABLE_TYPE == "live": print("D...
- 4003 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @David Martin Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers ...
- 0 kudos
| User | Count |
|---|---|
| 1636 | |
| 791 | |
| 515 | |
| 349 | |
| 287 |
