- 1258 Views
- 1 replies
- 2 kudos
Join our Community Social Group and Never Miss a Beat! Are you looking to connect with like-minded individuals and stay on top of the latest news and ...
Join our Community Social Group and Never Miss a Beat!Are you looking to connect with like-minded individuals and stay on top of the latest news and events in your community? Look no further than our special group on Community called the "Community S...
- 1258 Views
- 1 replies
- 2 kudos
- 2 kudos
@Rishabh Pandey​
- 2 kudos
- 14201 Views
- 5 replies
- 0 kudos
"Waiting to run" status of the cell
Whenever I tried to execute the cell, it is not executing and it says waiting to run and gets stuck there. I tried this with different clusters but the problem still persist.What should be the standard process if I create a new cluster and wants to a...
- 14201 Views
- 5 replies
- 0 kudos
- 0 kudos
Hello @Mahesh Chahare​ , The "Waiting to run" message appears usually when the cluster start or library installation is in progress. If you are running the commands on an interactive notebook, and you have clicked on "Run All", the commands will get ...
- 0 kudos
- 3671 Views
- 5 replies
- 1 kudos
Waiting Waiting Waiting.......Databricks voucher not received
Hello,I have attended the webinar Lakehouse Fundamentals Training in Feb 2023then Completed the Databricks Lakehouse fundamentals accreditation and submitted the survey.As per communication it is expected that I will receive Databricks Certification ...
- 3671 Views
- 5 replies
- 1 kudos
- 1 kudos
Hi @YOGESH SINGH​ I'm sorry you could not find a solution to your problem in the answers provided.Our community strives to provide helpful and accurate information, but sometimes an immediate solution may only be available for some issues.I suggest p...
- 1 kudos
- 3988 Views
- 3 replies
- 0 kudos
Display Html from dbfs files
Hi,I want to display some content from dbfs inside my notebook. Let's say I have a image under: /dbfs/mnt/test-bucket/test-custom/sample.pngI want to embed that into my Notebook Html Output like this:displayHTML("""<img src ='/dbfs/mnt/test-bucket/te...
- 3988 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @Vidula Khanna​ ,unfortunately not. I can access the file through the notebook via e.g. !ls /dbfs/mnt/test-bucket/test-custom/ but it cannot be displayed via the displayHTML, with the 401.
- 0 kudos
- 7732 Views
- 2 replies
- 0 kudos
Access DataBricks file system and transfer files
Is there a solution to access the files in DataBricks file system and transfer them to another directory, local or elsewhere. In other words, is there a FileZilla type solution? where can i find instructions to ssh into the cluster from my mac termin...
- 7732 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi, There are different options around this. You can start with https://docs.databricks.com/files/index.html. Please let us know if this helps. Also, please tag @Debayan​ with your next response which will notify me. Thank you!
- 0 kudos
- 4489 Views
- 3 replies
- 1 kudos
Issue with VS Code extension repo.
I successfully installed the extension and connected it to my databricks account. But when I try to select the repo (which already exists under repos in my databricks account) for syncing , I don't see it. My company uses Azure Devops (Git repo) as s...
- 4489 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi, was the repo already added?You can check on this : https://learn.microsoft.com/en-us/azure/databricks/repos/git-operations-with-reposAlso, please tag @Debayan Mukherjee​ with your next response which will notify me. Thank you!
- 1 kudos
- 5366 Views
- 2 replies
- 1 kudos
Understand if the configs I use to SparkSession.builder still make sense for Databricks 10+
Hi! I currently have this as an old generic template with amends over time to optimize Databricks Spark execution, can you help me to know if this still makes sense for v10-11-12 or if there are new recommendations? Maybe some of this is making my pr...
- 5366 Views
- 2 replies
- 1 kudos
- 1 kudos
@Alejandro Martinez​ :Hi! Your template seems to be a good starting point for configuring a SparkSession in Databricks. However, there are some new recommendations that you can consider for Databricks runtime versions v10-11-12. Here are some suggest...
- 1 kudos
- 8647 Views
- 1 replies
- 1 kudos
Error when using SAS token to connect to azure Storage Account: Unable to load SAS token provider class: java.lang.IllegalArgumentException
Hi, I am trying to connect to the Storage Account using the SAS token, and receive this error: Unable to load SAS token provider class: java.lang.IllegalArgumentException - more on the picture.I couldnt find anything on the web for this error.I also ...
- 8647 Views
- 1 replies
- 1 kudos
- 1 kudos
@Retko Okter​ :It seems that there is an issue with the SAS token provider class. This error can occur when the SAS token is not correctly formatted or is invalid.Here are some steps you can try to resolve the issue:Verify that the SAS token is corre...
- 1 kudos
- 9767 Views
- 1 replies
- 1 kudos
plotly express choropleth map not rendering in jupyter notebook
I have the following code which should render a choropleth map. import plotly.express as px import geopandas as gpd # Example GeoJSON file with polygon geometries geojson_file = 'example.geojson' # Read GeoJSON file into GeoDataFrame *** = gpd.re...
- 9767 Views
- 1 replies
- 1 kudos
- 1 kudos
@Keval Shah​ :There could be several reasons why the choropleth map is not rendering in your Jupyter notebook. Here are a few things you could try:Check that the GeoJSON file is loaded correctly: Make sure that the GeoDataFrame has been loaded correc...
- 1 kudos
- 7442 Views
- 7 replies
- 0 kudos
is there a read only option in jdbc driver?
Is there a "read only" option when using databricks sql using jdbc driver?I'm looking for an equivalent to this:https://docs.aws.amazon.com/redshift/latest/mgmt/jdbc20-configuration-options.html#jdbc20-readonly-optionThanks!
- 7442 Views
- 7 replies
- 0 kudos
- 0 kudos
Hi @Nativ Issac​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so w...
- 0 kudos
- 2217 Views
- 2 replies
- 0 kudos
take() ooperation is throwing error
Traceback (most recent call last): File "/usr/local/spark/python/pyspark/serializers.py", line 458, in dumps return cloudpickle.dumps(obj, pickle_protocol) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/spark/python/pyspa...
- 2217 Views
- 2 replies
- 0 kudos
- 0 kudos
@Shelly Bhardwaj​ :The error message you provided seems to be incomplete, as it only shows the traceback of a serialization error. Can you provide the full error message or describe the issue in more detail?Regarding the code you provided, it looks c...
- 0 kudos
- 8354 Views
- 3 replies
- 0 kudos
Creating a Api links by url & list from a saved df
I have 106,000 + api's I need to call, so instead of calling them one by one I would like to create a loop as I have the list of location Id's which I've called from there api locations list and these will sit at the end of the url to get more info o...
- 8354 Views
- 3 replies
- 0 kudos
- 0 kudos
@Kay Connolly​ :It looks like you are trying to concatenate a string with a column object, which is causing the error. You need to convert the column object to a string first before concatenating it to the URL. Here's a modified code snippet that sho...
- 0 kudos
- 3504 Views
- 2 replies
- 2 kudos
Pyspark insertinto on Hive external table doesn't work if overwrite is true
Getting an Hive Exception​
- 3504 Views
- 2 replies
- 2 kudos
- 2 kudos
@ppatel:If you are using insertInto with overwrite=True on a Hive external table in PySpark, it might not work as expected. This is because Hive external tables are not managed by Hive and the table data is stored externally. When you use overwrite=T...
- 2 kudos
- 44934 Views
- 9 replies
- 2 kudos
Resolved! Unable to clear cache using a pyspark session
Hi all,I am using a persist call on a spark dataframe inside an application to speed-up computations. The dataframe is used throughout my application and at the end of the application I am trying to clear the cache of the whole spark session by calli...
- 44934 Views
- 9 replies
- 2 kudos
- 2 kudos
No solution yet:Hi @Suteja Kanuri​ ,Thank you for thinking along and replying!Unfortunately, I have not found a solution yet.I am getting an error that there exists no ```.getCache()``` method on a spark context. Also note that I have tried to do som...
- 2 kudos
- 15834 Views
- 4 replies
- 3 kudos
Best way to install and manage a private Python package that has a continuously updating Wheel
I'm trying to setup a Workspace Library that is used internally within our organization. This is a Python package, where the source is available on a private GitHub repository, and not accessible on PyPi or the wider internet / surface web. I managed...
- 15834 Views
- 4 replies
- 3 kudos
- 3 kudos
@Eshwaran Venkat​ You can use the Databricks CLI to automate the process of cloning the private GitHub repository and building/uploading the Python package to DBFS as a wheel file. You can schedule this process to run periodically, such as once a day...
- 3 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 573 | |
| 349 | |
| 287 |
