- 7845 Views
- 9 replies
- 1 kudos
Unable to launch notebook
I have created a 12.2 LTS cluster and trying to launch notebook attached to this cluster. But unable to launch, it is not giving any error instead it is still showing the same home page.
- 7845 Views
- 9 replies
- 1 kudos
- 1 kudos
Hi @Muskan Bansal​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers...
- 1 kudos
- 5106 Views
- 3 replies
- 0 kudos
Resolved! Command output disappearing (Not sure what's the root cause)
I have a workflow which will run every month and it will create a new notebook containing the outputs from the main notebook. However, after some time, the outputs from the created notebook will disappear. Is there anyway I can retain the outputs?
- 5106 Views
- 3 replies
- 0 kudos
- 0 kudos
@Shaun Ang​ :There are a few possible reasons why the outputs from the created notebook might be disappearing:Notebook permissions: It's possible that the user or service account running the workflow does not have permission to write to the destinati...
- 0 kudos
- 810 Views
- 1 replies
- 2 kudos
www.databricks.com
Dear Community - @Youssef Mrini​ will answer all your questions on April 19, 2023 from 9:00am to 10:00am GMT during the Databricks EMEA Office Hours.Make sure to join this amazing 'Ask Me Anything' session by Databricks - https://www.databricks.com/r...
- 810 Views
- 1 replies
- 2 kudos
- 2 kudos
It was a successful office hours. Make sure to join the next one.
- 2 kudos
- 13251 Views
- 4 replies
- 5 kudos
How can i use the same spark session from onenotebook to another notebook in databricks
I want to use the same spark session which created in one notebook and need to be used in another notebook in across same environment, Example, if some of the (variable)object got initialized in the first notebook, i need to use the same object in t...
- 13251 Views
- 4 replies
- 5 kudos
- 5 kudos
You can use %run and then use the location of the notebook - %run "/folder/notebookname"
- 5 kudos
- 7767 Views
- 1 replies
- 1 kudos
Testing framework using Databricks Notebook and Pytest.
Hi Friends,I am designing a Testing framework using Databricks and pytest. Currently stuck with report generation, that is generating blank with only default parameters only .for ex :-testsuites><testsuite name="pytest" errors="0" failures="0" skippe...
- 7767 Views
- 1 replies
- 1 kudos
- 1 kudos
@Vijaya Palreddy​ :There are several testing frameworks available for data testing that you can consider using with Databricks and Pytest:Great Expectations: Great Expectations is an open-source framework that provides a simple way to create and main...
- 1 kudos
- 1206 Views
- 1 replies
- 2 kudos
Best Suited Language To Parallelize Notebook
I would like to know if we are writing code for parallel execution on notebook which language is best suited for that Python or Scala.
- 1206 Views
- 1 replies
- 2 kudos
- 2 kudos
You need to test in Python and scala based on the complexity one of it outperforms the other. In few cases Python was faster where as in other Scala. It is all about the efficiency of the code
- 2 kudos
- 6361 Views
- 2 replies
- 0 kudos
`moduleNotFoundError` when attempting to enable a jupyter notebook extension
I am running a set of commands and to run `pydeck` on jupyter notebook as per the documentation here: https://pydeck.gl/installation.html#enabling-pydeck-for-jupyterHowever, it throws an `moduleNotFoundError` exception. !pip install pydeck !jupyter n...
- 6361 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Keval Shah​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers yo...
- 0 kudos
- 3453 Views
- 3 replies
- 5 kudos
How to exit the entire job in the orchestration scenario?
Hi, can anybody answer this question I posted on StackOverflow? https://stackoverflow.com/questions/73314048/databricks-how-to-exit-the-entire-job-in-the-notebooks-orchestration-scenario
- 3453 Views
- 3 replies
- 5 kudos
- 5 kudos
@Vidula Khanna​ @Vidula Khanna​ We are experiencing the same issue in our Workflows and I was wondering if there has been any update.We need the functionality to call a method similar to `dbutils.notebook.exit` in a notebook that will cancel the exec...
- 5 kudos
- 42288 Views
- 9 replies
- 6 kudos
Resolved! How to access the result of a %sql cell from python
I see the way to move from python to sql is to create a temp view, and then access that dataframe from sql, and in a sql cell. Now the question is, how can I have a %sql cell with a select statement in it, and assign the result of that statement to ...
- 42288 Views
- 9 replies
- 6 kudos
- 6 kudos
Results from an SQL cell are available as a Python DataFrame. The Python DataFrame name is _sqldf.To save the DataFrame, run this code in a Python cell:df = _sqldfKeep in mind that the value in _sqldf is held in memory and will be replaced with the m...
- 6 kudos
- 12236 Views
- 2 replies
- 0 kudos
Combine Python + R in data manipulation in Databricks Notebook
Want to combine Py + Rfrom pyspark.sql import SparkSessionspark = SparkSession.builder.appName("CreateDataFrame").getOrCreate()# Create a sample DataFramedata = [("Alice", 25), ("Bob", 30), ("Charlie", 35), ("Oscar",36), ("Hiromi",41), ("Alejandro", ...
- 12236 Views
- 2 replies
- 0 kudos
- 0 kudos
@Oscar CENTENO MORA​ :To combine Py and R in a Databricks notebook, you can use the magics command %python and %rto switch between Python and R cells. Here's an example of how to create a Spark DataFrame in Python and then use it in R:from pyspark.sq...
- 0 kudos
- 2603 Views
- 2 replies
- 0 kudos
Run pytest inside repos and store the results in dbfs
Hi everyone!I am trying to run pytest inside a notebook on repos and store the results inside dbfs but i am getting an error stating permission denied, does anyone know why this happens and the solution. Error:
- 2603 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @João Peixoto​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so ...
- 0 kudos
- 3706 Views
- 3 replies
- 1 kudos
How to get url of the dashboard corresponding to a notebook?
I have several notebooks and their corresponding dashboards. I was able to get the url of the notebooks via the workspace api by concatenating the static part of the url with the object ids, but I am struggling to find a way to do the same for their ...
- 3706 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @Akshaya Choudhary​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best ans...
- 1 kudos
- 4899 Views
- 5 replies
- 3 kudos
Give customized notebook run id/name in console using dbutils.notebook.run()
I am running below code in Azure databricks:Now the running status shows as highlighted in yellow. I want to give a custom name to this running status instead of Notebook job #<job id>. This will help me identify which status is for which notebook so...
- 4899 Views
- 5 replies
- 3 kudos
- 3 kudos
Hi @Gaurav Tanwar​ I'm sorry you could not find a solution to your problem in the answers provided.Our community strives to provide helpful and accurate information, but sometimes an immediate solution may only be available for some issues.I suggest...
- 3 kudos
- 4578 Views
- 1 replies
- 3 kudos
TimestampFormat issue
The databricks notebook failed yesterday due to timestamp format issue. error:"SparkUpgradeException: You may get a different result due to the upgrading of Spark 3.0: Fail to parse '2022-08-10 00:00:14.2760000' in the new parser. You can set spark.s...
- 4578 Views
- 1 replies
- 3 kudos
- 3 kudos
You must have solved this issue by now but for the sake of those that encounter this again, here's the solution that worked for me:spark.sql("set spark.sql.legacy.timeParserPolicy=LEGACY")
- 3 kudos
- 3944 Views
- 4 replies
- 16 kudos
ipywidget not able to start running another notebook
I am trying to add a button in a notebook to trigger an execution of another notebook, but it does not respond to the command. Any idea why? The run command works if I run in a separate cell
- 3944 Views
- 4 replies
- 16 kudos
- 16 kudos
Did you end up finding an answer to this? I am running into the same problem
- 16 kudos