- 27890 Views
- 4 replies
- 2 kudos
ValueError: not enough values to unpack (expected 2, got 1)
Code:Writer.jdbc_writer("Economy",economy,conf=CONF.MSSQL.to_dict(), modified_by=JOB_ID['Economy'])The problem arises when i try to run the code, in the specified databricks notebook, An error of "ValueError: not enough values to unpack (expected 2, ...
- 27890 Views
- 4 replies
- 2 kudos
- 2 kudos
Hey Databricks Community,The error "ValueError: not enough values to unpack (expected 2, got 1)" typically occurs when Python is trying to unpack a certain number of values, but the data it is processing does not contain the expected number. This err...
- 2 kudos
- 8157 Views
- 8 replies
- 8 kudos
Resolved! Executing Notebooks - Run All Cells vs Run All Below
Due to dependencies, if one of our cells errors then we want the notebook to stop executing.We've noticed some odd behaviour when executing notebooks depending on if "Run all cells in this notebook" is selected from the header versus "Run All Below"....
- 8157 Views
- 8 replies
- 8 kudos
- 8 kudos
Has this been implemented? I have created a job using notebook. My notebook has 6 cells and if the code in first cell fails it should not run the rest of the cells
- 8 kudos
- 8381 Views
- 4 replies
- 1 kudos
Running unit tests from a different notebook (using Python unittest package) doesn't produce output (can't discover the test files)
I have a test file (test_transforms.py) which has a series of tests running using Python's unittest package. I can successfully run the tests inside of the file with expected output. But when I try to run this test file from a different notebook (run...
- 8381 Views
- 4 replies
- 1 kudos
- 1 kudos
Try this:import unittestimport os # Set the path to the directory containing the test filetest_dir = '/dbfs/mnt/repository/yunjchen/customer_ai_project/packages/visualization/test/model_performance' # Add the test directory to the Python pathos.sys.p...
- 1 kudos
- 11805 Views
- 8 replies
- 6 kudos
Resolved! Downloading sharepoint lists using python
Hello, I am trying to download lists from SharePoint into a pandas dataframe. However I cannot get any information successfully. I have attempted many solution mentioned in stackoverflow. Below is one of those attempts: # https://pypi.org/project/sha...
- 11805 Views
- 8 replies
- 6 kudos
- 6 kudos
The error "<urlopen error [Errno -2] Name or service not known>" suggests that there's an issue with the server URL or network connectivity. Double-check the server URL to ensure it's correct and accessible. Also, verify that your network connection ...
- 6 kudos
- 7468 Views
- 6 replies
- 2 kudos
Resolved! issue with group by
I am trying to group by a data frame by "PRODUCT", "MARKET" and aggregate the rest ones specified in col_list. There are much more column in the list but for simplification lets take the example below.Unfortunatelly I am getting the error:"TypeError:...
- 7468 Views
- 6 replies
- 2 kudos
- 2 kudos
The error you're encountering, "TypeError: unhashable type: 'Column'," is likely due to the way you're defining exprs. In Python, sets use curly braces {}, but they require their items to be hashable. Since the result of sum(x).alias(x) is not hashab...
- 2 kudos
- 6396 Views
- 7 replies
- 4 kudos
Resolved! Saving complete notebooks to GitHub from Databricks repos.
When saving notebook to GiHub repo, it is stripped to Python source code. Is it possible to save it in the ipynb formt?
- 6396 Views
- 7 replies
- 4 kudos
- 4 kudos
When I save+commit+push my .ipynb file to my linked git repo, I noticed that only the cell inputs are saved, not the output. This differs from the .ipynb file I get when I choose "File / Export / iPython Notebook". Is there a way to save the cell o...
- 4 kudos
- 2900 Views
- 2 replies
- 0 kudos
- 2900 Views
- 2 replies
- 0 kudos
- 0 kudos
I have tried to connect the mysql database using above code but failed to connect.getting this as error : DatabaseError: 2003 (HY000): Can't connect to MySQL server on 'localhost:3306' (111)and after using host as 127.0.0.1 getting this as error : Da...
- 0 kudos
- 9354 Views
- 11 replies
- 20 kudos
Resolved! Debugging!
Hi ML folks, We are using Databricks to train deep learning models. The code, however, has a complex structure of classes. This would work fine in a perfect bug-free world like Alice in Wonderland. Debugging in Databricks is awkward. We ended up do...
- 9354 Views
- 11 replies
- 20 kudos
- 20 kudos
Has this been solved yet; a mature way to debug code on databricks. I'm running in the same kind of issue.Variable explorer can be used and pdb, but not the same really..
- 20 kudos
- 14831 Views
- 3 replies
- 4 kudos
Resolved! AttributeError: 'DataFrame' object has no attribute 'rename'
Hello, I am doing the Data Science and Machine Learning course. The Boston housing has unintuitive column names. I want to rename them, e.g. so 'zn' becomes 'Zoning'. When I run this command: df_bostonLegible = df_boston.rename({'zn':'Zoning'}, axi...
- 14831 Views
- 3 replies
- 4 kudos
- 4 kudos
If df_boston is a DataFrame, but you still face issues, try an alternative syntax: df_boston = df_boston.rename(columns={'zn': 'Zoning'}).Make sure df_boston is a proper DataFrame and you're using a recent version of Pandas.
- 4 kudos
- 2138 Views
- 3 replies
- 1 kudos
URGENT HELP NEEDED: Python functions deployed in the cluster throwing the error
Hi,I have created a python wheel with the following code. And the package name is rule_engine"""The entry point of the Python Wheel"""import sysfrom pyspark.sql.functions import expr, coldef get_rules(tag): """ loads data quality rules from a table ...
- 2138 Views
- 3 replies
- 1 kudos
- 1 kudos
You can find more details and examples here https://docs.databricks.com/en/workflows/jobs/how-to/use-python-wheels-in-workflows.html#use-a-python-wheel-in-a-databricks-job
- 1 kudos
- 3320 Views
- 9 replies
- 3 kudos
Databricks Certified Associate Developer for Apache Spark 3.0
Databricks Certified Associate Developer for Apache Spark 3.0
- 3320 Views
- 9 replies
- 3 kudos
- 3 kudos
Hey I am looking for sample papers for the above exam other than the one provided by databricks do any one have any idea about it
- 3 kudos
- 3506 Views
- 7 replies
- 1 kudos
Resolved! examples on python sdk for install libraries
Hi Everyone,I'm planning to use databricks python cli "install_libraries"can some one pls post examples on function install_libraries https://github.com/databricks/databricks-cli/blob/main/databricks_cli/libraries/api.py
- 3506 Views
- 7 replies
- 1 kudos
- 1 kudos
Here you go using Python SDKfrom databricks.sdk import WorkspaceClientfrom databricks.sdk.service import computew = WorkspaceClient(host="yourhost", token="yourtoken")# Create an array of Library objects to be installedlibraries_to_install = [compute...
- 1 kudos
- 2988 Views
- 4 replies
- 0 kudos
Execute databricks notebook form a python source code.
Hello,I 'm trying to execute databricks notebook form a python source code but getting error.source code below------------------from databricks_api import DatabricksAPI # Create a Databricks API client api = DatabricksAPI(host='databrick_host', tok...
- 2988 Views
- 4 replies
- 0 kudos
- 0 kudos
The error you are encountering indicates that there is an issue with establishing a connection to the Databricks host specified in your code. Specifically, the error message "getaddrinfo failed" suggests that the hostname or IP address you provided f...
- 0 kudos
- 24084 Views
- 13 replies
- 3 kudos
Resolved! displayHTML can't seem to be used from Python code, only hand typed into a cell???
Trying to use displayHTML from w/in a Python module gets a Python exception:NameError: name 'displayHTML' is not definedand I've found no way around this. It seems to be something at the UI layer or something, not a Python function that can be refere...
- 24084 Views
- 13 replies
- 3 kudos
- 3 kudos
Holy Guacamole Batman! It works finally!!!! Wow, thanks @ptweir That's awesome! I can go back and update my doc (and code, to just use databricks the same, now, and Jupyter!) and it'll work by default. It's great they fixed it, shame they never told ...
- 3 kudos
- 10302 Views
- 3 replies
- 1 kudos
Resolved! How to kill the execution of a notebook on specyfic cell?
Let's say I want to check if a condition is false then stop the execution of the rest of the script. I tried with two approaches:1) raising exceptionif not data_input_cols.issubset(data.columns): raise Exception("Missing column or column's name mis...
- 10302 Views
- 3 replies
- 1 kudos
- 1 kudos
In Jupyter notebooks or similar environments, you can stop the execution of a notebook at a specific cell by raising an exception. However, you need to handle the exception properly to ensure the execution stops. The issue you're encountering could b...
- 1 kudos