- 10573 Views
- 6 replies
- 2 kudos
Resolved! MergeSchema Not Working
Hi All, I have a scenario where my Exisiting Delta Table looks like below:Now I have an incremental data with an additional column i.e. owner:Dataframe Name --> scdDFBelow is the code snippet to merge Incremental Dataframe to targetTable, but the new...
- 10573 Views
- 6 replies
- 2 kudos
- 2 kudos
In Databricks Runtime 15.2 and above, you can specify schema evolution in a merge statement using SQL or Delta table APIs:MERGE WITH SCHEMA EVOLUTION INTO targetUSING sourceON source.key = target.keyWHEN MATCHED THENUPDATE SET *WHEN NOT MATCHED THENI...
- 2 kudos
- 31052 Views
- 4 replies
- 2 kudos
ValueError: not enough values to unpack (expected 2, got 1)
Code:Writer.jdbc_writer("Economy",economy,conf=CONF.MSSQL.to_dict(), modified_by=JOB_ID['Economy'])The problem arises when i try to run the code, in the specified databricks notebook, An error of "ValueError: not enough values to unpack (expected 2, ...
- 31052 Views
- 4 replies
- 2 kudos
- 2 kudos
Hey Databricks Community,The error "ValueError: not enough values to unpack (expected 2, got 1)" typically occurs when Python is trying to unpack a certain number of values, but the data it is processing does not contain the expected number. This err...
- 2 kudos
- 1962 Views
- 2 replies
- 1 kudos
Resolved! If someone saves a flat file in a Databricks notebook without specifying a location, where does it go?
I ran the code block below and now I can't find the file. Where would this get saved since no location was specified?
- 1962 Views
- 2 replies
- 1 kudos
- 17485 Views
- 4 replies
- 4 kudos
How can I connect to an Azure SQL db from a Databricks notebook?
I know how to do it with spark, and read/write tables (like https://docs.microsoft.com/en-gb/azure/databricks/data/data-sources/sql-databases#python-example )But this time, I need to only update a field of a specific row in a table. I do not think I ...
- 17485 Views
- 4 replies
- 4 kudos
- 4 kudos
thanks for the link.I am maybe wrong, but they describe how to connect with spark. They do not provide a connection engine that we could use directly (like with pyodbc) or an engine that we could use in pandas, for example.
- 4 kudos
- 5779 Views
- 5 replies
- 4 kudos
The driver could not establish a secure connection to SQL Server by using Secure Sockets Layer (SSL) encryption
I am trying to connect to SQL through JDBC from databricks notebook. (Below is my notebook command)val df = spark.read.jdbc(jdbcUrl, "[MyTableName]", connectionProperties) println(df.schema)When I execute this command, with DBR 10.4 LTS it works fin...
- 5779 Views
- 5 replies
- 4 kudos
- 4 kudos
Try to add the following parameters to your SQL connection string. It fixed my problem for 13.X and 12.X;trustServerCertificate=true;hostNameInCertificate=*.database.windows.net;
- 4 kudos
- 8365 Views
- 5 replies
- 3 kudos
Resolved! Split a code cell at cursor position? Add a cell above/below?
In JupyterLab notebooks, we can --In edit mode, you can press Ctrl+Shift+Minus to split the current cell into two at the cursor position In command mode, you can click A or B to add a cell Above or Below the current cellare there equivalent shortcuts...
- 8365 Views
- 5 replies
- 3 kudos
- 3 kudos
What's the status of the ctrl-alt-minus shortcut for splitting a cell? That keyboard combination does absolutely nothing in my interface (running Databricks via Chrome on GCP).
- 3 kudos
- 26814 Views
- 24 replies
- 18 kudos
Resolved! Errors Using Selenium/Chromedriver in DataBricks
Hello,I’m programming in a notebook and attempting to use the python library Selenium to automate Chrome/chromedriver. I’ve successfully managed to install selenium using%sh pip install seleniumI then attempt the following code, which results in the...
- 26814 Views
- 24 replies
- 18 kudos
- 18 kudos
I also tried the script and am getting similar error. Can anyone please give some resolution for it?Error in Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/s/systemd/udev_245.4-4ubuntu3.18_amd64.deb and Unable to fetch some archives
- 18 kudos
- 6915 Views
- 7 replies
- 4 kudos
Resolved! Saving complete notebooks to GitHub from Databricks repos.
When saving notebook to GiHub repo, it is stripped to Python source code. Is it possible to save it in the ipynb formt?
- 6915 Views
- 7 replies
- 4 kudos
- 4 kudos
When I save+commit+push my .ipynb file to my linked git repo, I noticed that only the cell inputs are saved, not the output. This differs from the .ipynb file I get when I choose "File / Export / iPython Notebook". Is there a way to save the cell o...
- 4 kudos
- 2311 Views
- 3 replies
- 1 kudos
How to connect Analysis Service Cube with Databricks notebook
I am trying to connect AS Cube with Databricks notebook but unfortunately didn't find any solution yet. is there any possible way to connect AS cube with databricks notebook? if yes can someone please guide me
- 2311 Views
- 3 replies
- 1 kudos
- 1 kudos
I am able to connect Azure analysis services using Azure Analysis services rest api. is yours on-prem?
- 1 kudos
- 43932 Views
- 7 replies
- 11 kudos
Resolved! Automate the Databricks workflow deployment
Hi everyone,Do you guys know if it's possible to automate the Databricks workflow deployment through azure devops (like what we do with the deployment of notebooks)?
- 43932 Views
- 7 replies
- 11 kudos
- 11 kudos
Did you get a chance to try Brickflows - https://github.com/Nike-Inc/brickflowYou can find the documentation here - https://engineering.nike.com/brickflow/v0.11.2/Brickflow uses - Databricks Asset Bundles(DAB) under the hood but provides a Pythonic w...
- 11 kudos
- 3522 Views
- 4 replies
- 0 kudos
Is there a way to catch the cancel button or the interrupt button in a Databricks notebook?
I'm running oracledb package and it uses sessions. When you cancel a running query it doesn't close the session even if you have a try catch block because a cancel or interrupt issues a kill command on the process. Is there a method to catch the canc...
- 3522 Views
- 4 replies
- 0 kudos
- 2455 Views
- 2 replies
- 1 kudos
How to connect to IBM MQ from Databricks notebook?
We are trying to connect to IBM MQ and post message to MQ, which eventually consumed by mainframe application.What are the IBM MQ clients .jars / libraries installed in cluster ? if you have any sample code for connectivity that would be helpful.
- 2455 Views
- 2 replies
- 1 kudos
- 1 kudos
Kindly update if you are able to connect to MQ from Databricks. I am working on same but no luck as I’m unable to install pymqi library on the cluster as its showing error as MQ library could not be found
- 1 kudos
- 30358 Views
- 7 replies
- 1 kudos
How to open Streamlit URL that is hosted by Databricks in local web browser?
I have run this webapp code on Databricks notebook. It works properly without any errors. With databricks acting as server, I am unable open this link on my browser for this webapp.But when I run the code on my local IDE, I am able to just open the U...
- 30358 Views
- 7 replies
- 1 kudos
- 1 kudos
Hi @Akshay Aravinnakshan​ Thank you for posting your question in our community!Your input matters! Help our community thrive by coming back and marking the most helpful and accurate answers. Together, we can make a difference!
- 1 kudos
- 14819 Views
- 5 replies
- 10 kudos
Resolved! SFTP Connect
How I can connect sftp server from databricks. So I can write files into tables directly?
- 14819 Views
- 5 replies
- 10 kudos
- 10 kudos
The classic solution is to copy data from FTP to ADLS storage using Azure Data Factory, and after the copy is done in the ADF pipeline, trigger the databricks notebook.
- 10 kudos
- 4021 Views
- 7 replies
- 0 kudos
- 4021 Views
- 7 replies
- 0 kudos
- 0 kudos
Hi @Ratna Chaitanya Raju Bandaru​Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best? If not, please tell us so we can help you.Thanks!
- 0 kudos