cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
bkbarunkalita64
by • New Contributor II
  • 1278 Views
  • 2 replies
  • 2 kudos

Update query from Azure databricks to Oracle table

I need to update a single row on a on-prem Oracle table via jdbc connection.Please note, I don't want to append, just have to update a row, is it possible ?

Data Engineering
Reply
  • 1278 Views
  • 2 replies
  • 2 kudos
View Replies
Latest Reply
Kaniz_Fatma
Community Manager
  • 2 kudos

Hi @Tony S​ , This article mentioned all the steps required to establish a successful connection with the database using Python.However, you can try this code :- df.write.format("jdbc") \ .option("url", dbServerJdbcUrl) \ .option("user", username...

  • 2 kudos
1 More Replies
Teja07
by • New Contributor II
  • 6632 Views
  • 0 replies
  • 0 kudos

Ingesting data from oracle to databricks through IICS

While ingesting the data from oracle to databricks through IICS, target table were created however data is not getting inserted. Below is the error. Could someone please help meException occurred when initializing data session. Root cause: java.lang....

Data Engineering
Reply
  • 6632 Views
  • 0 replies
  • 0 kudos
elgeo
by • Valued Contributor II
  • 7666 Views
  • 4 replies
  • 0 kudos

Function returns UNSUPPORTED_CORRELATED_SCALAR_SUBQUERY

Hello experts. The below function in Databricks gives UNSUPPORTED_CORRELATED_SCALAR_SUBQUERY error. We didn't have this issue though in Oracle. Is this a limitation of Databricks? Just to note the final result returns only one row. Thank you in advan...

Data Engineering
Reply
  • 7666 Views
  • 4 replies
  • 0 kudos
View Replies
Latest Reply
TheofilosSt
New Contributor II
  • 0 kudos

Hello @Suteja Kanuri​  can we have any respond on the above?Thank you.

  • 0 kudos
3 More Replies
Anonymous
by • Not applicable
  • 5829 Views
  • 8 replies
  • 0 kudos

Not able to connect to On-Prem Oracle from Databricks cluster

Hi Everyone,I was trying to connect to Oracle Instance from Databricks cluster and it is giving below error:java.sql.SQLTimeoutException: ORA-12170: Cannot connect. TCP connect timeout of 30000ms for host xx.x.x.*** port 1521. (CONNECTION_ID=CgM7V7UB...

Data Engineering
Reply
  • 5829 Views
  • 8 replies
  • 0 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 0 kudos

@Satya89:The error message you received indicates that the TCP connection to the Oracle database timed out. This could be caused by a number of factors such as network issues, firewall restrictions, or the database being overloaded.Here are a few ste...

  • 0 kudos
7 More Replies
quakenbush
by • Contributor
  • 6511 Views
  • 1 replies
  • 0 kudos

Is there something like Oracle's VPD-Feature in Databricks?

Since I am porting some code from Oracle to Databricks, I have another specific question.In Oracle there's something called Virtual Private Database, VPD. It's a simple security feature used to generate a WHERE-clause which the system will add to a u...

Data Engineering
Reply
  • 6511 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 0 kudos

@Roger Bieri​ :In Databricks, you can use the UserDefinedFunction (UDF) feature to create a custom function that will be applied to a DataFrame. You can use this feature to add a WHERE clause to a DataFrame based on the user context. Here's an exampl...

  • 0 kudos
MaheshDR
by • New Contributor II
  • 8011 Views
  • 6 replies
  • 1 kudos

Open firewall to Azure Databricks workspace from AWS RDS machine/EC2 machine

Hi All,As part of our solution approach, we need to connect to one of our AWS RDS Oracle databases from Azure Databricks notebook.We need your help to understand which IP range of Azure Databricks to consider to whitelist them on AWS RDS security gro...

Data Engineering
Reply
  • 8011 Views
  • 6 replies
  • 1 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Mahesh D​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 1 kudos
5 More Replies
pandu
by • New Contributor II
  • 2013 Views
  • 2 replies
  • 3 kudos

connect to Oracle database using JDBC and perform merge condition

I would like to connect to oracle database using JDBC driver and write a code to perform merge condition using python.

Data Engineering
Reply
  • 2013 Views
  • 2 replies
  • 3 kudos
View Replies
Latest Reply
Vartika
Moderator
  • 3 kudos

Hi @Venkata Krishna Jonnalagadda​ Hope you are well.Just checking in. If @John Lourdu​'s answer helped, would you let us know and mark the answer as best? If not, would you be happy to give us more information?Thanks!

  • 3 kudos
1 More Replies
Raghu101
by • New Contributor III
  • 13401 Views
  • 6 replies
  • 8 kudos

Resolved! Databricks to Oracle

How to write data from Databricks SQL to Oracle DB

Data Engineering
Reply
  • 13401 Views
  • 6 replies
  • 8 kudos
View Replies
Latest Reply
ramravi
Contributor II
  • 8 kudos

we can use JDBC driver to write dataframe to Oracle tables. Every database will use jdbc connect to connect & access database. You can follow same process for connecting to any database.Download Oracle ojdbc6.jar JDBC DriverYou need an Oracle jdbc dr...

  • 8 kudos
5 More Replies
quakenbush
by • Contributor
  • 3258 Views
  • 4 replies
  • 5 kudos

Resolved! Does Databricks offer something like Oracle's dblink?

I am aware, I can load anything into a DataFrame using JDBC, that works well from Oracle sources. Is there an equivalent in Spark SQL, so I can combine datasets as well?Basically something like so - you get the idea...select lt.field1, rt.fie...

Data Engineering
Reply
  • 3258 Views
  • 4 replies
  • 5 kudos
View Replies
Latest Reply
Kaniz_Fatma
Community Manager
  • 5 kudos

Hi @Roger Bieri​  (Customer)​, I appreciate your attempt to choose the best answer for us. I'm glad you got your query resolved. @Joseph Kambourakis​ and @Adrian Łobacz​, Thank you for giving excellent answers .

  • 5 kudos
3 More Replies
Dinu2
by • New Contributor III
  • 1621 Views
  • 2 replies
  • 1 kudos

base64 encode is not matching with Oracle's base64 encode

Hi , base64 encode is not matching with Oracle's base64 encode. please see below result. Could anyone help me on this?In Azure Databricks: encoded= base64.b64encode(b'952B8D04E5CFB9BE')output is - b'OTUyQjhEMDRFNUNGQjlCRQ=='In Oracle: select utl_enco...

Data Engineering
Reply
  • 1621 Views
  • 2 replies
  • 1 kudos
View Replies
Latest Reply
Kaniz_Fatma
Community Manager
  • 1 kudos

Hi @Dinu Sukumara​, We haven’t heard from you since the last response from @Daniel Sahal​, and I was checking back to see if his suggestions helped you.Or else, If you have any solution, please do share that with the community as it can be helpful to...

  • 1 kudos
1 More Replies
Raghu101
by • New Contributor III
  • 4448 Views
  • 6 replies
  • 3 kudos

How to Call Oracle Stored Procedures from Databricks?

How to Call Oracle Stored Procedures from Databricks?

Data Engineering
Reply
  • 4448 Views
  • 6 replies
  • 3 kudos
View Replies
Latest Reply
sher
Valued Contributor II
  • 3 kudos

https://datathirst.net/blog/2018/10/12/executing-sql-server-stored-procedures-on-databricks-pyspark/try this link. this may help you

  • 3 kudos
5 More Replies
prasannar
by • New Contributor II
  • 2824 Views
  • 4 replies
  • 3 kudos

How to write dataframe to Oracle from Databricks ?

Data Engineering
Reply
  • 2824 Views
  • 4 replies
  • 3 kudos
View Replies
Latest Reply
sher
Valued Contributor II
  • 3 kudos

Df.write.format('jdbc').options( url='jdbc:oracle:thin:@192.168.11.100:1521:ORCL', driver='oracle.jdbc.driver.OracleDriver', dbtable='testschema.test', user='testschema', password='password').mode('overwrite').save()try ...

  • 3 kudos
3 More Replies
RKNutalapati
by • Valued Contributor
  • 3083 Views
  • 4 replies
  • 4 kudos

Read and saving Blob data from oracle to databricks S3 is slow

I am trying to import a table from oracle which has around 1.3 mill rows and one of the column is a Blob, the total size of data on oracle is around 250+ GB. read and save to S3 as delta table is taking around 60 min. I tried with parallel(200 thread...

Data Engineering
Reply
  • 3083 Views
  • 4 replies
  • 4 kudos
View Replies
Latest Reply
User16829050420
New Contributor III
  • 4 kudos

Hello @Rama Krishna N​ - We will need to check the task on the Spark UI to validate if the operation is a read from oracle database or write into S3. The task should show the specific operation on the UI.Also, the active threads on the Spark UI will ...

  • 4 kudos
3 More Replies
User15725630784
by • New Contributor II
  • 1389 Views
  • 1 replies
  • 0 kudos

Spark JDBC query isn't working for Oracle Databases

I am trying to read with the following syntaxval df = spark.read .format("jdbc") .option("url", "<url>") .option("query", "SELECT * FROM oracle_test_table)") .option("user", "<user>") .option("password", "<password>") .option("driver", "oracle...

Data Engineering
Reply
  • 1389 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
User15725630784
New Contributor II
  • 0 kudos

https://kb.databricks.com/data-sources/query-option-not-work-oracle.html#problem-apache-spark-jdbc-datasource-query-option-doesnt-work-for-oracle-database

  • 0 kudos
Labels