cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
RKNutalapati
by • Valued Contributor
  • 4058 Views
  • 5 replies
  • 4 kudos

Read and saving Blob data from oracle to databricks S3 is slow

I am trying to import a table from oracle which has around 1.3 mill rows and one of the column is a Blob, the total size of data on oracle is around 250+ GB. read and save to S3 as delta table is taking around 60 min. I tried with parallel(200 thread...

Data Engineering
Reply
  • 4058 Views
  • 5 replies
  • 4 kudos
View Replies
Latest Reply
vinita_mehta
New Contributor II
  • 4 kudos

Any update on this topic what should be the best option to read from oracle and write in ADLS. 

  • 4 kudos
4 More Replies
bkbarunkalita64
by • New Contributor II
  • 1649 Views
  • 1 replies
  • 2 kudos

Update query from Azure databricks to Oracle table

I need to update a single row on a on-prem Oracle table via jdbc connection.Please note, I don't want to append, just have to update a row, is it possible ?

Data Engineering
Reply
  • 1649 Views
  • 1 replies
  • 2 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Tony S​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.

  • 2 kudos
Teja07
by • New Contributor II
  • 6791 Views
  • 0 replies
  • 0 kudos

Ingesting data from oracle to databricks through IICS

While ingesting the data from oracle to databricks through IICS, target table were created however data is not getting inserted. Below is the error. Could someone please help meException occurred when initializing data session. Root cause: java.lang....

Data Engineering
Reply
  • 6791 Views
  • 0 replies
  • 0 kudos
elgeo
by • Valued Contributor II
  • 8762 Views
  • 4 replies
  • 0 kudos

Function returns UNSUPPORTED_CORRELATED_SCALAR_SUBQUERY

Hello experts. The below function in Databricks gives UNSUPPORTED_CORRELATED_SCALAR_SUBQUERY error. We didn't have this issue though in Oracle. Is this a limitation of Databricks? Just to note the final result returns only one row. Thank you in advan...

Data Engineering
Reply
  • 8762 Views
  • 4 replies
  • 0 kudos
View Replies
Latest Reply
TheofilosSt
New Contributor II
  • 0 kudos

Hello @Suteja Kanuri​  can we have any respond on the above?Thank you.

  • 0 kudos
3 More Replies
Anonymous
by • Not applicable
  • 7841 Views
  • 8 replies
  • 0 kudos

Not able to connect to On-Prem Oracle from Databricks cluster

Hi Everyone,I was trying to connect to Oracle Instance from Databricks cluster and it is giving below error:java.sql.SQLTimeoutException: ORA-12170: Cannot connect. TCP connect timeout of 30000ms for host xx.x.x.*** port 1521. (CONNECTION_ID=CgM7V7UB...

Data Engineering
Reply
  • 7841 Views
  • 8 replies
  • 0 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 0 kudos

@Satya89:The error message you received indicates that the TCP connection to the Oracle database timed out. This could be caused by a number of factors such as network issues, firewall restrictions, or the database being overloaded.Here are a few ste...

  • 0 kudos
7 More Replies
quakenbush
by • Contributor
  • 6808 Views
  • 1 replies
  • 0 kudos

Is there something like Oracle's VPD-Feature in Databricks?

Since I am porting some code from Oracle to Databricks, I have another specific question.In Oracle there's something called Virtual Private Database, VPD. It's a simple security feature used to generate a WHERE-clause which the system will add to a u...

Data Engineering
Reply
  • 6808 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 0 kudos

@Roger Bieri​ :In Databricks, you can use the UserDefinedFunction (UDF) feature to create a custom function that will be applied to a DataFrame. You can use this feature to add a WHERE clause to a DataFrame based on the user context. Here's an exampl...

  • 0 kudos
MaheshDR
by • New Contributor II
  • 8910 Views
  • 6 replies
  • 1 kudos

Open firewall to Azure Databricks workspace from AWS RDS machine/EC2 machine

Hi All,As part of our solution approach, we need to connect to one of our AWS RDS Oracle databases from Azure Databricks notebook.We need your help to understand which IP range of Azure Databricks to consider to whitelist them on AWS RDS security gro...

Data Engineering
Reply
  • 8910 Views
  • 6 replies
  • 1 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Mahesh D​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 1 kudos
5 More Replies
pandu
by • New Contributor II
  • 2573 Views
  • 2 replies
  • 3 kudos

connect to Oracle database using JDBC and perform merge condition

I would like to connect to oracle database using JDBC driver and write a code to perform merge condition using python.

Data Engineering
Reply
  • 2573 Views
  • 2 replies
  • 3 kudos
View Replies
Latest Reply
Vartika
Databricks Employee
  • 3 kudos

Hi @Venkata Krishna Jonnalagadda​ Hope you are well.Just checking in. If @John Lourdu​'s answer helped, would you let us know and mark the answer as best? If not, would you be happy to give us more information?Thanks!

  • 3 kudos
1 More Replies
Raghu101
by • New Contributor III
  • 18551 Views
  • 5 replies
  • 8 kudos

Resolved! Databricks to Oracle

How to write data from Databricks SQL to Oracle DB

Data Engineering
Reply
  • 18551 Views
  • 5 replies
  • 8 kudos
View Replies
Latest Reply
ramravi
Contributor II
  • 8 kudos

we can use JDBC driver to write dataframe to Oracle tables. Every database will use jdbc connect to connect & access database. You can follow same process for connecting to any database.Download Oracle ojdbc6.jar JDBC DriverYou need an Oracle jdbc dr...

  • 8 kudos
4 More Replies
quakenbush
by • Contributor
  • 4182 Views
  • 3 replies
  • 4 kudos

Resolved! Does Databricks offer something like Oracle's dblink?

I am aware, I can load anything into a DataFrame using JDBC, that works well from Oracle sources. Is there an equivalent in Spark SQL, so I can combine datasets as well?Basically something like so - you get the idea...select lt.field1, rt.fie...

Data Engineering
Reply
  • 4182 Views
  • 3 replies
  • 4 kudos
View Replies
Latest Reply
quakenbush
Contributor
  • 4 kudos

Thanks everyone for helping.

  • 4 kudos
2 More Replies
Raghu101
by • New Contributor III
  • 5604 Views
  • 6 replies
  • 3 kudos

How to Call Oracle Stored Procedures from Databricks?

How to Call Oracle Stored Procedures from Databricks?

Data Engineering
Reply
  • 5604 Views
  • 6 replies
  • 3 kudos
View Replies
Latest Reply
sher
Valued Contributor II
  • 3 kudos

https://datathirst.net/blog/2018/10/12/executing-sql-server-stored-procedures-on-databricks-pyspark/try this link. this may help you

  • 3 kudos
5 More Replies
Dinu2
by • New Contributor III
  • 2048 Views
  • 1 replies
  • 1 kudos

base64 encode is not matching with Oracle's base64 encode

Hi , base64 encode is not matching with Oracle's base64 encode. please see below result. Could anyone help me on this?In Azure Databricks: encoded= base64.b64encode(b'952B8D04E5CFB9BE')output is - b'OTUyQjhEMDRFNUNGQjlCRQ=='In Oracle: select utl_enco...

Data Engineering
Reply
  • 2048 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
daniel_sahal
Esteemed Contributor
  • 1 kudos

Oracle handles base64 encoding a little bit differently.Please check this link to understand what's the difference:https://dba.stackexchange.com/a/129134

  • 1 kudos
prasannar
by • New Contributor II
  • 3524 Views
  • 3 replies
  • 3 kudos

How to write dataframe to Oracle from Databricks ?

Data Engineering
Reply
  • 3524 Views
  • 3 replies
  • 3 kudos
View Replies
Latest Reply
sher
Valued Contributor II
  • 3 kudos

Df.write.format('jdbc').options( url='jdbc:oracle:thin:@192.168.11.100:1521:ORCL', driver='oracle.jdbc.driver.OracleDriver', dbtable='testschema.test', user='testschema', password='password').mode('overwrite').save()try ...

  • 3 kudos
2 More Replies
User15725630784
by • Databricks Employee
  • 1941 Views
  • 1 replies
  • 0 kudos

Spark JDBC query isn't working for Oracle Databases

I am trying to read with the following syntaxval df = spark.read .format("jdbc") .option("url", "<url>") .option("query", "SELECT * FROM oracle_test_table)") .option("user", "<user>") .option("password", "<password>") .option("driver", "oracle...

Data Engineering
Reply
  • 1941 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
User15725630784
Databricks Employee
  • 0 kudos

https://kb.databricks.com/data-sources/query-option-not-work-oracle.html#problem-apache-spark-jdbc-datasource-query-option-doesnt-work-for-oracle-database

  • 0 kudos
Labels