Databricks Community

pavlosskev · 07-16-2025

Hi everyone,I'm trying to read data from an Oracle database into Databricks using JDBC with timestamp-based partitioning. However, it seems that the partitioning doesn't work as expected when I specify lowerBound and upperBound using timestamp string...

pavlosskev · 08-01-2024

This is not a question, this is just the solution to a problem we encountered in case someone from the community finds it useful.Recently we encountered an issue, where our users' jobs started failing out of nowhere on the following command, with the...

pavlosskev · 07-16-2025

@joshuat Hello. Did you manage to solve this issue? We are having a similar problem, where we can't read Oracle data partitioned, unless the lower and upper bounds are integrers.

pavlosskev · 10-25-2024

Fermin_vicente's answer is pretty good already. Below is how you can do something similar with conftest.py# conftest.py import pytest from unittest.mock import MagicMock from pyspark.sql import SparkSession @pytest.fixture(scope="session") def dbuti...

pavlosskev · 08-15-2024

I don't clearly understand your full problem. But I do know the following regarding UDFs:1. PySpark UDFs are extremely slow, because it needs to deserialize the Java Object (DataFrame), transform it with the Python UDF, then Serialize it back. This h...

pavlosskev · 08-02-2024

If the validation is fine on your colleague's laptop and not on yours, my first assumption would be that it's a version issue. Do you have the same Databricks CLI version as your colleagues? You can check with databricks --version Also according to...

Databricks Community

User Stats

User Activity

Oracle JDBC Load Fails with Timestamp Partitioning (lowerBound/upperBound)

Databricks java.util.NoSuchElementException: None.get Error on "SHOW TABLES IN" command

Re: How to partition JDBC Oracle read query and cast with TO_DATE on partition date field?

Re: Python mocking dbutils in unittests

Re: How to solve udf performance issue with databricks sql function?

Re: Complex type variable in Databricks.yml not working