I am getting NoneType error when running a query from API on cluster
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-17-2024 07:01 AM
When I am running a query on Databricks itself from notebook, it is running fine and giving me results. But the same query when executed from FastAPI (Python, using databricks library) is giving me "TypeError: 'NoneType' object is not iterable".
I can see both queries (from notebook and from API) from SparkUI list of jobs and both of them are marked as SUCCESS.
Differences between two jobs:
API one doesn't have any stdout logs (giving error "The requested Spark UI page does not exist."), whilst notebook one has logs.
API one has 3 stages in DAG visualization, whilst notebook one has 2 stages (additional stage is mapPartitionsInternal)
API is sending a query to "thriftsever-session-..." pool)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-08-2025 11:20 PM
Hi @Nastia , can you please share the entire stacktrace and the query that you are running.
There is currently not much detail with which I can help you understand this. But it is totally possible it is a bug that's causing this, because there should be a uniform behavior on both API and notebook.
