cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
NW1000
New Contributor III
since ‎09-04-2025
a week ago

User Stats

  • 8 Posts
  • 0 Solutions
  • 0 Kudos given
  • 0 Kudos received

User Activity

Use .R file in data pipeline

Tuesday
In general R pipeline, we can use source("abc.R") file. However, it does not work in Databricks. I got errors: source("./abc.R"), but the error say: Error in file(filename, "r", encoding = encoding) : cannot open the connectionHow best to build a pip...

Shorten Classic Cluster start up time

2 weeks ago
We use R notebooks to generate workflow. Thus we have to use classic clusters. And we need roughly 10 additional R packages in addition to 2 pyPI packages. It takes at least 10-20 min to start the cluster. We found the most time taken were the packag...

Unable to access files using a classic cluster

3 weeks ago
I used the same code with the classic cluster (RunTime 17.3LTS ML, with spark config: "spark.databricks.workspace.fileSystem.enabled true"), not able to access files in workspace with the following python code: import os# Check if source exists and w...

Drop table not working consistently

4 weeks ago
During development, I drop table table_frq in SQL query mannually. Then run a python notebook using serverless compute, use spark.catalog.tableExists(table_frq) as the condition. Last week, after drop table, spark.catalog.tableExists(table_frq) showe...