Enable programmatically writing to files

[rubenteixeira]

Hi everyone. I'm training a time series forecasting model in Azure Databricks. When I try to parallelize, it gives me this error:

I have Contributor permission on the Databricks service, on the Azure Portal, and I'm an admin inside the Databricks workspace. What permissions am I missing?

[Hubert-Dudek]

Can't you use data frames API? Using RDD directly is an outdated method as it is not optimized. Dataframe is automatically distributed (parallelized using partitions and cores). Additionally, how big is a dataset? as running collect() is also a killer and should not be used in production or to return more than a dozen rows.

[Debayan]

Hi,

The error says the below filesystem is read-only and not RW.

[jose_gonzalez]

Hi @Rúben Teixeira​,

Just a friendly follow-up. Did any of the responses help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.