cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
Eelke
by • New Contributor II
  • 1897 Views
  • 2 replies
  • 2 kudos

I would like to create a schedule in Databricks that runs a job every two weeks on Monday night 0:00

This seems impossible with the cron that databricks is using but maybe I am wrong? However, if this is not possible it seems to me a missing feature, and thereby would like to suggest this feature to you

Data Engineering
Reply
  • 1897 Views
  • 2 replies
  • 2 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Eelke van Foeken​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.T...

  • 2 kudos
1 More Replies
anonturtle
by • New Contributor
  • 1154 Views
  • 1 replies
  • 0 kudos

How does automl classify which feature is numeric or categorical?

When running automl on its UI, it classifies a feature "local_convenience_store" as both a numeric and categorical column. This affects the result as for numeric columns a scaler is used while in a categorical column it is one hot encoded. For contex...

Data Engineering
Reply
  • 1154 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 0 kudos

@hr then​ :The approach taken by AutoML to classify features as numeric or categorical depends on the specific AutoML framework or library being used, as different implementations may use different methods or heuristics to make this determination.In ...

  • 0 kudos
Aravind_P04
by • New Contributor II
  • 3009 Views
  • 3 replies
  • 4 kudos

Clarification on merging multiple notebooks and other

1. Do we have any feature like merge the cells from one or more notebooks into other notebook.2. Do we have any feature like multiple cells from excel is copied it into multiple cells in a notebook . Generally all excel data is copied it into one cel...

Data Engineering
Reply
  • 3009 Views
  • 3 replies
  • 4 kudos
View Replies
Latest Reply
youssefmrini
Honored Contributor III
  • 4 kudos

1) We can't merge cells right now2)We don't have this feature as well3) We don't have multiple editing right now4)You will know only if you face an error. A Notification will pop up5)You can"t keep running the execution because the cells can be linke...

  • 4 kudos
2 More Replies
laurencewells
by • New Contributor III
  • 3723 Views
  • 5 replies
  • 1 kudos

Autoloader and "cleanSource"

Hi All, We are trying to use the Spark 3 structured streaming feature/option ".option('cleanSource','archive')" to archive processed files. This is working as expected using the standard spark implementation, however does not appear to work using aut...

Data Engineering
Reply
  • 3723 Views
  • 5 replies
  • 1 kudos
View Replies
Latest Reply
-werners-
Esteemed Contributor III
  • 1 kudos

https://docs.databricks.com/ingestion/auto-loader/options.html#common-auto-loader-optionscleanSource is not a listed option so it won't do anything.Maybe event retention is something you can use?

  • 1 kudos
4 More Replies
tomsyouruncle
by • New Contributor III
  • 15967 Views
  • 20 replies
  • 4 kudos

Resolved! How do I enable support for arbitrary files in Databricks Repos? Public Preview feature doesn't appear in admin console.

"Arbitrary files in Databricks Repos", allowing not just notebooks to be added to repos, is in Public Preview. I've tried to activate it following the instructions in the above link but the option doesn't appear in Admin Console. Minimum requirements...

image repos
Data Engineering
Reply
  • 15967 Views
  • 20 replies
  • 4 kudos
View Replies
Latest Reply
Kaniz_Fatma
Community Manager
  • 4 kudos

Hi @Tom Turner​ , An admin can enable this feature as follows:Go to the Admin Console.Click the Workspace Settings tab.In the Repos section, click the Files in Repos toggle.After the feature has been enabled, you must restart your cluster and refresh...

  • 4 kudos
19 More Replies
Erik
by • Valued Contributor II
  • 1607 Views
  • 4 replies
  • 3 kudos

Feature request: It is possible to add comments to both databricks sql databases and tables. It would be really usefull if these comments could show u...

Feature request: It is possible to add comments to both databricks sql databases and tables. It would be really usefull if these comments could show up (if they are provided) in PowerBI when one connects to the Databricks SQL endpoint, e.g. in this w...

bilde
Data Engineering
Reply
  • 1607 Views
  • 4 replies
  • 3 kudos
View Replies
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 3 kudos

Nice idea!

  • 3 kudos
3 More Replies
User16826992666
by • Valued Contributor
  • 1010 Views
  • 1 replies
  • 0 kudos

Resolved! If I create a Feature Store, how is the underlying data actually saved?

And do I have any control over where and how it's saved?

Data Engineering
Reply
  • 1010 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
sajith_appukutt
Honored Contributor II
  • 0 kudos

The offline store is backed by Delta tables . In AWS we support Amazon Aurora (MySQL-compatible) & Amazon RDS MySQL and in Azure we support Azure Database for MySQL and Azure SQL Database as as online stores https://docs.microsoft.com/en-us/azure/d...

  • 0 kudos
Labels