cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
User16826990884
by • Databricks Employee
  • 2281 Views
  • 1 replies
  • 0 kudos

Disable managed tables on Azure Databricks

When a user creates a table without a path, it writes it as a managed table in the root bucket. Can this functionality be disabled so users are forced to provide a storage path and follow our organization best practices?

Data Engineering
Reply
  • 2281 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
florent
New Contributor III
  • 0 kudos

Hi,I couldn't find the option to turn it off. It might be worth banning dbfs. However, you can configure the default location using a cluster policy by adding the following configuration:"spark_conf.spark.sql.warehouse.dir": {  "type": "fixed",  "val...

  • 0 kudos
rbarata
by • New Contributor II
  • 3413 Views
  • 1 replies
  • 5 kudos

Resolved! Pyspark environment and python packages on executors

At my company we use conda-pack to make certain packages available on the spark executors. Is there a better a better alternative to get away from creating a new environment and pack it every time I need a new python lib to be available for the execu...

Data Engineering
Reply
  • 3413 Views
  • 1 replies
  • 5 kudos
View Replies
Latest Reply
-werners-
Esteemed Contributor III
  • 5 kudos

databricks provides library installation in the form of PyPi packages, or wheel/egg.If you install the packages like that on the cluster, they are automatically sent to all executors.

  • 5 kudos
JBOCACHICA
by • Databricks Partner
  • 1461 Views
  • 0 replies
  • 1 kudos

Primer vez en este evento, la verdad muy buen evento, aunque este foro no tiene contenido en español, la comunidad hispano parlante está creciendo y e...

Primer vez en este evento, la verdad muy buen evento, aunque este foro no tiene contenido en español, la comunidad hispano parlante está creciendo y esperamos poder aportar en el desarrollo de nuestros paises a traves de la tecnologia!.First time on ...

Data Engineering
Reply
  • 1461 Views
  • 0 replies
  • 1 kudos
Jingalls
by • New Contributor II
  • 1177 Views
  • 1 replies
  • 2 kudos

The Data + AI summit is a blast so far. There are so many new technologies being released such as Delta Lake ​2.0 being open source.

The Data + AI summit is a blast so far. There are so many new technologies being released such as Delta Lake ​2.0 being open source.

Data Engineering
Reply
  • 1177 Views
  • 1 replies
  • 2 kudos
View Replies
Latest Reply
Zzof
New Contributor II
  • 2 kudos

Agreed! You should check out the Azure booth if you haven't already they have a really cool demo.​

  • 2 kudos
christys
by • Databricks Employee
  • 1682 Views
  • 1 replies
  • 7 kudos

Databricks SQL Serverless Now Available on AWS First announcement at Data + AI Summit 2022 is on the availability of serverless compute for Databricks...

Databricks SQL Serverless Now Available on AWSFirst announcement at Data + AI Summit 2022 is on the availability of serverless compute for Databricks SQL (DBSQL) in Public Preview on AWS! DB SQL Serverless makes it easy to get started with data wareh...

Data Engineering
Reply
  • 1682 Views
  • 1 replies
  • 7 kudos
View Replies
Latest Reply
js54123875
New Contributor III
  • 7 kudos

Awesome! We definitely want to join this preview. This is the missing piece for us in making data accessible from the data lake to end users, because it will help with the compute costs. I can't wait to learn more. #Summit22​ 

  • 7 kudos
Gchenzs
by • Databricks Partner
  • 657 Views
  • 0 replies
  • 2 kudos

1198646F-C951-4C40-AC6B-EE741EEF4F4C

As a SI partner of Databricks, we are excited to attend the summit and co-present with our client Amgen. Come see us in action at 3PM, check the app for the location!#ss22

Data Engineering
Reply
  • 657 Views
  • 0 replies
  • 2 kudos
sbo
by • New Contributor III
  • 1555 Views
  • 1 replies
  • 0 kudos

Resolved! What does Zorder do?

Data Engineering
Reply
  • 1555 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
Advana
New Contributor III
  • 0 kudos

Speeds up data retrieval when filling in provided fields by collocating data

  • 0 kudos
BradC26
by • New Contributor III
  • 2002 Views
  • 2 replies
  • 0 kudos

Resolved! Can I change the experiment location for a model?

We are looking to move some of the artifacts to S3 from dbfs.

Data Engineering
Reply
  • 2002 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
sbo
New Contributor III
  • 0 kudos

Yes you can do it via the mlflow python API. However know that when you do that and move the experiment location to S3 the Databricks UI interface doesnt allow interaction with the artifacts etc. the mlflow python API is rich with the abilities to in...

  • 0 kudos
1 More Replies
Labels