cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Data + AI Summit 2024 - Data Engineering & Streaming

Forum Posts

Start a conversation
brickster_2018
by • Databricks Employee
  • 918 Views
  • 1 replies
  • 1 kudos

Can I run the Python2 Pyspark application in my Databricks GCP workspace

Data Engineering
Reply
  • 918 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
aladda
Databricks Employee
  • 1 kudos

For Databricks Runtime 5.5 LTS, Spark jobs, Python notebook cells, and library installation all support both Python 2 and 3.The default Python version for clusters created using the UI is Python 3. In Databricks Runtime 5.5 LTS the default version fo...

  • 1 kudos
User16869509994
by • New Contributor II
  • 1141 Views
  • 1 replies
  • 1 kudos

In Databricks UI /Workspace and /Repos are in same level but while reading a CSV file in Repos Notebooks why do we need to give the path as /Workspace/Repos...?

Data Engineering
Reply
  • 1141 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
aladda
Databricks Employee
  • 1 kudos

Can you provide an example of what exactly do you mean? If the reference is to how "Repos" shows up in the UI, that's more for a Ux convenience. Repos as such are designed to be a container for version controlled notebooks that live in the Git reposi...

  • 1 kudos
User16790091296
by • Contributor II
  • 1824 Views
  • 2 replies
  • 0 kudos

What is the difference between delta lake (Databricks) and Delta Lake (Open Source -Maven) ?

Data Engineering
Reply
  • 1824 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
aladda
Databricks Employee
  • 0 kudos

Delta Lake on Databricks has added runtime optimizations of the Delta Engine that further enhance the performance and scale of the open source Delta Format. In additional you also get access to a whole host of capabilities available on the Databricks...

  • 0 kudos
1 More Replies
MoJaMa
by • Databricks Employee
  • 3339 Views
  • 1 replies
  • 1 kudos

Can ADF pass values/info back to ADF from the Notebook?

Data Engineering
Reply
  • 3339 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
MoJaMa
Databricks Employee
  • 1 kudos

Yes, you can pass parameters from ADF —> Azure Databricks.https://docs.microsoft.com/en-us/azure/data-factory/solution-template-databricks-notebook#how-to-use-this-templateYou can also pass values back from the Notebook --> ADF via the dbutils.notebo...

  • 1 kudos
brickster_2018
by • Databricks Employee
  • 1253 Views
  • 1 replies
  • 0 kudos

Maven library conflicts

The import statements work fine with maven libraries freshly installed on a running cluster and on restart of the same cluster, it fails to import the classes.

Data Engineering
Reply
  • 1253 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
brickster_2018
Databricks Employee
  • 0 kudos

When you install maven libraries one by one in running clusters, it is resolved and gets downloaded individually. The behavior would be different when you restart the clusters since the libraries are resolved together and can run into issues due to c...

  • 0 kudos
MoJaMa
by • Databricks Employee
  • 922 Views
  • 1 replies
  • 0 kudos

I'm cleaning up users in my workspace. For Clusters, the message mentions the cluster will be terminated but does the cluster also get deleted? Or can someone with Can Manage permissions just re-start the cluster?

Data Engineering
Reply
  • 922 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
MoJaMa
Databricks Employee
  • 0 kudos

The cluster will be terminated. But anyone with CAN_MANAGE can continue to start it, or clone it, and then delete it.

  • 0 kudos

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels