cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
Anonymous
by • Not applicable
  • 843 Views
  • 0 replies
  • 0 kudos

Detailed logs for R process

We have a user notebook in R that reliably crashes the driver. Are detailed logs from the R process stored somewhere on drivers/workers?

Data Engineering
Reply
  • 843 Views
  • 0 replies
  • 0 kudos
User16790091296
by • Contributor II
  • 2609 Views
  • 1 replies
  • 0 kudos

Resolved! How can I use a Python function defined in my git-repo module within the DB notebook?

I have a function within a module in my git-repo. I want to import that to my DB notebook - how can I do that?

Data Engineering
Reply
  • 2609 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
aladda
Databricks Employee
  • 0 kudos

Databricks Repos allows you to sync your work in Databricks with a remote Git repository. This makes it easier to implement development best practices. Databricks supports integrations with GitHub, Bitbucket, and GitLab. Using Repos you can bring you...

  • 0 kudos
Anonymous
by • Not applicable
  • 1667 Views
  • 0 replies
  • 0 kudos

Seeing all columns

I have a dataframe with a lot of columns (20 or so) and 8 rows. Part of the output is being cutoff and I can scroll to the right to see the rest of the columns but I was just wondering if it was possible to somehow "zoom out" of the table so I can se...

Data Engineering
Reply
  • 1667 Views
  • 0 replies
  • 0 kudos
MallikSunkara
by • New Contributor II
  • 8896 Views
  • 4 replies
  • 0 kudos

how to pass arguments and variables to databricks python activity from azure data factory

how to pass arguments and variables to databricks python activity from azure data factory

Data Engineering
Reply
  • 8896 Views
  • 4 replies
  • 0 kudos
View Replies
Latest Reply
CristianIspan
New Contributor II
  • 0 kudos

try importing argv from sys. Then if you have the parameter added correctly in DataFactory you could get it in your python script typing argv[1] (index 0 is the file path).

  • 0 kudos
3 More Replies
User16790091296
by • Contributor II
  • 1114 Views
  • 0 replies
  • 0 kudos

docs.databricks.com

What is Databricks Database?A Databricks database is a collection of tables. A Databricks table is a collection of structured data. You can cache, filter, and perform any operations supported by Apache Spark DataFrames on Databricks tables. You can q...

Data Engineering
Reply
  • 1114 Views
  • 0 replies
  • 0 kudos
Joseph_B
by • Databricks Employee
  • 2261 Views
  • 1 replies
  • 0 kudos

How can I implement a custom graph algorithm on top of Apache Spark? Can I use GraphX or GraphFrames?

Data Engineering
Reply
  • 2261 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
Joseph_B
Databricks Employee
  • 0 kudos

You can implement custom algorithms for GraphFrames using either Scala/Java or Python APIs. GraphFrames provides some structures to simplify writing graph algorithms; the three primary options are as follow, with the best options first:Pregel: This i...

  • 0 kudos
User15725630784
by • Databricks Employee
  • 1990 Views
  • 1 replies
  • 0 kudos

Spark JDBC query isn't working for Oracle Databases

I am trying to read with the following syntaxval df = spark.read .format("jdbc") .option("url", "<url>") .option("query", "SELECT * FROM oracle_test_table)") .option("user", "<user>") .option("password", "<password>") .option("driver", "oracle...

Data Engineering
Reply
  • 1990 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
User15725630784
Databricks Employee
  • 0 kudos

https://kb.databricks.com/data-sources/query-option-not-work-oracle.html#problem-apache-spark-jdbc-datasource-query-option-doesnt-work-for-oracle-database

  • 0 kudos
User16790091296
by • Contributor II
  • 2693 Views
  • 1 replies
  • 0 kudos

When do I start getting charged for a DBU? Am I charged for cluster spin up time?

Data Engineering
Reply
  • 2693 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
User16790091296
Contributor II
  • 0 kudos

Databricks starts to charge for DBUs once the virtual machine is up and the Spark context is initialized, which may include a portion of start up costs, but not all. Init scripts are loaded before the Spark context is initialized, which therefore wou...

  • 0 kudos
User16790091296
by • Contributor II
  • 1235 Views
  • 1 replies
  • 0 kudos

Does the price increase as I attach more notebooks to the same cluster?

Databricks pricing related question - Do I consume more DBUs when I attach more notebooks to the same cluster?

Data Engineering
Reply
  • 1235 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
User16790091296
Contributor II
  • 0 kudos

Hey PJ, short answer is - No, attaching more notebooks does not increase the price of the cluster, which is solely based on compute power. Attaching more notebooks to the cluster is a value-add of the platform.If you're interested, you can find some ...

  • 0 kudos

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels