- 9872 Views
- 4 replies
- 0 kudos
how to pass arguments and variables to databricks python activity from azure data factory
how to pass arguments and variables to databricks python activity from azure data factory
- 9872 Views
- 4 replies
- 0 kudos
- 0 kudos
try importing argv from sys. Then if you have the parameter added correctly in DataFactory you could get it in your python script typing argv[1] (index 0 is the file path).
- 0 kudos
- 1582 Views
- 0 replies
- 0 kudos
docs.databricks.com
What is Databricks Database?A Databricks database is a collection of tables. A Databricks table is a collection of structured data. You can cache, filter, and perform any operations supported by Apache Spark DataFrames on Databricks tables. You can q...
- 1582 Views
- 0 replies
- 0 kudos
- 2971 Views
- 1 replies
- 0 kudos
- 2971 Views
- 1 replies
- 0 kudos
- 0 kudos
You can implement custom algorithms for GraphFrames using either Scala/Java or Python APIs. GraphFrames provides some structures to simplify writing graph algorithms; the three primary options are as follow, with the best options first:Pregel: This i...
- 0 kudos
- 1079 Views
- 0 replies
- 0 kudos
Can I run multiple spark-submits on a job cluster?
I'm trying to run multiple spark submits on a Databricks job cluster but can't figure out how. Any tips?
- 1079 Views
- 0 replies
- 0 kudos
- 2663 Views
- 1 replies
- 0 kudos
Spark JDBC query isn't working for Oracle Databases
I am trying to read with the following syntaxval df = spark.read .format("jdbc") .option("url", "<url>") .option("query", "SELECT * FROM oracle_test_table)") .option("user", "<user>") .option("password", "<password>") .option("driver", "oracle...
- 2663 Views
- 1 replies
- 0 kudos
- 0 kudos
https://kb.databricks.com/data-sources/query-option-not-work-oracle.html#problem-apache-spark-jdbc-datasource-query-option-doesnt-work-for-oracle-database
- 0 kudos
- 3534 Views
- 1 replies
- 0 kudos
- 3534 Views
- 1 replies
- 0 kudos
- 0 kudos
Databricks starts to charge for DBUs once the virtual machine is up and the Spark context is initialized, which may include a portion of start up costs, but not all. Init scripts are loaded before the Spark context is initialized, which therefore wou...
- 0 kudos
- 1663 Views
- 1 replies
- 0 kudos
Does the price increase as I attach more notebooks to the same cluster?
Databricks pricing related question - Do I consume more DBUs when I attach more notebooks to the same cluster?
- 1663 Views
- 1 replies
- 0 kudos
- 0 kudos
Hey PJ, short answer is - No, attaching more notebooks does not increase the price of the cluster, which is solely based on compute power. Attaching more notebooks to the cluster is a value-add of the platform.If you're interested, you can find some ...
- 0 kudos
- 723 Views
- 0 replies
- 0 kudos
Things I want admins to know!
Things I want admins to know!
- 723 Views
- 0 replies
- 0 kudos
- 16611 Views
- 7 replies
- 0 kudos
how to read schema from text file stored in cloud storage
I have file a.csv or a.parquet while creating data frame reading we can explictly define schema with struct type. instead of write the schema in the notebook want to create schema lets say for all my csv i have one schema like csv_schema and stored ...
- 16611 Views
- 7 replies
- 0 kudos
- 0 kudos
@shyampsr big thanks, was searching for the solution almost 3 hours _https://luckycanadian.com/
- 0 kudos
- 1705 Views
- 0 replies
- 0 kudos
foreachBatch in pyspark throwing OSError: [WinError 10022] An invalid argument was supplied
Hello Team,Since last 3 weeks I am trying to move my project from batch to structure streaming.But every time I am running my code I am getting below error:Traceback (most recent call last): File "C:\Users\avisriva1\git_4May2021\comint-ml-scores\src\...
- 1705 Views
- 0 replies
- 0 kudos
- 1050 Views
- 0 replies
- 0 kudos
I am not able to add picture to my profile
How can i add picture to my profile page. Will not save. #Bug​
- 1050 Views
- 0 replies
- 0 kudos
- 990 Views
- 0 replies
- 0 kudos
Kafka-StructuredStreaming-Elastic Search
Can anyone send me any online materials that describe detail end to end setup of Kafka-StructuredStreaming-Elastic Search data flow?
- 990 Views
- 0 replies
- 0 kudos
- 1569 Views
- 2 replies
- 0 kudos
How to retrieve the SQL queries owned by a user who left the organization in SQL Analytics?
How to retrieve the SQL queries owned by a user who left the organization in SQL Analytics?
- 1569 Views
- 2 replies
- 0 kudos
- 6753 Views
- 1 replies
- 0 kudos
- 6753 Views
- 1 replies
- 0 kudos
- 0 kudos
We can use the below steps to push Cluster Logs to Elastic Search:1. Download the log4j-elasticsearch-java-api repo and build the jar file:git clone https://github.com/Downfy/log4j-elasticsearch-java-api.git cd log4j-elasticsearch-java-api/ mvn clean...
- 0 kudos
- 12275 Views
- 1 replies
- 0 kudos
Resolved! How do I download maven libraries with dependencies?
I want to import a maven library with its dependencies. How to do it?
- 12275 Views
- 1 replies
- 0 kudos
- 0 kudos
I recommend creating a UBER jar or download jars offline use it in clusters when the maven becomes healthy again: 1. Install the MVN CLI tool on your local mac: brew install mvnvm2. Download the Artifact with all dependencies: mvn dependency:get -Dr...
- 0 kudos
| User | Count |
|---|---|
| 1619 | |
| 790 | |
| 480 | |
| 349 | |
| 287 |
