- 3880 Views
- 1 replies
- 0 kudos
- 3880 Views
- 1 replies
- 0 kudos
- 0 kudos
You can implement custom algorithms for GraphFrames using either Scala/Java or Python APIs. GraphFrames provides some structures to simplify writing graph algorithms; the three primary options are as follow, with the best options first:Pregel: This i...
- 0 kudos
- 1352 Views
- 0 replies
- 0 kudos
Can I run multiple spark-submits on a job cluster?
I'm trying to run multiple spark submits on a Databricks job cluster but can't figure out how. Any tips?
- 1352 Views
- 0 replies
- 0 kudos
- 3355 Views
- 1 replies
- 0 kudos
Spark JDBC query isn't working for Oracle Databases
I am trying to read with the following syntaxval df = spark.read .format("jdbc") .option("url", "<url>") .option("query", "SELECT * FROM oracle_test_table)") .option("user", "<user>") .option("password", "<password>") .option("driver", "oracle...
- 3355 Views
- 1 replies
- 0 kudos
- 0 kudos
https://kb.databricks.com/data-sources/query-option-not-work-oracle.html#problem-apache-spark-jdbc-datasource-query-option-doesnt-work-for-oracle-database
- 0 kudos
- 4516 Views
- 1 replies
- 0 kudos
- 4516 Views
- 1 replies
- 0 kudos
- 0 kudos
Databricks starts to charge for DBUs once the virtual machine is up and the Spark context is initialized, which may include a portion of start up costs, but not all. Init scripts are loaded before the Spark context is initialized, which therefore wou...
- 0 kudos
- 2121 Views
- 1 replies
- 0 kudos
Does the price increase as I attach more notebooks to the same cluster?
Databricks pricing related question - Do I consume more DBUs when I attach more notebooks to the same cluster?
- 2121 Views
- 1 replies
- 0 kudos
- 0 kudos
Hey PJ, short answer is - No, attaching more notebooks does not increase the price of the cluster, which is solely based on compute power. Attaching more notebooks to the cluster is a value-add of the platform.If you're interested, you can find some ...
- 0 kudos
- 986 Views
- 0 replies
- 0 kudos
Things I want admins to know!
Things I want admins to know!
- 986 Views
- 0 replies
- 0 kudos
- 20090 Views
- 7 replies
- 0 kudos
how to read schema from text file stored in cloud storage
I have file a.csv or a.parquet while creating data frame reading we can explictly define schema with struct type. instead of write the schema in the notebook want to create schema lets say for all my csv i have one schema like csv_schema and stored ...
- 20090 Views
- 7 replies
- 0 kudos
- 0 kudos
@shyampsr big thanks, was searching for the solution almost 3 hours _https://luckycanadian.com/
- 0 kudos
- 2064 Views
- 0 replies
- 0 kudos
foreachBatch in pyspark throwing OSError: [WinError 10022] An invalid argument was supplied
Hello Team,Since last 3 weeks I am trying to move my project from batch to structure streaming.But every time I am running my code I am getting below error:Traceback (most recent call last): File "C:\Users\avisriva1\git_4May2021\comint-ml-scores\src\...
- 2064 Views
- 0 replies
- 0 kudos
- 1325 Views
- 0 replies
- 0 kudos
I am not able to add picture to my profile
How can i add picture to my profile page. Will not save. #Bug​
- 1325 Views
- 0 replies
- 0 kudos
- 1230 Views
- 0 replies
- 0 kudos
Kafka-StructuredStreaming-Elastic Search
Can anyone send me any online materials that describe detail end to end setup of Kafka-StructuredStreaming-Elastic Search data flow?
- 1230 Views
- 0 replies
- 0 kudos
- 2073 Views
- 2 replies
- 0 kudos
How to retrieve the SQL queries owned by a user who left the organization in SQL Analytics?
How to retrieve the SQL queries owned by a user who left the organization in SQL Analytics?
- 2073 Views
- 2 replies
- 0 kudos
- 7616 Views
- 1 replies
- 0 kudos
- 7616 Views
- 1 replies
- 0 kudos
- 0 kudos
We can use the below steps to push Cluster Logs to Elastic Search:1. Download the log4j-elasticsearch-java-api repo and build the jar file:git clone https://github.com/Downfy/log4j-elasticsearch-java-api.git cd log4j-elasticsearch-java-api/ mvn clean...
- 0 kudos
- 13622 Views
- 1 replies
- 0 kudos
Resolved! How do I download maven libraries with dependencies?
I want to import a maven library with its dependencies. How to do it?
- 13622 Views
- 1 replies
- 0 kudos
- 0 kudos
I recommend creating a UBER jar or download jars offline use it in clusters when the maven becomes healthy again: 1. Install the MVN CLI tool on your local mac: brew install mvnvm2. Download the Artifact with all dependencies: mvn dependency:get -Dr...
- 0 kudos
- 2962 Views
- 1 replies
- 0 kudos
- 2962 Views
- 1 replies
- 0 kudos
- 0 kudos
Since the SparkContext is already up and running, it requires a restart. Technically, it might be possible to kill the JVM process and restart it but we do not recommend that approach. In this case, we recommend restarting the cluster so that the Sp...
- 0 kudos
- 1502 Views
- 0 replies
- 0 kudos
We can now launch pools on databricks with different instance types. Hybrid Pools allows customers to create clusters and select different Databricks ...
We can now launch pools on databricks with different instance types. Hybrid Pools allows customers to create clusters and select different Databricks pools for driver and workers. It provides a way to support driver vs. worker heterogeneity, and ther...
- 1502 Views
- 0 replies
- 0 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 581 | |
| 349 | |
| 287 |
