- 3773 Views
- 1 replies
- 2 kudos
Resolved! How to source control a Dashboard?
Is it possible to source control the dashboard along with a notebook code? When source controlling a python notebook it gets converted to *.py. It looks like the resulting *.py file loses the information about the dashboard cells. Thus, if this *.py ...
- 3773 Views
- 1 replies
- 2 kudos
- 2 kudos
No, you will need to save as another source, like DBC Archive, to replicate the Notebook features.
- 2 kudos
- 5474 Views
- 1 replies
- 1 kudos
How to provide custom class extending SparkPlugin/ExecutorPlugin in Databricks 7.3?
How to properly configure the jar containing the class and spark plugin in Databricks?During DBR 7.3 cluster creation, I tried by setting the spark.plugins, spark.driver.extraClassPath and spark.executor.extraClassPath Spark configs by copying the ja...
- 5474 Views
- 1 replies
- 1 kudos
- 1 kudos
Hello @Krishna Kashiv​ - I don't know if we've met yet. My name is Piper and I'm a community moderator here. Thank you for your new question. It looks thorough! Let's give it a while to see what our members have to say. Otherwise, we will circle back...
- 1 kudos
- 3870 Views
- 1 replies
- 0 kudos
Create gpg file and save to AWS s3 storage in scala
Hi - Could you please help me on how can I create a scala notebook to perform the below tasksEncrypt a text file using the gpgUpload the file to amazon s3 storageverify the file exists in amazon s3decrypt the encrypted file to verify no issuesApprec...
- 3870 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello! My name is Piper and I'm a community moderator for Databricks. Thanks for your question. Let's give it a bit more to see what our members have to say. If not, we'll circle back around.
- 0 kudos
- 11042 Views
- 11 replies
- 7 kudos
Resolved! What is the proper way to import the new pyspark.pandas library?
I am moving an existing, working pandas program into Databricks. I want to use the new pyspark.pandas library, and change my code as little as possible. It appears that I should do the following:1) Add from pyspark import pandas as ps at the top2) Ch...
- 11042 Views
- 11 replies
- 7 kudos
- 15487 Views
- 6 replies
- 2 kudos
CLI Command <databricks fs cp> Not Uploading Files to DBFS
Hi all, So far I have been successfully using the CLI interface to upload files from my local machine to DBFS/FileStore/tables. Specifically, I have been using my terminal and the following command: databricks fs cp -r <MyLocalDataset> dbfs:/FileStor...
- 15487 Views
- 6 replies
- 2 kudos
- 2 kudos
hi @Ignacio Castineiras​ ,If Arjun.kr's fully answered your question, would you be happy to mark their answer as best so that others can quickly find the solution?Please let us know if you still are having this issue.
- 2 kudos
- 963 Views
- 0 replies
- 0 kudos
1.   DIFFERENT TYPES OF TACTICAL GEAR 1. HARDWAREOptical hardware, for instance, cuffs, laser sights, optics, and night vision goggles accompany a hug...
1. DIFFERENT TYPES OF TACTICAL GEAR1. HARDWAREOptical hardware, for instance, cuffs, laser sights, optics, and night vision goggles accompany a huge group of features and capacities. Packs and pockets are made of climate-safe material planned to ke...
- 963 Views
- 0 replies
- 0 kudos
- 5071 Views
- 1 replies
- 0 kudos
Creating a table like in SQL with Spark
Hi !I'm working on a project at my company on Databricks using Scala and Spark. I'm new to Spark and Databricks and so I would like to know how to create a table on specific location (on the Delta Lake of my company). In SQL + some Delta features, I ...
- 5071 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Adrien MERAT​ ,I would like to share the following documentation that will provide examples on how to create Delta tables:Create Delta table linkDelta data types link
- 0 kudos
- 6078 Views
- 8 replies
- 0 kudos
Spark adding NUL
Hi I have a DF which contains Json string so the value is like {"key": Value, "anotherKey": anotherValue}, so when I am trying to write the DF containing this string to the CSV, spark is ​adding NUL character af the front of this line and at the end,...
- 6078 Views
- 8 replies
- 0 kudos
- 0 kudos
Hard to tell without having the code, but it might be the separator for the csv? You do have comma's in the string, and comma is the default separator for csv.
- 0 kudos
- 6308 Views
- 4 replies
- 3 kudos
Hi, I like to create a web form with displayHTML in a notebook cell and when the users presses the post button, i like to write the content of my text...
Hi,I like to create a web form with displayHTML in a notebook cell and when the users presses the post button, i like to write the content of my text area of my form back in to the code cell of the notebook.Example:displayHTML ("""<form><textarea> u...
- 6308 Views
- 4 replies
- 3 kudos
- 3 kudos
Hi @afshin riahi​ ,Did Dan's response helped you to solve your question? if it did, can you mark it as best answer? I will help to move the post to the top so other can quickly find the solution.
- 3 kudos
- 8275 Views
- 5 replies
- 2 kudos
Resolved! AWS VPC peering connection: can't make Databricks VPC reach our services on the accepter VPC
Hi,We followed this document (https://docs.databricks.com/administration-guide/cloud-configurations/aws/vpc-peering.html) describing how to establish a connection between two (or more) VPC in AWS, but so far we haven't been able to communicate with t...
- 8275 Views
- 5 replies
- 2 kudos
- 2 kudos
Hi @Martin Cigorraga​ ,If Huaming's fully answered your question, would you be happy to mark their answer as best so that others can quickly find the solution?
- 2 kudos
- 7603 Views
- 3 replies
- 0 kudos
- 7603 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @Ayman Alneser​ ,Did Huaming.lu's response worked for you? if it did, could you marked as the best solution so that other can quickly find it in the future.
- 0 kudos
- 20411 Views
- 6 replies
- 5 kudos
Resolved! Can you help with this error please? Issue when using a new high concurrency cluster
Hello, I am trying to use MLFlow on a new high concurrency cluster but I get the error below. Does anyone have any suggestions? It was working before on a standard cluster. Thanks.py4j.security.Py4JSecurityException: Method public int org.apache.spar...
- 20411 Views
- 6 replies
- 5 kudos
- 5 kudos
@Tom Soto​ We have a workaround for this. This cluster spark configuration setting will disable py4jSecurity while still enabling passthrough spark.databricks.pyspark.enablePy4JSecurity false
- 5 kudos
- 23183 Views
- 8 replies
- 2 kudos
Resolved! How many hours I can estimate to trainning in a Databricks Academy Self-Placed Trainning platform ?
I done the Data Engineering Profissional and others training in a Self-Placed Trainning (https://www.linkedin.com/posts/wscardua_data-engineering-professional-activity-6851487238774108160-IsTE) . How many hours can I estimate for this training (and o...
- 23183 Views
- 8 replies
- 2 kudos
- 3107 Views
- 2 replies
- 4 kudos
Multi-task Job Run starting point
Hi community!I would like to know if it is possible to start a Multi-task Job Run from and specific task. The use case is as follows:I have a 17 tasks JobA task in the middle, let's say a task after 2 dependencies, failsI found the error and now it i...
- 3107 Views
- 2 replies
- 4 kudos
- 4 kudos
+1 to what @Dan Zafar​ said. We're working **** ** this. Looking forward to bring this to you in the near future.
- 4 kudos
- 15611 Views
- 2 replies
- 0 kudos
How to read a fixed length file in Spark using DataFrame API and SCALA
I have a fixed length file ( a sample is shown below) and I want to read this file using DataFrames API in Spark using SCALA(not python or java). Using DataFrames API there are ways to read textFile, json file and so on but not sure if there is a wa...
- 15611 Views
- 2 replies
- 0 kudos
- 0 kudos
Find the below solution which can be used. Let us consider this is the data in the file. EMP ID First Name Last Name 1Chris M 2John ...
- 0 kudos
| User | Count |
|---|---|
| 1645 | |
| 793 | |
| 584 | |
| 349 | |
| 287 |
