- 3522 Views
- 1 replies
- 2 kudos
Resolved! list users having access to scope credentials
Hello!How do I list all the users or groups having access to the key-vault backed scope credentials?Let's say, I have a scope called MyScope for which all the secrets are stored in MyKeyVault.I would like to see what users have access there and ideal...
- 3522 Views
- 1 replies
- 2 kudos
- 2 kudos
@Bartosz Wachocki​ , As secrets use ACL for the scope, you need to make an API call (can be via CLI also) to list ACL for the given scope >> 2.0/secrets/acls/list more info here https://docs.databricks.com/dev-tools/api/latest/secrets.html#list-secre...
- 2 kudos
- 6679 Views
- 2 replies
- 2 kudos
Bronze silver gold layers
Is there a best practise guide on setting up the delta lake for these 3 layers. ​I'm looking for document or scripts to run that will assist me.
- 6679 Views
- 2 replies
- 2 kudos
- 2 kudos
hi @Lloyd Vickery​ ,I would highly recommend to use Databricks Delta Live Tables (DLT) docs here https://databricks.com/product/delta-live-tables
- 2 kudos
- 5649 Views
- 3 replies
- 0 kudos
Try catch multiple write streams on a job
We are having issues with checkpoints and schema versions getting out of date (no idea why), but it causes jobs to fail. We have jobs that are running 15-30 streaming queries, so if one fails, that creates an issue. I would like to trap the checkpo...
- 5649 Views
- 3 replies
- 0 kudos
- 0 kudos
The problem is that on startup if a stream fails, it would never hit the awaitAnyTermination? I almost want to take that while loop and put it on a background thread to start that at the beginning and then fire all the streams afterward... not sure ...
- 0 kudos
- 5437 Views
- 3 replies
- 3 kudos
Resolved! Turn spark.sql query into scala function
Hello,I'm learning Scala / Spark and try to understand what's wrong with my function:I have a spark.sql query, stored in a variable:val uViewName = spark.sql(""" SELECT v.Data_View_Name FROM apoHierarchy AS h INNER JOIN apoView AS v ON h.View_N...
- 5437 Views
- 3 replies
- 3 kudos
- 3 kudos
try add .first()(0) it will return only value from first row/column as currently you are returning Dataset: var uViewName = spark.sql(s""" SELECT v.Data_View_Name FROM apoHierarchy AS h INNER JOIN apoView AS v ON h.View_Name = v.Context_View_N...
- 3 kudos
- 4107 Views
- 2 replies
- 1 kudos
Resolved! How to test Kafka connectivity from a Databricks notebook
My structured streaming job is failing as it's unable to connect to Kafka. I believe the issue is with Spark. How can I isolate if it's a Spark library issue or an actual network issue.
- 4107 Views
- 2 replies
- 1 kudos
- 1 kudos
The below code snippet can be used to test the connectivityimport java.util.Arrays import java.util.Properties import org.apache.kafka.clients.admin.AdminClient import org.apache.kafka.clients.admin.AdminClientConfig import org.apache.kafka.clients.a...
- 1 kudos
- 7408 Views
- 5 replies
- 5 kudos
Resolved! Using shared python wheels for job compute clusters
We have a GitHub workflow that generates a python wheel and uploads to a shared S3 available to our Databricks workspaces. When I install the Python wheel to a normal compute cluster using the path approach, it correctly installs the Python wheel and...
- 7408 Views
- 5 replies
- 5 kudos
- 5 kudos
You can mount S3 as a DBFS folder then set that library in "cluster" -> "libraries" tab -> "install new" -> "DBFS"
- 5 kudos
- 3298 Views
- 2 replies
- 5 kudos
Resolved! Different configurations for same Databricks Runtime version
Hi all,On my DBR installations, s3a scheme is mapped to shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystem. On my customer's DBR installations it is mapped to com.databricks.s3a.S3AFileSystem.We both use the same DBR runtime, and none of us has...
- 3298 Views
- 2 replies
- 5 kudos
- 5 kudos
@Yoni Au​ , If both of you are using the same DBR version, then you should not find any difference. As @Hubert Dudek​ mentioned, there might be some spark configuration change made on one of the clusters. Also, it's worth checking for any cluster sco...
- 5 kudos
- 2389 Views
- 1 replies
- 2 kudos
I cannot access databricks community edition account
Last week, I cannot loginto https://community.cloud.databricks.com/login.html all of a sudden. I tried to set the password, also didn't receive the reset email. It says "Invalid email address or password Note: Emails/usernames are case-sensitive".I e...
- 2389 Views
- 1 replies
- 2 kudos
- 2 kudos
@Kaniz Fatma​ can help, additionally you can open ticket here https://help.databricks.com/s/contact-us
- 2 kudos
- 5347 Views
- 4 replies
- 8 kudos
Resolved! init_script error during cluster creation - 101: Network is unreachable
When I run the init_script during cluster creationapt-get update && apt-get install -y ffmpeg libsndfile1-devI get an error in cluster logs E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/universe/o/openal-soft/libopenal1_1.19.1-1_amd64.deb ...
- 5347 Views
- 4 replies
- 8 kudos
- 8 kudos
Hi @Sergii Ivakhno​ Could you please check if outbound TCP access for port 80 is allowed in the security group.
- 8 kudos
- 763 Views
- 0 replies
- 0 kudos
PROCESS OF APPLYING FOR OMAN E-VISA The Oman e-Visa was initiated in 2018, for making the process easy. Presently, 220 countries in the world are elig...
PROCESS OF APPLYING FOR OMAN E-VISAThe Oman e-Visa was initiated in 2018, for making the process easy. Presently, 220 countries in the world are eligible to apply for Oman e-Visa. Tourists can apply for visas online by submitting the Oman visa applic...
- 763 Views
- 0 replies
- 0 kudos
- 5825 Views
- 4 replies
- 4 kudos
Resolved! Cant find reason but suddenly new Jobs are getting huge job id numbers. example 945270539673815
Created Job ID is suddenly started to make huge numbers, and that is now making problems in Terraform plan, cause int is too big Error: strconv.ParseInt: parsing "945270539673815": value out of rangeIm new on the board and pretty new with Databricks ...
- 5825 Views
- 4 replies
- 4 kudos
- 4 kudos
Hi @Jere Karhu​ , In case you are using the Job/Run id in API, please be advised that you will need to change the client-side logic to process int64/long and expect a random number. In some cases, you just need to change the declared type in their so...
- 4 kudos
- 4432 Views
- 3 replies
- 3 kudos
Resolved! Importing MongoDB with field names containing spaces
I am currently using a Python notebook with a defined schema to import fairly unstructured documents in MongoDB. Some of these documents have spaces in their field names. I define the schema for the MongoDB PySpark connector like the following:Struct...
- 4432 Views
- 3 replies
- 3 kudos
- 3 kudos
Solution: It turns out the issue is not the schema reading in, but the fact that I am writing to Delta tables, which do not currently support spaces. So, I need to transform them prior to dumping. I've been following a pattern of reading in raw data,...
- 3 kudos
- 3502 Views
- 1 replies
- 2 kudos
Resolved! Issue when reading .wav file
Hi, I am developing notebook to read .wav files and build Speech Matching Scenario. I have saved files in "/FileStore/tables/doors_and_corners_kid_thats_where_they_get_you.wav".When I wrote code like thisfrom scipy.io import wavfileimport numpy as np...
- 3502 Views
- 1 replies
- 2 kudos
- 2 kudos
Try to prefix it with dbfs dbfs:/FileStore or /dbfs/FileStore
- 2 kudos
- 8018 Views
- 1 replies
- 5 kudos
Resolved! How to Overwrite Using pyspark's JDBC without loosing constraints on table columns
Hello,My table has primary key constraint on a perticular column, Im loosing primary key constaint on that column each time I overwrite the table , What Can I do to preserve it? Any Heads up would be appreciatedTried Belowdf.write.option("truncate", ...
- 8018 Views
- 1 replies
- 5 kudos
- 5 kudos
@Abeeya .​ , Mode "truncate", is correct to preserve the table. However, when you want to add a new column (mismatched schema), it wants to drop it anyway.
- 5 kudos
- 1864 Views
- 1 replies
- 0 kudos
How to resolve Quickbooks error 12007
QuickBooks error 12007 occurs when an update time out occurs. QuickBooks may encounter this error when it cannot connect to the internet if it's unable to access the server. If you want to know its solutions then check out our latest blog on this.
- 1864 Views
- 1 replies
- 0 kudos
- 0 kudos
How to Resolve QuickBooks Payroll Update Error 12007?For various possible causes of the QB payroll update error 12007, you need to perform different troubleshooting procedures. Follow the solutions in their given sequence to fix this QuickBooks error...
- 0 kudos
| User | Count |
|---|---|
| 1619 | |
| 790 | |
| 493 | |
| 349 | |
| 287 |
