- 8130 Views
- 3 replies
- 0 kudos
Data Engineer Associate V3 exam got suspended
Hi team,My Databricks Certified Data Engineer Associate exam got suspended within 40 minutes. I had also shown my exam room to the proctor. My exam got suspended due to eye movement. I was not moving my eyes away from laptop screen. Some questions ar...
- 8130 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @vivek2612 the support team did receive your case and is working on it now. Good luck on your exam!
- 0 kudos
- 3064 Views
- 0 replies
- 0 kudos
Cluster can't connect to redshift
I'm trying to connect to redshift with redshift-connector module, but it get 'Connection refused' error, redshift cluster and ec2 cluster are in the same VPC, security group of redshift cluster is open with inbound from VPC's IP range, i can connect ...
- 3064 Views
- 0 replies
- 0 kudos
- 9609 Views
- 0 replies
- 0 kudos
Left join with merging two streaming tables returns inner join instead
Hi all, Im trying to merge two streaming tables together with a left (outer) join, but it seems to somehow return all values from the left table that have matching values from the right table, instead of also appending the NULL values if there is no ...
- 9609 Views
- 0 replies
- 0 kudos
- 7614 Views
- 1 replies
- 1 kudos
Need direction
I want to create an online chat bot that answers questions related to the immigration visa category that we work in. I don’t have programming experience myself. I would like to find someone in this community that could lead our product development.
- 7614 Views
- 1 replies
- 1 kudos
- 1 kudos
I suppose you are looking to recruit someone? You might be lucky finding someone in here, but job boards might have a larger reach.
- 1 kudos
- 7779 Views
- 0 replies
- 0 kudos
How to use variable values in Parent Workflow in Child Workflow.
In ADF, a variable called Datetime was specified to be entered and executed in a workflow called Parent using Web Activity.There is a Child Workflow (two-step workflow) in Parent Workflow, and Datetime variables are available in Parent Workflow, but ...
- 7779 Views
- 0 replies
- 0 kudos
- 1491 Views
- 1 replies
- 0 kudos
init script doesn't finish its run when it is located in workspace. it runs fun when I have in bfs
the first attached image runs fine when cluster startsthis next image of the script does not finish when I include the -y parameter. if I include the -y I get failed: Script exit status is non-zero can someone please help
- 1491 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi, if I have understand your issue correctly, when you run directly in notebook it ends with an error but when you run it inside .sh with -y it runs okay?
- 0 kudos
- 558 Views
- 0 replies
- 0 kudos
Import notebooks in Databricks
I am using databricks-connect and VS Code to develop some python code for Databricks.I would like to code and run/test everything directly from VS Code using databricks-connect to avoid dealing with Databricks web IDE. For basic notebooks, it works j...
- 558 Views
- 0 replies
- 0 kudos
- 18298 Views
- 5 replies
- 3 kudos
Is there a way to use wild card expressions inside dbutils functions
I am working on a migration project to migrate the HDFS command executions I do inside my Python code via os.system() function by replacing it with dbutils functions.dbutils functions are working as expected if I pass the fully qualified path of a fi...
- 18298 Views
- 5 replies
- 3 kudos
- 3 kudos
As I mentioned in my problem statement, I have already implemented the required functionality with alternative approaches (AWS S3 API and BOTO3 API).Still, it is an outstanding issue with DBUTILS.
- 3 kudos
- 2256 Views
- 0 replies
- 0 kudos
Mounting adls gen2 from databricks RBAC issue
I am trying to mount my Gen 2 storage account in databricks. I have added permission of Storage blob data contributor to the storage account. But I am getting below error:Invalid permissions on the specified KeyVault https://kvmigrationnew.vault.azu...
- 2256 Views
- 0 replies
- 0 kudos
- 4039 Views
- 1 replies
- 0 kudos
Ydata-profiling on Spark cluster -working for you?
I am trying to profile my dataset using ydata-profiling.I constantly run into errors, even with simple datasets on my spark cluster.Profiling in Spark cluster erroring out · Issue #1350 · ydataai/ydata-profiling (github.com)Has anyone got ydata-profi...
- 4039 Views
- 1 replies
- 0 kudos
- 2597 Views
- 2 replies
- 0 kudos
Mosaic library not supported in db13 and unity catalog
I want to use gepatial functions which uses mosaic library. How can I use it without using mosaic library and import functions available ?Example st_aswkt
- 2597 Views
- 2 replies
- 0 kudos
- 5042 Views
- 5 replies
- 1 kudos
Access RDS Postgres DB via SSH Tunnel
Hello, How can I configure a foreign catalog connection to use SSH tunneling? I want to be able to use unity catalog
- 5042 Views
- 5 replies
- 1 kudos
- 1 kudos
Hi, in addition to our previous message, you can try https://docs.databricks.com/en/query-federation/foreign-catalogs.html and https://grant-6562.medium.com/connecting-to-sql-server-through-an-ssh-tunnel-with-python-17de859caca5. Also please tag @Deb...
- 1 kudos
- 7292 Views
- 1 replies
- 2 kudos
Using Azure Event Grid for structured streaming
Can anyone point me to any Databricks documentation (or other resources) for configuring structured streaming to use Azure Event Grid for a source/sink? I found examples for Kafka and EventHubs but Azure Event Grid is different than Azure Event Hubs....
- 7292 Views
- 1 replies
- 2 kudos
- 2 kudos
I must be missing something. I don't see how the examples in the referenced document can be applied to Azure Event Grid.Is there another example that shows how to subscribe to an Event Grid topic?
- 2 kudos
- 2775 Views
- 1 replies
- 0 kudos
Unity Catalog - Invalid configuration value detected for fs.azure.account.key
Hi there,I am having issue with writing a df to a table or display it. I have three dataframes that I have unioned and after I have done the union I cannot display the dataframe.df_table1 = spark.sql(f'SELECT * FROM {sql_full_name}')df_table2 = ...df...
- 2775 Views
- 1 replies
- 0 kudos
- 1589 Views
- 1 replies
- 0 kudos
sql end point and jdbc driver
When we try and connect to a sql warehouse endpoint with the databricks jdbc driver our query is failing if we use first_value(). We've rewritten the query to use limit 1, but we would like to understand if this is a gap in the simba/databricks driv...
- 1589 Views
- 1 replies
- 0 kudos
- 0 kudos
A sample error message when using first_value() is:An error occurred while calling o132.csv. [Databricks][JDBC](10140) Error converting value to BigDecimal.
- 0 kudos
