- 2387 Views
- 2 replies
- 3 kudos
Resolved! Voucher code error
Hi,I received a voucher in Nov 2020 for 'Databricks Certified Associate Developer for Apache Spark 3.0 exam' with an expiry date in Nov 2022. However I receive an error stating the promotion code is used. I have never used the code. Any idea who to c...
- 2387 Views
- 2 replies
- 3 kudos
- 3 kudos
If your voucher code is not working, it could be due to the following reasons:The voucher code has expiredProducts in your shopping cart are not eligible for the discountTerms and conditions are not metIf applying a "new user only voucher" and our sy...
- 3 kudos
- 6427 Views
- 3 replies
- 3 kudos
Resolved! Error Code: 3206 - Processed HTTP request failed.
The ADF(Azur Data Factory) pipelines jobs executing several Databricks Notebook activities in parallel have been failing regularly with the following error "Error Code: 3206 - Processed HTTP request failed." The issue gets resolved on its own upon re...
- 6427 Views
- 3 replies
- 3 kudos
- 3 kudos
Method 1 - Close Conflicting Programsdown voteWhen you get a runtime error, keep in mind that it is happening due to programs that are conflicting with each other. The first thing you can do to resolve the problem is to stop these conflicting program...
- 3 kudos
- 2845 Views
- 1 replies
- 2 kudos
Does Spark MLlib support Generalized Additive Modeling? How does one go about implementing GAM models in Spark?
I want to implement GAM (Generalized additive model) model in Spark. Based on my research on online forums, I could not find the implementation of GAM models on Spark. Has anyone in this community attempted this? Does Spark MLlib support GAM?
- 2845 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi @Kaniz Fatma​, thanks for sharing this. We ended up using pyGAM library in Python for this. This PDF is a good introduction. I will share my learnings once we complete our experiments.
- 2 kudos
- 435 Views
- 0 replies
- 0 kudos
The summit was great #Summit22​Â
The summit was great #Summit22​
- 435 Views
- 0 replies
- 0 kudos
- 1668 Views
- 1 replies
- 0 kudos
Is there a way to watch a forum post ?
Is there a way to watch a forum post ? I cant see one.
- 1668 Views
- 1 replies
- 0 kudos
- 0 kudos
Thanks, I was just wondering if there was a watch with notification function.
- 0 kudos
- 2630 Views
- 4 replies
- 0 kudos
Resolved! Add custom tags to DLT cluster
We have a policy in our AWS account that whenever an EC2 instance is created, there are 5 mandatory tags that need to be added. When we create an All-Purpose cluster in the Databricks console, we can easily add these under Advance Options > Tags. How...
- 2630 Views
- 4 replies
- 0 kudos
- 0 kudos
@Tomasz Bacewicz​ Perfect!! worked like a charm...thank you!!do you think auto loader or DLT is a better approach to implementing a CDC pipeline in Databricks?
- 0 kudos
- 5586 Views
- 3 replies
- 3 kudos
Delta Live Table Pipeline Load Error
I am getting this error when running Delta Live Table Pipelines pulling from a source in our sandbox's local dbfs folder. It says the user is not authorized to perform this operation, whereas I am able to see the data when I run a simple select state...
- 5586 Views
- 3 replies
- 3 kudos
- 3 kudos
Hey there @Abhay Sudhakaran​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution and mark an answer as best? Else please let us know if you need more help. We'd love to hear fr...
- 3 kudos
- 2825 Views
- 3 replies
- 8 kudos
Resolved! AWS Private Link for DB workspace
What's the best way to set up a private link connection to Databricks without loosing the current settins? workspace
- 2825 Views
- 3 replies
- 8 kudos
- 8 kudos
At the data and AI summit, there was a nice talk about AWS private link. Here is a link if you have access:https://dataaisummit.com/session-virtual/?v2477da705118cc74fd14460db021e1784e2eed5a7982c6482ec95cb2e86d259644b8741959f52a49e0e6908b82a9d860=C22...
- 8 kudos
- 3370 Views
- 3 replies
- 3 kudos
Resolved! How do I setup the right cluster?
- 3370 Views
- 3 replies
- 3 kudos
- 3 kudos
For general cluster decision making refer to this article https://docs.microsoft.com/en-gb/azure/databricks/clusters/cluster-config-best-practicesOnce you've selected a cluster that makes sense, run it and check your ganglia metrics to see whether yo...
- 3 kudos
- 3745 Views
- 5 replies
- 2 kudos
Resolved! DLT Pipeline and Job Cluster
We have written few python functions(methods within a class) and packaged them as a wheel library.In the as-is situation we use to install that wheel library in All-Purpose cluster that we already have created. It works fine.In the to-be situtation(D...
- 3745 Views
- 5 replies
- 2 kudos
- 2 kudos
Does it give you an error when running the DLT pipeline specifically on the %pip command or does it not work in some other way? If it's the former, could you share the path format that you're using for the %pip command path?
- 2 kudos
- 5809 Views
- 3 replies
- 4 kudos
Resolved! Spark persistent view on a partition parquet file
In Spark, is it possible to create a persistent view on a partitioned parquet file in Azure BLOB? The view must be available when the cluster restarted, without having to re-create that view, hence it cannot be a temp view.I can create a temp view, b...
- 5809 Views
- 3 replies
- 4 kudos
- 4 kudos
Here is what worked for me. Hope this helps someone else: https://stackoverflow.com/questions/72913913/spark-persistent-view-on-a-partition-parquet-file/72914245#72914245CREATE VIEW test as select * from parquet.`/mnt/folder-with-parquet-file(s)/`@Hu...
- 4 kudos
- 1192 Views
- 0 replies
- 0 kudos
BEGINNER
Hey i am beginner and i am intrested to learn data engineering path for azure databrics.Can some one please provide some guidance
- 1192 Views
- 0 replies
- 0 kudos
- 1700 Views
- 1 replies
- 2 kudos
How to get the normalized hours value for the Databricks EMR cluster?
Hi Hubert!I'm working on a usecase to get the compute usage stats. I used boto3 code and describe_cluster() function to get the normalized instance hours value for the EMR cluster. I would like to know if there is an equivalent for this normalized in...
- 1700 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi Bala, nice to meet you.Yes, there is an equivalent. For example, in Databricks, you need to use the get method in cluster API or CLI implementation of API.https://docs.databricks.com/dev-tools/api/latest/clusters.html#get
- 2 kudos
- 19005 Views
- 1 replies
- 2 kudos
Creating permanent views from dataframes?
Hi:It's possible to create temp views in pyspark using a dataframe (df.createOrReplaceTempView()), and it's possible to create a permanent view in Spark SQL. But as far as I can tell, there is no way to create a permanent view from a dataframe, somet...
- 19005 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi Kaniz:This is what I understood from the research I did, I was curious more as to why permanent views can't be created from dataframes and whether this is a feature that might be implemented by Databricks or Spark at some point. Temporary views ca...
- 2 kudos
- 1135 Views
- 0 replies
- 0 kudos
www.mca.gov.in
Today, the entire private limited company registration process and other regulatory filings are paperless; documents are filed electronically through the MCA website and is processed at the Central Registration Centre (CRC). Online Private Limited Co...
- 1135 Views
- 0 replies
- 0 kudos
| User | Count |
|---|---|
| 1611 | |
| 768 | |
| 348 | |
| 286 | |
| 252 |
