- 2174 Views
- 1 replies
- 1 kudos
Resolved! Selective column loader unity catalog
I am loading a table into a data frame using df = spark.table(table_name) Is there a way to load only the required columns? The table has more than 50+ columns and I only need a handful of column.
- 2174 Views
- 1 replies
- 1 kudos
- 1 kudos
@vk217 Simply just use select function, ex.df = spark.read.table(table_name).select("col1", "col2", "col3")
- 1 kudos
- 5243 Views
- 2 replies
- 2 kudos
Terraform provider, problemi in creating dependant task!
Hola all.I have a serious problem, perhaps I missed something, but can't find the solution. I need to push a job description to Databricks using TERRAFORM. I wrote the code, but there is no way to get a task dependant from two different tasks.Conside...
- 5243 Views
- 2 replies
- 2 kudos
- 2 kudos
@6502 You need to make multiple depends_on blocks for each dependency, ex.depends_on { task_key = "ichi" } depends_on { task_key = "ni" }
- 2 kudos
- 2465 Views
- 1 replies
- 0 kudos
Reading tables from different databricks clusters
Hello,My organization uses two cluster for dev and Prod. We mount our azure blobs on to delta lake to store the delta tables. Prod has bunch of data and dev has limited data. I want to move the data from prod to dev for testing purposes. How can I do...
- 2465 Views
- 1 replies
- 0 kudos
- 0 kudos
It depends on the current setup, how your clusters are working right now and how your data is stored. One alternative could be mount the Dev storage to the Prod cluster and execute a DEEP CLONE (https://docs.databricks.com/en/sql/language-manual/delt...
- 0 kudos
- 2593 Views
- 0 replies
- 0 kudos
Can not read data from GCS
I am trying to use Databricks to read data on Google Cloud Storage (GCS) with Databricks on Google Cloud. I followed the steps from https://docs.gcp.databricks.com/storage/gcs.html.I have tried Access GCS buckets using Google Cloud service accounts o...
- 2593 Views
- 0 replies
- 0 kudos
- 2813 Views
- 0 replies
- 0 kudos
Blue/Green Deployment, Table Cloning, and Delta live table pipelines
this a rather complex question that addresses Databricks users only. Let me recap a bit of the context that produced it. In the attempt to adopt the Blue/Green deployment protocol, we found good applications of the table cloning capabilities offered ...
- 2813 Views
- 0 replies
- 0 kudos
- 1336 Views
- 0 replies
- 0 kudos
Deployment of Databricks Artifact Allowlist in Terraform
I would like to inquire about the deployment schedule for the Databricks artifact allowlist within Terraform.
- 1336 Views
- 0 replies
- 0 kudos
- 2295 Views
- 0 replies
- 0 kudos
Repos section in Admin Settings Page is not visible
Hello,We have multitple workspace of Azure Databricks and we recently noticed that in some of the Administrator workspace settings, we are not able to Repos section (https://learn.microsoft.com/en-us/azure/databricks/repos/repos-setup#--restrict-usag...
- 2295 Views
- 0 replies
- 0 kudos
- 3173 Views
- 1 replies
- 0 kudos
Connection VPC AWS in Databricks for extract data Oracle Onpremise
Hi, help meHow can I consume a VPC when it is already anchored in the "network" of Databricks to extract information from a server that really made ping and works
- 3173 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @Retired_mod I have already configured in "Cloud resources"/ "Network" the Private endpoint links and the linked VPC in ibabricks. How can I connect to Oracle using the EC2 virtual machine?Thank u
- 0 kudos
- 4237 Views
- 0 replies
- 0 kudos
Can someone explain the key differences between Databricks
Hello Databricks community! I'm relatively new to Databricks and I'm trying to understand the distinctions between Databricks Community Edition and Databricks Workspace
- 4237 Views
- 0 replies
- 0 kudos
- 1450 Views
- 1 replies
- 0 kudos
Error while attempt to give Lakehouse fundamentals exam
I would like to know why I am getting this error when I tried to earn badges for lake house fundamentals. I can't access the quiz page. Can you please help on this?I am getting 403: Forbidden error.
- 1450 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @simran_27 could you please try using the link
- 0 kudos
- 2399 Views
- 2 replies
- 0 kudos
Resolved! Databricks Certificate
I cleared Databricks Associate level certification on 25th September. But, I am yet to receive my certificate from Databricks. I raised a ticket for the same but got no response from the suppport team.
- 2399 Views
- 2 replies
- 0 kudos
- 0 kudos
Hello There, Can you share the ticket number, note that we have a high influx of cases and the team is actively working on clearing the backlogs and you should be able to receive the reply in the next 48 hours. I appreciate the patience.
- 0 kudos
- 6630 Views
- 5 replies
- 5 kudos
Databricks workspace unstable
Our company's databricks workspace is unstable lately. It can't launch any compute cluster. I have never seen this issue. In addition to this issues, I have seen storage credential error on main unity catalog. Why would this happen AWS Databricks ins...
- 6630 Views
- 5 replies
- 5 kudos
- 5 kudos
Hello @Retired_mod and @jose_gonzalez ,I couldn't locate the support ticket we opened. How can we track that ticket down? It came from the peopletech.com domain. If it is more efficient to create another ticket, please let me know. Let us know the UR...
- 5 kudos
- 2817 Views
- 1 replies
- 6 kudos
Webinar: Accelerate Data and AI Projects With Databricks Notebooks
Register now: October 24, 2023 | 8:00 AM PT Use new capabilities in Databricks Notebooks to speed up innovation. This webinar will walk you through the features that are designed to take the manual effort and delays out of building and deploying dat...
- 2817 Views
- 1 replies
- 6 kudos
- 6 kudos
This will help databricks user to speed up development.
- 6 kudos
- 6645 Views
- 3 replies
- 3 kudos
Application extracting Data from Unity Catalogue
Dear Databricks community,I'm seeking advice on the best method for applications to extract data from the Unity catalogue. One suggested approach is to use JDBC, but there seems to be a dilemma. Although using a job cluster has been recommended due t...
- 6645 Views
- 3 replies
- 3 kudos
- 3 kudos
What exacltly do you mean by 'extracting'? If you want to load tables defined in Unity into a database, I would indeed do this using job clusters and a notebook.If you want to extract some data once in a while into a csv f.e., you could perfectly do...
- 3 kudos
- 5676 Views
- 4 replies
- 2 kudos
Resolved! Error on Workflow
Hi , I have some mysteries situation here My workflow (job) ran and got an error -> [INVALID_IDENTIFIER] The identifier transactions-catalog is invalid. Please, consider quoting it with back-quotes as `transactions-catalog`.(line 1, pos 12) == SQL ==...
- 5676 Views
- 4 replies
- 2 kudos
- 2 kudos
Jobs are just notebooks executed in background, so if the notebook is the same between interactive (manual) and job run, there should be no difference.So I don't see what is wrong. Is the job using DLT perhaps?
- 2 kudos
