Not having a standard height becomes quite a challenge for many growing adolescents, which is said to be a condition of idiopathic short stature. So, while we can think it will be “ok” with time, we can’t ignore the elephant in the room. To diagnose ...
Inventory Management System Handling the inventory and keeping track of everything is the most difficult, yet important aspect of leading a business. Inventory never stays the same. Due to sales, return, purchases, etc your business’ inventory is alw...
Hello, I'm very much new to Databricks and I'm finding it hard if it's right solution for our needs.Requirement:We have multiple data sources spread across AWS S3 and Postgres. We need a common SQL endpoint that can be used to write queries to join d...
Yes you can. You can ETL to data lake storage register your tables to metastore and register your SELECT with JOINS as VIEW or even better create additionally jobs and store your JOINED table. From BI you can connect to databricks sql or to data lake...
import matplotlib.pyplot as pltimport seaborn as snsimport pandas as pdimport numpy as npprob = np.random.rand(7) + 0.1prob /= prob.sum()df = pd.DataFrame({'department': np.random.choice(['helium', 'neon', 'argon', 'krypton', 'xenon', 'radon', 'ogane...
@Maria Bruevich - Do either of these answers help? If yes, would you be happy to mark one as best so that other members can find the solution more quickly?
HI,I am running a Notebook job calling a JAR code (application code implmented in C#). in the Spark UI page for almost 2 hrs, it'w not showing any tasks and even the CPU usage is below 20%, memory usage is very small. Before this 2 hr window it shows...
Hi,I have a bunch of scripts in Databricks that perform a decent amount of data-wrangling. All of these scripts contain sensitive information and I have no intention of making them public.I would like to provide a service to my customers - so they ca...
@Tamas D I understood your concern. For cluster creation in different subscription I think that's by design at this moment. But I would like to request you to add your use case to https://feedback.azure.com/d365community/forum/2efba7dc-ef24-ec11-b6...
Hi all,I'm having some trouble with my Certification Transcript in the Academy Portal. I've passed "Databricks Certified Associate Developer for Apache Spark 3.0" last year and everything seemed fine (apart from the fact that I've been issued two sep...
Hey @Piper Wilson ! Thank you for your response. Unfortunately, I already created a support ticket through the address provided in this post you mentioned. And I got a 'case closed' e-mail after over two weeks with no response and no fix (certificat...
We are running multiple Databricks job via ADF. I was wondering which option out of the below is a cheaper route for databricks notebook processing from ADF. When I create a ADF linked service, which should I use to lower my cost.New Job Cluster opti...
the instance pool will be cheaper if you use spot instances. But only if you size your instance pool correctly. (number of workers and scale down time)AFAIK you cannot use spot instances for job clusters in ADF
I created a Databricks job with multiple tasks. Is there a way to pass variable values from one task to another. For example, if I have tasks A and B as Databricks notebooks. Can I create a variable (e.g. x) in notebook A and later use that value in ...
you could also consider using an orchestration tool like Data Factory (Azure) or Glue (AWS). there you can inject and use parameters from notebooks.The job scheduling of databricks also has the possibility to add parameters, but I do not know if yo...
Hello, I am developing an application that runs multiple processes that write their results to a common delta table as blind appends. According to the docs I've read online: https://docs.databricks.com/delta/concurrency-control.html#protocolchangedex...
I think you are right, the mergeSchema will change the schema of the table, but if you both write to that same table with another schema, which one will it be?Can you check if both of you actually write the same schema, or remove the mergeschema?
I created a delta table through a cluster over a dbfs location .Schema :create external table tmp_db.delta_data(delta_id int ,delta_name varchar(20) , delta_variation decimal(10,4) ,delta_incoming_timestamp timestamp,delta_date date generated always ...
varchartype is only available as from spark 3.1 I think.https://spark.apache.org/docs/latest/sql-ref-datatypes.htmlThe link is for spark 3.2, and 3.1 also has varchartype. So can you check your spark version?Also if the table definition still exists...
We are planning to migrate from standard to premium workspaceWe need to know if below artifacts will be maintainedneed to check on streaming Job DowntimeAccess token DBFS Access Production Cluster /JobsCluster ID Job ID and other properties like URL ...
hi @Kaniz Fatma then I can assume there wont be any impact on metastore and all the metadata(table definition,schema ) will be available post upgradation
I've been experimenting with DLT and it works well. I'd like to understand where can I see details of which records didn't meet the quality critera?
Hello, @Paresh J! Welcome and thank you for asking! My name is Piper, and I'm a moderator for Databricks.Let's give the community some time to help before we circle back to you. Thanks in advance for your patience.
Link UG300 ada menyediakan depo tipe e- wallet yang dapat dipakai unyuk dapat melaksanakan pembelian ataupun top up saldo ke e- wallet tujuan yang telah ada di dalam web. Adanya berita gembira buat kalian yang tidak mempunyai rekening bank, Jika kali...
Hello, @Ravi Param - My name is Piper, and I'm one of the moderators here. Thank you for your question! Let's give the community a chance to respond and then we'll circle back if necessary.
| User | Count |
|---|---|
| 1616 | |
| 790 | |
| 442 | |
| 349 | |
| 287 |
