- 6569 Views
- 6 replies
- 1 kudos
Resolved! Group vs User rights
I have a small question: How does the combination of group and user rights work? Is it like in azure, that if i have for example databricks sql acces threw a (databricks) group i am member of, but in my personal account databricks sql is not enabled...
- 6569 Views
- 6 replies
- 1 kudos
- 1 kudos
@Mirko Ludewig - Good morning (or evening depending on where you hail), would you be happy to mark whichever answer resolved the problem for you as best? That helps other members find the solutions more quickly.
- 1 kudos
- 2776 Views
- 0 replies
- 2 kudos
Spark Structred Streaming, An Aggregation DF with Watermark in Append mode to Delta table is not writing the most recent aggregation to the Delta table even after crossing the water mark boundary. This is causing dataloss
Team, I am struggling with a unique issue. I am not sure if my understanding is wrong or this is a bug with spark. I am reading a stream from events hub ( Extract) Pivoting and Aggregating the above dataframe ( Transformation). This is a WATERMARKED...
- 2776 Views
- 0 replies
- 2 kudos
- 8182 Views
- 9 replies
- 0 kudos
Resolved! Query AWS Redshift from Databricks SQL
Greetings,In the documentation for Databricks SQL it states that it supports JDBC connections, however when connecting to AWS Redshift via the built in PostgreSQL driver ("CREATE TABLE sample USING JDBC" and "jdbc://postgresql:/..." URI) I'm getting ...
- 8182 Views
- 9 replies
- 0 kudos
- 0 kudos
@Bilal Aslam anytime! Is there a place where customers could follow the timeline when such features are introduced?
- 0 kudos
- 7681 Views
- 7 replies
- 3 kudos
Resolved! Issue with AWS Glue metacatalogue and DBR 9.1 ... 10.1
I have a simple SparkSQL Select statementoffers_df = (spark.sql(""" SELECT * FROM delta.`{}` """.format(TABLE_LOCATION)))It runs under DBR 9.0 and previous. When changing the DBR to 9.1 ...10.1 I get an exceptionorg.apache.spark.SparkException: Una...
- 7681 Views
- 7 replies
- 3 kudos
- 3 kudos
Retest in DBR 10.3 beta SUCCESSFUL.Problem does not occur in DBR 10.3 beta
- 3 kudos
- 47223 Views
- 4 replies
- 4 kudos
Resolved! Spark is not able to resolve the columns correctly when joins data frames
Hello all, I m using pyspark ( python 3.8) over spark3.0 on Databricks. When running this DataFrame join:next_df = days_currencies_matrix.alias('a').join( data_to_merge.alias('b') , [ days_currencies_matrix.dt == data_to_merge.RATE_DATE, days...
- 47223 Views
- 4 replies
- 4 kudos
- 4 kudos
@Alessio Palma - Howdy! My name is Piper, and I'm a moderator for the community. Would you be happy to mark whichever answer solved your issue so other members may find the solution more quickly?
- 4 kudos
- 8758 Views
- 6 replies
- 4 kudos
Resolved! SQL Analytics: Is it possible to configure the default number format
HiWhen querying an integer value, the default format is '0.0' which results in an integer value 202111, displayed as 202,111. I can resolve the issue by updating the visualisation or using formatnumber in my query, however I would like to set a defa...
- 8758 Views
- 6 replies
- 4 kudos
- 4 kudos
@Bilal Haniff - Would you be happy to mark whichever answer helped you the most as best? That helps others find solutions more quickly.
- 4 kudos
- 2706 Views
- 1 replies
- 15 kudos
Resolved! Write to Azure Delta Lake - optimization request
Databricks/Delta team could optimize some commands which writes to Azure Blob Storage as Azure display that message:
- 2706 Views
- 1 replies
- 15 kudos
- 15 kudos
Hey there. Thank you for your suggestion. I'll pass this up to the team.
- 15 kudos
- 5063 Views
- 2 replies
- 4 kudos
Resolved! How to show the properties of the folders/files from DBFS ?
Hello,How to show the properties of the folders/files from DBFS ?Currently i am using this command :display(dbutils.fs.ls("dbfs:/"))But it only shows :pathnamesizeHow to show these properties ? : CreatedBy (Name)CreatedOn (Date)ModifiedBy (Name)Modi...
- 5063 Views
- 2 replies
- 4 kudos
- 4 kudos
Only one idea is to use %sh magic command but there is no name (just root)
- 4 kudos
- 1046 Views
- 0 replies
- 0 kudos
Not having a standard height becomes quite a challenge for many growing adolescents, which is said to be a condition of idiopathic short stature. So, ...
Not having a standard height becomes quite a challenge for many growing adolescents, which is said to be a condition of idiopathic short stature. So, while we can think it will be “ok” with time, we can’t ignore the elephant in the room. To diagnose ...
- 1046 Views
- 0 replies
- 0 kudos
- 1934 Views
- 0 replies
- 0 kudos
Inventory Management System Handling the inventory and keeping track of everything is the most difficult, yet important aspect of leading a business...
Inventory Management System Handling the inventory and keeping track of everything is the most difficult, yet important aspect of leading a business. Inventory never stays the same. Due to sales, return, purchases, etc your business’ inventory is alw...
- 1934 Views
- 0 replies
- 0 kudos
- 3390 Views
- 1 replies
- 3 kudos
Resolved! Can I use Databricks to join data from S3 and Postgres using SQL?
Hello, I'm very much new to Databricks and I'm finding it hard if it's right solution for our needs.Requirement:We have multiple data sources spread across AWS S3 and Postgres. We need a common SQL endpoint that can be used to write queries to join d...
- 3390 Views
- 1 replies
- 3 kudos
- 3 kudos
Yes you can. You can ETL to data lake storage register your tables to metastore and register your SELECT with JOINS as VIEW or even better create additionally jobs and store your JOINED table. From BI you can connect to databricks sql or to data lake...
- 3 kudos
- 5131 Views
- 4 replies
- 2 kudos
Resolved! Value Labels fail to display in Databricks notebook but they are displayed ok in Jupyter
import matplotlib.pyplot as pltimport seaborn as snsimport pandas as pdimport numpy as npprob = np.random.rand(7) + 0.1prob /= prob.sum()df = pd.DataFrame({'department': np.random.choice(['helium', 'neon', 'argon', 'krypton', 'xenon', 'radon', 'ogane...
- 5131 Views
- 4 replies
- 2 kudos
- 2 kudos
@Maria Bruevich - Do either of these answers help? If yes, would you be happy to mark one as best so that other members can find the solution more quickly?
- 2 kudos
- 8461 Views
- 4 replies
- 1 kudos
Resolved! Saprk UI not showing any running tasks
HI,I am running a Notebook job calling a JAR code (application code implmented in C#). in the Spark UI page for almost 2 hrs, it'w not showing any tasks and even the CPU usage is below 20%, memory usage is very small. Before this 2 hr window it shows...
- 8461 Views
- 4 replies
- 1 kudos
- 4207 Views
- 4 replies
- 0 kudos
Resolved! Customer deployment
Hi,I have a bunch of scripts in Databricks that perform a decent amount of data-wrangling. All of these scripts contain sensitive information and I have no intention of making them public.I would like to provide a service to my customers - so they ca...
- 4207 Views
- 4 replies
- 0 kudos
- 0 kudos
@Tamas D I understood your concern. For cluster creation in different subscription I think that's by design at this moment. But I would like to request you to add your use case to https://feedback.azure.com/d365community/forum/2efba7dc-ef24-ec11-b6...
- 0 kudos
- 2275 Views
- 2 replies
- 0 kudos
Hi all, I'm having some trouble with my Certification Transcript in the Academy Portal. I've passed "Databricks Certified Associate Devel...
Hi all,I'm having some trouble with my Certification Transcript in the Academy Portal. I've passed "Databricks Certified Associate Developer for Apache Spark 3.0" last year and everything seemed fine (apart from the fact that I've been issued two sep...
- 2275 Views
- 2 replies
- 0 kudos
- 0 kudos
Hey @Piper Wilson ! Thank you for your response. Unfortunately, I already created a support ticket through the address provided in this post you mentioned. And I got a 'case closed' e-mail after over two weeks with no response and no fix (certificat...
- 0 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 580 | |
| 349 | |
| 287 |
