cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
adrianna2942842
by • New Contributor III
  • 21628 Views
  • 3 replies
  • 7 kudos

Resolved! Exception "java.nio.charset.MalformedInputException: Input length = 1" when creating data profile on Docker Container Service (10.4 LTS)

I am encountering an issue while attempting to create a data profile on clusters using Docker Container Service (version 10.4 LTS). I keep receiving the following exception:java.nio.charset.MalformedInputException: Input length = 1What's puzzling is ...

Data Engineering
Reply
  • 21628 Views
  • 3 replies
  • 7 kudos
View Replies
Latest Reply
Vartika
Databricks Employee
  • 7 kudos

Hi @Adrianna Klank​,We haven't heard from you since the last response from @Akash Bhat​​, and I was checking back to see if the suggestion helped you.Or else, If you have any solution, please share it with the community, as it can be helpful to other...

  • 7 kudos
2 More Replies
Kash
by • Contributor III
  • 1707 Views
  • 1 replies
  • 0 kudos

Data-quality help: Save Data Profile dbutils.data.summarize(df) to table

Hi there,We would like to create a data quality database that helps us understand how complete our data is. We would like to run a job each day that basically outputs the same table data as dbutils.data.summarize(df) for a given table and save it to ...

Data Engineering
Reply
  • 1707 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
daniel_sahal
Esteemed Contributor
  • 0 kudos

From what I know there's no easy way to save dbutils.data.summarize() into a df.You can still create your custom python/pyspark code to profile your data and save the output.

  • 0 kudos
shrutis23
by • New Contributor III
  • 3893 Views
  • 4 replies
  • 4 kudos

How to use delta live table with google cloud storage

Hi Team I have been working on a POC exploring delta live table with GCS location. I have some doubts :how to access the gcs bucket. We have connection established using databricks service account. In a normal cluster creation , we go to cluster page...

Data Engineering
Reply
  • 3893 Views
  • 4 replies
  • 4 kudos
View Replies
Latest Reply
Senthil1
Contributor
  • 4 kudos

Kindly mount the DBFS location to GCS cloud storage, see belowMounting cloud object storage on Databricks | Databricks on Google Cloud

  • 4 kudos
3 More Replies
Abel_Martinez
by • Contributor
  • 1793 Views
  • 1 replies
  • 1 kudos

Resolved! Create data bricks service account

Hi all, I need to create service account users who can only query some delta tables. I guess I do that by creating the user and granting select right to the desired tables. But Data bricks requests a mail account for these users. Is there a way to cr...

Data Engineering
Reply
  • 1793 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
Abel_Martinez
Contributor
  • 1 kudos

HI @Kaniz Fatma​ , I've checked the link but the standard method requires a mailbox and the user creation using SCIM API looks too complicated. I solved the issue, I created a mailbox for the service account and I created the user using that mailbox....

  • 1 kudos
-werners-
by • Esteemed Contributor III
  • 2084 Views
  • 5 replies
  • 22 kudos

Look what I just saw appearing in my notebook:a data histogram of your dataframe!

Look what I just saw appearing in my notebook:a data histogram of your dataframe!

image
Data Engineering
Reply
  • 2084 Views
  • 5 replies
  • 22 kudos
View Replies
Latest Reply
-werners-
Esteemed Contributor III
  • 22 kudos

you heard it first in here!https://databricks.com/blog/2021/12/07/introducing-data-profiles-in-the-databricks-notebook.html

  • 22 kudos
4 More Replies
Labels