cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

adrianna2942842
by New Contributor III
  • 21791 Views
  • 3 replies
  • 7 kudos

Resolved! Exception "java.nio.charset.MalformedInputException: Input length = 1" when creating data profile on Docker Container Service (10.4 LTS)

I am encountering an issue while attempting to create a data profile on clusters using Docker Container Service (version 10.4 LTS). I keep receiving the following exception:java.nio.charset.MalformedInputException: Input length = 1What's puzzling is ...

  • 21791 Views
  • 3 replies
  • 7 kudos
Latest Reply
Vartika
Databricks Employee
  • 7 kudos

Hi @Adrianna Klank​,We haven't heard from you since the last response from @Akash Bhat​​, and I was checking back to see if the suggestion helped you.Or else, If you have any solution, please share it with the community, as it can be helpful to other...

  • 7 kudos
2 More Replies
Kash
by Contributor III
  • 1712 Views
  • 1 replies
  • 0 kudos

Data-quality help: Save Data Profile dbutils.data.summarize(df) to table

Hi there,We would like to create a data quality database that helps us understand how complete our data is. We would like to run a job each day that basically outputs the same table data as dbutils.data.summarize(df) for a given table and save it to ...

  • 1712 Views
  • 1 replies
  • 0 kudos
Latest Reply
daniel_sahal
Esteemed Contributor
  • 0 kudos

From what I know there's no easy way to save dbutils.data.summarize() into a df.You can still create your custom python/pyspark code to profile your data and save the output.

  • 0 kudos
shrutis23
by New Contributor III
  • 3919 Views
  • 4 replies
  • 4 kudos

How to use delta live table with google cloud storage

Hi Team I have been working on a POC exploring delta live table with GCS location. I have some doubts :how to access the gcs bucket. We have connection established using databricks service account. In a normal cluster creation , we go to cluster page...

  • 3919 Views
  • 4 replies
  • 4 kudos
Latest Reply
Senthil1
Contributor
  • 4 kudos

Kindly mount the DBFS location to GCS cloud storage, see belowMounting cloud object storage on Databricks | Databricks on Google Cloud

  • 4 kudos
3 More Replies
Abel_Martinez
by Contributor
  • 1811 Views
  • 1 replies
  • 1 kudos

Resolved! Create data bricks service account

Hi all, I need to create service account users who can only query some delta tables. I guess I do that by creating the user and granting select right to the desired tables. But Data bricks requests a mail account for these users. Is there a way to cr...

  • 1811 Views
  • 1 replies
  • 1 kudos
Latest Reply
Abel_Martinez
Contributor
  • 1 kudos

HI @Kaniz Fatma​ , I've checked the link but the standard method requires a mailbox and the user creation using SCIM API looks too complicated. I solved the issue, I created a mailbox for the service account and I created the user using that mailbox....

  • 1 kudos
-werners-
by Esteemed Contributor III
  • 2104 Views
  • 5 replies
  • 22 kudos

Look what I just saw appearing in my notebook:a data histogram of your dataframe!

Look what I just saw appearing in my notebook:a data histogram of your dataframe!

image
  • 2104 Views
  • 5 replies
  • 22 kudos
Latest Reply
-werners-
Esteemed Contributor III
  • 22 kudos

you heard it first in here!https://databricks.com/blog/2021/12/07/introducing-data-profiles-in-the-databricks-notebook.html

  • 22 kudos
4 More Replies
Labels