Databricks Community

dream · 07-01-2024

I have two external locations. On both of these locations I have `ALL PRIVILEGES` access.I am creating a table on the first external location using the following command:%sqlcreate or replace table delta.`s3://avinashkhamanekar/tmp/test_table_origina...

dream · 08-05-2023

So I was comparing schemas of two different dataframe using this code: >>> df1.schema == df2.schema Out: False But the thing is, both the schemas are completely equal.When digging deeper I realized that some of the StructFields() that should have bee...

dream · 07-03-2024

Also the delta log couldn't have been deleted (or vaccumed) since I am running all three commands one by one.

dream · 07-03-2024

I got these errors while running both commands[UC_COMMAND_NOT_SUPPORTED.WITH_RECOMMENDATION] The command(s): input_file_name are not supported in Unity Catalog. Please use _metadata.file_path instead. SQLSTATE: 0AKUCCould you please also try this as ...

dream · 07-18-2023

yes df.cache() will work

dream · 07-17-2023

just a slight correction but you can also pass parameters when using %run

dream · 07-17-2023

access_key = dbutils.secrets.get(scope = "aws", key = "aws-access-key") secret_key = dbutils.secrets.get(scope = "aws", key = "aws-secret-key") encoded_secret_key = secret_key.replace("/", "%2F") aws_bucket_name = "<aws-bucket-name>" mount_name = "<m...

Databricks Community

User Stats

User Activity

Accessing shallow cloned data through an External location fails

Comparing schemas of two dataframes

Re: Accessing shallow cloned data through an External location fails

Re: Accessing shallow cloned data through an External location fails

Re: What is the best approach to display DataFrame without re-executing the logic each time we displ

Re: Run a multiple %command in the same cell

Re: Data Pull from S3