Databricks Community

RiyazAli · 01-21-2025

I'm trying to read the data which has Japanese headers, might as well have Japanese data. Currently when I say header is True, I see all jumbled characters. Can any one help how can I parse these Japanese characters correctly?

RiyazAli · 01-07-2025

Hi there,As a compliance measure, I'm tasked with masking the PII data starting from bronze to silver and all the tables and views downstream. I suggested my clients to use row filters and column masks as mentioned in the doc.However, when a user who...

RiyazAli · 04-10-2024

Hi Community,I'm trying to load data from the landing zone to the bronze layer via DLT- Autoloader, I want to add a column record_id to the bronze table while I fetch my data. I'm also using file arrival trigger in the workflow to update my table inc...

RiyazAli · 11-09-2022

Hello all,I've been tasked to convert a Scala Spark code to PySpark code with minimal changes (kinda literal translation).I've come across some code that claims to be a list comprehension. Look below for code snippet:%scala val desiredColumn = Seq("f...

RiyazAli · 07-03-2022

The very first notebook of the dbc notebook which is a setup cell fails.

RiyazAli

Thank you, @Avinash_Narala I definitely used the encoding options to parse the data again but this time I used an encoding called `SHIFT_JIS` to solve the problem. Appreciate the quick response.!

RiyazAli · 01-09-2025

Hi @Walter_C, When you mask a column, the column will still be visible, but the data in that masked column will be encrypted (or masked) right?My requirement is that, the non compliant user, should never know that the particular masked column exists ...

RiyazAli · 12-30-2024

Hi @Shivaprasad If you think the above response answered your question, please mark it as a solution. Best,

RiyazAli · 12-29-2024

Hello @Shivaprasad Expanding on the code snippet provided above. Once you run the Rest Api code provided above, you will get the statement id. Use this statement_id to get the query results using the same statements api.code snippet is as below:impor...

RiyazAli · 12-29-2024

Hello @Shivaprasad Looks like the error is associated with the way you're trying to retrieve the data from the Unity Storage location.As an alternative, let's try to use the 3 level namespace UC has to submit a query and retrieve the data.e.g: instea...

Databricks Community

User Stats

User Activity

Parsing Japanese characters in Spark & Databricks

Requirement to remove/skip column(s) in the downstream tables/views while PII data masking

Unable to create a record_id column via DLT - Autoloader

Converting a transformation written in Spark Scala to PySpark

Issue with .dbc in the Advanced Data Engineering course in Databricks Academy

Re: Parsing Japanese characters in Spark & Databricks

Re: Requirement to remove/skip column(s) in the downstream tables/views while PII data masking

Re: Accessing delta tables using API outside azure (Workiva)

Re: Accessing delta tables using API outside azure (Workiva)

Re: Accessing delta tables using API outside azure (Workiva)