Databricks Community

Hi, we are also in the same exact situation. Were you able to solve the problem? Or a workaround maybe.

We are also seeing this error in 14.3 LTS from a simple example:from pyspark.sql.functions import coldf = spark.table('things')things = df.select(col('thing_id')).collect()[NOT_COLUMN_OR_STR] Argument `col` should be a Column or str, got Column.  

Hello @Phani1 ,These doc pages might be useful for you: Set up and manage Unity CatalogSync users and groups from Microsoft Entra ID 

Hi @ismaelhenzel, if you want to use the "MERGE INTO" sql command, you must turn-off rls. This is by design.

Hi @Mathias,  I'd say that watermarking might be a good solution for your use case. Please check Control late data threshold with multiple watermark policy in Structured Streaming.  If you want to dig-in further there's also: Spark Structured Streami...

Hi @EcuaCrisCar,  To query the number of records in your test table, you can use SQL or DataFrame APIs in Databricks.Next, you’ll need to check if the record count falls within the specified range (80,000 to 90,000). If it does, proceed with the note...

Hi @filipjankovic, Since you have multiple tables and need dynamic schema inference, I recommend using the following approach: Schema Inference from JSON String: You can infer the schema from the JSON string and then create a DataFrame. Schema I...

Hi @NikhilK1998, I'm sorry to hear your exam was suspended. Thank you for filing a ticket with our support team. Please allow the support team 24-48 hours to resolve. In the meantime, you can review the following documentation: Room requirements Beh...

Hi @Avinash_Narala, The error message you provided indicates that the verification of the instance profile failed due to an AWS authorization issue. Specifically, the user associated with the assumed role arn:aws:sts::755231362028:assumed-role/databr...