I know that I can do a DESCRIBE DETAIL on a table to get current delta table version details. If I want to get these same details on a previous version, how can I do that?
You would use a combination of the functions: pyspark.sql.functions.from_unixtime(timestamp, format='yyyy-MM-dd HH:mm:ss') (documentation) and pyspark.sql.functions.unix_timestamp(timestamp=None, format='yyyy-MM-dd HH:mm:ss') (documentation)from pysp...
Hi @Mike K.,
As spark is of a different distributed architecture than traditional RDBMS, you'll certainly see that some functionality isn't supported 1:1. For your particular issue, I would suggest performing the conditional logic and filtering into...
Hi @Paul Hernandez, please try adding a preceding slash before the files path:
displayHTML("<img src ='/files/Omnetric_logo.jpg'>")
Just to add as a design pattern - I like to have any static content contained in a markdown block. Most notebooks/wo...
It is possible to use existing Python or Scala libraries to send an email from a Databricks notebook (e.g. python's smtplib). The answer in this forum question does a good job explaining further: https://forums.databricks.com/questions/17785/databri...
You can use the length function for this
from pyspark.sql.functions import length
mock_data = [('TYCO', '1303'),('EMC', '120989'), ('VOLVO', '102329'),('BMW', '130157'),('FORD', '004')]
df = spark.createDataFrame(mock_data, ['col1', 'col2'])
df2 = d...
