Good afternoon,
Attempting to run this statement:
%sql
CREATE EXTERNAL TABLE IF NOT EXISTS dev_user_login (
event_name STRING,
datetime TIMESTAMP,
ip_address STRING,
acting_user_id STRING
)
PARTITIONED BY
(date DATE)
STORED AS
PARQUET
...
Hello,
I am attempting to append new json files into an existing parquet table defined in Databricks.
Using a dataset defined by this command (dataframe initially added to a temp table):
val output = sql("select headers.event_name, to_date(from_unix...
Many thanks! The above worked and I was able to create the table with the timestamp data type. Appreciate the automatic partition discovery also! Ill focus on using the Dataframes vs Hive implementation going forward.
Thanks for responding. However, I have since moved away from this approach and made heavy modifications to the notebook of interest as I did not see a way to define a specific path in conjunction with saveAsTable. I would like it to point to a S3 di...
