- 29304 Views
- 5 replies
- 2 kudos
I am trying to create a JAR for a Azure Databricks job but some code that works when using the notebook interface does not work when calling the library through a job. The weird part is that the job will complete the first run successfully but on an...
- 29304 Views
- 5 replies
- 2 kudos
Latest Reply
I am facing similar issue when trying to use from_utc_timestamp function. I am able to call the function from databricks notebook but when I use the same function inside my java jar and running as a job in databricks, it is giving below error. Analys...
4 More Replies
by
osoucy
• New Contributor II
- 1234 Views
- 0 replies
- 1 kudos
ObjectiveWithin the context of a delta live table, I'm trying to merge two streams aggregation, but run into challenges. Is it possible to achieve such a join?ContextAssume- table trades stores a list of trades with their associated time stamps- tabl...
- 1234 Views
- 0 replies
- 1 kudos
- 1372 Views
- 0 replies
- 2 kudos
Team, I am struggling with a unique issue. I am not sure if my understanding is wrong or this is a bug with spark. I am reading a stream from events hub ( Extract) Pivoting and Aggregating the above dataframe ( Transformation). This is a WATERMARKED...
- 1372 Views
- 0 replies
- 2 kudos
- 4304 Views
- 2 replies
- 0 kudos
Data has 2 columns:
|requestDate|requestDuration|
| 2015-06-17| 104|
Here is the code:
avgSaveTimesByDate = gridSaves.groupBy(gridSaves.requestDate).agg({"requestDuration": "min", "requestDuration": "max","requestDuration": "avg"})
avgSaveTimesBy...
- 4304 Views
- 2 replies
- 0 kudos
Latest Reply
My guess is that the reason this may not work is the fact that the dictionary input does not have unique keys. With this syntax, column-names are keys and if you have two or more aggregation for the same column, some internal loops may forget the no...
1 More Replies
