- 2506 Views
- 2 replies
- 4 kudos
Resolved! Streaming data to CosmosDB
Hello everyone,Here is the problem I am facing. I'm currently working on streaming data to DataBricks, my goal is to create a data stream on a first notebook, and then on a second notebook to read this data stream, add all the new rows to a dataFrame...
- 2506 Views
- 2 replies
- 4 kudos
- 4 kudos
Problem solved!Instead of trying to do everything directly with the .writeStream options I used the .forEachBatch() function which allows me to call a function outside the .writeStream().In this function I get a dataFrame in parameter which is my str...
- 4 kudos
- 2655 Views
- 2 replies
- 1 kudos
Resolved! Trying to Flatten My Json using CosmosDB Spark connector - Azure Databricks
Hi,Using the below cosmos DB query it is possible to achieve the expected output, but how can I do the same with spark SQL in Databricks.COSMOSDB QUERY : select c.ReportId,c.ReportName,i.price,p as provider from c join i in in_network join p in i.pr...
- 2655 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Abhishek Tomar​ , If you want to get it from Cosmos DB, use the connector with a custom query https://github.com/Azure/azure-cosmosdb-sparkIf you want to have JSON imported directly by databricks/spark, please go with the below solution:SELECT ...
- 1 kudos