I'm trying to ingest streaming HL7 messages and planned to use Kafka with Smolder to achieve this. I'm having a little trouble understanding how Kafka can run on databricks. So far I've installed Kafka and started the cluster but the start command just spins forever. Am I crazy to think this should be possible? Any resources someone can share on how to accomplish this?
@Jose Gonzalezโ Sorry for the delay. I've actually already looked at the provided links. The first one is for AWS. I'm currently using Azure. The KAFKA setup is working..I think? I'm stuck at ok so I have a server listening now what? How do I send/retrieve messages? Do I need to configure some IP settings on the compute so it works properly in Azure? What do I provide to the interface server so it can send messages to the compute?
Welcome to Databricks Community: Lets learn, network and celebrate together
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.