Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-17-2023 09:47 AM
I tried using below snippet to receive messages only from partition id=0
ehName = "<<EVENT-HUB-NAME>>"
# Create event position for partition 0
positionKey1 = {
"ehName": ehName,
"partitionId": 0
}
eventPosition1 = {
"offset": "@latest",
"seqNo": -1,
"enqueuedTime": None,
"isInclusive": True
}
# Put the rules into a map. The position key dictionaries must be made into JSON strings to act as the key.
positionMap = {
json.dumps(positionKey1) : eventPosition1
}
# Place the map into the main Event Hub config dictionary
ehConf["eventhubs.startingPositions"] = json.dumps(positionMap)
ehConf["eventhubs.connectionString"] = <eventhub_connection_string>
ehConf["eventhubs.consumerGroup"] = <eventhub_consumer_group>
df = spark \
.readStream \
.format("eventhubs") \
.schema(jsonSchema) \
.options(**conf) \
.load()But still i receive messages from other partitions as well.
Can anyone guide where am i missing ?