spark-streaming read from specific event hub partition

Sandesh87
New Contributor III

The azure event hub "my_event_hub" has a total of 5 partitions ("0", "1", "2", "3", "4")

The readstream should only read events from partitions "0" and "4"

event hub configuration as streaming source:-

val name = "my_event_hub"
val connectionString = "my_event_hub_connection_string"
val max_events = 50
 
 
val positions = Map(
  new NameAndPartition(name, 0) -> EventPosition.fromEndOfStream,
  new NameAndPartition(name, 4) -> EventPosition.fromEndOfStream
)
 
val eventHubsConf = EventHubsConf(connectionString)
                    .setStartingPositions(start)
                    .setMaxEventsPerTrigger(max_Events)

official doc for structured-streaming-eventhubs-integation: https://github.com/Azure/azure-event-hubs-spark/blob/master/docs/structured-streaming-eventhubs-inte...

Using the above configuration the streaming application reads from all 5 partitions of the event hub. Can we read from specific partitions only?

For example read events only from 2 partitions "0" and "4" with the checkpoint and offsets pointed to the specific partitions.