public class HDFSOffsetReader extends AbstractOffsetReader
Constructor and Description |
---|
HDFSOffsetReader(org.apache.hadoop.conf.Configuration config,
org.apache.hadoop.fs.Path baseOffsetStoragePath)
Creates a reader instance for interacting with the storage specified by the
config and with
the base storage path of baseStoragePath . |
Modifier and Type | Method and Description |
---|---|
void |
close() |
List<Long> |
getStoredOffsetPersistenceTimes()
Returns the list of available persistence times offsets have been written to the underlying storage mechanism.
|
Map<org.apache.kafka.common.TopicPartition,Long> |
readLatestOffsets()
Reads the last stored offsets.
|
Map<org.apache.kafka.common.TopicPartition,Long> |
readOffsets(long persistedOffsetTime)
Reads the offsets for a given
persistedOffsetTime . |
public HDFSOffsetReader(org.apache.hadoop.conf.Configuration config, org.apache.hadoop.fs.Path baseOffsetStoragePath)
config
and with
the base storage path of baseStoragePath
.config
- the config for interacting with the underlying data store.baseOffsetStoragePath
- the base storage path for offset information. If the path does not exist it will
be created.IllegalArgumentException
- if either argument is null
.public Map<org.apache.kafka.common.TopicPartition,Long> readLatestOffsets() throws IOException
OffsetReader
IOException
- if there is an error reading from the underlying storage.public Map<org.apache.kafka.common.TopicPartition,Long> readOffsets(long persistedOffsetTime) throws IOException
OffsetReader
persistedOffsetTime
. Note that not all storage mechanisms support
complete historical offset information. Use the OffsetReader.getStoredOffsetPersistenceTimes()
to find valid values
to specify for persistedOffsetTime
.readOffsets
in interface OffsetReader
readOffsets
in class AbstractOffsetReader
persistedOffsetTime
- the persistence time when offsets were written to the underlying storage system.persistedOffsetTime
. If no offsets were persisted
at that time or available to be retrieved then null
will be returned.IOException
- if there is an error reading from the underlying storage.public List<Long> getStoredOffsetPersistenceTimes() throws IOException
OffsetReader
getStoredOffsetPersistenceTimes
in interface OffsetReader
getStoredOffsetPersistenceTimes
in class AbstractOffsetReader
empty list
is returned.IOException
- if there is an error reading from the underlying storage.public void close() throws IOException
IOException
Copyright © 2017 The Apache Software Foundation. All rights reserved.