public class KafkaUtils extends Object
KafkaSource
instance.Modifier and Type | Field and Description |
---|---|
static String |
KAFKA_EMPTY_RETRY_ATTEMPTS_KEY
Configuration property for the number of retry attempts that will be made to Kafka in the event of getting empty
responses.
|
static int |
KAFKA_RETRY_ATTEMPTS_DEFAULT
Default number of retry attempts.
|
static String |
KAFKA_RETRY_ATTEMPTS_DEFAULT_STRING |
static String |
KAFKA_RETRY_ATTEMPTS_KEY
Configuration property for the number of retry attempts that will be made to Kafka.
|
static int |
KAFKA_RETRY_EMPTY_ATTEMPTS_DEFAULT
Default number of empty retry attempts.
|
static String |
KAFKA_RETRY_EMPTY_ATTEMPTS_DEFAULT_STRING |
Constructor and Description |
---|
KafkaUtils() |
Modifier and Type | Method and Description |
---|---|
static org.apache.hadoop.conf.Configuration |
addKafkaConnectionProperties(Properties properties,
org.apache.hadoop.conf.Configuration config)
Adds the
properties to the provided config instance. |
static Map<org.apache.kafka.common.TopicPartition,Long> |
getBrokerOffsets(Properties properties,
long time,
String... topics)
Retrieves the offset values for an array of topics at the specified time.
|
static Properties |
getKafkaConnectionProperties(org.apache.hadoop.conf.Configuration config)
Converts the provided
config into a Properties object to connect with Kafka. |
public static final String KAFKA_RETRY_ATTEMPTS_KEY
public static final int KAFKA_RETRY_ATTEMPTS_DEFAULT
public static final String KAFKA_RETRY_ATTEMPTS_DEFAULT_STRING
public static final String KAFKA_EMPTY_RETRY_ATTEMPTS_KEY
public static final int KAFKA_RETRY_EMPTY_ATTEMPTS_DEFAULT
public static final String KAFKA_RETRY_EMPTY_ATTEMPTS_DEFAULT_STRING
public static Properties getKafkaConnectionProperties(org.apache.hadoop.conf.Configuration config)
config
into a Properties
object to connect with Kafka.config
- the config to read propertiesconfig
.public static org.apache.hadoop.conf.Configuration addKafkaConnectionProperties(Properties properties, org.apache.hadoop.conf.Configuration config)
properties
to the provided config
instance.properties
- the properties to add to the config.config
- the configuration instance to be modified.public static Map<org.apache.kafka.common.TopicPartition,Long> getBrokerOffsets(Properties properties, long time, String... topics)
Retrieves the offset values for an array of topics at the specified time.
If the Kafka cluster does not have the logs for the partition at the specified time or if the topic did not exist at that time this will instead return the earliest offset for that partition.
properties
- the properties containing the configuration for kafkatime
- the time at which we want to know what the offset values weretopics
- the topics we want to know the offset values ofIllegalArgumentException
- if properties is null
or if topics is null
or empty or if any of
the topics are null
, empty or blank, or if there is an error parsing the
properties.IllegalStateException
- if there is an error communicating with the Kafka cluster to retrieve information.Copyright © 2017 The Apache Software Foundation. All rights reserved.