How to configure generic parameters in Storm Flux

345 views Asked by At

I am a new guy on Storm Flux, and now confused by how to configure generic parameters in Storm Flux. For example, the org.apache.storm.kafka.spourt.KafkaSpout is defined as below:

 public class KafkaSpout<K, V> extends BaseRichSpout {

    ....

 }

And there is a example YAML file using this class:

  components:
    - id: "onlyValueRecordTranslator"
      className: "org.apache.storm.flux.examples.OnlyValueRecordTranslator"

    - id: "spoutConfigBuilder"
      className: "org.apache.storm.kafka.spout.KafkaSpoutConfig$Builder"
      constructorArgs:
         - "localhost:9092"
         - ["myKafkaTopic"]
      properties:
         - name: "firstPollOffsetStrategy"
           value: EARLIEST
         - name: "recordTranslator"
           ref: "onlyValueRecordTranslator"
      configMethods:
         - name: "setProp"
           args:
           - {
                 "key.deserializer" : "org.apache.kafka.common.serialization.StringDeserializer",
                 "value.deserializer": "org.apache.kafka.common.serialization.StringDeserializer"
             }

         - id: "spoutConfig"
           className: "org.apache.storm.kafka.spout.KafkaSpoutConfig"
           constructorArgs:
              - ref: "spoutConfigBuilder"

   config:
        topology.workers: 1

   # spout definitions
   spouts:
       - id: "kafka-spout"
         className: "org.apache.storm.kafka.spout.KafkaSpout"
         constructorArgs:
             - ref: "spoutConfig"

My question is why there is not any configuration for the generic paratmers of KafkaSpout?

Thanks!

1

There are 1 answers

2
Stig Rohde Døssing On

I don't think anyone has found it necessary to implement generics support in Flux yet.

Recall that in Java you are free to use "raw types" (though it's usually a bad idea), i.e. you can do new ArrayList(), and it'll be like you wrote new ArrayList<Object>(). Generics can help you be more specific in which types you expect to put in the list, and let you catch type errors at compile time.

Flux is more like an interpreted language in that you have to submit your topology to Flux to know whether your yaml is valid. Generics wouldn't really have much of a point then, especially because Flux is pretty liberal with type coercion as well.