Does HDFS sink support writing to sequence file in Spring XD?

Question

I would like to use HDFS sink in stream processing for writing data of binary key-value pair to HDFS. As a result, a thin wrapper format like sequence file will suffice. Does the HDFS sink module support writing key-value to HDFS sequence file at all?

Thanks in advance!

Answer 1

Looks like this is yet to be addressed: https://jira.spring.io/browse/XD-992. You can vote up. If you want to contribute, you are welcome. Please refer to Spring XD documentation and https://github.com/spring-projects/spring-xd-modules for examples on how to develop modules.

Also, FYI: there is hdfs-dataset sink module that supports Avro/Parquet format. https://github.com/spring-projects/spring-xd/blob/master/modules/sink/hdfs-dataset/config/hdfs-dataset.xml