I would like some technology thoughts on a comparison use case. we process million of data points / sec in time series data. We have a ingestion endpoint service which hands it over to a compute intensive service which indices these points. We do gRPC hand off today. Some folks in the team want to move to async streaming processing model with Kafka which give immediate fault tolerance etc. Some team members claim adding a messaging layer adds latency to affect the processing of indices. What are trade off there?
Kafka streaming vs gRPC batching time series data
122 views Asked by Manoj Ramakrishnan At
0
There are 0 answers
Related Questions in APACHE-KAFKA
- No method found for class java.lang.String in Kafka
- How to create beans of the same class for multiple template parameters in Spring
- Troubleshoot .readStream function not working in kafka-spark streaming (pyspark in colab notebook)
- Handling and ignore UNKNOWN_TOPIC_OR_PARTITION error in Kafka Streams
- Connect Apache Flink with Apache kudu as sink using Pyflink
- Embedded Kafka Failed to Start After Spring Starter Parent Version 3.1.10
- Producer Batching Service Bus Vs Kafka
- How to create a docker composer environment where containers can communicate each other?
- Springboot Kafka Consumer unable to maintain connect to kafka cluster brokers
- Kafka integration between two micro service which can respond back to the same function initiated the request
- Configuring Apache Spark's MemoryStream to simulate Kafka stream
- Opentelemetry Surpresses Kafka Produce Message Java
- Kafka: java.lang.NoClassDefFoundError: Could not initialize class org.apache.logging.log4j.core.appender.mom.kafka.KafkaManager
- MassTransit Kafka producers configure to send several events to the same Kafka topic
- NoClassDefFoundError when running JAR file with Apache Kafka dependencies
Related Questions in GRPC
- gRPC: Asynchronous client thread safety clarification
- gRPC client is slower in C++ than using `ghz` with the same count of threads
- Incorrect/incomplete python pb2 file generation with buf
- Logging interceptor using grpc.aio python
- Shutting down a channel of a client from Server using grpc-java
- Can I create a gRPC recipe for Python4Android to build a firestore kivy app?
- gRPC async server : bad thread scailibility
- Error during signUp: Error: 13 INTERNAL: Received RST_STREAM with code 2 triggered by internal client error: Protocol error
- Should I use string or Value as type in a message for a serialized object?
- Is there a way that allows us to queue multiple grpc requests?
- How do I programmatically enable InternalLogLevel.TRACE for Netty's Http2FrameLogger, relied upon by grpc-java?
- Grpc Java - Failed to set up TLS using boring ssl on Redhat
- how can i use googlemap gRPC in android?
- Least Connection Load balancing using Grpc
- Missing packages in github.com/grpc-ecosystem/go-grpc-middleware upgrading from v1 to v2
Related Questions in STREAM-PROCESSING
- Running distributed applications using stream processing?
- Order Preserving Union/Merge Executor in Flink
- State Store or SQL Database?
- Using KafkaStreams TimeWindows for detecting timeout and expiry
- Apache Beam portable runner to run golang based job on flink
- Kafka streaming vs gRPC batching time series data
- Server Logs aggregation/reduction with KsqlDB
- Are checkpoints needed for a stream processed in databricks job running via continuous trigger?
- Apache flink job not printing anything to stdout and not sending to Kafka sink topic
- How to access the results from ProcessFunction?
- Good way to parallel process elements of 'stream' while keeping output in order
- Unable to perform 'maxBy' on a Flink windowed stream of type <SampleClass> by using the SampleClass's field as a parameter does not work
- Is there a suitable python library for doing stream processing with Kafka topics?
- In Kafka Streams, how do you parallelize complex operations (or sub-topologies) using multiple topics and partitions?
- Is it possable to stream images to a PDF using tesseract?
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
Popular Tags
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)