The implementations of Pregel I have seen so far (GraphX, Flink Gelly) assume a static graph model where we know the vertices and edges beforehand. I searched for any Pregel like framework that could process streaming data or dynamic graphs and was surprised that I could not come across any especially when according to it's paper Pregel can handle graph topology mutations:
As with messages, mutations become effective in the superstep after the requests were issued. Within that superstep removals are performed first, with edge removal before vertex removal, since removing a vertex implicitly removes all of its out-edges. Additions follow removals, with vertex addition before edge addition, and all mutations precede calls to Compute(). This partial ordering yields deterministic results for most con icts.
The remaining conflicts are resolved by user-defined handlers. If there are multiple requests to create the same vertex in the same superstep, then by default the system just picks one arbitrarily, but users with special needs may specify a better conflict resolution policy by defining an appropriate handler method in their Vertex subclass. The same handler mechanism is used to resolve conflicts caused by multiple vertex removal requests, or by multiple edge addition or removal requests. We delegate the resolution to handlers to keep the code of Compute() simple, which limits the interaction between a handler and Compute(), but has not been an issue in practice."
Correct me if I'm wrong but Apache Giraph also assumes the graph is static for running Pregel algorithms like SSSP and PageRank.
If there is any such framework or API in graph streaming that processes dynamic graphs? If not, is there any particular reason why vertex-centric Pregel approach has not been extended to graph streams?