I have an activity stream in my web app, which uses a setup almost identical to the one described here: How to implement the activity stream in a social network
In short: the data structure currently consists of just one long denormalized MySQL table.
The above post also suggests the possibility of using Redis as a cache for the most recent 100 or so activities for each user. I have already started developing this in a way where each user has a Redis list named something like "uid:123:activities" and each list item is a json encoded PHP array saved as a string. The JSON contains information like "user_id", "time", "photo_id" etc.
However, I seem to be faced with a problem. How can I remove activities from a users list in Redis when one of a participants in that activity is deleted from the application? e.g. If someone you follow deletes their account, any activities involving them need to be removed from your stream. I see two potential approaches, neither of which seem fantastic:
- Every time something is deleted, loop through all users lists and list items looking for references to it and if one exists, delete it. (This probably isn't realistic)
- When building the steam from the list, check if the item exists and if it doesn't make sure to delete that item from the list.
A lot of people suggest using Redis for activity streams structured in similar ways, but I haven't seen the issue of keeping the list in sync with the original data source addressed.
Does anyone have any other suggestions before I go down the route of option 2?
I'm one of the authors of Stream-Framework and getstream.io The most common approach is to:
1.) Fanout deletes and remove them from every feed, and:
2.) Add a filtering layer while doing reads
This second step is useful in case people submit inappropriate content, you have advanced privacy settings, or for some other reason you want to remove the data quickly.