Expiring old entries - ever growing database problem

The database is constantly growing, even with one active user, not sharing with and being shared with much.

Since I have started running my pod, according to phpMyAdmin the whole database has grown to ~2,661,012 entries with a data usage of 2.8 GiB (!) and only my “little” pod.

The feature should be configurable through yaml config or admin interface.

Updated from comment poster:
–>Only the pod of the original poster is obliged to store the post forever. The other pods can expire the post if no user fetched it for a certain time period. If the pod expires the post it removes everything except the posts metadata from the database. The metadata includes all the stuff needed to keep the post searchable (id, posting user, hashtags). If the pod ever needs the post again it will fetch it from the original posters pod.

This should significantly reduce the database size (especially for small pods) while keeping all posts accessible if needed.<–


Note: This discussion was imported from Loomio. Click here to view the original discussion.

I could think of the following system: Only the pod of the original poster is obliged to store the post forever. The other pods can expire the post if no user fetched it for a certain time period. If the pod expires the post it removes everything except the posts metadata from the database. The metadata includes all the stuff needed to keep the post searchable (id, posting user, hashtags). If the pod ever needs the post again it will fetch it from the original posters pod.

This should significantly reduce the database size (especially for small pods) while keeping all posts accessible if needed.

I like this idea, proposal is edited with your idea.

You may be interested in this: https://github.com/diaspora/diaspora/issues/4920
In short: The problem is known and being worked on. It actually seems not to be caused by the number of posts but by other meta data.

Also, although currently postponed, a re-fetch feature has been talked about. It may get implemented in the future.

Maybe there could be a way to set an expiry on a post, for example if I post about an event next week, there could be a way to set event expires and as poster it can be deleted.

For your information, #4920 (linked above by @CSammy) is now closed in the develop branch. The next diaspora* major release (0.6.0.0) will drastically reduce the disk space used by the database.