Sounds awesome But personally, I find that the diaspora* core should do one thing well and only that - act as a social media server.
IMHO directories for d* could be small server apps themselves that could be hosted by anyone and in pod settings podmins could have a config setting for the "main" or first directory to use. Directories could learn from each other etc, but otherwise, they would just be directories and offer directory services. Pod users could query them, add to them, etc. Directories would also offer a search endpoint that pods could hook up to (chosen in settings).
I know this isn't a purely decentralized model, but as said before, the diaspora core server is already too heavy. Personally I wouldn't add anything more to it. It's not just performance, but it's also code burden. IMHO a component architecture is better than do-it-all, for diaspora. Redmatrix I understand, since it's more of a do-it-all solution for federated social spaces. But diaspora* is more like a traditional social media server app.
But the problem Sean describes is a huge one for diaspora. Discovery is really limited and should definitely be improved (including searching by metadata, like other socnet usernames, for example). But should that part be a part of the core? IMHO no.