One of the most difficult challenges that most decentralized social networks face is that of content and user discovery. Everybody and all of their stuff lives in different places on the network, and finding anything often requires being aware of where that thing lives in the first place. It’s a growing pain that only gets more pronounced as the network gets bigger. That’s why the Mastodon project hopes to tackle this problem head on with Fediverse Discovery Providers.

What is it?

This initiative, funded by an NGI Search grant, aims to develop infrastructure and a reference implementation for “pluggable services” that people can use with their instances to find stuff. A few interesting use-cases are highlighted on the project repository:

• Search and Discovery: No single instance has a complete view of the full fediverse, and if it had, that would be very expensive and result in a lot of duplicated content overall.
• Spam Detection: This is not an easy problem to solve and every single Fediverse software having to implement their own solution seems wasteful. Also, it really helps to fight this efficiently if you are able to work on data from multiple servers.
• Block synchronization: Right now, most instances coordinate blocks either through manual curation of shared lists, and this can bring a number of inefficiencies and conflicts, especially when only a handful of admins are involved.
• Link Preview generation: All instances fetching a web page to generate a link preview once a single post with a link is being federated, means the web site gets hammered with a lot of requests in a short timeframe.
• Helpdesk integration: if there are problems related to a given service, it usually relies on manual outreach and reporting, which can have a number of barriers to entry. Being able to directly report issues at the service level with necessary context could speed up turnaround time, and make it easier to fix problems.

Looking at these examples, it’s clear that the project aims to provide a service infrastructure framework that any Fediverse instance might be able to take advantage of.

If we have not made it clear enough before: Fediverse Discovery Providers are not a project just for Mastodon.

We would like to make them usable and useful to as many Fediverse services as possible. For this we require feedback and engagement from fellow implementers.

Project FAQ

If you looked at this and thought “gee, this sounds like what Nostr relays and Bluesky’s various layers do”, you’re right! Fediverse Discovery Providers would basically act as ad-hoc services that instances could leverage.

Prior Efforts

It’s worth pointing out that a number of different efforts exist in the network today that try to tackle specific domain areas:

• Discovery: Pixelfed uses a microservice called Beagle for surfacing trending posts between instances, and Newsmast has done some incredibly interesting work with Patchwork, a pluggable service for Mastodon that allows for special curated feeds.
• Blocklists / Moderation: IFTAS has done a lot of groundbreaking development with CARIAD, which serves as the basis for FediCheck. For the Lemmy community, db0 also develops FediSeer for instance curation.
• Content / User Search: After Mastodon released a robust update to their full-text search system that respects user privacy, it’s become somewhat easier to find things across the network. PeerTube also introduced Sepia Search, a type of global index of PeerTube videos, in a bid to aid user discovery. Finally, NeoDB is doing some really interesting work with creating data catalogues of just about anything you could ever write a review for.
• Instance / Service Discovery: FediDB does some incredible work with tracking the growth and adoption of different platforms and instances, and can provide a comprehensive breakdown of the network. Just about every platform’s project site also offers an instance list for newcomers, so this might be a way for them to also highlight available servers across the network.

These are all worthwhile efforts that attempt to chip away at different parts of the discovery problem, but most of them are platform-specific. In the future, it may be possible that these all become Fediverse Discovery Providers. Being platform-agnostic would likely extend the benefits of each effort to be used by instances across the entire network, regardless of platform.

What’s Next?

These are still very much early days for the project, but there’s already active discussions on how the network could move to make Fediverse Discovery Providers a reality. One area that the project ought to dig into is existing Fediverse Enhancement Proposals, such as these:

• FEP-eb22: Supported ActivityStreams types with NodeInfo
• FEP-5e53: Opt-out Preference Signals
• FEP-9fde: Mechanism for servers to expose supported operations
• FEP-6481: Specifying ActivityPub extension support with NodeInfo
• FEP-400e: Publicly-appendable ActivityPub collections

These are just a few ideas to get started with. There’s a big opportunity to advance community standards in a way that works for everybody, and enriches the network to the fullest while following best practices for user consent and privacy. Here’s hoping that it bears fruit.