It's been a long time since there has been any news on the state of federation, so here's an update on where MediaGoblin's at and some technical aspects of federation. We've been working with the W3C Social Working Group to define the future of federation, and part of my work there has been to work on the ActivityPump standard. There's more to say on that and why we're investing time there, but this blogpost will mostly be about MediaGoblin and federation from a technical perspective.
Over the last year I've been building up the foundations of federation, creating the necessary APIs and infrastructure needed. I am currently finishing the last of that, modifying the models to handle federated data. This is a challenge as the models were designed without federation in mind. There is a lot of new data that is being stored as well as remote and local versions of the models.
Once I've finished working on the models I'll be starting the work on actually getting the media and comments federating across instances.
The core of the federation and APIs are "Activities", these are created all the time right now in 0.8.0. Every time someone posts media, comments, tags or updates anything. These serve two main purposes:
- Command language: instructing the server to do something
- Log: telling clients and other servers what has been done
The activities at their core at subject predicate object, for example:
Chris posts an image
The activity also usually contains the audience targeting data (i.e. who it's for). These are represented in JSON with elements being nested as other JSON objects, lets take a look:
{
"id": "https://mediagoblin.com/api/activity/1"
"verb": "post",
"objectType": "activity",
"actor": {
"id": "acct:cwebber@mediagoblin.com",
"displayName": "Christopher Allan Webber",
"objectType": "person"
},
"object": {
"id": "https://mediagoblin.com/api/image/1",
"objectType": "image",
"url": "https://mediagoblin.com/u/cwebber/m/mediagoblin-shirt-close-up-and-badge/",
"fullImage": "https://mediagoblin.com/mgoblin_media/media_entries/814/IMG_1011_modified.jpg"
},
"to": [
{
"id": "https://mediagoblin.com/api/user/cwebber/followers",
"objectType": "collection"
}
]
}
This might look overwhelming but it's the same basic subject predicate object sentence above. Chris (the actor) is posting (the verb) an image (the object), it also contains some extra information like IDs so they can be referenced and the audience which is a a collection of people that follow Chris.
Once we have this activity, what actually happens, you ask? Each user has an inbox and outbox like you do with email. Step by step walkthrough of what would happen when someone posts an image from a desktop or mobile client:
1 Client to server
back a preliminary object to use in the API.
- The client creates an activity that at minimum would look like this:
{
"verb": "post",
"objectType": "activity",
"object": {
"id": "https://mediagoblin.com/api/image/1",
"objectType": "image",
},
"to": [
{
"id": "https://mediagoblin.com/api/user/cwebber/followers",
"objectType": "collection"
}
]
}
- This activity is then posted to the client's inbox.
- The server receives the activity, creates an ID, and attaches the actor and other metadata, then saves it.
2 Federation!
where we're all on our own servers. This is what the server (once it's finished) will do once it gets the client's request:
- The server looks for whom it's sent to (the "to" property) to get a list of people that the object has to be sent to.
- The activity is then sent to each persons inbox.
As one of Chris' followers you can now open up the website or any client and see Chris' image. Commenting on the image will produce an activity which is sent to Chris, who subsequently sends it as an update to everyone else.
Phew! we're done, that's how the API and federation works in MediaGoblin. We've successfully posted an image from a client and had it sent around and hopefully you're able to see how a comment or something else would work in this system too.
This is just a high level overview, there are many technical problems such as access control (who's able to view content), verification of objects (checking what we've been sent is correct), etc. Federation doesn't come easy but we think it's well worth it.