From c698cf41fb91431379585e99c0f0666a545bc8b8 Mon Sep 17 00:00:00 2001 From: lda Date: Sat, 5 Oct 2024 15:40:28 +0200 Subject: [PATCH 01/12] Add basic WebXDC draft It is most likely full of errors --- proposals/xxxx-webxdc.md | 156 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 156 insertions(+) create mode 100644 proposals/xxxx-webxdc.md diff --git a/proposals/xxxx-webxdc.md b/proposals/xxxx-webxdc.md new file mode 100644 index 00000000000..4be1c79f543 --- /dev/null +++ b/proposals/xxxx-webxdc.md @@ -0,0 +1,156 @@ +# MSCXXXX: WebXDC on Matrix + + +In Matrix, users may want to send each other interactive content (for example a game +in a groupchat, or maybe a collaborative board). Previously, integrations were used +to provide such content, but they are not in-band (thus needs another server, which +tends to be unmaintained as of now). As such, this proposal aims to resolve this by +adding [WebXDC](https://webxdc.org/) into the Matrix client ecosystem, with no +server-side changes or additions needed. + +## Proposal + +A common method, with a well-defined API and that isn't locked to the Matrix ecosystem +(which may be useful for, e.g bridges, which would greatly benefit from such a common +system, and even client developers, who can simply rely on pre-existing tooling) and +that wouldn't require server administrators to either host another software solution +or users to resort to a centralised hub(as integrations) would be highly appreciable. + +(Please note that the following simply describes how WebXDC calls transmate to +Matrix events. More information is available at WebXDC's own specification) + +To start a WebXDC event, a client may start by sending a `m.webxdc.start` event, +formatted as such: +```json +{ + "content": { + "name": "Storm on Mt. Ooe", // optional, must be a string + "url": "mxc://hell.club/WebXDCFileForClients", // MUST be a .xdc file + "icon": "mxc://hell.club/AnIconForTheWebXDC", // optional, + // content.icon_mime must exist + // iff present + "icon_mime": "image/png" // present IFF content.icon is present + }, + "room_id": "!former:hell.club", + "event_id": "$WebXDCStart1", + "sender": "@yuugi:hell.club", + "type": "m.webxdc.start" + ... +} +``` +where the `content.url` field must be a MXC URI pointing to the WebXDC media. +Once a user starts, a client can show the event(alongside `content.name`, and +`content.icon`/`content.icon_mime`) as an invitation to enter the WebXDC context. + +If so, it MUST set `window.webxdc.selfAddr` and `window.webxdc.selfName` to the +user's MXID and name in the room(or the global name stored in the profile API if +unavailable, otherwise fallback to the MXID), respectively. + +Whenever a client calls the JavaScript `window.webxdc.sendUpdate(update, desc)` +function, it MUST send a `m.room.message` with a relation to the original event +with `rel_type=m.webxdc`, the `content.body` field being the description if existent, +and the *required* `m.webxdc.data` containing a JSON-encoded object as a representation +of `update`, which must be stored as a string if it contains floating-point numbers, +like so: +```js +// Client tries to send an update +window.webxdc.sendUpdate( + { + payload: {graze: 430, score: 5300}, + info: window.webxdc.selfName + ' got over the Spellcard!', + summary: "Score: 5300" + }, + "New Score on Mt. Ooe" +); +``` + +```json +{ + "content": { + "m.relates_to": { "event_id": "$WebXDCStart1", "rel_type": "m.webxdc" }, + "body": "Marisa got over the Spellcard!", + "m.webxdc.data": { + "payload": { + "graze": 430, + "score": 5300 + }, + "info": "Marisa got over the Spellcard!", + "summary": "Score: 5300" + } + }, + "sender": "@marisa:magic.forest", + "room_id": "!former:hell.club", + "event_id": "$WebXDCEvent", + "type": "m.room.message", + ... +} +``` + +Clients receiving such an event(including themselves) must process it with +the listener set with `window.webxdc.setUpdateListener`, with the update's +`payload` being the `content->m.webxdc.data` object, it's `info`, `document` +and `summary` being the `content->m.webxdc.data->info`, `content->m.webxdc.data->document`, +and `content->m.webxdc.data->summary` fields respectively, the +`serial`/`max_serial` fieldd being the current and last event's `origin_server_ts` +fields respecticely. They must also check for such events (ideally via the +relations API). + +The `window.webxdc.sendToChat(message)` function shall send a regular event to the +current room with the `content->body` field being set to the `message.text` value, +and being sent as the correct type of media if the `media.file` value exists. + + +## Potential issues + +Non WebXDC-capable clients currently do not get a fallback for WebXDC start events, +which means that such messages will either not display, or will be hidden. Note that +this isn't a big issue, since clients that simply do not handle WebXDC generally have +no real reason to have the file in the first place. + +Handling of floating-point values within events within the spec(which effectively do +not allow such things) may be an issue for WebXDC applications which require it. + +Servers may be out of sync with the `origin_server_ts`, thus causing conditions where +two WebXDC updates `A` and `B`, such that `A` comes before `B`, may be processed in the +order `B` *then* `A`. + +As it is experimental, this proposal does not provide a mapping between +`window.webxdc.joinRealtimeChannel` and a real, Matrix-side construct. The author also +doesn't see a good way to make it function alongside the existing event system properly, +and recommends waiting until it is stable within the WebXDC API before making an MSC +which supports it. As such, clients MUST set the function to the JavaScript `undefined`. + + +## Alternatives + +Another way to have interactive content in rooms would be to use an integration manager. +However, these would require an additional server to be setup(which are not maintained), +and would effectively be limited to the Matrix ecosystem only, instead of being platform- +agnostic. +Widgets(MSC1236), while powerful, also suffer from a similar agnosticity problem with +other platforms. + +## Security considerations + +WebXDC still suffers from some issues, due to its nature, which is effectively a packaged web +application. As such, client developers will need to be careful to properly conceal secrets +(like a user's access token), and must follow [the WebXDC constraints](https://webxdc.org/docs/spec/messenger.html#webview-constraints-for-running-apps). +A malicious user could also use WebXDC as a vector for existing vulnerabilities in, e.g, a +browser/JavaScript engine, thus being able to break out of sandboxes. Clients must first +consult the user on whenever they want to start a WebXDC session, and must warn of potential +risks, especially with untrusted entities. + + +## Unstable prefix + +Until this MSC is merged, client shall replace the following namespaces: +`m.webxdc`->`at.kappach.at.webxdc` +`m.webxdc.start`->`at.kappach.at.webxdc.start` +`m.webxdc.data`->`at.kappach.at.webxdc.data` + +As this proposal acts at the event layer, there is no need to ask the server for if it has +support for this proposal. + +## Dependencies + +This MSC is independent of any other unmerged MSCs. From 44a359758c03fe1d6dceba063e87392361d27172 Mon Sep 17 00:00:00 2001 From: lda Date: Sat, 5 Oct 2024 15:49:50 +0200 Subject: [PATCH 02/12] Rename WebXDC proposal --- proposals/{xxxx-webxdc.md => 4190-webxdc.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename proposals/{xxxx-webxdc.md => 4190-webxdc.md} (100%) diff --git a/proposals/xxxx-webxdc.md b/proposals/4190-webxdc.md similarity index 100% rename from proposals/xxxx-webxdc.md rename to proposals/4190-webxdc.md From 73de554a4a3d6cd6a3438f6f084208a8035be6a0 Mon Sep 17 00:00:00 2001 From: lda Date: Sat, 5 Oct 2024 15:50:41 +0200 Subject: [PATCH 03/12] Rename title --- proposals/4190-webxdc.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4190-webxdc.md b/proposals/4190-webxdc.md index 4be1c79f543..40ba4e72af2 100644 --- a/proposals/4190-webxdc.md +++ b/proposals/4190-webxdc.md @@ -1,4 +1,4 @@ -# MSCXXXX: WebXDC on Matrix +# MSC4190: WebXDC on Matrix In Matrix, users may want to send each other interactive content (for example a game From 4c1f3f2f9c0d31aad86f6779065f5d07b5f1d840 Mon Sep 17 00:00:00 2001 From: lda Date: Sat, 5 Oct 2024 15:56:41 +0200 Subject: [PATCH 04/12] Give the MSC a proper title --- proposals/{4190-webxdc.md => 4211-webxdc.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename proposals/{4190-webxdc.md => 4211-webxdc.md} (99%) diff --git a/proposals/4190-webxdc.md b/proposals/4211-webxdc.md similarity index 99% rename from proposals/4190-webxdc.md rename to proposals/4211-webxdc.md index 40ba4e72af2..43b0e9bda01 100644 --- a/proposals/4190-webxdc.md +++ b/proposals/4211-webxdc.md @@ -1,4 +1,4 @@ -# MSC4190: WebXDC on Matrix +# MSC4211: WebXDC on Matrix In Matrix, users may want to send each other interactive content (for example a game From b9b0ee125d1ea0be8427f9071cb72807ef14412f Mon Sep 17 00:00:00 2001 From: lda Date: Sat, 5 Oct 2024 16:02:29 +0200 Subject: [PATCH 05/12] Give back MSC number You can tell I'm not good at this. Also signed off. Signed-off-by: lda --- proposals/{4211-webxdc.md => xxxx-webxdc.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename proposals/{4211-webxdc.md => xxxx-webxdc.md} (99%) diff --git a/proposals/4211-webxdc.md b/proposals/xxxx-webxdc.md similarity index 99% rename from proposals/4211-webxdc.md rename to proposals/xxxx-webxdc.md index 43b0e9bda01..4be1c79f543 100644 --- a/proposals/4211-webxdc.md +++ b/proposals/xxxx-webxdc.md @@ -1,4 +1,4 @@ -# MSC4211: WebXDC on Matrix +# MSCXXXX: WebXDC on Matrix In Matrix, users may want to send each other interactive content (for example a game From 38eb76a80ced1e939d185d452674cf2ede58aca1 Mon Sep 17 00:00:00 2001 From: lda Date: Sat, 5 Oct 2024 16:07:40 +0200 Subject: [PATCH 06/12] Actually listen to Tulir :( Signed-off-by: lda --- proposals/{xxxx-webxdc.md => 4211-webxdc.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename proposals/{xxxx-webxdc.md => 4211-webxdc.md} (99%) diff --git a/proposals/xxxx-webxdc.md b/proposals/4211-webxdc.md similarity index 99% rename from proposals/xxxx-webxdc.md rename to proposals/4211-webxdc.md index 4be1c79f543..43b0e9bda01 100644 --- a/proposals/xxxx-webxdc.md +++ b/proposals/4211-webxdc.md @@ -1,4 +1,4 @@ -# MSCXXXX: WebXDC on Matrix +# MSC4211: WebXDC on Matrix In Matrix, users may want to send each other interactive content (for example a game From 9ed7764ca78ca38be32fb995ee003bf4dbd33622 Mon Sep 17 00:00:00 2001 From: lda Date: Sat, 5 Oct 2024 16:15:13 +0200 Subject: [PATCH 07/12] Use json with comments* --- proposals/4211-webxdc.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/4211-webxdc.md b/proposals/4211-webxdc.md index 43b0e9bda01..b658188f666 100644 --- a/proposals/4211-webxdc.md +++ b/proposals/4211-webxdc.md @@ -21,7 +21,7 @@ Matrix events. More information is available at WebXDC's own specification) To start a WebXDC event, a client may start by sending a `m.webxdc.start` event, formatted as such: -```json +```jsonc { "content": { "name": "Storm on Mt. Ooe", // optional, must be a string @@ -64,7 +64,7 @@ window.webxdc.sendUpdate( ); ``` -```json +```jsonc { "content": { "m.relates_to": { "event_id": "$WebXDCStart1", "rel_type": "m.webxdc" }, From 2ecdc8473af6e067d70b1a13b79c418eaf06aaab Mon Sep 17 00:00:00 2001 From: lda Date: Sat, 5 Oct 2024 16:22:31 +0200 Subject: [PATCH 08/12] Use a list --- proposals/4211-webxdc.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/4211-webxdc.md b/proposals/4211-webxdc.md index b658188f666..1a2b426d1f7 100644 --- a/proposals/4211-webxdc.md +++ b/proposals/4211-webxdc.md @@ -144,9 +144,9 @@ risks, especially with untrusted entities. ## Unstable prefix Until this MSC is merged, client shall replace the following namespaces: -`m.webxdc`->`at.kappach.at.webxdc` -`m.webxdc.start`->`at.kappach.at.webxdc.start` -`m.webxdc.data`->`at.kappach.at.webxdc.data` +- `m.webxdc`->`at.kappach.at.webxdc` +- `m.webxdc.start`->`at.kappach.at.webxdc.start` +- `m.webxdc.data`->`at.kappach.at.webxdc.data` As this proposal acts at the event layer, there is no need to ask the server for if it has support for this proposal. From 2da0a4749a8d0f8a6531483d262e9b4d43cd0966 Mon Sep 17 00:00:00 2001 From: lda Date: Sat, 5 Oct 2024 20:47:23 +0200 Subject: [PATCH 09/12] Expand on alternatives --- proposals/4211-webxdc.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/proposals/4211-webxdc.md b/proposals/4211-webxdc.md index 1a2b426d1f7..c96b7e046d6 100644 --- a/proposals/4211-webxdc.md +++ b/proposals/4211-webxdc.md @@ -128,7 +128,9 @@ However, these would require an additional server to be setup(which are not main and would effectively be limited to the Matrix ecosystem only, instead of being platform- agnostic. Widgets(MSC1236), while powerful, also suffer from a similar agnosticity problem with -other platforms. +other platforms, and has been noted as not having a "[...] canonical document to describe +[it]" last year. WebXDC also has stricter security concerns(e.g: clients cannot have +sensitive information leaked, unlike widgets), thus making it safer. ## Security considerations From 8f9fceb98bb6fddc46f7b935de223c49c279d896 Mon Sep 17 00:00:00 2001 From: lda Date: Tue, 8 Oct 2024 06:51:13 +0200 Subject: [PATCH 10/12] a bit of reviewing, eh? --- proposals/4211-webxdc.md | 49 +++++++++++++++++++++------------------- 1 file changed, 26 insertions(+), 23 deletions(-) diff --git a/proposals/4211-webxdc.md b/proposals/4211-webxdc.md index c96b7e046d6..d220437500a 100644 --- a/proposals/4211-webxdc.md +++ b/proposals/4211-webxdc.md @@ -6,7 +6,7 @@ in a groupchat, or maybe a collaborative board). Previously, integrations were u to provide such content, but they are not in-band (thus needs another server, which tends to be unmaintained as of now). As such, this proposal aims to resolve this by adding [WebXDC](https://webxdc.org/) into the Matrix client ecosystem, with no -server-side changes or additions needed. +server-side changes or additions needed to existing Matrix interfaces. ## Proposal @@ -17,7 +17,8 @@ that wouldn't require server administrators to either host another software solu or users to resort to a centralised hub(as integrations) would be highly appreciable. (Please note that the following simply describes how WebXDC calls transmate to -Matrix events. More information is available at WebXDC's own specification) +Matrix events. More information is available at WebXDC's +[own specification](https://webxdc.org/docs/spec/index.html)) To start a WebXDC event, a client may start by sending a `m.webxdc.start` event, formatted as such: @@ -42,9 +43,10 @@ where the `content.url` field must be a MXC URI pointing to the WebXDC media. Once a user starts, a client can show the event(alongside `content.name`, and `content.icon`/`content.icon_mime`) as an invitation to enter the WebXDC context. -If so, it MUST set `window.webxdc.selfAddr` and `window.webxdc.selfName` to the -user's MXID and name in the room(or the global name stored in the profile API if -unavailable, otherwise fallback to the MXID), respectively. +If the client is in the context, it MUST set `window.webxdc.selfAddr` and +`window.webxdc.selfName` to the user's MXID and name in the room(or the global +name stored in the profile API if unavailable, otherwise fallback to the MXID), +respectively. Whenever a client calls the JavaScript `window.webxdc.sendUpdate(update, desc)` function, it MUST send a `m.room.message` with a relation to the original event @@ -91,7 +93,7 @@ the listener set with `window.webxdc.setUpdateListener`, with the update's `payload` being the `content->m.webxdc.data` object, it's `info`, `document` and `summary` being the `content->m.webxdc.data->info`, `content->m.webxdc.data->document`, and `content->m.webxdc.data->summary` fields respectively, the -`serial`/`max_serial` fieldd being the current and last event's `origin_server_ts` +`serial`/`max_serial` field being the current and last event's `origin_server_ts` fields respecticely. They must also check for such events (ideally via the relations API). @@ -118,34 +120,35 @@ As it is experimental, this proposal does not provide a mapping between `window.webxdc.joinRealtimeChannel` and a real, Matrix-side construct. The author also doesn't see a good way to make it function alongside the existing event system properly, and recommends waiting until it is stable within the WebXDC API before making an MSC -which supports it. As such, clients MUST set the function to the JavaScript `undefined`. +which supports it. As such, clients MUST set the function to the JavaScript `undefined` +until a MSC which explicitely declares support for it is merged into the specification. ## Alternatives -Another way to have interactive content in rooms would be to use an integration manager. -However, these would require an additional server to be setup(which are not maintained), -and would effectively be limited to the Matrix ecosystem only, instead of being platform- -agnostic. -Widgets(MSC1236), while powerful, also suffer from a similar agnosticity problem with -other platforms, and has been noted as not having a "[...] canonical document to describe -[it]" last year. WebXDC also has stricter security concerns(e.g: clients cannot have -sensitive information leaked, unlike widgets), thus making it safer. +Widgets([MSC1236](https://github.com/matrix-org/matrix-spec-proposals/issues/3803)), while powerful, +also suffer from a similar agnosticity problem with other platforms, and effectively requires an actively +online, external server. WebXDC also has stricter security concerns(e.g: clients cannot have sensitive +information leaked, unlike widgets), thus making it safer, though a system that would also widgets to act +as basic WebXDC containers (allowing clients that already have support for the more complicated widgets, +while allowing others to focus on the WebXDC subset) would be an interesting compromise. ## Security considerations -WebXDC still suffers from some issues, due to its nature, which is effectively a packaged web -application. As such, client developers will need to be careful to properly conceal secrets -(like a user's access token), and must follow [the WebXDC constraints](https://webxdc.org/docs/spec/messenger.html#webview-constraints-for-running-apps). -A malicious user could also use WebXDC as a vector for existing vulnerabilities in, e.g, a -browser/JavaScript engine, thus being able to break out of sandboxes. Clients must first -consult the user on whenever they want to start a WebXDC session, and must warn of potential -risks, especially with untrusted entities. +WebXDC still suffers from some issues, due to its nature, which is effectively a packaged web application. +As such, client developers will need to be careful to properly conceal secrets (like a user's access token), +and must follow [the WebXDC constraints](https://webxdc.org/docs/spec/messenger.html#webview-constraints-for-running-apps). +A malicious user could also use WebXDC as a vector for existing vulnerabilities in, e.g, a browser/JavaScript +engine, thus being able to break out of sandboxes. Clients must first consult the user on whenever they want to +start a WebXDC session, and must warn of potential risks, especially with untrusted entities. A user could also +craft WebXDC events(not created through the application itself, or with a modified version of the application). +WebXDC app developers must take into consideration the source of a payload(and possibly take measures against, +but the author finds that discussing said measures falls outside the MSC's own scope). ## Unstable prefix -Until this MSC is merged, client shall replace the following namespaces: +Until this MSC is merged, clients shall replace the following namespaces accordingly: - `m.webxdc`->`at.kappach.at.webxdc` - `m.webxdc.start`->`at.kappach.at.webxdc.start` - `m.webxdc.data`->`at.kappach.at.webxdc.data` From 2151fb49025f22fa51f9c2eb1c3ee505737d8e2c Mon Sep 17 00:00:00 2001 From: lda Date: Tue, 8 Oct 2024 06:52:57 +0200 Subject: [PATCH 11/12] fix typos, eh? --- proposals/4211-webxdc.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4211-webxdc.md b/proposals/4211-webxdc.md index d220437500a..8780e3d7a9f 100644 --- a/proposals/4211-webxdc.md +++ b/proposals/4211-webxdc.md @@ -121,7 +121,7 @@ As it is experimental, this proposal does not provide a mapping between doesn't see a good way to make it function alongside the existing event system properly, and recommends waiting until it is stable within the WebXDC API before making an MSC which supports it. As such, clients MUST set the function to the JavaScript `undefined` -until a MSC which explicitely declares support for it is merged into the specification. +until a MSC which explicitly declares support for it is merged into the specification. ## Alternatives From f0c0d279c1d101ee4bdb0aeda26af4c232f4576a Mon Sep 17 00:00:00 2001 From: lda Date: Thu, 10 Oct 2024 15:46:02 +0200 Subject: [PATCH 12/12] Update known Widget-based MSCs --- proposals/4211-webxdc.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/4211-webxdc.md b/proposals/4211-webxdc.md index 8780e3d7a9f..60b879f1a40 100644 --- a/proposals/4211-webxdc.md +++ b/proposals/4211-webxdc.md @@ -126,7 +126,8 @@ until a MSC which explicitly declares support for it is merged into the specific ## Alternatives -Widgets([MSC1236](https://github.com/matrix-org/matrix-spec-proposals/issues/3803)), while powerful, +Widgets([MSC1236](https://github.com/matrix-org/matrix-spec-proposals/issues/3803) and +[MSC4214](https://github.com/matrix-org/matrix-spec-proposals/pull/4214)), while powerful, also suffer from a similar agnosticity problem with other platforms, and effectively requires an actively online, external server. WebXDC also has stricter security concerns(e.g: clients cannot have sensitive information leaked, unlike widgets), thus making it safer, though a system that would also widgets to act