Derive "mentions" (for other accounts) from post message body #3

thisisaaronland · 2024-03-15T16:46:55Z

For example:

https://github.com/sfomuseum/go-activitypub/blob/main/app/post/create/create.go#L116-L126

thisisaaronland · 2024-04-18T00:43:58Z

First step, extracting addresses from text: #4

thisisaaronland · 2024-04-22T18:57:09Z

This is working except for the fact that @{OBJECT_ID}@ collection.sfomuseum.org links/mentions in the body of a post are not triggering the "profile page" dialog in clients like Ivory but are instead opening the links in a web browser.

For example, this message is not handled as expected:

{
   "@context" : "https://www.w3.org/ns/activitystreams",
   "actor" : "https://collection.sfomuseum.org/ap/testbot",
   "id" : "https://collection.sfomuseum.org/ap/@testbot/posts/1782474364306853888#15485f46-4e48-44d0-99ea-1d57c7399385",
   "object" : {
      "attributedTo" : "https://collection.sfomuseum.org/ap/@testbot",
      "content" : "This is a fourth test mentioning <a href=\"https://collection.sfomuseum.org/ap/@1880246575\">@[email protected]</a> with a link – <a href=\"https://collection.sfomuseum.org/objects/1880246575/\">https://collection.sfomuseum.org/objects/1880246575/</a>",
      "id" : "https://collection.sfomuseum.org/ap/@testbot/posts/1782474364306853888",
      "published" : "Mon, 22 Apr 2024 11:19:39 GMT",
      "tag" : [
         {
            "href" : "https://collection.sfomuseum.org/ap/1880246575",
            "name" : "@[email protected]",
            "type" : "Mention"
         }
      ],
      "to" : [
         "https://www.w3.org/ns/activitystreams#Public"
      ],
      "type" : "Note",
      "url" : "https://collection.sfomuseum.org/ap/@testbot/posts/1782474364306853888"
   },
   "to" : [
      "[email protected]"
   ],
   "type" : "Create"
}

However this message is working (as in: clicking on @[email protected] in Ivory opens the in-app "profile" dialog for the account):

{
   "@context" : "https://www.w3.org/ns/activitystreams",
   "actor" : "https://collection.sfomuseum.org/ap/testbot",
   "id" : "https://collection.sfomuseum.org/ap/@testbot/posts/1782476063658479616#ae835106-67fd-49ad-91f6-db97e2c4c38e",
   "object" : {
      "attributedTo" : "https://collection.sfomuseum.org/ap/@testbot",
      "content" : "This is YA a test mentioning <a href=\"https://orthis.social/@thisisaaronland\">@[email protected]</a> with a link: <a href=\"https://collection.sfomuseum.org/objects/1880246575/\">https://collection.sfomuseum.org/objects/1880246575/</a>",
      "id" : "https://collection.sfomuseum.org/ap/@testbot/posts/1782476063658479616",
      "published" : "Mon, 22 Apr 2024 11:26:25 GMT",
      "tag" : [
         {
            "href" : "https://orthis.social/users/thisisaaronland",
            "name" : "@[email protected]",
            "type" : "Mention"
         }
      ],
      "to" : [
         "https://www.w3.org/ns/activitystreams#Public"
      ],
      "type" : "Note",
      "url" : "https://collection.sfomuseum.org/ap/@testbot/posts/1782476063658479616"
   },
   "to" : [
      "[email protected]"
   ],
   "type" : "Create"
}

Unless I am missing something both messages honour the syntax described here:

https://www.w3.org/TR/activitystreams-vocabulary/#h-microsyntaxes

Which suggests that the post itself is correct but maybe the client (Ivory) is fetching the profile and comparing it to the mention? For example:

$> curl -s -H 'Accept: application/ld+json; profile="https://www.w3.org/ns/activitystreams"' https://collection.sfomuseum.org/ap/@1880246575 | jq '.id'
"https://collection.sfomuseum.org/ap/1880246575"

$> curl -s -H 'Accept: application/ld+json; profile="https://www.w3.org/ns/activitystreams"' https://orthis.social/@thisisaaronland | jq '.id'
"https://orthis.social/users/thisisaaronland"

But then if I do this:

$> curl -s -H 'Accept: application/ld+json; profile="https://www.w3.org/ns/activitystreams"' https://orthis.social/@thisisaaronland | jq '.name'
"thisisaaronland"

$> curl -s -H 'Accept: application/ld+json; profile="https://www.w3.org/ns/activitystreams"' https://collection.sfomuseum.org/ap/@1880246575 | jq '.name'
"Airsickness bag: Virgin America (2023.018.407)"

The name for the object is "Airsickness bag: Virgin America (2023.018.407)" rather than "1880246575" as noted in the post. That shouldn't be a problem, as I understand, the spec since {ACTOR}.preferredUsername is the unique account name (1880246575) and {ACTOR}.name is the display/full name.

$> curl -s -H 'Accept: application/ld+json; profile="https://www.w3.org/ns/activitystreams"' https://collection.sfomuseum.org/ap/@1880246575 | jq '.preferredUsername'
"1880246575"

$> curl -s -H 'Accept: application/ld+json; profile="https://www.w3.org/ns/activitystreams"' https://orthis.social/@thisisaaronland | jq '.preferredUsername'
"thisisaaronland"

Mentions for posts are added here:

https://github.com/sfomuseum/go-activitypub/blob/main/post.go#L59-L94

And validated again, before delivery, here:

https://github.com/sfomuseum/go-activitypub/blob/main/app/post/deliver/deliver.go#L180-L200

ThisIsMissEm · 2024-04-22T20:03:46Z

{
   "@context" : "https://www.w3.org/ns/activitystreams",
   "actor" : "https://collection.sfomuseum.org/ap/testbot",
   "id" : "https://collection.sfomuseum.org/ap/@testbot/posts/1782474364306853888#15485f46-4e48-44d0-99ea-1d57c7399385",
   "object" : {
      "attributedTo" : "https://collection.sfomuseum.org/ap/@testbot",
      "content" : "This is a fourth test mentioning <a href=\"https://collection.sfomuseum.org/ap/@1880246575\">@[email protected]</a> with a link – <a href=\"https://collection.sfomuseum.org/objects/1880246575/\">https://collection.sfomuseum.org/objects/1880246575/</a>",
      "id" : "https://collection.sfomuseum.org/ap/@testbot/posts/1782474364306853888",
      "published" : "Mon, 22 Apr 2024 11:19:39 GMT",
      "tag" : [
         {
            "href" : "https://collection.sfomuseum.org/ap/1880246575",
            "name" : "@[email protected]",
            "type" : "Mention"
         }
      ],
      "to" : [
         "https://www.w3.org/ns/activitystreams#Public"
      ],
      "type" : "Note",
      "url" : "https://collection.sfomuseum.org/ap/@testbot/posts/1782474364306853888"
   },
   "to" : [
      "[email protected]"
   ],
   "type" : "Create"
}

actor and attributedTo are mismatched, one contains an @ the other doesn't. You have two to fields, one on the Activity and one on the Note; You need more @context than just activitystreams.

Mastodon's mentions in the HTML specifically look like:

<span class="h-card" translate="no"><a href="#{h(url)}" class="u-url mention">@<span>#{h(display_username)}</span></a></span>

Perhaps that's a factor there?

The href's also don't match:

<a href=\"https://collection.sfomuseum.org/ap/@1880246575\">

vs

"href" : "https://collection.sfomuseum.org/ap/1880246575",

thisisaaronland · 2024-04-22T20:17:02Z

@ThisIsMissEm Thanks for the pointers. I guess the thing I don't understand is why the second message with a mention for @thisisaaronland, which contains all the same glitches, works fine. As in, clicking on @[email protected] loads the profile "inline" (in both Mastodon and Ivory) rather than opening a new browser window/tab with the link.

To confuse matters further, I noticed that this post which contains both a mention and a separate link behaves unexpectedly:

https://mastodon.social/deck/@[email protected]/112316250991932010

Specifically, the not-a-mention link opens the account profile "inline" in Mastodon rather than opening the object page itself while the @ mention link continues to open a new tab (rather than the profile).

Currently the mention blocks use the {ACTOR}.id property for the {MENTION}.url property but maybe it should be {ACTOR}.url ?

$> curl -s -H 'Accept: application/ld+json; profile="https://www.w3.org/ns/activitystreams"' https://collection.sfomuseum.org/ap/@1880246575 | jq '.id'
"https://collection.sfomuseum.org/ap/1880246575"

$> curl -s -H 'Accept: application/ld+json; profile="https://www.w3.org/ns/activitystreams"' https://collection.sfomuseum.org/ap/@1880246575 | jq '.url'
"https://collection.sfomuseum.org/objects/1880246575/"

$> curl -s -H 'Accept: application/ld+json; profile="https://www.w3.org/ns/activitystreams"' https://orthis.social/@thisisaaronland | jq '.id'
"https://orthis.social/users/thisisaaronland"

$> curl -s -H 'Accept: application/ld+json; profile="https://www.w3.org/ns/activitystreams"' https://orthis.social/@thisisaaronland | jq '.url'
"https://orthis.social/@thisisaaronland"

ThisIsMissEm · 2024-04-22T21:08:48Z

That might be it? I'm honestly not sure. Also, I'd only recommend testing this in mastodon's web UI, not in Ivory (since it's closed source and they might be doing something "special")

snarfed · 2024-04-22T22:33:47Z

I haven't fully read everything here, but it took me a while to get mentions to register in Mastodon too. Here are the conclusions that are currently working for me: snarfed/bridgy-fed#493 (comment)

snarfed · 2024-04-22T22:38:42Z

Also here's my code for extracting HTML links from content into AS1 mention tags. Converting those from AS1 to AS2 is pretty trivial, they just go into tag instead of tags and have type: Mention instead of objectType: mention and href instead of url.

thisisaaronland · 2024-04-23T00:57:04Z

I continue to be stumped. I can see that Mastodon is treating some of the mentions as "unhandled-link" (s) while others load their "profile" tab as expected.

Basically the problem seems to be any account from collection.sfomuseum.org, even ones that are already being followed. The "Create" Activity used to produce the screenshot above is included below.

It does not appear as though the mastodon.social servers are trying to fetch any data from the collection.sfomuseum.org servers when the post is sent so maybe it's not an Actor JSON problem? Or maybe it is but the problem has been cached (by Mastodon) ?

{
   "@context" : "https://www.w3.org/ns/activitystreams",
   "actor" : "https://collection.sfomuseum.org/ap/testbot",
   "cc" : [
      "https://collection.sfomuseum.org/ap/1913687651",
      "https://orthis.social/users/orthat",
      "https://collection.sfomuseum.org/ap/onview",
      "https://orthis.social/users/thisisaaronland"
   ],
   "id" : "https://collection.sfomuseum.org/ap/@testbot/posts/1782569705924464640#a5f0026b-4a6f-47cd-9414-0b84305f6ec5",
   "object" : {
      "attributedTo" : "https://collection.sfomuseum.org/ap/@testbot",
      "cc" : [
         "https://collection.sfomuseum.org/ap/1913687651",
         "https://orthis.social/users/orthat",
         "https://collection.sfomuseum.org/ap/onview",
         "https://orthis.social/users/thisisaaronland"
      ],
      "content" : "(9) Another test mentioning <a href=\"https://orthis.social/@orthat\">@[email protected]</a>, <a href=\"https://collection.sfomuseum.org/ap/onview\">@[email protected]</a>, <a href=\"https://orthis.social/@thisisaaronland\">@[email protected]</a> and <a href=\"https://collection.sfomuseum.org/ap/1913687651@\">@[email protected]</a> with a link – <a href=\"https://collection.sfomuseum.org/objects/1913687637/\">https://collection.sfomuseum.org/objects/1913687637/</a>",
      "id" : "https://collection.sfomuseum.org/ap/@testbot/posts/1782569705924464640",
      "published" : "Mon, 22 Apr 2024 17:38:31 GMT",
      "tag" : [
         {
            "href" : "https://collection.sfomuseum.org/ap/1913687651",
            "name" : "@[email protected]",
            "type" : "Mention"
         },
         {
            "href" : "https://orthis.social/users/orthat",
            "name" : "@[email protected]",
            "type" : "Mention"
         },
         {
            "href" : "https://collection.sfomuseum.org/ap/onview",
            "name" : "@[email protected]",
            "type" : "Mention"
         },
         {
            "href" : "https://orthis.social/users/thisisaaronland",
            "name" : "@[email protected]",
            "type" : "Mention"
         }
      ],
      "to" : [
         "https://www.w3.org/ns/activitystreams#Public"
      ],
      "type" : "Note",
      "url" : "https://collection.sfomuseum.org/ap/@testbot/posts/1782569705924464640"
   },
   "to" : [
      "@[email protected]"
   ],
   "type" : "Create"
}

thisisaaronland · 2024-04-23T01:16:23Z

Looking through the Mastodon source code I can see this:

let mention = this.props.announcement.get('mentions').find(item => link.href === item.get('url'));

It appears to be comparing the href property a given link to the url property in the Mention? Which is weird since Mentions don't have a url property. That suggests that props.announcement.get('mentions') is actually a list of "Actor" instances that have been derived from the mentions before template processing?

So, I guess the next step is ensure that all the {ACTOR}.id stuff for mentions with {ACTOR}.url,

https://github.com/mastodon/mastodon/blob/483fabf48a0a89aed405e6472ad07552bc8befd5/app/javascript/mastodon/components/status_content.jsx#L117-L128

https://github.com/mastodon/mastodon/blob/483fabf48a0a89aed405e6472ad07552bc8befd5/app/javascript/mastodon/features/getting_started/components/announcements.jsx#L71-L84

…ested

…ot ivory

thisisaaronland · 2024-04-23T21:03:16Z

Closing this issue as I am able to make Mastodon (www) do the right thing which requires:

Assigning the value of {ACTOR}.url property to a.href property in the HTML link for the mention
Assign the value of the {ACTOR}.id property to href property of the "Mention" tag passed around the (AP) post / create activity.

This still doesn't work for posts in Ivory but I will open a separate issue for that.

ThisIsMissEm · 2024-04-23T22:07:36Z

It might be an idea for you to contribute your knowledge around mentions to the ActivityPub Primer, perhaps in the "Addressing" section? https://www.w3.org/wiki/ActivityPub/Primer/Addressing

(Anyone with a W3C account (free) can participate in that wiki)

thisisaaronland pushed a commit that referenced this issue Apr 22, 2024

default to text logging; various tweaks trying to debug issue #3

d0a48e4

thisisaaronland pushed a commit that referenced this issue Apr 22, 2024

notes wrt/ {ACTOR}.Id or {ACTOR}.URL for mentions per issue #3

c6b95f3

thisisaaronland pushed a commit that referenced this issue Apr 23, 2024

issue #3 - update to use ACTOR.url (rather than id) for mentions, unt…

1405b01

…ested

thisisaaronland pushed a commit that referenced this issue Apr 23, 2024

issue #3 - sort out mentions and profile tab for mastodon (www) but n…

32f796d

…ot ivory

thisisaaronland closed this as completed Apr 23, 2024

thisisaaronland mentioned this issue Apr 23, 2024

Figure out why mention-ed accounts in post body are treated as links in Ivory #7

Open

thisisaaronland mentioned this issue Apr 24, 2024

Preview cards for posts with multiple links #8

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Derive "mentions" (for other accounts) from post message body #3

Derive "mentions" (for other accounts) from post message body #3

thisisaaronland commented Mar 15, 2024

thisisaaronland commented Apr 18, 2024

thisisaaronland commented Apr 22, 2024 •

edited

Loading

ThisIsMissEm commented Apr 22, 2024 •

edited

Loading

thisisaaronland commented Apr 22, 2024 •

edited

Loading

ThisIsMissEm commented Apr 22, 2024

snarfed commented Apr 22, 2024

snarfed commented Apr 22, 2024

thisisaaronland commented Apr 23, 2024 •

edited

Loading

thisisaaronland commented Apr 23, 2024

thisisaaronland commented Apr 23, 2024

ThisIsMissEm commented Apr 23, 2024 •

edited

Loading

Derive "mentions" (for other accounts) from post message body #3

Derive "mentions" (for other accounts) from post message body #3

Comments

thisisaaronland commented Mar 15, 2024

thisisaaronland commented Apr 18, 2024

thisisaaronland commented Apr 22, 2024 • edited Loading

ThisIsMissEm commented Apr 22, 2024 • edited Loading

thisisaaronland commented Apr 22, 2024 • edited Loading

ThisIsMissEm commented Apr 22, 2024

snarfed commented Apr 22, 2024

snarfed commented Apr 22, 2024

thisisaaronland commented Apr 23, 2024 • edited Loading

thisisaaronland commented Apr 23, 2024

thisisaaronland commented Apr 23, 2024

ThisIsMissEm commented Apr 23, 2024 • edited Loading

thisisaaronland commented Apr 22, 2024 •

edited

Loading

ThisIsMissEm commented Apr 22, 2024 •

edited

Loading

thisisaaronland commented Apr 22, 2024 •

edited

Loading

thisisaaronland commented Apr 23, 2024 •

edited

Loading

ThisIsMissEm commented Apr 23, 2024 •

edited

Loading