change: allow blank nodes to be inserted in N3 Patches

[jeswr]

In todays CG and SolidOS calls we discussed this change, to allow blank node insertions in N3 patches.

In both cases, all present members saw no reason why blank node insertions should be forbidden. @bourgeoa indicated that it was likely a mistake, rather than an intentional decision to forbid blank node insertions in the spec.

If you have any objections to this change, please raise them in this PR. If no objections are raised, this PR will be merged 10 days from now.

Context:

• Inserts in n3patch may contain Blanknodes : Forbidden by CSS from N3PATCH spec linkeddata/rdflib.js#659 (comment)
• N3 Patch contradiction about blank nodes: can they be part of ?insertions? #704

cc @csarven @bourgeoa @CxRes @timbl @lecoqlibre

[csarven]

Some comments and a question before I can proceed with a technical review:

In https://lists.w3.org/Archives/Public/public-solid/2025Jan/0028.html , I asked:

Has the Solid CG recently decided to continue working on (some or all) items that are also part of the LWS WG charter deliverables?

The position of the Solid CG was stated in https://lists.w3.org/Archives/Public/public-solid/2025Jan/0044.html :

The general understanding is that only the core Solid Protocol report is currently an input of the WG.

My understanding of this is that the Solid Protocol is not something the Solid CG will continue to work on. However, this PR, along with the decision made during the Solid CG meeting on 2025-02-05 (minutes under review), appears to conflict with that understanding as per the Solid CG charter's Out of Scope section:

Community Group’s Work Items that have transitioned to a deliverable of an active Working Group.

This is further complicated by an unnecessary/redundant "decision" - since the charter had already ruled it out of scope - made in the 2024-09-11 meeting, literally:

PROPOSAL: CG stops working on the Solid Protocol draft

• MdJ: +1
• eP: +1
• PAC: +1

RESOLUTION: CG stops working on the Solid Protocol draft

This sends mixed signals. I ask once again: Is the Solid CG working on reports like the Solid Protocol or not?

• If yes, the CG's "final" decision should also be reflected in the CG charter and communicated to LWS WG. I'm happy to review if that understanding is made clear, with prior consent to review and edit.
• If not, then this PR can remain open but unmerged (while still collecting data, additional implementation reports, reviews etc.)

Please also note that as per Solid CG Contribution Guidelines, I believe this PR qualifies as a Class 3 change. If you believe it falls under a different category, please indicate that in the PR comments. Assuming it is a Class 3 change, it would be best to follow the guidelines to processing PRs - in this case, allowing "within 10 days or 2 meetings" to provide sufficient time for review.

[jeswr]

"within 10 days or 2 meetings" to provide sufficient time for review.

Good shout - I've updated the timeline in the original message.

Is the Solid CG working on reports like the Solid Protocol or not?

My interpretation (including the 2024-09-11 resolution) is that the CG should focus energy on deliverables outside of LWS scope; and thus not be actively working on the CG spec - however, this does not mean that the CG spec is locked.

This change would fix an active blocker in the solidcommunity.net migration of NSS -> CSS, and land before the LWS starts to actively use the input documents such as the CG spec - since LWS is still in use-case and requirements stages; so pragmatically I think it is a good idea for the change to be made now if there are no objections on technical grounds.

[elf-pavlik]

I would propose to keep the issue focused on technical conversation and move all the process-related chat to mailing list thread https://lists.w3.org/Archives/Public/public-solid/2025Feb/0019.html

[csarven]

Jesse, I find your interpretation of the charter's out of scope section and the resolution "CG stops working on the Solid Protocol draft" as meaning the spec isn't locked rather unusual. If the intent was simply to indicate that the spec isn't locked so to speak, there could've been many other, clearer ways to express that. See also this bit leading to that proposal/resolution:

• MdJ: We always said we just keep going until WG gets formed. Would the CG stop working on the core spec? I think that would make sense.
• PAC: Formally there is specific version of the Solid Protocol that is mentioned in the charter. It makes sense not to update that report. Some nit-picky people might object taking a more recent version. We don't need this kind of discussion. I think this is only one that should be frozen. The rest is mentioned as tentative and WG will decide as they go.

But, who exactly shouldn't be actively working on it? https://github.com/solid/specification/commits/main/ED/protocol.html

And if that's the case, isn't it strange that only three people in the entire CG voted "CG stops working on the Solid Protocol draft" on the spot, without consulting the actual primary contributors to the specification? I'm not asking you to justify it. If the intent wasn't to lock, what did the resolution even achieve given CG charter's out of scope?

This seems part of a broader pattern, particularly with the reluctance to move things forward in the Solid CG before the LWS WG charter was approved. For instance, see the concerns as mentioned here:
#227 (comment)

---

Pavlik, there's no reason why process-related comments can't be made here especially in this case when I'm raising concerns about mixed signals regarding the process for handling updates to certain specs. And when merge dates are being mentioned, it's better to err on the side of caution.

In any case, I've proposed a simple approach for mutual understanding between the LWS WG and Solid CG here: https://lists.w3.org/Archives/Public/public-lws-wg/2025Feb/0000.html

That'd be all I have to say on process here and happy to follow up once the mutual understanding is accepted/rejected.

[csarven]

Leaning on SPARQL Update and SPARQL Query for some guidance pertaining to INSERT and blank nodes, SPARQL Update doesn't prohibit INSERT's with blank nodes, but there are some expectations on how they are applied, e.g.:

https://www.w3.org/TR/2013/REC-sparql11-update-20130321/#insertData states:

Blank nodes in QuadDatas are assumed to be disjoint from the blank nodes in the Graph Store, i.e., will be inserted with "fresh" blank nodes.

Only blank nodes seem to be disallowed in DELETE, https://www.w3.org/TR/2013/REC-sparql11-query-20130321/#sparqlGrammar :

8. Blank node syntax is not allowed in DELETE WHERE, the DeleteClause for DELETE, nor in DELETE DATA.
9. Rules for limiting the use of blank node labels are given in section 19.6.

https://www.w3.org/TR/2013/REC-sparql11-query-20130321/#grammarBNodes :

The same blank node label can not be used in:
two INSERT DATA operations within a single SPARQL Update request

https://www.w3.org/TR/2013/REC-sparql11-update-20130321/#deleteInsert :

Blank nodes that appear in an INSERT clause operate similarly to blank nodes in the template of a CONSTRUCT query, i.e., they are re-instantiated for any solution of the WHERE clause;

So, as far as I can tell (and my hunch), removing ?insertions would at least better align with SPARQL Update.

---

As I see it, this is a class 3 change:

makes conforming data, processors, or other conforming agents become non-conforming according to the new version
makes non-conforming data, processors, or other agents become conforming

So, it is not a conformance issue for conforming client implementations that were not using blank nodes in INSERT. They can continue to work as they currently do but now have the option to include blank nodes in their INSERT queries.

As for previously non-conforming servers, i.e., the ones that were processing INSERT queries with blank nodes, with this change, they become conforming.

As for previously conforming servers, i.e., the ones that were not processing INSERT queries with blank nodes, with this change, they would become non-conforming. They have two general options: either implement to allow blank nodes in INSERT (successful response) or continue to run with reduced capability but make sure to reject payloads including INSERT and blank nodes with 405 or 415.

---

It is useful to be able to insert statements with blank nodes. Approving the PR based on the above. If there is information that's being overlooked as to why this change shouldn't be made, it'd be good to review that as well.

---

If not already, perhaps the N3 Patch tests can take the following into account from SPARQL Update's INSERT DATA ( https://www.w3.org/TR/2013/REC-sparql11-update-20130321/#insertData ).

Blank nodes in QuadDatas are assumed to be disjoint from the blank nodes in the Graph Store, i.e., will be inserted with "fresh" blank nodes.

[melvincarvalho]

During the SolidOS call, @bourgeoa mentioned that this might be seen as a bug fix rather than a spec change. Considering that this repo has 180 open issues, it's likely there are still edge-case bugs that will only surface during migrations and broader usage. I propose we allow some flexibility to address these issues on a case-by-case basis as we move forward.

[RubenVerborgh]

Removing this statement introduces ambiguity and incompatibility, as blank node scoping is not defined in N3, and existing implementations are known to differ. Additionally, it creates a multiplicity problem.

The blank node restriction was deliberately added to the document after earlier experimentation was found to trigger such problems in existing implementations. Instead, patches should use variables, with the spec-mandated restriction that those should also occur in the conditions block, such that the spec-mandated procedure for dealing with multiplicities can be used.

[bourgeoa]

Instead, patches should use variables, with the spec-mandated restriction that those should also occur in the conditions block, such that the spec-mandated procedure for dealing with multiplicities can be used

That is already the case in RDFlib.js CommunitySolidServer/CommunitySolidServer#1998 (comment)

[lecoqlibre]

Removing this statement introduces ambiguity and incompatibility, as blank node scoping is not defined in N3, and existing implementations are known to differ. Additionally, it creates a multiplicity problem.

@RubenVerborgh The proposal here still forbids to use blank nodes in the ?deletions (solid:deletes ). It just allows to have them in the ?insertions (solid:inserts):

• Current Solid spec: The ?insertions and ?deletions formulae MUST NOT contain blank nodes.
• Proposal: The ?deletions formulae MUST NOT contain blank nodes.

Note that the current Solid spec also states: The triples resulting from ?insertions are to be added to the RDF dataset, with each blank node from ?insertions resulting in a newly created blank node. This suggests that blank nodes are allowed in ?insertions, right? If we do not want to allow blank nodes in ?insertions I think the second part of this statement should also be removed.

The blank node restriction was deliberately added to the document after earlier experimentation was found to trigger such problems in existing implementations. Instead, patches should use variables, with the spec-mandated restriction that those should also occur in the conditions block, such that the spec-mandated procedure for dealing with multiplicities can be used.

I can understand that variables allow to clearly identify a blank node when deleting. But when inserting, the blank node does not exist yet in the patched document and thus it can't be identified in the ?conditions.

[RubenVerborgh]

@lecoqlibre There are still issues with blank node interpretation in N3 inside and outside of formulas.
We’d have to fix N3 before we can resolve this.

In any case, the current PR is missing a statement about uniqueness of any blank nodes that would occur in the insertions graph. And even if such a statement were to be added, the lack of consistent blank node semantics in N3 would prevent verification or enforcement thereof, so it would be void.

There are various cans of worms here; it's not an easy fix. N3Patch was always a stopgap; we knew this from its inception.

[bourgeoa]

@RubenVerborgh Can you explain what is different when using non spec PATCH with SPARQL update which is used in NSS and CSS ? what are the limitations that allow bnodes in INSERT ?

NSS made the choice to map n3-patch content to a SPARQL update PATCH content. With this there is a unique entry point in Rdflib.js.
At the same time the Rdflib.js updateManager uses the same algorithm to create the PATCH WHERE clause from inserts and deletes clauses.
I was with the idea that n3-patch at first (before future spec development) would only do what the actual and may be badly named SPARL update was doing in actual implementations.

I think that at this point it is this minimal PATCH that we would like to see implemented in all SOLID servers.

[lecoqlibre]

@RubenVerborgh I hoped there was an easy fix :)

So in the current Solid protocol we have no way to insert blank nodes with a PATCH request. We are then forced to do a PUT request. If a PATCH can't handle blank nodes it seems to loose its interest as blank nodes are a core feature of RDF. I think I already saw Solid apps that are only using PUT request even to patch documents. This workaround does the job but it's not satisfying and also leads to performance issues (transferring the whole document instead of just what needs to be changed).

N3Patch was always a stopgap; we knew this from its inception.

If this issue was known from the beginning, did someone already work on trying to resolve the problem?

We’d have to fix N3 before we can resolve this.

Would you have a plan? Could you provide more details about "issues with blank node interpretation in N3 inside and outside of formulas"? Is "the lack of consistent blank node semantics in N3" the only problem?

FYI @bourgeoa, Tim. Berners-Lee indicated that the main problem of SPARQL-update was the impossibility to make a lock (see this comment. He also added that N3 provides more features like to have more metadata about who made the change, etc.

[bourgeoa]

FYI @bourgeoa, Tim. Berners-Lee indicated that the main problem of SPARQL-update was the impossibility to make a lock (#715 (comment). He also added that N3 provides more features like to have more metadata about who made the change, etc.

@lecoqlibre I finally found the discussion. From what I understand of @timbl comment is that with the SPARQL specification and using a SPARQL endpoint there is no lock if you follow the spec.
The NSS implementation did a lock and was implemented as a divergence of SPARQL and was used for a few years without returned issues. I don't know the situation with CSS, but I think that is is also diverging and lock the file.

It seems that the actual point raised by @RubenVerborgh is with N3.js and marginally with the spec.

@timbl fully agreed with the need of bnodes.

Tim Berners-Lee
Yes the Solid Protocol says you just break the SPARQL rules. Which our servers did. Move to N3Patch removes that tension.

Yes, blank nodes should be allowed in a N3Patch insert clauses. And you can modify triples with blank nodes by identifying them indirectly with a where clause.

[RubenVerborgh]

the actual point raised by @RubenVerborgh is with N3.js

No, it is with the (lack of) a Notation3 specification/standard implemented by all parties.

Could you provide more details

Existing Notation3 implementations have diverging interpretations of how to handle blank nodes in Notation3 documents, when those blank nodes occur in different scopes/formulas (as would be the case with this proposal). Even worse, any constraint the Solid spec would impose on such blank nodes cannot be implemented consistently because of those diverging interpretations. We'd essentially ask to operate on a subset of N3 that we cannot properly define without changing N3 and updating existing implementations.

The original phrasing sidestepped that problem by sticking to a commonly agreed subset of Notation3 supported by all known parsers, at the cost of limiting functionality (superficially/cosmetically, in the sense that bnode: IRIs or similar could be used).

Workarounds are possible; they always are. A proper fix is not within this group's mandate, as it pertains to Notation3.

[bourgeoa]

@RubenVerborgh

In any case, the current PR is missing a statement about uniqueness of any blank nodes that would occur in the insertions graph. And even if such a statement were to be added, the lack of consistent blank node semantics in N3 would prevent verification or enforcement thereof, so it would be void.

Do you have a concrete example of what you mean by uniqueness.
Do you mean checking that the PATCH relates to only one document ?
or something else ?

[RubenVerborgh]

@bourgeoa

Do you have a concrete example of what you mean by uniqueness.

_:b1 a solid:InsertDeletePatch;
  solid:where   { _:b1 ex:familyName "Garcia". };
  solid:inserts { _:b1 ex:givenName "Alex". }.

The issue is that different parsers will disagree on whether the 3 syntactical occurrences of _:b1 refer to the same node or a different one. By the time the Notation3 syntax is parsed, that information is / must be presumed lost. Fixing requires changing Notation3 spec and parsers; not within this spec's mandate.

Time constraints might prevent me from further engaging in this thread, so I will halt my participation here.
TL;DR: Multiple cans of worms.

[bourgeoa]

 _:b1 a solid:InsertDeletePatch;
  solid:where   { _:b1 ex:familyName "Garcia". };
  solid:inserts { _:b1 ex:givenName "Alex". }.

This is interesting. From the spec I understand that this should not be a valid n3-patch even after the proposed modification
This would be a valid one

_:b1 a solid:InsertDeletePatch;
solid:where   { ?_:b1 ex:familyName "Garcia". };
solid:inserts { ?_:b1 ex:givenName "Alex". }.

I suppose that you find this as subject to interpretation

_:b1 a solid:InsertDeletePatch;
 solid:inserts { _:b1 ex:givenName "Alex". }.

Even if they relate to 2 different documents.
Could this resolve the issue ?

<> a solid:InsertDeletePatch;
  solid:inserts { _:b1 ex:givenName "Alex". }.

And there can be a provision to say that the
subject of this triple ?patch rdf:type solid:InsertDeletePatch MUST no be a blankNode

[lecoqlibre]

@RubenVerborgh I had in mind that we should allow this one:

1. Insert a new blank node:

@prefix solid: <http://www.w3.org/ns/solid/terms#>.
@prefix ex: <http://www.example.org/terms#>.

_:insert a solid:InsertDeletePatch;
  solid:inserts { ex:product ex:hasPrice [ ex:value "5.5"; ex:currency "euro". ] . }.

[RubenVerborgh]

@lecoqlibre We totally should—but we can't (easily).

A different syntactical representation of the above Notation3 document is:

_:p a solid:InsertDeletePatch;
  solid:inserts { ex:product ex:hasPrice _:v. _:v ex:value "5.5"; ex:currency "euro". }.

And, according to some (!) parsers, another one is:

_:p a solid:InsertDeletePatch;
  solid:inserts { ex:product ex:hasPrice _:p. _:p ex:value "5.5"; ex:currency "euro". }.

And sure, we can start restricting the syntax of N3Patch documents.
Meaning that Solid implementations also need a custom Notation3 parser specific to N3Patch.

We can do those things.

But none of them are a simple fix.

Better to build a proper patch format to replace N3Patch.

But again, not a simple fix.

[doerthe]

Dear all,

I am very late in reading the discussion but I am pleased to read that you discuss to use N3 for patches here. I was co-chairing the N3 community group and we released a first draft for the semantics where especially the meaning of blank nodes was important (all documents can be found here: https://w3c.github.io/N3/spec/).
We would like to go for a working group if there is enough interest. Would you be interested in having N3 pushed further. Would you have further use cases?

I would also be open to give more feedback to the patches as they are once I better understand (I am a little bit late in that conversation :) ).

[doerthe]

@RubenVerborgh

And, according to some (!) parsers, another one is:

_:p a solid:InsertDeletePatch;
  solid:inserts { ex:product ex:hasPrice _:p. _:p ex:value "5.5"; ex:currency "euro". }.

I do not know any implementation which would rename a blank node [] to another existing balnk node _:p, where does that come from and what is the reasoning for that? It looks like that is contradicting RDF?

[RubenVerborgh]

I do not know any implementation which would rename a blank node [] to another existing balnk node _:p

There are none.

The claim wasn't about renaming; the claim is about there existing different syntaxes to write blank nodes. IF we restrict the syntax of patches to [] for new blank nodes, then we don't run into such problems.

The argument is that syntax restrictions suddenly open up a whole new can of worms. N3Patch is currently defined on top of a shape-defined subset of Notation3 that is interoperable with a wide range of (old and new) parsers. Mitigations include:

• upgrading all parsers to the latest Notation3 specification
• defining a syntactical (rather than shape-based) subset of Notation3

Both require significantly more effort for implementers compared to what we have today. I.e., the small change in spec text proposed above, has disproportionally huge implications for implementers. None of those are easy fixes, and presumably exceed the mandate of the CG.

At this point, it's easier to define a custom patch format without such dependencies (or use any of the available RDF patch formats, bearing in mind their semantics). But that also does not qualify as an easy fix and is presumed to be outside of the mandate.

[doerthe]

Ah, then I misunderstood your comment here, sorry. Of course blank nodes are always complicated. I thought that it was simply about allowing blank nodes in insert-statements if you are sure that these blank nodes are "fresh".

[michielbdejong]

mind your Latin plurals there ;)

formulae -> formula

[michielbdejong]

Suggested change

	<li id="server-patch-n3-blank-nodes" rel="spec:requirement" resource="#server-patch-n3-blank-nodes"><span property="spec:statement">The <code>?deletions</code> formulae <span rel="spec:requirementLevel" resource="spec:MUSTNOT">MUST NOT</span> contain blank nodes.</span></li>
	<li id="server-patch-n3-blank-nodes" rel="spec:requirement" resource="#server-patch-n3-blank-nodes"><span property="spec:statement">The <code>?deletions</code> formula <span rel="spec:requirementLevel" resource="spec:MUSTNOT">MUST NOT</span> contain blank nodes.</span></li>

[TallTed]

Is this actually singular formula (being only ?deletions, and not ?insertions and ?deletions) or might there be multiple ?deletions formulae (which is how I read it)?

[jeswr]

We would like to go for a working group if there is enough interest. Would you be interested in having N3 pushed further.

Hi @doerthe I think the short answer is - yes, there would be interest in giving N3 WG status. I would endorse / support such a WG; and encourage my institution to vote in favour of a charter.

I am interested is in seeing a stable WG specification so there is no ambiguity on how blank nodes are interpreted. As far as I understand, there is no need from the Solid side for much further evolution of the specs - just a need for stability.