Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow tracking of json node location information #1046

Merged
merged 1 commit into from
Jun 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 56 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -341,8 +341,8 @@ Assertions contains the following additional information
| Arguments | The arguments used for generating the message.
| Type | The keyword that generated the message.
| Property | The property name that caused the validation error for example for the `required` keyword. Note that this is not part of the instance location as that points to the instance node.
| Schema Node | The `JsonNode` pointed to by the Schema Location.
| Instance Node | The `JsonNode` pointed to by the Instance Location.
| Schema Node | The `JsonNode` pointed to by the Schema Location. This is the schema data that caused the input data to fail. It is possible to get the location information by configuring the `JsonSchemaFactory` with the `LocationJsonNodeFactoryFactory` and using `JsonNodes.tokenLocationOf(schemaNode)`.
| Instance Node | The `JsonNode` pointed to by the Instance Location. This is the input data that failed validation. It is possible to get the location information by configuring the `JsonSchemaFactory` with the `LocationJsonNodeFactoryFactory` and using `JsonNodes.tokenLocationOf(instanceNode)`.
| Details | Additional details that can be set by custom keyword validator implementations. This is not used by the library.

Annotations contains the following additional information
Expand All @@ -351,6 +351,58 @@ Annotations contains the following additional information
|-------------------|-------------------
| Value | The annotation value generated

##### Line and Column Information

The library can be configured to store line and column information in the `JsonNode` instances for the instance and schema nodes. This will adversely affect performance and is not configured by default.

This is done by configuring a `LocationJsonNodeFactoryFactory` on the `JsonSchemaFactory`. The `JsonLocation` information can then be retrieved using `JsonNodes.tokenLocationOf(jsonNode)`.

```java
String schemaData = "{\r\n"
+ " \"$id\": \"https://schema/myschema\",\r\n"
+ " \"properties\": {\r\n"
+ " \"startDate\": {\r\n"
+ " \"format\": \"date\",\r\n"
+ " \"minLength\": 6\r\n"
+ " }\r\n"
+ " }\r\n"
+ "}";
String inputData = "{\r\n"
+ " \"startDate\": \"1\"\r\n"
+ "}";
JsonSchemaFactory factory = JsonSchemaFactory.getInstance(VersionFlag.V202012,
builder -> builder.jsonNodeFactoryFactory(LocationJsonNodeFactoryFactory.getInstance()));
SchemaValidatorsConfig config = new SchemaValidatorsConfig();
config.setPathType(PathType.JSON_POINTER);
JsonSchema schema = factory.getSchema(schemaData, InputFormat.JSON, config);
Set<ValidationMessage> messages = schema.validate(inputData, InputFormat.JSON, executionContext -> {
executionContext.getExecutionConfig().setFormatAssertionsEnabled(true);
});
List<ValidationMessage> list = messages.stream().collect(Collectors.toList());
ValidationMessage format = list.get(0);
JsonLocation formatInstanceNodeTokenLocation = JsonNodes.tokenLocationOf(format.getInstanceNode());
JsonLocation formatSchemaNodeTokenLocation = JsonNodes.tokenLocationOf(format.getSchemaNode());
ValidationMessage minLength = list.get(1);
JsonLocation minLengthInstanceNodeTokenLocation = JsonNodes.tokenLocationOf(minLength.getInstanceNode());
JsonLocation minLengthSchemaNodeTokenLocation = JsonNodes.tokenLocationOf(minLength.getSchemaNode());

assertEquals("format", format.getType());
assertEquals("date", format.getSchemaNode().asText());
assertEquals(5, formatSchemaNodeTokenLocation.getLineNr());
assertEquals(17, formatSchemaNodeTokenLocation.getColumnNr());
assertEquals("1", format.getInstanceNode().asText());
assertEquals(2, formatInstanceNodeTokenLocation.getLineNr());
assertEquals(16, formatInstanceNodeTokenLocation.getColumnNr());
assertEquals("minLength", minLength.getType());
assertEquals("6", minLength.getSchemaNode().asText());
assertEquals(6, minLengthSchemaNodeTokenLocation.getLineNr());
assertEquals(20, minLengthSchemaNodeTokenLocation.getColumnNr());
assertEquals("1", minLength.getInstanceNode().asText());
assertEquals(2, minLengthInstanceNodeTokenLocation.getLineNr());
assertEquals(16, minLengthInstanceNodeTokenLocation.getColumnNr());
assertEquals(16, minLengthInstanceNodeTokenLocation.getColumnNr());
```


#### Output formats

Expand Down Expand Up @@ -479,9 +531,9 @@ The following is sample output from the Hierarchical format.

When the library creates a schema from the schema factory, it creates a distinct validator instance for each location on the evaluation path. This means if there are different `$ref` that reference the same schema location, different validator instances are created for each evaluation path.

When the schema is created, the library will automatically preload all the validators needed and resolve references. At this point, no exceptions will be thrown if a reference cannot be resolved. If there are references that are cyclic, only the first cycle will be preloaded. If you wish to ensure that remote references can all be resolved, the `initializeValidators` method needs to be called on the `JsonSchema` which will throw an exception if there are references that cannot be resolved.
When the schema is created, the library will by default automatically preload all the validators needed and resolve references. This can be disabled with the `preloadJsonSchema` option in the `SchemaValidatorsConfig`. At this point, no exceptions will be thrown if a reference cannot be resolved. If there are references that are cyclic, only the first cycle will be preloaded. If you wish to ensure that remote references can all be resolved, the `initializeValidators` method needs to be called on the `JsonSchema` which will throw an exception if there are references that cannot be resolved.

The `JsonSchema` created from the factory should be cached and reused. Not reusing the `JsonSchema` means that the schema data needs to be repeated parsed with validator instances created and references resolved.
Instances for `JsonSchemaFactory` and the `JsonSchema` created from it are designed to be thread-safe provided its configuration is not modified and should be cached and reused. Not reusing the `JsonSchema` means that the schema data needs to be repeated parsed with validator instances created and references resolved. When references are resolved, the validators created will be cached. For schemas that have deeply nested references, the memory needed for the validators may be very high, in which case the caching may need to be disabled using the `cacheRefs` option in the `JsonSchemaValidatorsConfig`. Disabling this will mean the validators from the references need to be re-created for each validation run which will impact performance.

Collecting annotations will adversely affect validation performance.

Expand Down
2 changes: 1 addition & 1 deletion src/main/java/com/networknt/schema/JsonMetaSchema.java
Original file line number Diff line number Diff line change
Expand Up @@ -369,7 +369,7 @@ public static JsonMetaSchema getV202012() {
* Use {@link #getV4()} for the Draft 4 Metaschema, or if you need a builder based on Draft4, use
*
* <code>
* JsonMetaSchema.builder("http://your-metaschema-iri", JsonMetaSchema.getDraftV4()).build();
* JsonMetaSchema.builder("http://your-metaschema-iri", JsonMetaSchema.getV4()).build();
* </code>
*
* @param iri the IRI of the metaschema that will be defined via this builder.
Expand Down
43 changes: 43 additions & 0 deletions src/main/java/com/networknt/schema/JsonNodeLocation.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
/*
* Copyright (c) 2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.networknt.schema;

/**
* JsonNode location information.
*/
public class JsonNodeLocation {
private final int line;
private final int column;

public JsonNodeLocation(int line, int column) {
super();
this.line = line;
this.column = column;
}

public int getLine() {
return line;
}

public int getColumn() {
return column;
}

@Override
public String toString() {
return "JsonNodeLocation [line=" + line + ", column=" + column + "]";
}
}
16 changes: 6 additions & 10 deletions src/main/java/com/networknt/schema/JsonSchema.java
Original file line number Diff line number Diff line change
Expand Up @@ -16,15 +16,13 @@

package com.networknt.schema;

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.node.ObjectNode;
import com.networknt.schema.SpecVersion.VersionFlag;
import com.networknt.schema.serialization.JsonMapperFactory;
import com.networknt.schema.serialization.YamlMapperFactory;
import com.networknt.schema.utils.JsonNodes;
import com.networknt.schema.utils.SetView;

import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.net.URLDecoder;
import java.util.ArrayList;
Expand All @@ -46,6 +44,9 @@
* This is the core of json constraint implementation. It parses json constraint
* file and generates JsonValidators. The class is thread safe, once it is
* constructed, it can be used to validate multiple json data concurrently.
* <p>
* JsonSchema instances are thread-safe provided its configuration is not
* modified.
*/
public class JsonSchema extends BaseJsonValidator {
private static final long V201909_VALUE = VersionFlag.V201909.getVersionFlagValue();
Expand Down Expand Up @@ -871,15 +872,10 @@ public <T> T validate(ExecutionContext executionContext, JsonNode node, OutputFo
*/
private JsonNode deserialize(String input, InputFormat inputFormat) {
try {
if (InputFormat.JSON.equals(inputFormat)) {
return JsonMapperFactory.getInstance().readTree(input);
} else if (InputFormat.YAML.equals(inputFormat)) {
return YamlMapperFactory.getInstance().readTree(input);
}
} catch (JsonProcessingException e) {
return this.getValidationContext().getJsonSchemaFactory().readTree(input, inputFormat);
} catch (IOException e) {
throw new IllegalArgumentException("Invalid input", e);
}
throw new IllegalArgumentException("Unsupported input format "+inputFormat);
}

public ValidationResult validateAndCollect(ExecutionContext executionContext, JsonNode node) {
Expand Down
Loading