JSON Schema | TypeBox Benchmark

[sinclairzx81]

This PR implements benchmarks for JSON Schema comparative benchmarking. The benchmarks are implemented using the TypeBox type compiler for reference. As TypeBox generates JSON Schema Draft 6 compatible schemas, it should be possible to pass these schemas directly to Ajv for compilation also (see update below)

Note: This PR also provides a small fix for the ZodUltimateUnion by adding an additional literal type ajv for the Application purpose property. This should enable Zod to participate in the ultimate union benchmark.

This PR is submitted both for comparative benchmarking, and to potentially provide a reference model for JSON Schema generation in typescript-json. The new benchmarks should pass all current is, validate and assert measurements, however it would be good to replace the TypeBox type compiler with Ajv to get good visibility on performance deltas there.

Submitting for consideration

Update: Have since tested the schemas against Ajv version 6.12.6 (currently installed) and all benchmarks are passing with results (including the ultimate union). It should be possible to compare the TB schemas with the output of TSON.application<[...], "ajv">(). There may be a way to update the schema generation in typescript-json to get better Ajv compatibility for some of these more complex types. I'm hopeful these reference TB schemas may help.

[samchon]

Wow, it seems exciting. I'll update README's benchmark graphs soon

[sinclairzx81]

All good :)

Hey, just for Ajv benchmarking, you should be able to replace the following code....

export const __TypeBoxUltimateUnion = Type.Array(Application(Schema));
export const TypeBoxUltimateUnion = TypeCompiler.Compile(
    __TypeBoxUltimateUnion,
);

With this code...

import Ajv from 'ajv'
// ...
export const __TypeBoxUltimateUnion = Type.Array(Application(Schema));
const ajv = new Ajv()
const check = ajv.compile(__TypeBoxUltimateUnion)
export const TypeBoxUltimateUnion = {
   Check: (input: unknown) => check(input) as boolean
}

This should be enough to swap from the TypeCompiler to Ajv for testing. I'm hoping this PR helps you out with testing Ajv compatibility moving forward. It took me quite a while to get good schema alignment with Ajv in TypeBox, so happy to offer TypeBox as a reference point for TypeScript > Json Schema representation mapping in typescript-json.

Good work on TSON btw, it's looking good :)

[samchon]

Your typebox is really fast. About ultimate type, I will fix it. Thanks

[samchon]

I understood the reason why your library is much faster. It sometimes do not wrap an object type as a function.

However, I can show you how dangerous such optimization can be. I wanna repeat that again, typebox also should implement union speciaization algorithm. May I demonstrate your code using my test program's data spoilers like below?

https://github.com/samchon/typescript-json/blob/master/test/features/is/test_is_object_union_implicit.ts

[sinclairzx81]

May I demonstrate your code using my test program's data spoilers like below?

Sure. But you may want to include the additionalProperties constraint for ObjectUnionImplicit tests. See here for documentation.

const Line = Type.Object({
    p1: Point,
    p2: Point,
}, { additionalProperties: false }); // <-- include here

However, I can show you how dangerous such optimization can be. I wanna repeat that again, typebox also should implement union speciaization algorithm.

TB is somewhat tied to the validation capabilities of the JSON Schema specification, so not sure it's possible to update the existing algorithm, but happy to take a look. Do you have a link / reference to the algo?

[samchon]

Will change assert benchmark like below:

[samchon]

https://github.com/samchon/typescript-json/blob/master/src/programmers/helpers/UnionPredicator.ts

This would be helpful

[samchon]

@sinclairzx81

This code never be finished. Is it bug? or have I developed something wrong?

[samchon]

Oops, understood.

[sinclairzx81]

@samchon TypeBox returns an Symbol.iterator for Errors. So, you can use the spread operator to read them all out.

return [...TypeBoxObjectHeirarchical.Errors(input)]

However, it's going to be REALLY slow to do that. Note that TypeBox treats errors as a optional query you would run upon on validation failure, and it performs a secondary dynamic validation to gather errors (so no optimizations). You can include this if you want (tho assert usually means to throw) :D

[samchon]

Then only validate() benchmark would be meaningful. I'll send a PR and review it soon. Review it please after 15m later.