-
-
Notifications
You must be signed in to change notification settings - Fork 172
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Developing #257 - spoiler functions are reusable #258
Conversation
There is no
Can you please provide a repro for the breaking case using rest parameters? |
function check(value) {
return (
(typeof value === "object" &&
value !== null &&
!Array.isArray(value) &&
Object.keys(value).length === 2 &&
typeof value.x === "number" &&
typeof value.y === "number") ||
(typeof value === "object" &&
value !== null &&
!Array.isArray(value) &&
Object.keys(value).length === 3 &&
typeof value.x === "number" &&
typeof value.y === "number" &&
typeof value.z === "number")
);
}
About damaging data, I've started from assigning undefined value, but such damaging is also possible with non-undefined value. As you know, until |
This is the compiled validation routine. {
__cache: undefined,
} If you compare both TypeCompiler and Ajv, you will see they will both produce the same result. Specifying function test(value: { x: number, y: number }) {}
test({
x: 1,
y: 2,
__cache: undefined // error!
})
// Argument of type '{ x: number; y: number; __cache: undefined; }' is not assignable to
// parameter of type '{ x: number; y: number; }'. Object literal may only specify known
// properties, and '__cache' does not exist in type '{ x: number; y: number; }' Are you sure that TSON should be permissive of the |
const suprlus = {
x: 3,
y: 4,
note: "A 2d point"
};
const p: Point2D = surplus; // possible
TSON.is<Point2D>(surplus); // true
TSON.is<Point2D>(p); // true
TSON.is<Point2D>({
x: 1,
y: 2,
...{
surplus: "proeprty"
}
}); // true and can be Point2D type TypeScript also can possible to assign surplus property like above. Also, using However, this is |
Sure, If |
My json schema generator does not support the |
In the previous PR of yours (#256), I merged it without deep consideration. It is my mistake, but I repeatedly asked you that really is it okay and you told me that it is okay. At that scene, I told only undefined value assigned case. However, learning and reviewing your project, I undetstood that what I have missed. I am sorry for late understanding. I may got it too late because I was impressed by hyper performance shown in the |
Well, the schemas were fine until you updated the tests afterwards to invalidate them. While I think it's good to be able to identify where disparities in validation logic exist (and TSON being permissive of additional properties with I mean, Ajv can produce results for those failing tests, but has been disqualified on a technicality unrelated to the actual test being run. it's a bit of a shame, but oh well.
Yes, That's fine. And you're welcome to use any of TypeBox's optimizations in TSON if you find them useful. All the best! |
Heya, I've had a look at fixing up these benchmarks. Unfortunately, only Ajv's AjvAjvArrayRecursiveUnionImplicit.tsimport { TSchema, Type } from "@sinclair/typebox";
import Ajv from "ajv";
const ImageFile = Type.Object(
{
id: Type.Number(),
name: Type.String(),
path: Type.String(),
width: Type.Number(),
height: Type.Number(),
url: Type.String(),
size: Type.Number(),
},
{
additionalProperties: false,
},
);
const TextFile = Type.Object(
{
id: Type.Number(),
name: Type.String(),
path: Type.String(),
size: Type.Number(),
content: Type.String(),
},
{
additionalProperties: false,
},
);
const ZipFile = Type.Object(
{
id: Type.Number(),
name: Type.String(),
path: Type.String(),
size: Type.Number(),
count: Type.Number(),
},
{
additionalProperties: false,
},
);
const Shortcut = <T extends TSchema>(bucket: T) =>
Type.Object(
{
id: Type.Number(),
name: Type.String(),
path: Type.String(),
target: bucket,
},
{
additionalProperties: false,
},
);
const Directory = (bucket: TSchema) =>
Type.Object(
{
id: Type.Number(),
name: Type.String(),
path: Type.String(),
children: Type.Array(bucket),
},
{
additionalProperties: false,
},
);
const SharedDirectory = (bucket: TSchema) =>
Type.Object(
{
id: Type.Number(),
name: Type.String(),
path: Type.String(),
children: Type.Array(bucket),
access: Type.Union([Type.Literal("read"), Type.Literal("write")]),
},
{
additionalProperties: false,
},
);
const Bucket = Type.Recursive((bucket) =>
Type.Union([
ImageFile,
TextFile,
ZipFile,
Shortcut(bucket),
Directory(bucket),
SharedDirectory(bucket),
]),
);
export const __AjvArrayRecursiveUnionImplicit = Type.Array(Bucket);
const ajv = new Ajv();
const validate = ajv.compile(__AjvArrayRecursiveUnionImplicit);
export const AjvArrayRecursiveUnionImplicit = {
Check: (input: unknown) => validate(input) as boolean,
}; TypeBox
TypeBoxArrayRecursiveUnionImplicit.tsimport { TSchema, Type } from "@sinclair/typebox";
import { TypeCompiler } from "@sinclair/typebox/compiler";
const ImageFile = Type.Object(
{
id: Type.Number(),
name: Type.String(),
path: Type.String(),
width: Type.Number(),
height: Type.Number(),
url: Type.String(),
size: Type.Number(),
},
{
additionalProperties: false,
},
);
const TextFile = Type.Object(
{
id: Type.Number(),
name: Type.String(),
path: Type.String(),
size: Type.Number(),
content: Type.String(),
},
{
additionalProperties: false,
},
);
const ZipFile = Type.Object(
{
id: Type.Number(),
name: Type.String(),
path: Type.String(),
size: Type.Number(),
count: Type.Number(),
},
{
additionalProperties: false,
},
);
const Shortcut = <T extends TSchema>(bucket: T) =>
Type.Object(
{
id: Type.Number(),
name: Type.String(),
path: Type.String(),
target: bucket,
},
{
additionalProperties: false,
},
);
const Directory = (bucket: TSchema) =>
Type.Object(
{
id: Type.Number(),
name: Type.String(),
path: Type.String(),
children: Type.Array(bucket),
},
{
additionalProperties: false,
},
);
const SharedDirectory = (bucket: TSchema) =>
Type.Object(
{
id: Type.Number(),
name: Type.String(),
path: Type.String(),
children: Type.Array(bucket),
access: Type.Union([Type.Literal("read"), Type.Literal("write")]),
},
{
additionalProperties: false,
},
);
const Bucket = Type.Recursive((bucket) =>
Type.Union([
ImageFile,
TextFile,
ZipFile,
Shortcut(bucket),
Directory(bucket),
SharedDirectory(bucket),
]),
);
export const __TypeBoxArrayRecursiveUnionImplicit = Type.Array(Bucket);
export const TypeBoxArrayRecursiveUnionImplicit = TypeCompiler.Compile(
__TypeBoxArrayRecursiveUnionImplicit,
); TypeBoxObjectUnionImplicit.tsimport { Type } from "@sinclair/typebox";
import { TypeCompiler } from "@sinclair/typebox/compiler";
const Point = Type.Object(
{
x: Type.Number(),
y: Type.Number(),
slope: Type.Optional(Type.Union([Type.Null(), Type.Number()])),
},
{ additionalProperties: Type.Undefined() },
);
const Line = Type.Object(
{
p1: Point,
p2: Point,
distance: Type.Optional(Type.Union([Type.Null(), Type.Number()])),
},
{ additionalProperties: Type.Undefined() },
);
const Triangle = Type.Object(
{
p1: Point,
p2: Point,
p3: Point,
width: Type.Optional(Type.Union([Type.Null(), Type.Number()])),
height: Type.Optional(Type.Union([Type.Null(), Type.Number()])),
area: Type.Optional(Type.Union([Type.Null(), Type.Number()])),
},
{ additionalProperties: Type.Undefined() },
);
const Rectangle = Type.Object(
{
p1: Point,
p2: Point,
p3: Point,
p4: Point,
width: Type.Optional(Type.Union([Type.Null(), Type.Number()])),
height: Type.Optional(Type.Union([Type.Null(), Type.Number()])),
area: Type.Optional(Type.Union([Type.Null(), Type.Number()])),
},
{ additionalProperties: Type.Undefined() },
);
const Polyline = Type.Object(
{
points: Type.Array(Point),
length: Type.Optional(Type.Union([Type.Null(), Type.Number()])),
},
{ additionalProperties: Type.Undefined() },
);
const Polygon = Type.Object(
{
outer: Polyline,
inner: Type.Optional(Type.Array(Polyline)),
area: Type.Optional(Type.Union([Type.Null(), Type.Number()])),
},
{ additionalProperties: Type.Undefined() },
);
const Circle = Type.Object(
{
centroid: Type.Optional(Point),
radius: Type.Number(),
area: Type.Optional(Type.Union([Type.Null(), Type.Number()])),
},
{ additionalProperties: Type.Undefined() },
);
const Union = Type.Union([
Point,
Line,
Triangle,
Rectangle,
Polyline,
Polygon,
Circle,
]);
export const __TypeBoxObjectUnionImplicit = Type.Array(Union);
export const TypeBoxObjectUnionImplicit = TypeCompiler.Compile(
__TypeBoxObjectUnionImplicit,
); TypeBoxUltimateUnion.tsimport { TSchema, Type } from "@sinclair/typebox";
import { TypeCompiler } from "@sinclair/typebox/compiler";
const Attribute = {
description: Type.Optional(Type.String()),
"x-tson-metaTags": Type.Optional(
Type.Array(
Type.Object({
// @todo - must be specified, but too hard
kind: Type.String(),
}),
),
),
"x-tson-jsDocTags": Type.Optional(
Type.Array(
Type.Object({
name: Type.String(),
text: Type.Optional(
Type.Array(
Type.Object({
text: Type.String(),
kind: Type.String(),
}),
),
),
}),
),
),
};
const Unknown = Type.Object(
{
...Attribute,
},
{ additionalProperties: Type.Undefined() },
);
const NullOnly = Type.Object(
{
type: Type.Literal("null"),
...Attribute,
},
{ additionalProperties: Type.Undefined() },
);
const Atomic = (literal: string, type: () => any) => {
return Type.Object(
{
type: Type.Literal(literal),
nullable: Type.Boolean(),
default: Type.Optional(type()),
...Attribute,
},
{ additionalProperties: Type.Undefined() },
);
};
const Constant = (literal: string, type: () => any) =>
Type.Intersect(
[
Atomic(literal, type),
Type.Object({
enum: Type.Array(type()),
}),
],
{ additionalProperties: Type.Undefined() },
);
const Array = <T extends TSchema>(schema: T) =>
Type.Object(
{
type: Type.Literal("array"),
items: schema,
nullable: Type.Boolean(),
...Attribute,
},
{ additionalProperties: Type.Undefined() },
);
const Tuple = <T extends TSchema>(schema: T) =>
Type.Object(
{
type: Type.Literal("array"),
items: Type.Array(schema),
nullable: Type.Boolean(),
...Attribute,
},
{ additionalProperties: Type.Undefined() },
);
const Reference = Type.Object(
{
$ref: Type.String(),
...Attribute,
},
{ additionalProperties: Type.Undefined() },
);
const RecursiveReference = Type.Object(
{
$recursiveRef: Type.String(),
...Attribute,
},
{ additionalProperties: Type.Undefined() },
);
const OneOf = <T extends TSchema>(schema: T) =>
Type.Object(
{
oneOf: Type.Array(schema),
...Attribute,
},
{ additionalProperties: Type.Undefined() },
);
const ObjectDef = <T extends TSchema>(schema: T) =>
Type.Object(
{
$id: Type.Optional(Type.String()),
type: Type.Literal("object"),
nullable: Type.Boolean(),
properties: Type.Record(Type.String(), schema),
patternProperties: Type.Optional(
Type.Record(Type.String(), schema),
),
additionalProperties: Type.Optional(schema),
required: Type.Optional(Type.Array(Type.String())),
description: Type.Optional(Type.String()),
"x-tson_jsDocTags": Type.Optional(Type.Array(Type.Any())),
$recursiveAnchor: Type.Optional(Type.Boolean()),
},
{ additionalProperties: Type.Undefined() },
);
const Components = <T extends TSchema>(schema: T) =>
Type.Object(
{
schemas: Type.Record(Type.String(), ObjectDef(schema)),
},
{ additionalProperties: Type.Undefined() },
);
const Application = <T extends TSchema>(schema: T) =>
Type.Object(
{
schemas: Type.Array(schema),
components: Components(schema),
purpose: Type.Union([Type.Literal("swagger"), Type.Literal("ajv")]),
prefix: Type.String(),
},
{ additionalProperties: Type.Undefined() },
);
const Schema = Type.Recursive((schema) =>
Type.Union([
Atomic("boolean", () => Type.Boolean()),
Atomic("integer", () => Type.Number()),
Atomic("bigint", () => Type.Number()),
Atomic("number", () => Type.Number()),
Atomic("string", () => Type.String()),
Constant("boolean", () => Type.Boolean()),
Constant("integer", () => Type.Number()),
Constant("bigint", () => Type.Number()),
Constant("number", () => Type.Number()),
Constant("string", () => Type.String()),
Array(schema),
Tuple(schema),
Reference,
RecursiveReference,
OneOf(schema),
Unknown,
NullOnly,
]),
);
export const __TypeBoxUltimateUnion = Type.Array(Application(Schema));
export const TypeBoxUltimateUnion = TypeCompiler.Compile(
__TypeBoxUltimateUnion,
); Cheers |
I'll try the new benchmark about It may be next weekend. Thanks for the amazing patch. |
This question is just by my curious about the Does |
Yes, it has first class support TypeScript Link Here import { Type, Static } from '@sinclair/typebox'
const Point = Type.Object({
x: Type.Number(),
y: Type.Number(),
slope: Type.Optional(Type.Union([Type.Null(), Type.Number()])),
});
type Point = Static<typeof Point> // <-- hover Point
// type Point = {
// slope?: number | null | undefined;
// x: number;
// y: number;
// } |
Hi @samchon. Hey, Just looking at the benchmarks you've included in your readme, I do see TB is performing quite well (which is nice, and certainly appreciate you including TB in the performance graphs), but how you would feel about removing TB from the performance graphs, and just updating the following chart? To be honest, it was this chart in particular that prompted the initial PR's. I submitted TypeBox and Ajv to demonstrate it was possible for JSON Schema to express these data structures, and to offer a reference model for future updates to Again, I do appreciate you including TB so prominently in your benchmarks, but feel free to remove if you wish. I'd much rather see TSON / Typia performance shine against those other ecosystem libraries (which are way more popular than TB); and for you to use TB as a tool to help produce reference schemas for some of these more complex data structures (like UltimateUnion). Btw, Have you considered maybe moving the performance benchmarks into a separate project? The current benchmark site everyone uses is https://github.com/moltar/typescript-runtime-type-benchmarks, but note there isn't much variation between the schemas (the schemas are very simple), and they don't really highlight the functional differences between each library. If you wanted to set something up, I'd be happy to help contribute TB and other libraries for testing. Cheers dude! |
The reason why
Also, you showed me a nice idea avoiding failure case of implicit union type by using
Therefore, only wrong information from that table is Object (hierarchical) type about |
Well, these would the configurations for the varying tests with respect to additional properties. I think they could all be implemented (but would need separate schemas per test, so would be a pain to implement). The schemas up there should match the rules of the allow superfluous propertiesconst T = Type.Object({
x: Type.Number(),
y: Type.Number(),
})
{ x: 1, y: 2 } // ok
{ x: 1, y: 2, z: 3 } // ok
{ x: 1, y: 2, __cache: undefined } // ok disallow superfluous properties (implicit for union)const T = Type.Object({
x: Type.Number(),
y: Type.Number(),
}, { additionalProperties: false })
{ x: 1, y: 2 } // ok
{ x: 1, y: 2, z: 3 } // fail
{ x: 1, y: 2, __cache: undefined } // fail allow superfluous properties of undefined only (implicit for union with permissive __cache)const T = Type.Object({
x: Type.Number(),
y: Type.Number(),
}, { additionalProperties: Type.Undefined() })
{ x: 1, y: 2 } // ok
{ x: 1, y: 2, z: 3 } // fail
{ x: 1, y: 2, __cache: undefined } // ok |
Really is it possible to support all the data case? Whether possible or not, it just sounds like a crazy behavior for me. At one time, I had considered to mark You might have complain about what I'd kept changing the union implicit DTO types for creating failure cases. But on the other hand, it also means that Implementing union type specialization algorithm is not difficult. I just hope you to implement it and come back again. |
How? The schemas were written to match the cases you were testing for. You added additional tests after the fact, so I updated the schemas to conform to the specifications of the new tests. JSON Schema doesn't assume any default validation behavior, this is the programmers job to produce schemas with the desirable constraints. The goal of these PR's were to find these constraints, and I think that was largely achieved?
I'm not complaining, I'm trying to help. The updated tests with spoiler data revealed disparities between the validation behaviors of Ajv, So, assuming these schemas are aligned, the only thing left to do would be implement the same JSON schema for
Yes, I have reviewed the union specialization algorithm, unfortunately it violates many compositional components for both JSON Schema and JSON Type Definition (JTD), so can't be implemented in TypeBox. This doesn't mean you shouldn't implement in TSON, it's a good algorithm. |
I repeat that your special logic const raw = {
...props of ObjectUnionImplicit.IPolyline
...{
[RandomGenerator.string()]: "some random value",
}
};
const polyline: ObjectUnionImplicit.IPolyline= raw; // possible You can yield the above code is really crazy. However, this In such reason, for me, normal validation means supporting superfluous properties. Also, when a validation function being broken by those superfluous properties, I consider it as vulnerable. I've made On the other hand, I appreciate your contributions and special guidances for the JSON schema spec. I've learned my things from them. |
Cool, I'm glad :D Based on these schemas that have been setup thus far though. Would it be possible to have Inlineimport TSON from 'typescript-json'
interface Bar {
a: string,
b: string
}
interface Foo {
x: number,
y: number,
bar: Bar
}
const Foo = TSON.application<[Foo], 'ajv'>()
// const Foo = {
// $id: 'Foo',
// type: 'object',
// required: ['x', 'y'],
// properties: {
// x: { type: 'number' }
// y: { type: 'number' }
// bar: {
// $id: 'Bar',
// type: 'object',
// required: ['a', 'b'],
// properties: {
// a: { type: 'string' }
// b: { type: 'string' }
// }
// }
// }
// } Normalizedimport TSON from 'typescript-json'
type Ref<T> = T // marker
interface Bar {
a: string,
b: string
}
interface Foo {
x: number,
y: number,
bar: Ref<Bar>
}
const Foo = TSON.application<[Foo], 'ajv'>()
// const Foo = {
// $id: 'Foo',
// type: 'object',
// required: ['x', 'y'],
// properties: {
// x: { type: 'number' }
// y: { type: 'number' }
// bar: { $ref: 'Bar' }
// }
// }
const Bar = TSON.application<[Bar], 'ajv'>()
// const Bar = {
// $id: 'Bar',
// type: 'object',
// required: ['a', 'b'],
// properties: {
// a: { type: 'string' }
// b: { type: 'string' }
// }
// } Generic NormalizedUsing a convention that generic arguments be delimited with import TSON from 'typescript-json'
type Ref<T> = T // marker
interface Bar<T0, T1> {
a: T0,
b: T1,
}
interface Foo {
x: number,
y: number,
bar: Ref<Bar<string, number>>
}
const Foo = TSON.application<[Foo], 'ajv'>()
// const Foo = {
// $id: 'Foo',
// type: 'object',
// required: ['x', 'y'],
// properties: {
// x: { type: 'number' }
// y: { type: 'number' }
// bar: { $ref: 'Bar_string_number' }
// }
// }
const Bar = TSON.application<[Bar<string, number>], 'ajv'>()
// const Bar = {
// $id: 'Bar_string_number',
// type: 'object',
// required: ['a', 'b'],
// properties: {
// a: { type: 'string' }
// b: { type: 'number' }
// }
// } Recursiveimport TSON from 'typescript-json'
interface Node {
id: string
nodes: Node[]
}
const Node = TSON.application<[Node], 'ajv'>()
// const Node = {
// $id: 'Node',
// type: 'object',
// required: ['id', 'nodes'],
// properties: {
// id: { type: 'string' }
// nodes: {
// type: 'array',
// items: { $ref: 'Node' }
// }
// }
// } |
@sinclairzx81 Reviewing your special logic descriminating union type through
Object.keys().length
, no matther much I think about it, it dosn't make sense. The logic can be easily broken by object construction with rest parameters, commonly used by many TypeScript developers.Therefore, I added a logic breaking your
TypeBoxObjectUnionImplicit
type validator. Until you do not implement special algorithm for specializing union type, I can prepare inifite real use-cases which can break theadditionalProperties: false
option. I hope you to consider such environments andTypeBox
to be enhanced much powerfully.Also, the benchmark is comparing
is()
function. TheadditionalProperties: false
option is suitable forequals()
function. In such reasons, I brokeTypeBoxObjectUnionImplicit
andTypeBoxUltimateUnion
type and theadditionalProperties: false
would be removed soon.Instead, I recommend you to run
equals()
benchmark and it would be much faster than mine.