Coref model skipping long documents during training

[501Good]

I have been investigating the code for the coreference model to better understand its inner workings.

One thing that caught my attention is that during training documents longer than 5,000 subtokens are skipped without any warning.

stanza/stanza/models/coref/model.py

Lines 458 to 460 in af3d42b

	# skip very long documents during training time
	if len(doc["subwords"]) > 5000:
	continue

I think this behavior should be optional, or at least notify the user with a message when a document is skipped. Especially since some datasets (e.g. French-Democrat) have quite long documents (>10,000 tokens), which could lead to unexpectedly low performance.

[AngledLuffa]

Aye, that seems like a reasonable request. Let me see if I can make it an option.

[AngledLuffa]

Someone had sent me a coref dataset for Tamil a little while back, so I brushed that off and added the flag you were looking for as part of that.

#1467