-
Notifications
You must be signed in to change notification settings - Fork 723
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Introducing TF v2 wrapper, ModelSignatureManager, and easy import of TF v2 models #2870
Merged
maziyarpanahi
merged 123 commits into
310-release-candidate
from
feature/saved-model-bundle-auto-wrapper
May 18, 2021
Merged
Introducing TF v2 wrapper, ModelSignatureManager, and easy import of TF v2 models #2870
maziyarpanahi
merged 123 commits into
310-release-candidate
from
feature/saved-model-bundle-auto-wrapper
May 18, 2021
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Conflicts: # docs/licensed/api/com/johnsnowlabs/license/Keys$.html # docs/licensed/api/com/johnsnowlabs/license/LicenseValidationException.html # docs/licensed/api/com/johnsnowlabs/license/LicenseValidator$.html # docs/licensed/api/com/johnsnowlabs/license/Licensed.html # docs/licensed/api/com/johnsnowlabs/ml/search/JSerializableKDTree.html # docs/licensed/api/com/johnsnowlabs/ml/search/Recipe.html # docs/licensed/api/com/johnsnowlabs/ml/search/SerializableKDTree.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/AssertionBatch.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/AssertionDatasetEncoder.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/TensorflowAssertion.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/TensorflowBertRE.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/TensorflowGenericClassifier.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/text2sql/ActionInfo.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/text2sql/Beam.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/text2sql/DatabaseDefinition.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/text2sql/HypMeta.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/text2sql/PreprocessedDB.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/text2sql/SchemaJSON.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/text2sql/Sem2SQL.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/text2sql/rule/Edge.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/text2sql/rule/Grammar.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/text2sql/rule/Graph.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/text2sql/rule/Node.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/text2sql/rule/Utils$.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/text2sql/rule/Utils.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/text2sql/rule/action/A.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/text2sql/rule/action/Action.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/text2sql/rule/action/C.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/text2sql/rule/action/Filter.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/text2sql/rule/action/N.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/text2sql/rule/action/Order.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/text2sql/rule/action/Root.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/text2sql/rule/action/Root1.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/text2sql/rule/action/Sel.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/text2sql/rule/action/Sup.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/text2sql/rule/action/T.html # docs/licensed/api/com/johnsnowlabs/ml/tensorflow/text2sql/rule/lf.html # docs/licensed/api/com/johnsnowlabs/nlp/FeaturesAssembler.html # docs/licensed/api/com/johnsnowlabs/nlp/HasFeaturesJsl.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/Chunk2Token.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/DrugNormalizer$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/DrugNormalizer.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/assertion/Datapoint.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/assertion/dl/AssertionDLApproach$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/assertion/dl/AssertionDLApproach.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/assertion/dl/AssertionDLModel$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/assertion/dl/AssertionDLModel.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/assertion/dl/ReadablePretrainedAssertionDL.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/assertion/dl/ReadsAssertionGraph.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/assertion/logreg/AssertionLogRegApproach$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/assertion/logreg/AssertionLogRegApproach.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/assertion/logreg/AssertionLogRegModel$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/assertion/logreg/AssertionLogRegModel.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/assertion/logreg/Datapoint.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/assertion/logreg/NegexDatasetReader.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/assertion/logreg/ReadablePretrainedAssertionLogReg.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/assertion/logreg/Windowing$VectorizedChunk.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/assertion/logreg/Windowing.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/chunker/AssertionFilterer$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/chunker/AssertionFilterer.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/chunker/ChunkFilterer$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/chunker/ChunkFilterer.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/classification/DocumentLogRegClassifierApproach.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/classification/DocumentLogRegClassifierModel$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/classification/DocumentLogRegClassifierModel.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/classification/ReadablePretrainedDocumentLogRegClassifierModel.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/context/ContextualParserApproach$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/context/ContextualParserApproach.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/context/ContextualParserModel$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/context/ContextualParserModel$MatchedToken.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/context/ContextualParserModel.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/context/Dictionary.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/context/EntityDefinition.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/datasets/AssertionAnnotationAndText.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/datasets/AssertionAnnotationWithLabel$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/datasets/AssertionAnnotationWithLabel.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/datasets/IndexedChunk.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/deid/DeIdentification$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/deid/DeIdentification.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/deid/DeIdentificationModel$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/deid/DeIdentificationModel$StringReplacement.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/deid/DeIdentificationModel.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/deid/DeIdentificationParams.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/deid/DefaultRegex$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/deid/Obfuscator$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/deid/ObfuscatorAnnotatorApproach$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/deid/ObfuscatorAnnotatorApproach.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/deid/ObfuscatorAnnotatorModel.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/deid/ObfuscatorParams$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/deid/ObfuscatorParams.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/deid/ReadablePretrainedDeId.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/deid/SentenceMaxException.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/deid/StructuredDeid.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/deid/StructuredDeidentification.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/deid/TextToDocumentColumns.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/deid/faker/Age$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/deid/faker/Company$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/deid/faker/Contact$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/deid/faker/Faker.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/deid/faker/ID$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/deid/faker/Location$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/deid/faker/Person$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/deid/faker/Profession$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/deid/faker/URL$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/NerDisambiguator$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/NerDisambiguator.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/NerDisambiguatorModel$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/NerDisambiguatorModel.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/SimpleDisambiguationPipeline.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/datatypes/ContextMappingsSchema$$Record.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/datatypes/ContextMappingsSchema$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/datatypes/DisambiguationDataTypes$$DisambiguatedSentence.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/datatypes/DisambiguationDataTypes$$Disambiguation.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/datatypes/DisambiguationDataTypes$$DisambiguationResult.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/datatypes/DisambiguationDataTypes$$Position.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/datatypes/DisambiguationDataTypes$$RawDisambiguationResult.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/datatypes/DisambiguationDataTypes$$Sentence.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/datatypes/DisambiguationDataTypes$$SentenceEmbeddingWithType.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/datatypes/DisambiguationDataTypes$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/datatypes/LinalgAliases$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/linalg/package$$ManifoldTopologyBreeze$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/logic/AnnotationLogic.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/logic/Knowledge.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/logic/KvKnowledgeExtractor.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/logic/Mappings.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/logic/PredictionLogic$Prediction.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/logic/PredictionLogic.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/logic/embeddings/EmbeddingsExtractor.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/logic/embeddings/PoolingLogic.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/logic/embeddings/impl/BowPoolingExtractor$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/logic/embeddings/impl/BowPoolingExtractor.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/logic/embeddings/impl/PoolingLogicBase.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/logic/embeddings/impl/SentenceEmbeddingsExtractor$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/logic/embeddings/impl/SentenceEmbeddingsExtractor.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/logic/embeddings/impl/SwitchableEmbeddingsExtractor.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/util/CategoryToDomainMapping$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/util/DisambiguatorModelParams.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/util/SwitchableSentenceEmbeddings$$SentenceEmbeddingColumns.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/util/SwitchableSentenceEmbeddings$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/util/linalg/CosineMetrics.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/util/linalg/CosineMetricsBreezeImpl.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/util/linalg/Topology.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/util/read/Reader.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/disambiguation/util/read/RocksDbReader.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/generic_classifier/GenericClassifierApproach$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/generic_classifier/GenericClassifierApproach.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/generic_classifier/GenericClassifierDataEncoder.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/generic_classifier/GenericClassifierModel$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/generic_classifier/GenericClassifierModel.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/generic_classifier/ReadablePretrainedGenericClassifier.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/generic_classifier/ReadsGenericClassifierGraph.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/merge/ChunkMergeApproach.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/merge/ChunkMergeModel$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/merge/ChunkMergeModel.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/merge/ReadablePretrainedChunkMerge.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/ner/IOBTagger$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/ner/IOBTagger.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/ner/NamedEntityConfidence.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/ner/NerChunker$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/ner/NerChunker.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/ner/NerConverterInternal$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/ner/NerConverterInternal.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/ner/NerTaggedInternal$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/ner/NerTagsEncodingInternal$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/re/BertREConfig.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/re/DLRelationInstance.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/re/PosologyREModel.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/re/REDataEncoder.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/re/REFeatureGenerator$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/re/RENerChunksFilter$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/re/RENerChunksFilter.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/re/ReadRelationExtractionDLModelTensorflowModel.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/re/ReadablePretrainedRelationExtractionDLModel.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/re/RelationExtractionApproach.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/re/RelationExtractionDLModel$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/re/RelationExtractionDLModel.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/re/RelationExtractionModel$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/re/RelationExtractionModel.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/re/RelationInstance.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/resolution/BigChunkEntityResolverApproach.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/resolution/BigChunkEntityResolverModel$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/resolution/BigChunkEntityResolverModel.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/resolution/BigFoundData.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/resolution/ChunkEntityResolverApproach.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/resolution/ChunkEntityResolverModel$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/resolution/ChunkEntityResolverModel.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/resolution/ConfidenceFunction$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/resolution/DistanceFunction$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/resolution/DistanceResult.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/resolution/JDataReader.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/resolution/JTreeComponent.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/resolution/JTreeData.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/resolution/JTreeReader.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/resolution/JTreeWriter.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/resolution/PoolingStrategy$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/resolution/ReadablePretrainedBigChunkEntityResolver.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/resolution/ReadablePretrainedChunkEntityResolver.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/resolution/ReadablePretrainedSentenceEntityResolver.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/resolution/SentenceEntityResolverApproach.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/resolution/SentenceEntityResolverModel$.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/resolution/SentenceEntityResolverModel.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/resolution/TreeData.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/resolution/params/ResolverParams.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/resolution/params/SentenceResolverParams.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/resolution/util/StringFunctions.html # docs/licensed/api/com/johnsnowlabs/nlp/annotators/resolution/util/StructFeature_HadoopFix.html # docs/licensed/api/com/johnsnowlabs/nlp/eval/BaseChunk.html # docs/licensed/api/com/johnsnowlabs/nlp/eval/BaseToken.html # docs/licensed/api/com/johnsnowlabs/nlp/eval/EvaluationMetrics$TpFnFp.html # docs/licensed/api/com/johnsnowlabs/nlp/eval/EvaluationMetrics.html # docs/licensed/api/com/johnsnowlabs/nlp/eval/EvaluationModes$.html # docs/licensed/api/com/johnsnowlabs/nlp/eval/HasBeginEnd.html # docs/licensed/api/com/johnsnowlabs/nlp/eval/POSEvaluation.html # docs/licensed/api/com/johnsnowlabs/nlp/eval/ner/NerCrfEvaluation.html # docs/licensed/api/com/johnsnowlabs/nlp/eval/ner/NerDLEvaluation.html # docs/licensed/api/com/johnsnowlabs/nlp/eval/ner/NerDLMetrics.html # docs/licensed/api/com/johnsnowlabs/nlp/eval/resolution/EvalEntityResolver.html # docs/licensed/api/com/johnsnowlabs/nlp/eval/spell/NorvigSpellEvaluation.html # docs/licensed/api/com/johnsnowlabs/nlp/eval/spell/SymSpellEvaluation.html # docs/licensed/api/com/johnsnowlabs/nlp/eval/util/GoldTokenizer.html # docs/licensed/api/com/johnsnowlabs/nlp/eval/util/LoggingData.html # docs/licensed/api/com/johnsnowlabs/nlp/eval/util/TagsMetrics$.html # docs/licensed/api/com/johnsnowlabs/nlp/eval/utils$.html # docs/licensed/api/com/johnsnowlabs/nlp/jsl/AnnotatorType$.html # docs/licensed/api/com/johnsnowlabs/nlp/jsl/SparkNLP$.html # docs/licensed/api/com/johnsnowlabs/nlp/jsl/functions$.html # docs/licensed/api/com/johnsnowlabs/nlp/pretrained/InternalsPythonResourceDownloader$.html # docs/licensed/api/com/johnsnowlabs/nlp/serialization/ResolverMapFeature.html # docs/licensed/api/com/johnsnowlabs/nlp/training/AnnotationDefinition.html # docs/licensed/api/com/johnsnowlabs/nlp/training/AnnotationToolJsonReader$.html # docs/licensed/api/com/johnsnowlabs/nlp/training/AnnotationToolJsonReader.html # docs/licensed/api/com/johnsnowlabs/nlp/training/AnnotationValue.html # docs/licensed/api/com/johnsnowlabs/nlp/training/AnnotationValueChoices.html # docs/licensed/api/com/johnsnowlabs/nlp/training/AnnotationValueLabel.html # docs/licensed/api/com/johnsnowlabs/nlp/training/CantemistReader.html # docs/licensed/api/com/johnsnowlabs/nlp/training/CodiEspReader.html # docs/licensed/api/com/johnsnowlabs/nlp/training/CompletionDefinition.html # docs/licensed/api/com/johnsnowlabs/nlp/training/NerAnnotationDefinition.html # docs/licensed/api/com/johnsnowlabs/nlp/training/RelAnnotationDefinition.html # docs/licensed/api/com/johnsnowlabs/nlp/training/TaskDataDefinition.html # docs/licensed/api/com/johnsnowlabs/nlp/training/TaskDefinition.html # docs/licensed/api/com/johnsnowlabs/storage/JSLDatabase$.html # docs/licensed/api/com/johnsnowlabs/util/distances/StringDistances$.html # docs/licensed/api/com/johnsnowlabs/util/external/wmd4j/Cosine$.html # docs/licensed/api/com/johnsnowlabs/util/external/wmd4j/Euclidean$.html # docs/licensed/api/com/johnsnowlabs/util/wmd4j/FrequencyVector.html # docs/licensed/api/com/johnsnowlabs/util/wmd4j/SparkNLPWordVectors.html # docs/licensed/api/com/johnsnowlabs/util/wmd4j/WordMovers.html # docs/licensed/api/com/johnsnowlabs/util/wmd4j/WordVectors.html # docs/licensed/api/com/johnsnowlabs/util/wmd4j/emd/EarthMovers.html # docs/licensed/api/index.html # docs/licensed/api/index.js # docs/licensed/api/lib/diagrams.css # docs/licensed/api/lib/diagrams.js # docs/licensed/api/lib/index.css # docs/licensed/api/lib/index.js # docs/licensed/api/lib/ref-index.css # docs/licensed/api/lib/scheduler.js # docs/licensed/api/lib/template.css # docs/licensed/api/lib/template.js
BPE, input IndexToken, Output Array[TokenPiece]
Bpe based tokenizers
End Index changed, prepend space for token without one
- remove cache until the Scala collection ops actually faster than re-doing the calculations again - change merges type, it will be the same from the beginning to avoid splitting and converting into Map on every token
- also disabling a MosesTokenizerTest test
…-zero-index Added fix for ending zero index in single char initial token in positional mask algorithm
Introducing RoBertaEmbeddings annotator based on RoBERTa models
Introducing DistilBertEmbeddings annotator based on DistilBERT models
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces the following features:
last_hidden_state
orpooler_output
) for SavedModel in TF v2 (currently only supports TF v1 with hard-coded placeholders for inputs and outputs)Annotators adapted to these changes to support TF v2 SavedModel:
NOTE: All the previous models exported from TF v1 or uploaded as pretrained are still compatible. This PR will extend the support to TF v2 SavedModel especially the use of
loadSavedModel
in Python/Scala to easily import models from TF Hub and HuggingFace for the listed annotators.NOTE2: There are other branches using TF v2 models which rely on this branch. (it was the only way to have support for TF v2 SavedModel for development)
########
This PR introduces a new annotator for RoBertaEmbeddings based on RoBERTa models (for instance roberta-base).
Import the following models:
roberta-base
roberta-large
distilroberta-base
########
This PR introduces a new annotator for DistilBertEmbeddings based on DistilBERT models (for instance distilbert-base-cased).
Benchmarks against
bert_base_cased
have shown better speed and very close accuracy in NER (faster convergence).Import the following models:
distilbert-base-cased
distilbert-base-uncased
distilbert-base-multilingual-cased
########
Other PRs