SPARKNLP-1109 Adding Extractor to Sparknlp

[danilojsl]

Description

This PR introduces two new SparkNLP powered annotators: Extractor and Cleaner.

Extractor Annotator:
Enables seamless extraction of key information (e.g., dates, emails, IP addresses) from various data sources such as .eml files. This simplifies data parsing workflows by isolating relevant details automatically.

Cleaner Annotator:
Removes unnecessary or undesirable content from datasets, such as bullets, dashes, and non-ASCII characters, enhancing data consistency and readability.

These annotators are designed for scalable, high-performance data processing, leveraging Apache Spark to handle large datasets efficiently.

Motivation and Context

In many real-world NLP tasks, raw data often contains noise, such as irrelevant symbols, inconsistent formatting, or extraneous details. Similarly, extracting structured information from unstructured data sources can be complex and inefficient without dedicated tools. These new annotators aim to solve these challenges by providing built-in, scalable capabilities for data preprocessing, reducing manual effort and boilerplate code while improving pipeline performance and accuracy.

How Has This Been Tested?

Screenshots (if appropriate):

• Local test
• Google colab notebooks

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• Code improvements with no or little impact
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have read the CONTRIBUTING page.
• I have added tests to cover my changes.
• All new and existing tests passed.